That's correct. It'll also show a "1/2" over shared tiles, if any exist.
As long as the resource is somewhere in that worked area, you'll get the benefit. In the case of shared mithral and adamantine resources, the cities split the income benefit but both will be able to generate mithral/adamantine-equipped units.
It's also worth noting that the worked tiles are blockaded by any units that are not your own. Even those of pacted/allied wizards will prevent a tile from being worked. This is also why other wizards get very upset if your units hang around near their cities, especially if you have a treaty with them, since blockading a city's tile is a violation of the treaty. (They're not quite as scrupulous about avoiding your cities' tiles, of course...)