If I throw something away in a trash can in Tarant (or any city for that matter), does it stay there forever, or does it eventually disappear?

It won't stay - trashcans respawn every day. If you want to make safe stashes, use chests in inns (those that stand by the beds in rented rooms). You can use them even if you didn't pay for a room. Other chests - some respawn, some don't. I never bothered to explore that topic further, because inns' chests work just fine and they're usually conveniently placed in any city.

Unlike in most games, trash cans in Tarant are indeed useful for disposing of items you don't want. I can't remember if trash cans in other cities all work the same; I really need to play Arcanum again.

Regarding stash spots in Tarant, I always liked that small shed next to the University's steamrail station. Doesn't respawn and is quick to get to.

You can store things in any of the trash cans in the warehouse by the docks, once you've cleared the place of rats.