I'm a great fan of games with strange, moody, fantastical worlds to explore, such as Morrowind. Planescape: Torment, Metroid Prime, and most old dungeon crawlers (ie. Dungeon Master and Eye of the Beholder). Even the original Quake I would lump into this category. These worlds I find really stimulate my imagination and help me get much more immersed than even very atmospheric games such as the Stalker and Metro series.

Arx Fatalis is on my wishlist, but beyond that I'm not sure where to look so I would appreciate suggestions.

Note: the only game in the Ultima series I have played is the 7th iteration, which sadly really disappointed me so I have not looked into any of the others.

I really liked Outcast for it's world, might not be quite what you had in mind though...

Septerra Core is a good one.

Vangers? (Okay, maybe not the moody kind of world the OP has in mind. Probably strange enough, though.)

abe's oddysee - thats one messed up world

oddworld is the BEST.

Zeno Clash and Abyss Odyssey definitely apply.

This is what I wanted to write. Very interesting world and immersive.

Alice: Madness Returns. The game is not on GOG, unfortunately. But the whole world(s) in game is extraordinary and really frightening sometimes.

Arx Fatalis - although it is pretty much just a pretext for a dungeon crawler you're right to look to it for what you want.
Machinarium - probably about half of what I love about this game is the world it's in.
Bioshock - I guess that all the Bioshocks qualify. Although Infinite would have been a lot better if it gave you what you want and let you just explore a bit more.
Zeno Clash - highly imaginative to the point of almost making no sense. Worth a look.
NaissanceE - totally and utterly about exploring a strange world at the expense of everything else it seems. I haven't played it but it looks to me to check the right box for you.

Aside from the already mentioned Oddworld series, I'd say:

Anachronox
[url=http://www.gog.com/game/beneath_a_steel_sky ]Beneath a Steel Sky [/url]
Beyond Good and Evil
Brütal Legend
Dragonsphere
Legacy of Kain series
Primordia
[url=http://www.gog.com/game/rayman_2_the_great_escape ]Rayman series [/url](particularly Rayman 2)
Realms of the Haunting
Unreal

Terraria (maybe?)

I would also suggest checking out the Ultima: Worlds of Adventure spin-offs "Savage Empire" and "Martian Dreams", which have really interesting and weird scenarios.

Morrowind
Shadowman
Soul Reaver
Myst

Hm. La-Mulana might suit the bill if you like Indiana Jones and Metroid. Psychonauts may also work for you, on account of entering the minds of various people who have mental baggage.

Aaah, too many suggestions, can't keep up. Many of these games I have already played and loved, others I have been put off by reviews or for other reasons. That leaves:

Zeno Clash: I had not heard of these games, and there is even a demo for the original. Downloading it now. Thanks!
NaissanceE: wow, reading the description, it's like the devs made this game specifically for me. It got some pretty mixed reviews though so I'll wait for a sale. Thank you so much!

edit: and Outcast too looks very interesting, even if it doesn't look very moody.

psychonauts is also a great choice; both the real world and the mental world are brimming with color and personality.