Zrevnur: About the bolded part: Is this a general thing that Nvidia is bad on a Linux gaming machine?
No. NVidia is better for gaming overall. Its weak parts (like Wayland support, initrd integration or Optimus technology on notebooks) have nothing to do with gaming.
Zrevnur: Probably due to no open source drivers?
NVidia provides excellent closed source drivers in time. That should be enough.
Zrevnur: What kind of gfx card do I optimally get? My priorities are in this order:
1. Reliability - it should last as long as possible not have a 50% chance to drop dead after 1 year of gaming
Go nVidia. Even 10 year cards are still supported.
Zrevnur: 2. Compatibility (GOG games mostly)
Go nVidia. Some (hastily ported) games, like "Divinity: Original Sin", do not work on AMD cards.
As you have already guessed it, go nVidia. :) It provides the most performative cards (like RTX 2080Ti) and quality drivers to boot (in order to provide the exact same level of performance as on Windows™). And if you want the most performance per watt, nVidia also wins here.
Personally I have settled for "GTX 1660Ti". I don't believe in ray-tracing for now (even if if technically works on Linux, I don't know any games that use it).
Simply put, nVidia "Just Works"™ while with AMD you are pretty much obliged to learn how to compile Mesa and LLVM (I don't even know what it is! :D ) from Git repository, to install custom-built beta kernel and, most importantly, how to
wait. Navi cards are not supported
even now! while some cards like "Radeon VII" are already discontinued (to be manufactured).