So if anyone wants to try that one day:

You install Oblivion in your wine prefix or through PlayOnLinux, play the game and see that it all works fine. Now you want to start modding and fire up the CS which is included in the installation. You load the base game (Oblivion.esm) and some of the DLC that came with it (eg. DLCShiveringIsles.esp). Now you see, that the CS seems to work perfectly fine, seeing everything, changes made are correctly stored in your own .esp etc. Now you want to add something to Shivering Isles but... Everything's purple and there's yellow boxes with exclamation marks all over the place.

While on Windows the full installation of both Oblivion and CS through GOG's installer works just fine, installing it in Wine on Linux comes with the problem mentioned above. Fortunately, there's an easy solution to that (thanks to ladyonthemoon from TESAlliance):

Get the "original" CS from the Nexus sites and install it on top of Oblivion (even if it already included the CS):
https://www.nexusmods.com/oblivion/mods/11367/

This will set the proper configuration to wine/registry and the CS, so it's not missing BSA files while loading other ESPs into the CS.