Am I missing something or is there no way for me to "swim down" in this game?

Instead I am left to just sink very slowly to the bottom in order to the collect gems there. Pressing a button will just swim "up" even when pressing down as well. If the feature does not exist, could it be implemented or at least allow the Giana Sisters to sink faster as it takes forever the way it is now.

Have you tried just swimming down and no other button?

In addition to Jump+Down, I tried with all other buttons as well but nothing happens. I just keep sinking slowly downwards. I'm using a 360 controller.

Don't use jump. That will move you up. I play with keyboard myself so I can't give you an exact answer but if the controls are similar to the way they work on the keyboard, then you should be pressing nothing other than the directional arrow for down (or analogue stick or whatever), in the same way as you would crouch.

Going to start up the game and search for some water now.
edit:
Ok It tried it again. You may be right and there is no actual downward swimming. Thing is though that in my game I "sink" quite fast. Deffinitly not at a speed you'd want to accelerate. And "sinking" seems to accelerate the longer you sink.

It seems this ended up being a bug in the game itself. I just downloaded the recent update and now my character sinks at a reasonable speed whereas before it was painfully slow. I actually don't need to press anything.