The layout is still the same in French, indeed.
If it's a bug, well then:
Scroll down the main page and find 'Langue' in the bottom right corner. If you still have issues, well 'Add to cart' is 'Ajouter au panier', 'Check out now' is translated as 'Acheter' (= to buy), then the big green button below your payment info is also exactly the same thing, confirm payment. 'Remove' is 'Retirer'.
I actually had to check these a bit as I only use GOG in English ^^
Usually, when you change the language once logged in, GOG sticks to the new one. But if you're just browsing without being logged in, it can be whatever's detected from your location I think. I've also noticed that changing language when you're not on the main page may not trigger a full change at once.
----
A hint to learn French? Who would be so devious as to hint at such a thing? :D
Anyway, something to get you started - somewhat accurate French subtitles, historical accuracy, and a real (and useful!) French sentence too:
https://www.youtube.com/watch?v=DGXx56WqqJw&feature=youtu.be&t=57s