You have to activate controller support in option menus.

It's either KB+M or controller.
Sound dumb but if I can save people time

I don't mind doing it, I just wish I knew the option exist in the first place.