So I downloaded dgVoodoo and despite the game launching, it crashed to desktop. Changing the compatibility settings did not work either.

So I had a look at thaum.err and it turned out that DINPUT.DLL was causing issues.
The solution is simple:

Just download https://github.com/elishacloud/dinputto8/releases and copy the dinput.dll to the game dir.

I'm on Windows 10.

Nice, thanks for sharing this.