I'm downloading the witcher 3 DLCs and patch update. It was about 36GB and has been downloading for 18 hours. At 96.4% it quits and says server problem (using the GOG app).

It will let me retry and picks up at 96.3% then quits again with the same error. I have tried it a few times. Any ideas?

I can't start over and wait 24 hours just to get screwed again..

It's called Galaxy, not "the GOG app", and that is your problem. This forum is rife with the exact same complaint from people using Galaxy. The only option you have (that I know of) is to try again later, since it's a server issue.