Protoss: Go to the installation folder.
Edit the file dosboxMOM.conf
change the line
fullscreen=true
to
fullscreen=false
Or overwrite your old file with the attached one (rename the ending jpg to conf, sorry @GOG but you should really allow conf files to be attached)
As a side note: This goes for all DOSBOX games on GoG or elsewhere (Steam has some) - just modify the relevant .conf file and change the fullscreen option. There are a few other options you can change in there too - if you want deep control you can look into one of the many DOSBOX frontend software apps (I use D-Fend Reloaded), they make configuring various dos games much easier (and you can use them w/ GoG games easily)