I found when I windows-buttoned out of the game, I could not get it to go back to full fullscreen. So in messing around with the dosboxULTIMA1.conf I made some changes that worked real well for me. changing:

fullresolution=original
windowresolution=original

these options in the first block of the config file, to a set resolution will allow you to play it in a window of your choosing (ie your native resolution) or like i did with my native resolution on my laptop is 1366x768 and I found changing them to 1024x768 gave me a nice sized window that i could see everything and move around easily.

I also recommend changing:

autolock=true

to false if you are playing in a window, that way you don't have to button out to jump between windows

edit: to fix the problem I originally set out to find (going back into full-screen non windowed mode) The solution is to just hit alt-enter.

There should be a "Graphic mode setup" shortcut in your startmenu that allows you to set most of this up (well, not the autolock part, but resolution, fullscreen/windowed, scalling engine, etc.)

This is the way to do it. Windows will auto-window any Dosbox game if you alt+tab out. Just alt+enter, and you'll go right back to having full screen!