I had that problem while messing with the DOSBOX CONFIGURATOR via GOG GALAXY ("GRAPHIC MODE SETUP") and running it on Windows 8.1 64 Bit.
It appears that it has to do with starting the game and at the first screen using the mouse click to continue... so try using the keyboard instead (e.g. "space" key) until you get to the main menu screen (3 key presses). Possible Solution:
Try unticking "Double Buffer" and SAVE the configuration. Then try changing the window resolution from "Original" to "320x200", SAVE. Set one last time "Original" and SAVE. Run the game. If that fails, delete the game via Galaxy and also the remaining folder (your saves are stored on the "SAVEGAM0.WL6" "SAVEGAM1.WL6" etc format and can be copied to another location then pasted back to the original one).
My current settings are:
Basic Settings Graphic Mode: Direct3D
Full Screen: Tick.
Advanced Settings Keep Aspect Ratio: Tick.
Double Buffering:
No Tick. Scaling Engine: hq3x
Mouse Sensitivity: 100
Window Resolution:
Original or if that fails
320x200 (game won't look any different with the other resolutions and might keep showing the Quit Dialog).
BONUS SETTING You could edit the dosbox_wolf3d.conf file (Make sure you backup it before doing that) and change cycles=auto to cycles=max so that everything runs smoother. REDVWIN
wtfpantera: Whenever I try running Wolfenstein 3D, I get through the loading screen and title cards but as soon as I reach the main menu, I'm asked one of the "Are you sure you want to quit? Y/N" questions. I keep pressing N, but the new variant immediately appears, until I give up and press Y.
Meaning - I can't play the game.
Help?