I can think of a few ways to achieve better scaling. If you are dead set on integer scaling, there are builds of DOSBox that are patched to implement integer scaling, such as DOSBox ECE (Enhanced Community Edition) with its "pixel perfect" patch. Indeed, DOSBox ECE has some great features in general. However, it has been discontinued so, if you want to try it, you might need to hunt around for the binaries. The last official revision/build of ECE was r4482.
I would actually suggest against using integer scaling for Stonekeep and shall explain why. Stonekeep - like many MS-DOS games - uses a VGA mode having a resolution of 320x200 pixels. On a real VGA card, this mode produces non-square pixels! The presumption was that the monitor would contort the image into the expected 4:3 aspect ratio (something that a CRT monitor is able to make light work of). So, let's say that you use DOSBox ECE and enable pixel-perfect scaling. What will happen is that 320x200 will be scaled to 1600*1000 but, since the pixels on an LCD monitor are square, the aspect ratio will be wrong; the image will be vertically shorter than it ought to be. As such, it will not constitute an authentic experience.
So, what can one do? I imagine that you've dabbled with the scaling options of the coventional DOSBox builds that GOG ship and - quite understandably - found them to be wholly unsatisfactory. As such, I would highly recommend that you use a build of DOSBox that is able to support pixel shaders. DOSBox ECE happens to be among the builds that can load shaders. Specifically, it supports a "glshader" configuration key which can be given the path to a file containing a shader in the "glsl" language.
The effect of a good pixel shader is truly extraordinary. You may think that it is impossible to perform fractional scaling without ruining the graphics, but you will almost certainly change your mind once you see the effect in action. In particular, there are several mature shaders available that simulate the effect of using a CRT monitor while managing to scale 200p to 1080p beautifully, all while preserving the correct aspect ratio. The one that I last used for my playthrough of Stonekeep is named "crt-hylian.glsl". I have attached a 1920x1080 screenshot of the game in action with it. Naturally, the screenshot should be viewed at its exact size in order to be able to evaluate the proper effect of the shader.
If you're already got the game up and running in RetroArch, you might be interested to know that RetroArch is capable of downloading shader packs. Shaders can then be selected and applied from the Quick Menu (poke around under the "crt" sub-directory of wherever the shaders were extracted to). The Hylian shader is included, along with many others.
EDIT: The forum silently disregarded my attachment, possibly because it's too large. I'll try linking to it instead:
https://0x0.st/Xxg5.png
