I went throughe games in this collection, found the DosBox config tool and changed the graphic settings (full-screen to windowed), but did not see any audio-related settings beyond those that pop up when you launch the title.

Changing the sound blaster type, IRQ, DMA channel, etc. did not change anything: there is no sound at all. No music or SFX.

Is there a way to make audio work with the GoG DosBox, or is it easier to just drag the game files over to a custom DosBox installation?

GoG's DOSBox is not that different from the mainline one. Manual setting are the same, just different defaults. What game exactly are you trying to setup? And am I correct considering that there were no sound related problems before you switched from full screen to windowed mode?

In terms of the best practice it of course would be better to have own custom DOSBox environment if you play old games often.
I'd recommend latest DOSBox ECE with pixel-perfect upscaler patch for pure 2d graphics games; and somehow otudated ykhwong's DOSBox for games with vector, flat shaded, ray casting (this includes Heretic and HeXen) and 3dfx accelerated graphics.

I had no audio at any point. I do have DosBox installed regularly, along with a graphical frontend. I have sound with that DosBox. For some reason the DosBox that is provided with Heretic, Hexen, etc. does not have sound.

I could debug this further, but I think I'll probably just end up using the working DosBox installation, or use ZDoom or similar as alternative.

Thanks for the response :)