I just got this game, and the frame rate is absolutely atrocious. The game can't seem to handle anything more stressful than a couple enemies at a time without serious mouse lag occurring.

Posting your system specs would go a long way in identifying your problem. I'm sure they are way beyond the requirements for the original game upon release back in 1998, but the game as released on GOG uses a wrapper to emulate old video cards and this process is resource heavy.

Is the framerate persistently low, or does it just break in sometimes? I had an issue where the game would only run at 5-10 fps when I would shoot the assault rifle. Turned out that nGlide was the problem. Turned it off, and then everything worked fine.

A general advice is to make sure that all your drives are update, the issue might also be the wrapper

If your game lag where muzzle flashes appear, you shooting at enemies or them at you the solution is to change r_dynamic "1" to "0" in the config.cfg in your player profile. That's confirmed problem not related to nglide but how game treats some features in 3dfx itself on modern machines, even with emulation isn't exactly optimized.

But that would turn off dynamic lightning, so why don't do yourself a favour and use OpenGL instead?

Well, game and gunplay looks the same to me in 3DFX while r_dynamic is off so I wonder if it ever worked and tried to render something tha's not visible in game while all main menu video options are still set to max

Default OpenGL has no special lighting and shadows, slower input response, also has darker screen overall, it's also possible in default that some levels reveal bugs since retail which were fixed only in 3dfx (famous elevator and several others).