OpenGL is an API for rendering 2D and 3D graphics. It is part of your graphics card drivers, so where software mode will most likely utilize CPU power, OpenGL (and similarly DirectX) will utilise your graphics card power.

With OpenGL you'll get fancy graphics like dynamic lights, shaders for lighting, glow, fog, etc., as well as texture resizing.

So, if you want the game to look like it originally did, then go with ZDoom as you don't need all the OpenGL options. But if you'd like to have improved lighting or other effects, or you want to play mods that require them, go with GZDoom.

Note that all of the OpenGL features can be turned off in GZDoom, making it identical to ZDoom, should you wish to have different setups but want to use just one source port. (You can store the different settings in separate GZDoom.ini files).