Open up the file:
<Unreal Folder>\System\Unreal.ini
In the section entitled [Engine.Engine] there is a line for the selected render device:
GameRenderDevice=
You can change the value to any of the available render devices - more render devices are available if you apply unofficial extras, such as Kentie's DirectX10 renderer.
The render devices are listed in sections within that same ini file. For me I have the following render devices:
SoftDrv.SoftwareRenderDevice
GlideDrv.GlideRenderDevice
D3DDrv.D3DRenderDevice
D3D8Drv.D3D8RenderDevice
D3D9Drv.D3D9RenderDevice
OpenGLDrv.OpenGLRenderDevice
I use the unofficial 227 patch for Unreal Gold and as such Kentie's DX10 renderer is not compatible. As such I'm using the OpenGL renderer.
Note that each one of these renderers has settings within the Unreal.ini file that you can edit. Once you get the game launched you can also use the TAB key from the menu screen and enter the word PREFERENCES. When you then press Enter the game minimizes and opens an advanced settings dialog box where you can change various settings that take effect in the Unreal.ini file.