Both solutions sadly don't work for me.
If I rename olwin the game just crashes before I even get to the main menu.
If I delete the entries in the registry like jnisbet suggested nothing happens, because the entry in layers just gets created new when I start the game.

That's strange. After you rename the .exe try to open olcfg.exe file in the Outlaws folder and change to D3D.

And... I'm running the olwin.exe with all the compatibilty settings UNchecked.

Okay, I am closer now.
Due to the change to D3D it works. BUT, now I have the problem, that the game starts glitching out whenever I look at a door or a window. The screen begins to shake and the gun freezes. When I look away from doors and windows it works fine. But considering how many of these exists in the game its not really acceptable.

I tried playing around with the advanced setting in olcfg.exe but it didn't help.

One thing I want to mention: In the windowed mode everything works alright. But you all know the thing with the mouse in that mode... does anyone know maybe a mod for this? Playing in a window is totally fine but I want to use the damn mouse. :D

An entry is recreated in Layers, but I believe that its value is different (equating to no compatibility settings applied, whereas, before, it equated to one or more compatibility settings).

Make sure that you deleted the actual "olwin.exe" key/folder inside Custom, not just the entries inside "olwin.exe."

If olwin.exe still crashes for you when you change its name, then I don't think that the registry has anything to do with it.

That happens to everyone. Direct3D isn't usable for that reason.

You need to switch to Glide in olcfg.exe. On that same page, you might go into the advanced settings for Glide and disable everything to rule out that it's one of the advanced settings. If it still crashes, try uninstalling and reinstalling the game.

Ok i think that this is the cause of music problem; i was having this problem and i have fixed it removing all Outlaws's keys from Layers and from Custom.

Next i have added "run as admin" to all .exe files of game.

i don't know if this is caused by all two entries (custom and layers) or only one of these (i think custom)

edit: HKEY_CURRENT_USER has some problems too, with DISABLEWDM and DISABLETHEMES, and these can disable music in-game... so don't use it, at least don't use it with windows 10 Anniversary Update. (i think it works fine with all older build)

This worked for me. It seems like Windows 10 does this for more games where i've had experienced problems. Previously i used this method with Bloodrayne after it automatically kept applying compatibility settings to the game and i've had to use this same method to prevent that from happening.

This game still not working. In 3dfx mode nothing happens and when I delete the entries nothing starts, only in D3D mode but then there is no sound. FIX IT!!!!!!!!!!!!!!!

Does your game folder have nglide_config.exe and glide.dll? If not, install nGlide from here and set the game to 3Dfx Glide in olcfg.exe.

As for sound (I assume you mean all sounds, not just music), have you also tried all three options on the Sound Configuration page of olcfg.exe? A3D probably doesn't work because it was a special driver, and DirectSound may not work because Microsoft removed it, starting with Vista. You can make both work, though, by simply downloading and adding a few files.

To make DirectSound mode work, try indirectSound. It consists of two files, dsound.dll and dsound.ini. Copy both to your Outlaws folder. Alternately, if you have a Creative card, you can download and install Creative Alchemy, but indirectSound is a simpler solution and works for all sound cards.

To make A3D mode work, try A3D-Live (you want the top link in the table, the "A3D-Live.exe" one). The site is in Russian and your antivirus might flag the file as containing a virus, but I think that it's safe and can confirm that it really does make A3D mode work. If you're still skeptical about running A3D-Live.exe, though, simply open it in a file compression program like WinRAR and extract out just a3d.dll and a3dapi.dll to your Outlaws folder (the other files aren't needed). If you have more than two speakers, however, also copying the indirectSound files from above may help (since the A3D files wrap to DirectSound and indirectSound restores DirectSound 3D, not just plain DirectSound).

Are you sure that all files are the same? (galaxy version and classic link from browser)

I'm using galaxy version that include all updates and it works fine.

Thank you very much for the help, but when I start with Direct3D mode everything works unless that it is quiet unstable. In 3dFX mode the game starts and I get a black screen. The sound ingame works but there is no music. Tried also the dll you have linked.

After installing it again and renaming olwin.exe music works, but now in direc3D mode game crashes every time in the beginning.

Direct3D mode is unusable. Even if you get it running, it'll trip out graphically whenever you try to enter a building. Give up on it.

You need to get 3Dfx Glide mode working. If you installed nGlide from the link that I gave, try deleting or moving the glide*.dll files in your game folder. If those are missing in the game folder, then the game will use the nGlide installed on your system, instead. It also wouldn't hurt to delete the 3DfxSpl*.dll files, since those come from nGlide, as well, though I believe that they're used only for showing the splash screen.

Thank you very much for your help. Now everything works:)
First I delete every entry from registry and make a new and clean install of the game. Next I have deleted the games own nglide from the directory and installed the official nglide 1.05. For the sound I renamed the olwin.exe to olwin_1.exe

I've tried everything here, but still havent managed to get the music to play. Sad because I seem to remember the soundtrack being quite good. :(

This is the what the dsound.log comes up with:
[i]
[[===========================================================================]]
This file is automatically generated every time that IndirectSound is used
[[===========================================================================]]

IndirectSound was loaded dynamically by a call to LoadLibrary()
The IndirectSound library being loaded is located at E:\Installert\Outlaws\DSOUND.dll
Version -- 0.12
The process loading IndirectSound is located at E:\Installert\Outlaws\olwin_1.exe
Product Name -- Outlaws
Product Version -- 2.0.1
File Description -- Outlaws
File Version -- 2.0.1
Initializing User Settings
(From the dsound.ini file)
hardwareBufferCount: 128
128 hardware-accelerated buffers will be emulated
eax1: true
EAX 1.0 support will be emulated
eax2: true
EAX 2.0 support will be emulated
monoSourceBehavior: "center"
2D sounds with mono sources will play out of the front center speaker if it exists
and the front left and right speakers otherwise
disabled3dBehavior: "2D"
3D sounds with disabled processing will play out of the same speakers that they would if they were 2D
Audio Device: Høyttalere (Realtek High Definition Audio)
Role: Global default device
Device ID: {0.0.0.00000000}.{a57470cc-05a5-486e-8afd-ae5241e0d7b4}
Speaker Configuration: Stereo
(Left and right speakers)
IndirectSound is being unloaded because the process is terminating[/i]