The Engine.dll for the new Master Server didn't help me much with direct IP connecting, but now I figured out what I did wrong all the time.
1. Forward the Ports 10480-10483 (maybe -10489)
2. Create LAN Server
3. Get external IP address on:
http://whatismyipaddress.com/
4. Get your server Port from:
SWAT4_InstallDir\Content\System\swat4.ini
SWAT4_InstallDir\ContentExpansion\System\swat4x.ini
Default Port = 10480 or 10481
5. Your buddies can now join with the direct IP option in the Server Browser and APPEND the Port with : like this
12.345.67.89:10481