yodojojo: I think the way GoG handles it is a reduced game wrapper based on the Boxer app. Haven't found any documentation so far for changing the emulation parameters.
I'm using 10.9.5. I haven't had any problems so far, but I haven't got to any missions with the Death Star yet either. I did change "cycles=max limit 60000" in the dosbox preferences to help a little with
this bug.
You can find the DOSBox preferences file here if you want to try fiddling around with it:
RIght-click on the game app and select "Show package contents", then /Contents/Resources/game/Star Wars™- TIE Fighter Special Edition.app ("show package contents" here again) /Contents/Resources/Star Wars™- TIE Fighter Special Edition.boxer ("show package contents").
The file is called "DOSBox Preferences.conf", open it with a text editor (I use TextEdit). You'll probably want to make a copy before you make any changes so you can go back to it if anything breaks. This page here describes all the lines and what you can change:
http://www.dosbox.com/wiki/Dosbox.conf