I run Rebel Galaxy v1.08b on a MacBook Air, OS X 10.9.5 (1366x768 resolution).
The game runs by default in Window mode in a higher resolution and I cannot change it. I wish to make it run full screen in 1366x768 but there is no such option in the settings. Also, the Launcher.app does nothing at all. Does any one know how to change these settings?

Not sure why the launcher isn't working for you, but that's where you change the settings for the game. It has options for resolutions, windowed/fullscreen and various graphical options(I have to run it on low on OSX or the opening cutscene stutters).

I don't know what to suggest as there's only the one file to download from GoG, which should start the launcher when you run it. Perhaps try getting it through GoG Galaxy or something, as the launcher runs every time I start the game.