Hi.
I'm making some tests with fluidsynth and several sound fonts.
Everyone seems to recommend FluidR3 soundfont. These are some fragments with that font (unfortunately they keep the 2 second silence gap at the beginning):
Fluid CR01
Fluid EM11
Fluid RB01
Fluid RB11

Personally, I think that the ChoriumRevA.SF2 soundfont fits much better to X-Wing. These are the same music fragments:
Chorium CR01
Chorium EM11
Chorium RB01
Chorium RB11

What do you think?
The midis should be edited to remove the silence at the beginning, and add a fixed amount of it after it ends to deal with the possible trailing reverb in a better way, as we discussed.

Wow, ChoriumRevA.SF2 sounds REALLY good, I like it!

I've already tried setting up a workflow in Reaper to render these out quickly (it takes a while to set up, since I have to do a bit of sorting and obviously removing the silence), but once set up it makes revisions to the files really easy.

Yeah, Chorium seems to be the best fit.

Anyway, something that might save time:

It seems like every music segment per prefix, except for transitions/intros, are the same amount of time. So everything DG01-32 is the same length of time. That will help later when we deal with trailing. (Note for that you'll need to record with something other than WMP because that seems to cut off hard at the end of playback. On the other hand that's a very good thing because that's a good way to easily record every segment without the trailing for now before that's handled!)

Second, the pre-music delay seems to be 1 measure long no matter the track.

So, this seems to be the best workflow for final recordings to handle trailing later on:

1. Remove MIDI pre-delays sequentially if possible:
2. Record in WMV to get the non-trailing time length first.
3. Record in a different player that doesn't cut off at the end
4. Use one of many various command line sound editing tools to add silence to the end of Recording From Step 3 to match the Length Of Recording From Step 2 + 1 second.

Also, since many concourse musics have an intro defined and then a looping segment within the same track, I propose that the Intro part and the Loop part are split into two separate files. That would provide the most seamless looping experience because whether or not the Intro part played before it would affect the trailing heard at the beginning of the Looping part.

That would be hard to do as a batch but only concourse music ever does that, so there aren't very many and would not be too hard to do manually.

Also, some warnings about concourse music:
-Like I said before, the format is Type 2, which means one MIDI file can contain several songs.
-The problem is that any additional songs past the first one do NOT seem to include the instrument assignment commands after being separated, if it is the same instrument as the first song in the file.
-So, somehow when making the concourse MIDIs before recording, you are going to somehow have to copy the instruments per channel from the first song into all of the subsongs, but only the ones that are not defined.

Example:
-tourdesk.GMD has two tracks.
-In Track 2, channels 7, 9, and 13 do not have instruments assigned. Channel 15 does.
-The solution is to use the instruments defined for channels 7, 9, and 13 in Track 1. Channel 15 is left alone.

I'm mostly useless in terms of my skill set for this project, but here are a few ideas I'll offer.

1. the name. How about OpenSfoil? in keeping with OpenXcom , plus it's a pun. The Internet loves puns!

2. In terms of creating immersion, we'll probably do well to make new cut-scenes. This is something i can actually do/help with.

3. I've been spending all my time with TIE98 lately, so i don't know... do we have high-quality voice files to work with in X-WING? If not (or if we want to implement more immersive radio messaging), I'd be happy to work on that as well. I know several amazing voice actors who've worked on games before and who'd jump at a project like this.

otherwise, put me down as a beta tester!

It sounds fun. However we are currently at a planing/analysis phase, while OpenXcom is basically finished and under several layers of polishing already. It feels like we would be making use of their popularity to promote ourselves.
I propose that once we have something further on in the development cycle, with at least a couple of iterations behind us, we vote for a name for the project.

I am with Tarvis here.
The only reason I am bothering to make wave files of the midi files is because at this point I find it faster to develop something that makes use of wave audio than starting to implement N different audio synthesis engines, that depend on X different variables in the files and Y other different variables on the user's computer in order to sound good at all.
We are making first a prototype of the remake, to show if we are going somewhere or there is nowhere to go.

I also fantasize with voice messages during missions, debriefings, HD cutscenes, etc. But all of that take us closer to other previous projects like X-Wing Redux.
Once we have the foundation ready, then we can start implementing extra optional features that you can select from a list like in OpenTTD: original, XW95, or XWAU models; MIDI music or digital music; original art (cutscenes, concourse) or HD remade art; original cockpits or 3D cockpits; etc.

Maybe we need to establish a roadmap. I would say that it should look a little bit like this:

1- Prototype the different "landmark" systems: dynamic music, in flight sim, menu navigation system, briefings, pilot handling.
2- If the prototypes are promising, rework or refine them and integrate them together. This might involve a change of framework if the current one proves unfit.
3- Complete the basic scope of the project, that is, X-Wing with the most immediate technological improvements, without altering the content itself: resolution, input, bugfixes, evident gameplay improvements.
4- Develop a mod system to allow for changing the content.

With "evident gameplay improvements" I mean stuff that doesn't really change the game much and the devs added or changed in TIE Fighter because it was just clearly better. These I plan to add from the beginning because there is no point on mimicking an old gameplay feature if it was soon after replaced by something better. For example:
- Targetting: Four brackets enclose the selected ship in the starfield, instead of it blinking red. It has better visibility at all distances and it is more friendly to colorblind people.
- Keymapping: Match speed with target (Enter key) was added in TIE Fighter, and later in XW95. I see no reason for not having it here. The same for the Q, E, U and other keys that just make your life easier without actually changing the game. Maybe also an Alt+T feature for fast forwarding until the next event happens.
- Pilot backup: Because otherwise people will need to use third party pilot revivers. Nobody liked losing their pilot everytime they failed at a particularly hard mission. And those missions you sometimes need to replay many times. It was annoying in X-Wing, not fun. They "patched" it in TIE Fighter by making it auto backup and restore after a capture or a death. They totally removed it in XvT and XWA because it never served any purpose.
I would instead account the number of death/captures and log it in the pilot log just in case someone wants to play in permadeath style.

Count on it! :)

Here's some other controls-related improvements that would make joystick users very happy:

-Full 32-step throttle axis instead of 33% increments (like XWA)
-Rudder roll axis (also like XWA)

-Decrease Laser/Shield Energy commands (take the recharge rate a step down instead of a step up) Could go on Shift-F9/F10 or Alt-F9/F10

-Direct Laser/Shield Recharge Axes: For use with rotaries for throttles. Basically, rotary at center detent: charging at maintenance level. Rotary all the way down = drain energy, rotary at max = charge energy. Should stay at 5 steps like the old games did [-2, -1, 0, 1, 2], I think. I don't see the benefit for finer recharge rate control and it would probably overcomplicate things.

-Support multiple joystick devices (the Windows games only supported one but most HOTAS are reported as separate devices)

-View position axes. Will be handy for things like TrackIR and possibly Oculus Rift. Even though it'll look janky it should also support the directional 2D cockpit views too, because it still has the ultimate function of looking to the left when you physically look left. Later 3D cockpits would allow for fine angular control instead of locking to 45 degree directions

As for mechanics, one thing in particular stands out to me: The Targeting Computer view mode. Frankly it seems useless to me, since it is no more accurate than the F and R radar views! I think ways to make it useful would make the "dot" correspond to where to aim to hit the target, and not the target's position, and second to make it more "zoomed in" for finer aiming. This would be a bit of a crutch though, so I understand not doing this since it would change gameplay balance.

And lastly...S-Foils. Did they have any functional purpose besides the novelty? I wonder if a speed boost when S-Foils are closed like Rogue Squadrion would be nice. Again this is one of those things that would change gameplay, though.

Pilot restoration: Even the DOS CD version of X-Wing added this, so it's a must. Automatically doing it and going back to the briefing TIE-Fighter style would be a help, though, instead of having to hit "Modify Pilot" and go back through the concourse.

Yes, I would totally love to see an improved version of this game done. If you are looking for voice over artists for characters myself and a couple other guys are pretty good with that kind of stuff.

I think you should form a team and get this done, there are plenty of Star Wars fans and X-Wing fans that would love to see this done as well.

Also, it wouldn't bet too hard to find people who have specific accents or talents with specific characters.

The Solomon's Revenge addon voice actors come to mind.

I can't post a link to my portfolio because gog doesn't let you post links until you are rep 20, but if you are interested message me.

Anyway, I envision like you a vastly improved version of this game and would love to see a version with superior graphics and gameplay elements as well as user inputs improved upon.

You can post links fine under 20 rep, just not if it's the first post in the thread.

Anyway, I figured out how the loops are stored in the MIDI files, and how the beginning song gaps are skipped. It is indeed a MIDI SysEx command in this format. It is a Jump command, telling the player to jump to a given position in the song. So, every MIDI has a Jump command in the beginning to tell it to skip the silence block, and at the end, a command to tell it to loop back to a certain point. (For reference: every MIDI in iMUSE is 480 ticks per beat)

SysEx: 7D 30 00 00 00 00 00 00 00 BEAT TICK

where BEAT and TICK tell the player which beat (INCLUDING the silence at the beginning) and tick to jump to.

They are in a bit tricky format like this: To "jump" to beat 16, tick 450, this is what must be done:
-Convert 16 and 450 to 16 bit big-endian numbers (C and 1C2)
-Pad each digit with a 0 (0C and 01 0C 02)
-Pad 0s so that each number takes up 8 bytes (00 00 00 0C and 00 01 0C 02)

Therefore, fully constructed SysEx for jumping to Beat 16 + Tick 450 is:

SysEx: 7D 30 00 00 00 00 00 00 00 00 00 00 0C 00 01 0C 02

You will find this command near the end of the first track of HALMARCH.GMD, for example. (It's not the last one, I'm not sure what the last one is, but it seems the engine will never get to that point because it reaches the Jump command above first)

Remember, the first beat is Beat 0, not Beat 1!

There are also other SysEx commands at the end of each bar that signify that it's OK for the engine to do a transition to a new room, for each possible transition. So those would have to be handled for the MIDI implementation of our iMUSE simulator. For now though, this can't be used with digital files, so we don't need to worry about that. Unless we make a system to track where each of these jump points can be in a digital recording, we'll just have to make it jump at the exact time it's asked for (for Concourse music. Inflight music is chopped up into separate bars so it's a different story there)

I don't know if you mean that Oculus and TrackIR should support the 2D cockpit view.
I don't see how that would work without inducing headache or claustrophobia. I think Oculus adn TrackIR will need to wait to 3D cockpits for usability reasons. Anyway, we are talking way in the future here now. :)

Do you have any nostalgia feeling for the XW targeting computer? Because even when I agree on the premise that we should stay true to the original game, I personally think that XW's targeting computer is absolutely useless, and the devs found out a better use for it in TIE Fighter, and their hands didn't shake a little when they retrofitted TIE's targeting computer to the rebel craft in XvT and XWA.

In my prototype I already experimentally added the aim hint thing, though. I mean, when you have a TIE Figher selected and in front of you, you get to see a little indicator telling you where to shoot at, accounting for your weapon speed and your target's current vector. It simple as hell to get that done with easy vector math, so I did it just to feel the gameplay with it.
It makes dogfighting much easier for newbies, and allow veterans to easily snipe TIEs from far away. So I think if we choose to keep it, it should be an optional setting, or some easy mode that grants a lower score in the mission.

I also added a target relative direction hint. It's an arrow that appears when your target isn't in front of you, and it points to the direction you need to turn to to face it. This one is maybe redundant with the F and R sensor spheres, but it helps a lot during close dogfights.

There is a big difference between what XWCD does and what TIE does. In XWCD you can revive the pilot, but it wipes out all your scores and demotes you to Officer. In TIE, if you die or are captured, the game rolls you back to the point just before you started the mission instead. So you don't get to keep any score or kills you did during the mission you went down, but neither you lose your previous score, rank and kill count.

It is funny that the XW developers put so much effort on making deaths or captures so punishing, to the point of the flight sim itself accesses your pilot file and kills it in the precise moment that your ship explodes (instead of waiting to the debriefing room, for example). So you couldnt kill the game in order to save your pilot.

I would go for just accomplishing what TIE fighter went after, only without the complex Backup/Restore mess they organized. Your pilot will never be killed or captured "permanently". You will just have to repeat the mission and you will not get to keep the score and kills you got during failed attempts. Maybe, I will reuse the Int16 value that currently stores the pilot status (alive, captured, killed) to record a counter of "deaths" or "captures".
If someone wants to keep the classic behavior (I don't know why. Maybe they like to challenge themseves to see how high rank can they achieve before dying) I could always add a little button to the register screen to reset the pilot death count in exchange of resetting also the rank and the score and kills. But I don't see many people using this.


Hi, thanks for the enthusiasm!
But before involving a lot of people to put their talents on something that big, we should have some solid game going on and working first. Otherwise there is a huge risk of breaking expectations and getting into problems.
What to say!? You are doing a lot of impressive reverse engineering there! I feel truly humbled.

But the more you discover about the midi iMuse, the more I feel like it's too complicated to get a midi synth running without getting ourselves into something much more complex that we should, at this stage, at least.
We need to note down all your discoveries because we definitely will come back to this when we are confortable and willing to implement a real musical synthetizer like ScummVM does (don't they use fluidsynth also?).

I experimented yesterday with the digital audio output in the project. Audio is represented by N channels of float values between -1 and 1. Before being sent to the audio hardware, this data can be processed easily by accessing the array of floats and doing whatever we feel with it. If we keep somewhere some metadata about the loops and tails, the program could just use that information to know what to send to the audio output. I don't think there is any need to separate the different parts in different files, as long as we keep the metadata at hand.
For example, if the concourse music is not yet ready for the concourse to change screen, we can register so in the metadata file, somehow.
Then the audio system could lift a flag like "Wait before moving on" that the main game would await to go away before moving on to the next screen or whatever.

The same could be done with the flight sim midis, unless all of them have the same durations, for skipping the silence or recognizing when the tail starts and it is okay to start merging the next queued chunk.

I don't know about having the lead indicator in the actual game view - forcing you to line up your shots yourself until the green light was a distinctive thing that always set X-Wing apart from Freespace and Wing Commander, I think. Like I said, that would be best used in the targeting computer only, and would give it a bit of a use. It fits the theme at the end of Star Wars Episode 4 that the targeting computer is a crutch and a disadvantage anyway. So, it would be used to physically show (on the CMD, not the actual world) where to aim to hit, at the cost of losing situational awareness if you're always staring at it instead of what's ahead.

The edge of screen target direction arrow is fine, though. XWA had that

Actually I think finding that Jump command makes the implementation a lot easier, because now we don't have to programmatically remove the delay anymore: there's a specific command in the file that gives you the real start position of the MIDI file. So upon reaching it in playback, just check for it to tell the MIDI player to skip ahead to that point.

The "wait before transition" thing doesn't apply to the inflight midis, because those are already chopped up into very short measures so we can just explicitly set it to only transition at the end of a file. You can't do that for concourse music because those are all one long file instead.

Also, you bringing up ScummVM made me realize that theoretically we could just use their iMUSE code. But, ScummVM is GPL so we can't really use that code with a Unity game, I think. I have no idea how licensing for Unity stuff works, but since Unity itself is proprietary I think it isn't allowed by GPL.

It depends on the kind of interaction.
I believe if we copy and user their code, then our code becomes GPL too. As far as I know, the game code is the one we write, not the entire Unity engine, so we could very well release the game code as GPL. To build it, you happen to need the Unity engine, but that component itself is not making use of the GPL code.
I mean, it's like if we used for our project a GPL chunk of code, and a unmodified unrelated propietary dll. We would be obliged to release our code as GPL, but not that of the propietary library.

Maybe I am mistaken, as usual. In any case, I have no problem with releasing the code of the game, of course.
And as I said before, I wouldn't oppose to change to another engine or framework once the game prototype is working as we intend, because the hardest part, the knowledge, would be documented already. In that case, though, I would not be the one coding the game becase I cannot afford the time at this point to learn another engine or even worse, start writing our own.

To be honest, I've always wanted to have just a simple iMUSE player for both TF's and XW's flight music. If you should be able to put together a working system, would it make sense to assemble a little tool where you can just have the music running endlessly and you can just click a button to transition into another state? This would make testing much easier, with the side effect of working as said iMUSE player ;-)

Also I'd say Unity is a good choice, because performance-wise it seems to be ahead of Unreal Engine 4 (and comes with Oculus support out of the box).

I suppose I can whip this up over the weekend or next week.

I've heard the opposite, and that Unity runs notoriously poorly for even very simple graphics. Much like Minecraft. Probably due more to it being frequently used by novices than the engine's fault, but still.

Now, if you said ahead of UE3 then I'd believe you :P

At the end of the day though, I think both are overkill. Because we'd still be writing our own physics, flight model, music engine, and eventually the full implementation of the concourse and cutscenes, I think we'd have written far more code than the engine would take care of.

Really, a home grown engine for the games before XWA would not need to be complex in terms of rendering. There are no special lighting effects other than light sources. Models are never transformed beyond rotation, just rendered as they are. There aren't even transparency effects! It's a pretty good thing that space sim games are so barren! Even the physics are simple. Nothing ever moves sideways, ships always fly in the direction they are facing. Nothing bounces off of each other, there are no newtonian physics.

Wow, that would be awesome!