They sound awesome. Good work there! :)

Okay, that went far over my head, so I guess my real question is, can you use my files the way they are now? ;)

@Azrapse: Thanks! Given that I didn't think I'd have time for them at all, I'm quite happy with the result as well :D

If I am not mistaken, what Tarvis said was basically a long "Yes". :)

Tarvis, so if I understood right, you rather keep all that musical timing information in the mid files than extracting it into another metafile. Other than for avoiding redundancy, why do you think that is the best solution?
Please, read no intention in my question. I trust you know what you are doing. However, it contrasts with your previous track of recommendation on cockpit files, instrument panel files, bflight.ovl or other resource files, where you have said that it would be best to just keep the metadata out in a file instead of reading the originals every time.

It's because the routines to do so would already be implemented, since we're supporting MIDI output too. For the cockpit layouts and things like ship stats the goal would be to not really read from the original files at all beyond using them to make our own data files or maybe determine exactly which version the game is. For those it is the best choice because it's modder-friendly.

Using the MIDI for timing would make life for potential modders easier, since they won't have to worry about extracting metadata and calculating endpoints and all that if they make new music for the game. The engine would figure it out instead of the composer working through each recorded file to make sure it has 3 beats of leadout. It just saves some effort.

In the case of the digital music recordings, don't think of it as a mod. Think of it as a sort of different soundcard output, like Adlib or MT-32, because all the recordings still correspond to the original game's music data. That's why it would be fine to rely on the original data for music - because it's already in a standard format we can understand. It is nothing more than Type 2 MIDI files that anybody can edit in a MIDI editor, so it's not an obscure format requiring an old converter like other assets would, beyond requiring an extra iMUSE header. But, from my studies, we don't really need any info from those headers, so we can support MIDIs without them, too.

Third, this method better supports tempo variation. While there's no real way to do it in the "leadin" part, it would work just fine during the actual segment. That's why I advocated for the leadout time being a fixed time instead of a number of beats (if we go the metadata route), because then it would be independent of tempo. You could instead use a fixed tempo for every inflight segment, but that would limit modding potential. Or, you could specify the segment's tempo with metadata, but that's exactly my point: it's just more work for the modder/composer/whatever.

I mean, both methods will work fine and perform the same in practice, so it really doesn't matter which method is used. I just think getting metadata from the MIDIs would be easier since it's less files to read and deal with.

In layman's terms, my idea would be to cue the next song segment when the MIDI reaches that "next segment" marker instead of when the digital recording reaches 3 beats from the end. Both would have the same effect in practice and have negligible performance impact.

Yeah. I think if we went the external metadata route instead of getting it from the MIDI engine though that the full 4 bars of leading silence should be kept. The reason for this is that many concourse tracks do not start at beat 4. Some start on 5, some start on 4 and 3/4th, etc. Others do some really tricky stuff: in the skipped silence, there could be jumps to other sections. This is used to skip to later parts of the song, because the song is first entered past that initial skip. Therefore it would be better to just leave the silence alone to make the metadata points a general solution instead of specifically declaring that inflight tracks start on beat 3. I was thinking a good format for the song marker metadata would be something like Essentially just a map for each file, from the MIDI positions to fixed time position in microseconds, to use with the recorded segments. It will work either way though, and we can worry about that later

For now, good job with the renditions. I can't wait to hear some of the concourse tracks.

Well, since right now all the WAVs were rendered at the same speed of 137.8 bpm, lead-in and lead-out have a fixed length (in milliseconds, bytes, whatever you need).

But do you really think you can make sure that both MIDI and digital audio will stay in sync all the time? If the MIDI commands just vary in speed for just a moment, the whole digital audio part will go awry as it won't line up anymore and the next cue will be triggered to late or too early. Having the actual audio files be their own triggers (by specifying the byte where the next cue is triggered) seems like a safer system. But again, I'm no programmer, so I'm just spitballing here.

Regarding speed changes, since DG-S leads into the "Success" cues (SU), do we need all the SU cues both at 137.8 and 133 bpm? I assume the SU cues don't necessarily need the DG-S transition to start, so they won't always get set to 133 bpm...

And regarding the concourse music, I don't know when I'll be able to do those. If I remember correctly you already split the MIDIs into their different pieces needed for interactivity? This seems to be getting really complicated...

Humm... You music people have certainly gone much faster than I expected. :) Unfortunately, I am not as fast as you are.

I would say, before spending many many days working on the digital concourse music, please wait for a while until we have something more to show.
I'm working on the flight sim part, and the combat music is for that part. So that is all right to have it now.

But I am pretty sure I want to have that part basically done and playable before moving onto the concourse part, and that can take many months yet. We will not need the concourse part ready until after that.
The concourse part seems to have a lot of work behind, and at the same time, is probably the part where the player spends the least amount of time. We will need to find out the format for the images, palettes, animations and interactive areas. I don't think anything for that has been done before, so we would be into totally unexplored territory here.
That means, it will take many months until we have something usable. Meanwhile I will make a barebones menu system in the style of XvT to select a pilot and launch the different missions.

So no hurries at all with the music for that part. In special if it proves to be much more complex than the in flight part.

All the SU parts also define their tempo at 133bpm. Additionally, the transition defined for every single other state is DG-S, so it is always used. So, SU is guaranteed to be 133bpm. Also: I talked with some of the ScummVM guys, and it turns out that each music segment is handled by its own parser and player. This means that any tempo changes in one segment actually do not affect any other currently playing segments.

I see no reason for the MIDI timers to go out of sync, more than likely it is going to be your synthesizer that has a hiccup instead, and that has no bearing on the internal timing.

Anyway, if we go with metadata, we can STILL extract the cue locations from the MIDI files without running the MIDI engine, so it can still be done without having our own external metadata files if we wanted to.

It's the same PLTT / DELT / ANIM / FILM format used for Dark Forces' cutscenes (and X-Wing's cutscenes) and there's plenty of specs out there for those.

FILM - specifies locations of each graphic, how they animate, etc. The layout of the concourse. The main difference from Dark Forces is likely that there is extra info about what clicking certain areas do. If necessary, this is one of those files we can define ourselves in a more friendly format.
PLTT - The palette
DELT - Static graphic file
ANIM - A collection of DELTs that form an animation

Each concourse room has its own .LFD. The main concourse for example is MAINMENU.LFD.

I don't know, why I didn't see that... either Reaper or I messed up. Now, outputting the SU tracks at 133 bpm doesn't really work now, since the lead-in and lead-out beats will be longer than in the 137.8-files, hence they will go out of sync during the "4 beat overlap".

But the way I understand you, the files I uploaded aren't usable for what you have in mind anyway. You need just exact copys of the MIDIs, timing-wise, right?

Well, since we discovered that next segments aren't started by the end of a MIDI but rather by a SysEx command, we can't rely on leadout for timing anymore. It has to be defined at a position with metadata. Therefore you can have as much leadout time as you want and it won't matter, no matter the tempo. Leadout is solely for letting instruments release instead of being cut off.

In your case the leadin is fine, because no matter what the tempo is the file will play from the beginning of the file. Since the sysex jump makes it start at Beat 4, and your recordings start at Beat 4, it will start on time.

I would prefer if it matched the MIDI timing-wise (plus leadout time), but with extra metadata information it doesn't have to (for example: we can define in the metadata what beat the recording starts on). Don't bother re-recording anything yet though, because implementation may change when I get to that point and then you'd have to do it all over again.

---------------------

Also, about concourse music being complex: it isn't as bad as you think.

Don't worry about the _0s and _1s. All those mean is that they were separate songs contained in the same MIDI file. They are totally discrete music pieces and aren't overlaid or appended into the recording or anything like that. Usually they're used for transitions (like taking the shuttle to another ship). As far as the engine is concerned, it's its own song.

_intro and _loop parts define looping. Due to the leadout issue, if we record a looping segment with an intro in the same file, and then just jump back in the file to make it loop, you get the leadout from the intro part when it loops, which may sound jarring. If this is still confusing I can make a recording to demonstrate just what I mean. Instead, it will be set up so that it plays the _intro part first, then loops the _loop part.

_ch# means that they are conditional instruments turned on or off by the engine. For example, HALLMARCH_0_loop_ch5 plays over HALLMARCH_0_loop while the Historical Missions door is open, and does not play otherwise. In MIDIs you can easily just mute specific channels to do that, but obviously for recorded music it can't be done that way. That's why they have to be split off into their own files. Silence should be kept, because it starts playing at the same time the base _loop plays, and is then muted/unmuted.

As far as recording them goes, just record each file. That's why they are split up the way they are, it's the ideal format for the recorded versions. But, again:: due to how implementation of jump points may change it might be too early to record them if you'e just going to have to do it again.

You make it sound simple. Well, that is definitely the spirit! :D

It really is. PLTT/DELT/ANIM has already been cracked. The FILM files are the only ones that would really need to be deciphered, because they'll have many commands that the Dark Forces ones lack (as Dark Forces only used them for cutscenes and not clickable menus)

But like I said, it's something we can re-write ourselves if we needed to, because we can already figure out where things are drawn by looking at the game. FILM is pretty much nothing more than "Draw these images at these x and y coordinates" and "play this sound at these intervals" and most likely "do this when moused over/clicked"

Me too! I got goosebumps specifically at 0:35 right when the french horns kick in and does the same medley as when the TIEs attack the falcon from Episode 4. Great job on this btw Laserschwert. You really outdid yourself and I totally can see this playing in an enhanced version of X-Wing.

You guys should set up a Skype chat or something to better communicate. If you guys want I'd love to keep in touch with what you guys are doing. That way live updates can be shared amongst the group. My Skype username is : thealtureus

I am trying to put as much time as I can on this. However that doesn't usually account for more than 1 or 2 hours a day.
But I am getting some progress.
At this moment, I am designing the structure of the flight sim's disparate components.
Why I am designing stuff instead of programming a bloody Y-Wing that you all can fly right away and shoot?
It's not easy to just get the shit done, because there is so few pieces of data that we can hold as empirical evidence on how Peter Lincroft designed it internally, that we don't actually know how the shit is actually supposed to be.

If I move too far from Lincroft's design, it will probably lead to the spacecraft behaving slightly different, and giving a feeling of "this is not the X-Wing I remember".

So these are my guesses and the aspects I am trying to implement:
- Flight groups have mainly orders, primary and secondary objectives, formation, and can be mission goals.
- Ships belong to flight groups and so they attempt to carry out their orders.
- Every order is carried out by dividing it in tasks. For example, for the "Fly Once" order, the first task consists on going to Waypoint 1 (if enabled), then when that is done, the next task is going to the Second waypoint, etc. Finally, the ship departs to hyperspace if allowed to.

These task can be interrupted by some emergency happenings, for example:
- There is an obstacle in the path: The ship must avoid it without straying too far from the path. I am using a Flow Field approach here (it consists on the "goal" of the ship attracting it like if it would be an opposite polarity electrical charge, and the different obstacles repelling it, like if they were same polarity charges).
- The ship is under attack: Unless it has order to ignore enemies, the ship must do some evasive maneuvers to dodge enemy fire until help comes. This will be done for a few seconds, then retake it's previous task.
- The ship is heavily damaged: Some ship types will cancel their mission when they are heavily damaged. For example, some TIEs will attempt to go back to hangar when their hull is damaged. (Only TIEs do this? Do the rebel do this also? Bigger ships?)

Tasks can also fail to be completed:
- This ship's current task goal is no more: It makes no sense for a waypoint, but if the ship would be chasing or following another ship, it is possible that the later has been destroyed, has entered hangar, or hypered out.
- This ship is unable to perform the task: It has become Disabled, for example.

The only evidence of all of this I have is inspecting what the AI ships do in TIE Fighter "Z" screen. They display there their current task, and the ETA for it.
I don't know if X-Wing was implemented in the same way, but I think it fits it.

The prototype currently makes every ship move towards their primary objectives, while avoiding collisions with each other.
It has quite an interesting math background in order to keep efficiency while searching the closest obstacles to avoid, for which I am planning to use k-d trees when the amount of ships is bigger than 10. Or brute force when 10 or under.