Hey everyone,

As I mentioned in some of the earlier forum topics, I started work on disassembling Starship Titanic, and reimplementing clean support for it in ScummVM. I've just made a blot posting about it to: my disassembly blog. A lot of the basic fundamentals already work, like scene display, cursors, and basic video/animations. The bulk of the game's game logic still remains, as does the PET interface, including the whole "talking to characters" system.

Wow, that’s great news! Especially as I have difficulties running the game through WINE ;-)
Is there any kind of help you’re looking for?

Programming help, mostly. Once I've got any remaining core functionality done, it'll still be required to implement all the logic for each object in each scene. So if you, or others, do have any C++ experience and even a passing knowledge of assembly, you drop in on the ScummVM group and try getting your feet wet, so to speak ;)

Sadly I code in shell only (yet)… But I’ll speak of your project around me, and try to find some experienced C++ and/or assembly coders to point your way.
And when you have something ready for testing I’ll happily jump aboard!

Indeed great news! I did get it running in Wine once, but a few system upgrades later it has stopped working.... =( Unfortunately, my skills are in C and Python (I have coded C++, but I code it like a C-programmer, which is generally not a good thing in that context...). I do have the opportunity to spend some time working on FOSS projects as part of my work though, so if there's anything you think I might get my hands on then I'd be happy to have a look at it! Where is this ScummVM group? IRC?

Yes, the group maintains a #scummvm channel on Freenode. You can also go to scummvm.org, the main website for the group/project. Any help is always welcome.

Just as an update, I've got the bulk of the game's initial conversation parsing reversed and reimplemented now. There's still a bunch of further specific logic for each NPC to handle as well. But I'm I'm very pleased with the progress, since I was afraid that understanding how conversations worked would be a lot more difficult then it was. Not that I really understand everything that the conversation parser does, but at least the classes all have relevant names, and a lot of the methods have something understandable.

At this rate, the conversation system will all be handled soon, and video playback will be the only remaining major unknown area remaining - many of the AVI video files for the game break the ScummVM AVI Decoder, so I'll need to figure out what's so special about the files, or whether perhaps they just happen to be malformed. And hand-in-hand with that is the classes for managing video playback in the game; there's a whole bunch of stuffing for queing videos, and handling videos with multiple video tracks that I'm not handling yet.

After that, it'll just be a matter of implementing all the hardcoded logic for each room. And then bob's your uncle :)

Thank you very much for your work on reimplementing the engine for ScummVM. Video game preservation is something we can only rely on the dedicated community members (like yourself and the ScummVM/DOSBox, etc teams) to handle along with some special places like GOG who actually make it commercially relevant once again. The more games are saved from inaccessibility for future generations the better!

Work on the reimplementation has been going fairly well. All the NPC-specific conversation stuff has been implemented, and I've now implemented the bulk of the sound code, though some of the sound-related game classes that make use of it still need to be done. I can now start the game and hear the looping background music whilst Douglas Adams on the TV scolds me. A lot of testing of the code will eventually be needed, even before a public testing period, but the massive code blocks associated with each NPC is at least done.

So really, there are only two areas remaining to do. First is the custom logic for all the various objects and interactive game elements. These will all be fairly straightforward at this point, since I've figured out and named pretty much all the core game functionality that they'll use. It'll just be a tedious matter of implementing each class one at a time. The other area is the star control hierarchy. It contains a whole hierarchy of classes for doing 3D projections and movements of the starfield. Over 30 different classes for just the one small bit of the game. I'm likely going to leave it as a low priority to begin with, and concentrate on implementing game logic and testing until I can get far enough in the game to need it.

So it may not be too much longer before we can dump the nasty original executable :)

Thank you! Can't wait to play it on Linux.

Work on the game inches ever closer to completion. I spent the last several weeks plowing through implementation of all the classes for objects within the game, which put together form the actual story and gameplay. And as of last weekend finally I completed them. This period was pretty boring.. as I previously mentioned, almost all the core engine methods the classes call were already known, so it was simply a matter of slowly converting each of the class's assembly to clean C++ code in turn. As part of this, I also tried to give meaningful names to each field in turn. A CToggleButton class with a single boolean could be fairly confidently call the field 'pressed' (for whether the button is pressed or not), but some of the classes have fields I wasn't sure about, so there'll a second round of clean up as I actually start working/playing my way through the game.

At the moment, I'm currently focusing on finally implementing code in various miscellaneous parts of the core engine that I'd previously left stubbed out. Things like handling credit text, some of the font drawing functionality, some of the music playback and so on. I don't think this is going to take very long, so I'll soon be starting to finally play my way through the game, fixing problems as I identity them. I can't wait to finally have the ship crash into my house and get on board :)

On a side note, I can confirm that the original executable had various game classes related to debugging functionality. However these aren't currently used - the way savegames work is that every game class in the executable is stored in a big list, and each has methods for saving and loading it's state. So starting a new game, or loading a savegame, involves specifying & creating instances of every single room, view, and object in one big hierarchy. The point is that the debugging room/dashboard was likely implemented as a special room definition during original development, and then removed in production from the newgame.st (which is the new game "savegame"). This isn't really a problem for me, since with modern systems I can easily set breakpoints in the code and then change values, and ScummVM comes with it's own debugger I can add debugging commands to as well. It's just a shame that I'll never see what their original debug layout looked like.

PS: Of course, there is also the work remaining to implement all the classes needed for the Star Control view, but after all this time, I think I'd like to be able to get part way through the game before I have to worry about it again. :)

Good news, bad news.

The bad news is that I wasn't quite as complete in the game coding (ignoring the star control) as I'd thought. One of the puzzles in the game involves a Music Room, where various controls for Bells, Snake, Piano, and Bass are combined to form music.. I think. Not having played through the game yet, I'd thought "music" for the game was just playing the relevant wave files, code for which I already had. And that I could ignore all the audio buffering the original had, and just use decoders provided by the ScummVM framework. Now that that's not the case, looks like I'll have to spend a little more time disassembling the audio code to figure out how to generate that room's music on the fly.

The good news is that I'm not waiting for that to be done before starting to play and debug the game. Just last night I got view change transitions and standard music working probably, so now I can officially get as far as the ship crashing into my house. It currently then hangs as the robot appears, but it's progress nonetheless. Soon he'll be introducing himself, and giving me the PET, and hopefully, before too long, I'll finally be able to board the ship :)

:D All sounds like great progress to me, and an interesting read.

Thanks for the support. :)

Since last pointing I've made some further progress. I was actually blocked for the past week, since as soon as the Doorbot exited the crashed ship and started talking to me, the game would crash. Turns out that that the NPC videos were using Indeo4 video encoding, which the ScummVM framework didn't support. It had a decoder for Indeo3 already, but not 4. So I had to spend the week pinching code from ffmpeg and building up a decoder that properly plugs into the existing ScummVM decoder framework, and meets the project's coding conventions. And on the weekend I finished it, and it was accepted into the codebase. No longer any crash \o/. Though the Doorbot isn't properly animating yet either. But that must be due to some other bug in my code, since a little test video playback loop I did outside of the engine verifies I can play the Doorbot's AVI file (y457.avi) just fine now.

The lack of transparency is also noticeable now for the Doorbot. It's looking more and more like the secondary video track that some of the AVIs have is used as a mask, to indicate which pixels are transparent (and shouldn't be drawn).. I'd previously avoided bothering to disassemble the drawing routines, and just used the ScummVM framework's drawing. Though ScummVM's drawing methods only allow for a designated transprency color, not an entire image mask. So it looks like I'll have to write some custom blitting code to handle the drawing.

The other thing that happened was that it was pointed out to me that GOG, bless them, offer the German version as a download option. It hadn't previously dawned on me - I'd been willing to wait until the English version was done and then hunt down a German release. But being able to download the version from GOG, now I don't. So I spent further time on the weekend, getting the German version to at least start. Now I can at least hear the German localisation of the message on the TV, shouting at me in German to, presumably, start playing the darn game :)

In case anyone's interested, the German version has the following differences:
1) Most obviously, the text strings for the messages throughout the game are different.
2) The German version has a "Translation" subview within the PET. Most noticeably, it has logic for showing German players what the various texts mean when the player is in the Arboretum.
3) There's new data fields for the CodeWheel and ArboretumGate game classes. I haven't yet delved into the changes to see what they do. At least the moment, I've merely implemented loading and saving of the new fields so the game at least starts up
4) Unfortunately, it looks like there was a widespread change of .wav filenames used for sound playback throughout the game. Considering the hundreds of tiimes the method is called, it's going to be a pain to slowly step through each call and do "if german then "file1.wav" else "file2.wav"" throughout the code. So I'm going to leave that for a future date too.

So good progress overall. Indeo video support implemented, and German version starts up. Now to get back to general bugfixing. I'm eager to finally be able to start talking to the Doorbot and finally get on board. :)

Progress continues on the engine. I now have transparencies working correctly. It turned out to be slightly more complicated than I'd anticipated. The second video track for some videos was the transparency "alpha" information, and part of the reason I couldn't properly understand it's data was due to a problem in ScummVM's video decoder.. when I'd initially added support for the second track, I'd accidentally set it to read data from the first track's offset in the AVI file, so the read data was of course not valid transparency information.

A second problem was that, presumably for optimization, the transparency track frames were compressed using a MS Run Length Encoding algorithm, and the original game had a special class that took the raw compressed data and allowed the drawing routines to scan to specific pixels to get transparency data without ever decompressing the data. Whereas ScummVM's video decoder always decodes frames and returns the decompressed data to the callers, so they don't need to worry about the underlying compression. This meant that I spent effort reimplementing a decompression that wasn't really needed. I ended up implementing my own "best guess" versions of various methods that test for transparencies, that work on uncompressed frame data.

So far, it all looks okay. The Doorbot now correctly displays on the screen, and the star-burst cursor you see when you hover over the TV buttons now looks how it does in the original. Well, almost. The original some some degree of transparency with the cursor images, and manually handled drawing the cursor on-screen each frame. With ScummVM, I'm using the built-in cursor manager that takes care of drawing the cursor. So cursor pixels which are mostly transparent are now wholly transparent. Personally, I don't think anyone will notice the difference.

On other fronts, I've still been somewhat stymied by getting the Doorbot conversation at the start of the game to work properly. Once he approaches you, there's supposed to be an initial cape animation. This doesn't happen, so it screws up the following conversation logic that's meant to be responsible for animating the Doorbot as he speaks. I'd previously tried debugging the original program in the IDA disassembler, to see if I could figure out what code fragment calls the movie playback method to do it, but it would keep crashing the game immediately with an exception. Even if I did a remote debug session using another computer. I've come to the conclusion that it may possibly be an earlier form of anti-debugging implemented either in some corner of the executable, or the third party libraries like QSound.

This had really been worrying me. Particularly now that the video decoding and display issues were out of the way, I'd be starting to debug the code I'd implemented for the conversation system. Without the ability to debug the original to see what "legitimate" values are meant to be at various places, it'd make fixing my implementation that much harder, since I wouldn't be able to easily narrow down to where I'd implemented something incorrectly.

Last night, though, I had a bit of an epiphany, and tried using Visual Studio's Attach to Process, and it worked. I was able to set a breakpoint and run the game again without it crashing. So I should be able to simply use Visual Studio, with my IDA disassembly of the game open in another window. The only problem now is that the graphics go all screwy trying to return to the game. I remember, though, mention in earlier forum threads about a tool that force full screen games into Windowed mode. I'll have to experiment with them.. hopefully together it'll properly let me debug that sucker, and finally progress further into the game. :)

Things have turned out well. I was able to use the DXWnd tool. To force the game into windowed. It's not perfect, and wouldn't recommend it for someone trying to properly play the game, but for the purposes of debugging, it works great. I've been able to use it in tandem with Visual Studio's Attach to Process to set breakpoints, and observe the game state in the original.

Thanks to that, I've been able to move ahead with debugging the operation of the Doorbot. As of now, the initial conversation with the Doorbot runs, you can now get the PET, answer "yes", and get the photograph off the mantle. In fact, the only thing left before I can leave with him, is the final animation of leaving the room :). There are also still some other glitches to look into - the Doorbot doesn't properly animate all the time whilst speaking, the dragged Photograph glyph doesn't disappear when you finish the drag, and the game itself doing the drag and drop of the photograph if you take too long doesn't work. But none of these are "blockers", so I still plan to get the game to the stage where you can properly get into the ship itself, before I come back to look into these more minor issues.

I'm still hopeful that my progress through the game will pick up once I'm on board the ship. At the moment, every time I make a change in the code, or want to test something in the original game, I have to let the entire crash sequence run each and every time. At this point, I can repeat word for word everything the Doorbot says :P. It'll save a lot of time overall if I could just load a savegame and be right at the place I want to test. I'm also thankful that all the code for conversation entry and parsing worked well enough the first time that it recognised my "yes" answer. Given how complicated the entire parsing system is, it could have easily ended up taking ages to debug problems had they been present. And having to redo the crash landing scene each and every time would have been a nightmare. But it didn't, so I'm happy :)