Hello.

One of the things I love most about GOG is that it lets us download offline installers and install it whenever we want, wherever we want. (Good for offline gaming machines!) This alone is the reason for me the competition will never surpass GOG in terms of helping the community in game preservation.

But recently I've been experimenting with some heavy GOG offline installer setups and I've come to the unfortunate conclusion that they are not as optmized as I expected for backups when you have limited digital space.

I understand that this is not news for those who actively study methods of 'agressive' compression, still, even a common user may use good tools like 7z for better compression than the offline installers currently have.

With two examples of my simple research I wish to share (and ask) for knowledge that is simple to use, yet, powerful in results in my humble opinion. :)

My method was:
- Download the games I tested using the amazing lgogdownloader tool (for easier and faster handling)
- Install them in a fully offline Windows 10 machine
- Delete unnecessary files and test the game
- Compress the installation folder using 7z
(I use 7z on Linux through command line, but you can achieve the same and better results in Windows with it's GUI options for sure too.)
(Command used: 7z a -t7z -m0=lzma2 -mx=9 -mfb=256 -md=2048m -mmc=10000 $arquive $directory)

Dragon's Dogma: Dark Arisen
Current GOG version 1.0.0.18 = 19.4 GBs. Setup + BINs.
Reduced from around 21.5 GBs installation
... to 9,3 GBs.

The Elder Scrolls V Skyrim Special Edition
Current GOG version 1.6.659.0.8 = 23.7 GBs. Setup + BINs.
Reduced from around 24 GBs installation
... to 11.3 GBs.

Another reason why it's 'only' 11.3 GBs:
You can free up to 12 GBs in your Skyrim installation.

Unfortunately, depending on which game you're going to backup using this idea, files present there cannot be compressed further, so this method vary on compressed percentage. I'd still say that this is better than most currently compressed GOG offline installer method - not trying to sound arrogant or superior here, Im not. I'm really just trying to share a knowledge that worked for me in my case and may help more people, or so do I hope.

What are your opinions on this method? Is it good? Is it bad? Can it be improved (No doubt it can :P)?
How you compress your offline backups? Do you do it at all?

Thing is with increasing compression: sure it'll save space, but the more you compress it the more time it takes to unpack it later. This is why for some of my Epic installations I didn't opt to compress the archive at all but rather just zipped it at STORE level as a simple means of neatly organising my collection and having one file for the checksumming operations. After all, if your collection isn't absolutely MASSIVE, external storage space is remarkably cheap if you're satisfied with slower HDD drives (rather than SSDs).

I don't usually deflate any archive beyond "normal" compression levels, and for compatibility sake between systems I forgo using 7z itself and just use zip format.

For me it's more of striking a balance between space and convenience. It's gotta be up to the user's needs obviously.

GOG has very little to do with game preservation, but doesn't your method go even further away from preserving games, if you delete some files from the game?
Of course, stuff like Galaxy files can be deleted, as it was never part of the game itself, but something else might be a different thing.

GOG has taken that approach too far with some games, like deleting exe files from some games that come bundled with ScummVM. Saves very little space, but ruins any attempts to run the game as originally intended. Not at all, I'm doing the complete opposite.

I back up all files from GOG exactly as they are, including gigabytes worth of such language versions that I never intend to use.
Not only that, but I also back up the installation folder in its unpacked, freshly installed, unmodified form. This is to make it easier to run the game later, possibly using other operating systems, so there's no need to re-extract the necessary files.

And if I do some heavy tweaking, then I also save the tweaked installation folder.


So, to give an example.
I recently started replaying the first Tomb Raider game.

Of that I have backed up all GOG files as they are when downloaded, the fresh GOG installation as it is before the game is launched for the first time, and because I used the widescreen Windows mod (link below for those interested), I also backed that up, so I don't need to do the same modification again, if I want to use it later again.

https://community.pcgamingwiki.com/files/file/883-tomb-raider-retail-fix/

GOG installers used to use RAR compression, which was pretty good, but as discussed in this thread, more recent ones have moved to a stream-like system (with lots of smaller parts offering less compression). Have you tried restoring from one of your backups? Most of the Windows applications on GOG store data in the Registry, so may not work if this hasn't been backed up (and restored) also. The better programmed games can re-create default Registry entries, but not all will. Many are non-obvious (and 64-bit Windows/UAC add further complications with Registry key redirection) so a snapshot based installation monitor may be needed to find all the keys involved.

For GNU/Linux software, you may need to check your home directory for game install/configuration details also. Plus any directories used by WINE to store their configuration details.

I agree. But please may I ask, what you mean by compatibility of 7z and Zip on systems? May you please explain it? Thank you.
I use 7z format on Windows and Linux without problems. I understand your point. It's true yes.
Of course I would never (I hope :P) delete necessary files for the game to work, but mostly text and uninstall files common to GOG installers, basically turning that folder a portable version of the games which can survive this kind of backup. :)

The point of my kind of backup is when you have limited digital space to work with. If you can handle store Terabytes of data, you would not need this kind of concern I think. In any way, they're great for preserving the full package indeed.

On the topic of game preservation:
It seems to me that the way GOG gives users the possibility to download and maintain their games fully is still the best in all the market - or, at least, the most organized way right now. (but you guys tell me, Im relatively new to GOG.)

Like I've said before in other topics, I came from Steam in 2020 after noticing the way Steam's DRM hinder us from actually playing our games except if we follow their arbitrary rules.
If you want to play your purchased Steam game on a PC without internet connection, you will probably need to crack the game you bought before you can play it. And They're open about this with their Denuvo, Steamworks, and so on DRMs, treating everybody that want to play fully disconnected as second class citizens. Which means, they not only control the game and company, but also the way you will play it. This is absurd. (Not all games there work this way, to be fair.) Thanks for the reference! Yup! That's one of my concerns when extracting them and trying to play them. The ones I tested worked fine though.

Curious. Going off the returned 'flags' used i extracted and built my own scripts which uses the following

7z a -mx=9 -mmt=3 -bb0 -ms=4g -md=1g -mfb=273 -mf=Delta -mf=BCJ2 "${X}.7z" "${X}"

Takes about 11Gb Ram to use, but I've seen good compression overall.

-mx - compression level, 9 being max
-mmt - threads, 3 is as many as you can do before ram requirements jump
-ms - blocksize the archive internally sets to.
-mfb - word match size that it tries for
-md - dictionary size, this alone will increase compression and/or ram requirements more than anything
-mf - Filter settings. BCJ is Branch Call Jump to compress executables better. Delta is the difference between the next byte(s) which can increase compression for a number of types of files.

That's probably the nicest way of saying "Galaxy ruined offline installer compression".

Anyway, since we're revisiting the topic, a better way to do it in these modern times would be using zstd -19, which has a good compression ration and blazingly fast decompression speed.

Oh almost forgot. I'm pretty sure GoG started incorporating all language packs into the same installers. This can sometimes result in the installed game being smaller than the installer, rather than bigger. That's due to video audio and language being installed and ignoring all the other ones.

Though they also include a bunch of dependencies and some games that means you might have a 300Mb installer for a 30Mb game.

This is the kind of stuff that makes it difficult to determine how best to approach the topic, I like preserving things as natural as possible but sometimes you're wasting space to backup the full installer.

Simple: 7z isn't natively supported by Windows iirc. You have to install 7-zip first - trivial, yes, but sometimes I just don't want extra bloat or don't have the time to bother installing extra sh*t on a machine just to unpack an archive. Sticking to zip which is currently Windows supported makes it certain that no-matter the system I'm working with I can unpack / repack the archive at a whim.

And it's future-proof to stick with a format widely supported. 7-zip might go poof one day and you may not find an installer online for it (not likely, but possible). Zip may not be fancy in comparison, but it is more likely to stay longer-term.

"Trivial" doesn't begin to describe how minor it is. The installation directory weighs in at around 5.5 megabytes. (Safe to say that "bloat" is also not really an applicable term in this case ;D ) Installation is, consequently, very brief.
7-Zip is free, open-source, and quite popular (as far as compression utilities go). Even if the creator died suddenly or something and the site eventually went down with no contingency, I'm sure various people who already had the source code would carry the torch with their own versions. (I'd be willing to bet there are already some forks or other derived utilities in existence.)

I understand not wanting to install every random "might come in handy at some point" piece of software you run across, and I, myself, most often just use the good ol' .ZIP format in the uncommon instances when I need to create a compressed archive. But if you even occasionally download other people's files (including DRM-free indie games from places like itch, or FOS utility software) from the Internet, you'll find that you're going to need to unpack .7Z archives (or .RARs, which are also not recognized by Windows "out of the box", AFAIK) at least once in a while. That's why I keep it installed.

Oh, of course. I have one version or another of 7zip installed on most of my recent hardware, but I have an old laptop or two which may/may not have it installed on it and those laptops get the occasional run when I don't want to drag my new one in front of the TV and do some file sorting while watching something else.

Keeping with one archive format for my own personal archiving means once I've downloaded the file, packaged it how I want and then popped it only my external HDDs, it's accessible for all my equipment or even a friend's equipment should they not have 7zip installed, ya know? And I've also found that some older versions of 7zip didn't like the new compression settings the newer versions had available (I heavily compressed a couple of installations and found a really old version of 7zip refuse to unpack it, despite being fully intact). So, yeah, sticking with a well-known and supported compression filetype was far more convenient to me than heavily compressing using a someone's unique compression algorithm and having to bother with tracking down an installer.

Of course I could also pop a CLI / portable version of 7-zip in storage with the files themselves just in case. But another what-if is "what if the 7-zip package gets corrupt on the drive but not the archives?". I know, nitpicking stupid. But it still stands: sticking with zip was just a simple choice for my personal archived files.
I use the term bloat not only to indicate size, but unwanted extra registry use and program integration into Windows itself. I try to keep my system as free of extra stuff as possible at all times, so anything extra is generally unwanted to me. I tend to like portable (non-installed) programs over installed ones for that reason as well. Fortunately 7-zip offers that as well iirc, but the context menu alone makes such an option less attractive XD.

First off, drop Windows 10, use InnoExtract. Second off, as pointed out, without saving registry and other data, you are going to ruin certain games to the point of unusable. And finally, this will never help against games like "Boomerang X" that for some odd reason had duplicate data in some complex way up until recently.

Another factor you do not count is deadly and random, the non-zero chance of your PC silently ruining a game because no ECC RAM (error correction). This is not widely known. Typical customer PCs are not enterprise-grade and the industry considers the chance of corruption being acceptable for non-enterprise users (I don't agree with that), as such you must do it at least twice or three times to make sure everything is alright, wasting a huge amount of time. I could go on and on on other aspects, considering I am an old-term Linux user with Python/Bash knowledge, but it'll overload the post too much.

On myself, I don't re-compress, I preserve the 15TB unchanged on 2 enterprise-grade HDDs (the main plus a copy) with ECC RAM, literally the entire Windows side of the store with Goodies, including all the removed products (but 1, impossible to get). I don't recommend anyone do that, it's a giant headache to maintain, update, improve; To choose a game makes me lost every time. Everybody with small collections, you are lucky in my opinion, so much easier to deal with, so much easier to pick a game out of other interesting. To own the full is both a blessing and a curse, folks. Not finished, the Github repo demonstrates new bugs and other things. Not too suited for long-term. What if it eventually breaks something in development, making the older archives unusable? I think a more stable format will be a better way to do. When ZSTD becomes more stable and around 14 years pass, sure, nothing to say against then. How much does it take per game? I am afraid the model is not very realistic for anyone with hundreds or thousands of games (where it could be the most beneficial). The fact I can hold 15TB right now doesn't mean "I do not need space" because the disks are 16TB each, GOG still continues to shovel new games on a weekly basis, with some being concerning sizes of over 100GB. No small or big collectioner is uninterested in having more space for their stuff when it's feasible, does fit into their requirements of "how" a collection should be ideally.
Popular doesn't always equal good, as happens with some few things.

For example XZ is popular, as a format for long-term is a disaster of badly thought design. Both use some LZMA underneath, which unsurprisingly means unpopular Open Source LZIP offers significantly better results due to it not being a container format (look up "tarlz" for that, it nicely integrates LZIP into TAR) and the design allowing multi-thread compression/decompression, better integrity checks with free recovery (no additional bytes) from small 1 byte corruption on a disk (For more recovery from bigger random corruption, use 1 block Parchive2 files, it'll take around 500MB per 1TB, much cheaper than what RAR could do). On additional integrity verification, the best on Windows would be SHA384 (because certain attacks, SHA512 is not a great choice), for Linux is BLAKE2b, for custom Linux it would be either "a little overkill" SKEIN-1024-1024 or speedy secure BLAKE3.

I personally keep the files raw for ease of access, packing RSA-signed checksums in archives, then make Parchive 2 files just in case.

Recompressing can be worth it. but with GoG installers, no. I tried remastering a installer from GoG with optimizations, and certain required install scripts it needed were missing. Making the whole thing moot.

Though if it's a difference between a zip/rar/tar file and recompressing to 7z i'll do that, potentially removing redundant files.