It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Community
General discussionFile integrity and backup(8 posts)(8 posts)
(8 posts)
Pages:
1
This is my favourite topic
Posted February 17, 2018
I just finished downloading my entire GOG library and want to back it up to an external hard drive, which I'd keep off-site. Before I do so, however, I'm concerned there may have been some errors in downloads. The total count is 3430 files and 916GB, so some of the files may have gotten corrupted in the download process. I'm wondering whether anyone could help me out with two questions regarding file integrity:

1. Verifying the downloads: Is there an easy way to check for file integrity, preferably a batch program? I already know to check for digital signatures of single .exe files, but for multipart downloads, I know of only enabling the file integrity check during installation. I don't want to spend the next two weeks installing and uninstalling games, though.

2. Any suggestions for a good backup software? Preferably one that checks that the file has been successfully copied over to the external drive. Backup scheduling is not necessary - I'll keep the drive off site and will bring it to manually back up my files only occasionally.
No posts in this topic were marked as the solution yet. If you can help, add your reply
Posted February 17, 2018
avatar
DrLove: I just finished downloading my entire GOG library and want to back it up to an external hard drive, which I'd keep off-site. Before I do so, however, I'm concerned there may have been some errors in downloads. The total count is 3430 files and 916GB, so some of the files may have gotten corrupted in the download process. I'm wondering whether anyone could help me out with two questions regarding file integrity:

1. Verifying the downloads: Is there an easy way to check for file integrity, preferably a batch program? I already know to check for digital signatures of single .exe files, but for multipart downloads, I know of only enabling the file integrity check during installation. I don't want to spend the next two weeks installing and uninstalling games, though.

2. Any suggestions for a good backup software? Preferably one that checks that the file has been successfully copied over to the external drive. Backup scheduling is not necessary - I'll keep the drive off site and will bring it to manually back up my files only occasionally.
This might be what you're looking for https://www.gog.com/forum/general/gogrepopy_python_script_for_regularly_backing_up_your_purchased_gog_collection_for_full_offline_e
Posted February 17, 2018
avatar
DrLove: 1. Verifying the downloads
How did you download them? I know with the old GoG downloader you can requeue the downloader links and it will check what you downloaded to make sure it's correct:

https://www.gog.com/downloader
Posted February 17, 2018
I don't think there is a way to validate them now you have downloaded them, unless you somehow extract the md5 checksums from gog, maybe look at gogrepo.

For the second, I would really suggest freefilesync. It can sync files across drives and is very easy to use. I have used it for quite a while now for all my backups.

I would also have a second back up, in fact have as many as you can afford. Hdd's do fail, and when they do they it's not always possible to recover. It's really bad when you lose, not just the files which you may be able to get again, but personal ones, and the effort you put in to organise.

Another tip, setup a structure that your happy with. It is so much easier to programmatically move or restructure this data later on if it is consistently stored first off - trust me, after a few reorganisations it really will save you days later on. Try to get it right first time, include saves, mods, patches etc. Anything you may want to keep.

Also, one other thing, get teracopy - it plugs into windows explorer and does copy/paste. It's far better than windows copy/paste, and it has an option which can check what was copied matches source, which is invaluable to get perfect copies.
Posted February 18, 2018
avatar
Melvinica: This might be what you're looking for https://www.gog.com/forum/general/gogrepopy_python_script_for_regularly_backing_up_your_purchased_gog_collection_for_full_offline_e
Thanks. I was looking at it, but I only run Linux in a VM, without access to most of my hard drive Not sure I'd be able to get it to work in Windows, so without a dedicated machine it's not helpful to me. But I'm keeping an eye on this, for when I get another PC in my man cave.
avatar
drmike: How did you download them? I know with the old GoG downloader you can requeue the downloader links and it will check what you downloaded to make sure it's correct:

https://www.gog.com/downloader
I downloaded everything manually. Never got the Downloader to accept my login info, so I gave up on it. I was under the impression that it was actually end of lifed after Galaxy was released.
avatar
nightcraw1er.488: I don't think there is a way to validate them now you have downloaded them, unless you somehow extract the md5 checksums from gog, maybe look at gogrepo.

For the second, I would really suggest freefilesync. It can sync files across drives and is very easy to use. I have used it for quite a while now for all my backups.

I would also have a second back up, in fact have as many as you can afford. Hdd's do fail, and when they do they it's not always possible to recover. It's really bad when you lose, not just the files which you may be able to get again, but personal ones, and the effort you put in to organise.

Another tip, setup a structure that your happy with. It is so much easier to programmatically move or restructure this data later on if it is consistently stored first off - trust me, after a few reorganisations it really will save you days later on. Try to get it right first time, include saves, mods, patches etc. Anything you may want to keep.

Also, one other thing, get teracopy - it plugs into windows explorer and does copy/paste. It's far better than windows copy/paste, and it has an option which can check what was copied matches source, which is invaluable to get perfect copies.
Many thanks! I'm already looking at Freefilesync and Teracopy; will definitely try them out. Unfortunately, I grabbed only one external HDD last Black Friday, but I do get your point. I'll try to get more in the future. As for the structure, I have a consistent file and folder structure throughout my GOG downloads. Not sure how I'd want to restructure it in the future, but I do like consistency even without its future-proofing advantage.
Post edited February 18, 2018 by DrLove
Posted February 18, 2018
avatar
DrLove: Never got the Downloader to accept my login info, so I gave up on it. I was under the impression that it was actually end of lifed after Galaxy was released. Many thanks!
The downloader is end-of-lifed so it might stop working in the future, but for now actually still works. It's quite a nice tool. The catch is that you can't log in if your password contains certain characters: > < & ' % " and maybe a couple of others, because of a small bug that didn't get fixed before Galaxy came along and replaced it. If you change your password to something without those characters you'll probably be able to use the old downloader.
Posted February 18, 2018
avatar
DrLove: I just finished downloading my entire GOG library and want to back it up to an external hard drive, which I'd keep off-site. Before I do so, however, I'm concerned there may have been some errors in downloads. The total count is 3430 files and 916GB, so some of the files may have gotten corrupted in the download process. I'm wondering whether anyone could help me out with two questions regarding file integrity:

1. Verifying the downloads: Is there an easy way to check for file integrity, preferably a batch program? I already know to check for digital signatures of single .exe files, but for multipart downloads, I know of only enabling the file integrity check during installation. I don't want to spend the next two weeks installing and uninstalling games, though.

2. Any suggestions for a good backup software? Preferably one that checks that the file has been successfully copied over to the external drive. Backup scheduling is not necessary - I'll keep the drive off site and will bring it to manually back up my files only occasionally.
You could just use an external downloader (not the GOG one) to get the files, they usually are much more trustworthy compared to browsers with downloading and verifying stuff.
Posted February 19, 2018
I use DownThemAll! browser add-on with Pale Moon and Firefox ESR (or older Firefox, non-Quantum versions).

Regarding the file integrity check, you can ask kbnrylaec. I didn't exactly understand how to get the xml files with the MD5 hash.

If you need a program to create and verify hashes, I recommend HashCheck, a very simple and fast Shell Extension which integrates itself in Windwos Explorer.

I've heard many times about TeraCopy, but never used it myself. I use Total Commander, a file manager with everything you need, including file verification and checksum files.
Pages:
1
This is my favourite topic
Community
General discussionFile integrity and backup(8 posts)(8 posts)
(8 posts)