It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Not sure what would be a proper site and forum to ask about this (Super User?), so in the GOG forum it goes.

So, I have a bunch of old disk images (some of them inside RAR files, some as ISOs or other formats).

I know you can open an .iso file (a CD or DVD ISO image) in e.g. 7-zip and then test its data integrity, just like you can for any RAR or 7Z or ZIP file. What kind of integrity check is that anyway (CRC32 checksum or somesuch?), and if it says ok, can I conclude the .iso image is fine, no unintended data corruption or bitrot in it?

The reason I am asking is that once in awhile, if I uncompress such iso file from a RAR file, 7-zip might report data corruption in the source RAR file (or the .ISO file which was uncompressed from it), but it uncompresses the ISO file anyway. Then when I check that uncompressed ISO file with 7-zip, it doesn't detect any data corruption.

Can I conclude then that the ISO file is fine after all, even if 7-zip reported some error(s) in the RAR file from which I uncompressed it? (This is not specific to 7-zip, I've also tried the same files on WinRAR and it reports errors in the same RAR file as well.)

Another question, do other image formats have some similar ways to check their data integrity? NRG, BIN/CUE, BWS, what have you... At least 7-zip recognizes and can open apparently only .ISO files, but are there some other tools to check the data integrity for other disc image formats?
This question / problem has been solved by AB2012image
low rated
ISO? hmm isnt that pirated game especially rar-ed?
imho if it says corrupted then it is has errors , while it can open the archive and iso the data there is corrupted
avatar
Orkhepaj: ISO? hmm isnt that pirated game especially rar-ed?
imho if it says corrupted then it is has errors , while it can open the archive and iso the data there is corrupted
No, it can also be an ISO image made from my own retail CD or DVD (of which I have in my physical collection quite a lot, and like to convert to digital format for easier and more future-proof archiving), or e.g. my old VHS home videos which I converted to DVD format at some point (with a dual VHS/DVD-R recorder that my brother fortunately has).

Anyway, that is beside the point, and your second comment doesn't answer my question at all, so I can't mark your answer as the solution. Thanks for participating though.

(For the record, I wasn't talking about only being able to open/mount the ISO. I was talking about running the data integrity test for the ISO with e.g. 7-zip. Maybe you don't understand what I am talking about, and that is quite ok I guess.)
Post edited March 08, 2020 by timppu
According to this:

https://superuser.com/questions/1393764/how-to-verify-integrity-of-iso-file-with-these-constraints

ISO format (raw disk image) does not include any way to check the data integrity.

Then again, that raises the question in my mind that why does 7-zip still offer an option to "test" ISO files, just like you can test zip, rar, 7z etc. files' data integrity with it? Does it tell anything? For e.g. .tar files, 7-zip has the test option grayed out, implying that it tells the user that one can't check the data integrity of tar files.
avatar
timppu: I know you can open an .iso file (a CD or DVD ISO image) in e.g. 7-zip and then test its data integrity, just like you can for any RAR or 7Z or ZIP file. What kind of integrity check is that anyway (CRC32 checksum or somesuch?), and if it says ok, can I conclude the .iso image is fine, no unintended data corruption or bitrot in it?

The reason I am asking is that once in awhile, if I uncompress such iso file from a RAR file, 7-zip might report data corruption in the source RAR file (or the .ISO file which was uncompressed from it), but it uncompresses the ISO file anyway. Then when I check that uncompressed ISO file with 7-zip, it doesn't detect any data corruption.
I'm not sure but I've had one or two errors when using RAR files in 7zip that I haven't had when using RAR in WinRAR. It's possible that 7zip vs WinRAR handles them differently. As for integrity checks, I don't know for sure what CRC, etc, WinRAR / 7zip uses, but what I personally do is use a tool like Hashcheck to create .md5 files that contain a checksum of the ISO itself (file is stored in the same folder as the ISO). To test you literally double click on the .md5 file and it will verify it. If that changes then the ISO has changed enough that I'd personally want to re-rip rather than try and manually hunt down the offending file within the archive. Likewise having 2-3 backups (on different drives) makes great insurance against "bit-rot" as it's extremely unlikely the same file in the same archive will change on each simultaneously.
avatar
timppu: Another question, do other image formats have some similar ways to check their data integrity? NRG, BIN/CUE, BWS, what have you... At least 7-zip recognizes and can open apparently only .ISO files, but are there some other tools to check the data integrity for other disc image formats?
I'm not sure about internal checksums, but .md5's will show if any changes occur in any file for any file format. It should also possible to "mount" the ISO as a virtual drive and create an .md5 file (within the ISO / NRG, etc) of checksums of each individual file within the ISO, then unmount the ISO, and you'd basically have a way of checking each file within the archive even if the format of the archive doesn't have any native internal integrity check support? To be honest, when faced with older proprietary archive formats like .ACE, .BIN/CUE, .NRG, .ICE, .PAK, .PEA, .UHA, etc, today I'd rather just convert them all to standard mountable ISO's (if the game needs to 'see' a CD/DVD-ROM drive to install or play), or if the game is "portable" and doesn't need installing, just directly zip them up into .7z files.

Edit: And if you kept backup ISO's of ripped discs in a single folder, then create an .md5 file in the same folder for a checksum of all of them, you'd then have a fast way of verifying the whole lot at once with one check, which if that comes back clear, saves a lot of time eliminating the need to mount / open and verify each ISO one at a time? I've done this with my DOS backups, where 200-odd disc backups are checked all at once with 2-clicks via HashCheck, instead of 1,000 clicks manually opening & testing each file one by one.
Post edited March 08, 2020 by AB2012
avatar
AB2012: I'm not sure but I've had one or two errors when using RAR files in 7zip that I haven't had when using RAR in WinRAR. It's possible that 7zip vs WinRAR handles them differently.
Yeah I was also suspecting that, so just in case I have also installed WinRAR (and also because I have some ancient .ace files that I want to convert to e.g. 7-zip, and 7-zip doesn't handle .ace files while WinRAR does).

So far for me though, all rar files that have failed the integrity check on 7-zip, have failed it also on WinRAR.

One difference in behavior between the two is that if WinRAR detects a file with an error, it doesn't uncompress it, while 7-zip still uncompresses it regardless of the error. Not sure what to make of that, I presume 7-zip just uncompresses the data from the file that it can, while rar doesn't even try. Or something.

Anyway, I put the same question also to Super User, if someone there has more details what exactly that 7-zip data integrity test for .iso files means in practice, what is it testing? I presumed there is some built-in CRC32 checksum thingie or something in .iso files.

I recall from the old days, trying to read some of my old CD-R discs, that when it failed to read correctly some part of the disc, IIRC it would complain about "cyclic redundancy check" (=CRC), but not sure if that was related to the CD ISO standard itself, or e.g. the ATA bus. But somehow it was able to detect that it can't read a file right from the CD.
avatar
AB2012: As for integrity checks, I don't know for sure what CRC, etc, WinRAR / 7zip uses, but what I personally do is use a tool like Hashcheck to create .md5 files that contain a checksum of the ISO itself
Too late for that now, I can't generate the correct checksums afterwards. Those files didn't have any extra checksums generated when they were created.

I do use other kinds of mass integrity checks generally for my archived files, like dvdsig (I think it uses MD5) and rhash (I often use SHA256 with it, but I guess MD5 would be enough to detect normal data corruption). These are not very useful though if the data in the archives changes and is updated often, as then you'd have to remember to always re-generate the checksums, but only for the altered or new files.

Whenever possible though, I like to compress my archived files with e.g. RAR or nowadays 7-zip because then I can always check the data integrity by testing the compressed files with 7-zip, or for lots of files in one swoop. For e..g. video and music files and photos this is not useful as you normally want to keep those in an easily accessible format (ie. uncompressed).

My longer term plan is to move my more important archives into OpenZFS or BtrFS filesystems, which are "data corruption resistant" and have all those file checksum generations etc. automated, included in the filesystem itself. They can also autofix corrupted files if I am so inclined.
avatar
AB2012: Edit: And if you kept backup ISO's of ripped discs in a single folder, then create an .md5 file in the same folder for a checksum of all of them, you'd then have a fast way of verifying the whole lot at once with one check, which if that comes back clear, saves a lot of time eliminating the need to mount / open and verify each ISO one at a time? I've done this with my DOS backups, where 200-odd disc backups are checked all at once with 2-clicks via HashCheck, instead of 1,000 clicks manually opening & testing each file one by one.
Yeah I use rhash and dvdsig for that same purpose, but unfortunately that doesn't help in this case where I need to check the data integrity of an .iso file, without an additional checksum file which was generated with the .iso file.

Still wondering what exactly does 7-zip tool do when it tests the data integrity of an .iso file. It does seem to go through the files within the iso file, but then I don't recall it ever complaining of any errors. With e.g. .tar files, 7-zip does not offer any way to test the data integrity, I presume tar format doesn't include anything like that, it just combines several files into one file without any compression etc. and wouldn't know if any of them have data corruption.
Post edited March 08, 2020 by timppu
avatar
timppu: Anyway, I put the same question also to Super User, if someone there has more details what exactly that 7-zip data integrity test for .iso files means in practice, what is it testing? I presumed there is some built-in CRC32 checksum thingie or something in .iso files.
From what I understand, ISO files are an uncompressed sector-by-sector image of the data on an optical disc (a copy of the ISO 9660 CD-ROM filesystem), stored within a binary file. I don't know if they use CRC32 (as zip files do), or some ISO Filesystem level check, but have you tried recovery software like IsoBuster (on both the ISO file and physical disc)?

Can you still re-rip the discs? That's definitely preferable. If you can't, then what happens if you try and mount ISO files using software like VirtualCloneDrive, and then copying the files from the Virtual "CD-ROM drive" to a folder on your HDD via Explorer? As a last resort, if you bought & paid for the games I wouldn't feel that guilty about "reacquiring" broken files from certain sites, especially if they're older disc based games that aren't for sale digitally anywhere.

Edit: When extracting in WinRAR, after you open the file and click "Extract To", there should be an option that says "Keep broken files" at the bottom left. Does ticking that make any difference?

Edit2: If you have a CD-R / DVD-R / BD-R drive, what happens if you try and select the ISO file and burn it to a blank disc using software like CDBurner? If you don't have an optical burner, then what happens if you mount the ISO using VirtualCloneDrive, and then use CDBurner to "rip" that Virtual drive to a new .ISO file? I'm thinking that if the corruption is due to a faulty CD file system header, then actual CD burner software may have a better chance at fixing broken ISO headers in the process of creating a new ISO file than file archival utilities like 7zip / WinRAR whose ISO capability is typically read-only.
Post edited March 08, 2020 by AB2012
Ok I guess I got the answer from the Superuser forum.

ISO images don't have any checksums so what the 7-zip tool is probably being able to test is that the ISO file is not truncated, ie. that there is no data missing at the end of the image (ie. if it was downloaded from somewhere, so that it would downloaded fully).

So it (apparently) can't detect if some of the data has changed within the ISO file.
Post edited March 09, 2020 by timppu
I've been doing research and trying to understand Solomon Reed to do some of my own.

Solomon Reed is an error detection/correction codes, embedded in CD's DVD's and other media. Get a scratch? The hardware when reading some sector will notice the errors, check against the table and codes it has and correct the error on the fly (doing some insane math). How much the codes take space and how many errors they can correct will depend. Computerphile did a piece where they drilled out a piece of a CD and then put it in and hit play...

It's probably fairly widespread, error detection of codes in media, hard drives, which can detect bit rot and correct it. Probably works better than you think.