No worries.

The Witcher 3: Wild Hunt - Complete Edition no longer fails for me when retrieving the manifest.

I just tried it now and I was about to say that. Maybe the game had some problem on their end. It has happened in the past that they've had dead links (not the weird notification kind, but the kind where there is an actual file missing) that they needed to solve.

Anyways, glad it is working fine now.

Every time I run this, I get manifest-generation-progress.json and manifest-generation-warnings.json, but no manifest.json.

to be clear, I'm trying to follow the instructions on GitHub (for a local path, but that's not key here, I don't think):

./gogcli manifest generate --lang=english --os=windows -b

Is it crashing before finishing, or is this manifest the same format as manifest.json? I've tried on Windows and Linux, same result.

Magnitus can reply in greater detail if he wants, but you should end up with a file called 'manifest.json'.

I use the following command-line within my GUI program.

gogcli.exe -c Cookie.txt manifest generate -l=english -o=windows -i=Quake

That gets me a single manifest entry for the game Quake.

I don't get a full manifest as that would take hours for me, so just grab for each game on demand.

Perhaps I am looking at an older command list, because I do not see the parameter '-b'.

You are I am presuming using a cookie file?

-b is what you two were talking about a page back or so. It ignores errors... which I figured I need.

Yup I have my cookies working. I would like to download all of my games, and I have most of them done with lgogdownloader but I can't get that working without a full Linux install (even Docker doesn't work). I'd like to scan for updates too.

Ah yes, a new parameter I'd forgotten about, that wasn't in the command list I quickly looked at.

I've never gotten around to testing it, let alone implementing it, and not really felt the need to.

Except once, recently, I haven't had any failures with the latest gogcli.exe for quite some time, and that one failure resolved itself within days, but I had meanwhile gotten what I needed by other means. I suspect the -b parameter would have helped in that instance, because the errors were related to other language versions of the game files, English wasn't impacted.

--------------------------------------

I don't recall off the top of my head, whether it does or not, and I've never used it in any case, but gogcli.exe might have a debug option that could show what is happening.

How long does your generate process take? It should take quite a while if you have lots of games. For me on this low power PC with almost 1700 games it would likely take several hours.

Not that long, 5-10 minutes?

Then it is not an immediate fail, so it would depend on how many games you have, whether it is crashing (exiting) before it finishes. I imagine the creation of the 'manifest.json' file is the very last process it does, which likely gets skipped if there is some fatal issue not covered by the -b parameter.

A really rough estimate, would be quite a bit longer than 10 minutes, if you have 500 games or more. That really is dependent on your PC specs though and web connection, program limits, etc.

I'm assuming you are running the latest version? You're downloading it from the releases or building it locally from the main branch?

I'm assuming that you are getting some progress output on your console? If so, does it complete?

I have a hint that there might be an interruption, based on the fact that the "manifest-generation-progress.json" file still exists (if it completes, the manifest generation should delete that file).

Btw, for manifest generation, the storage doesn't matter (the manifest is always generated locally first and is storage independent).

Edit:

fyi, I generated an abreviated manifest with only 1 game. see the command and output below.

./gogcli manifest generate --lang=english --os=windows --tolerate-bad-metadata --title="Master of Magic"
[manifest writer] Generating/Updating manifest for 1 games
[manifest writer] Got all info on game with id 1207658855. 0 games left to process

Hi Magnitus.

It's been a good long while since I have had an issue with getting manifest data for a game.

Unfortunately, gogcli.exe appears to be trapped in a retrying loop for the following game.

Encased: A Sci-Fi Post-Apocalyptic RPG

I'd just bought a DLC for it and needed to refresh the manifest. I tried a few times then gave up and manually canceled.

According to my own program which I think uses an older version of gogcli.exe, the following two files are the problem.

/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4

/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch6

Those two don't return a filename or checksum.

Working atm, but I'll check tonight. My guess is that it is probably a broken link on GOG's end (an unintentional one, not a "this file will come soon" intentional dead link). I usually add an exception for those in the skip url section of the manifest if the file is not important (ie, a patch) orthewise I report it to GOG. I guess I can add a command add url exceptions to the manifest if it doesn't exsists already.

Initially, I would automatically skip such files, but then, it happened in the past that an installer file temporarily was a broken link on GOG's end and it got deleted (technically, wouldn't have happened if I created and thoroughly checked a plan file before applying the manifest update, but I don't always do, I can get lazy). After that incident, I decided it would be better if the tool stopped in the case of an unintentional broken link on GOG's end and let the user decide whether he wanted to stop until GOG fixes it (if it is an important file) or just add an exception for it (if it is a patch) in his manifest.

Note that technically, you can put a regex in the skip url section so you could make it skip all patches. I've contemplated doing that (to save space if nothing else) and only the thought that some games might get a patch without an updated install have stopped me so far.

EDIT: I could probably add a "skip url if broken" part and put the patches there though.

Honestly, the more it goes, the more I'm thinking that the GOG Galaxy api might be the way to go to save a lot of hassle (I think you can get the offline installers through that as well). I just need to find an elegant way to solve the login page issue in a cross-platform way. I'm thinking Tauri might solve that problem elegantly, but I need to finish learning Rust. Wife is going back to Japan for a month soon. I guess I'll have more time to do that then.

Hi,

I'm generating a manifest just with encased and this is what I have so far:

./gogcli manifest generate --title="Encased"
[manifest writer] Generating/Updating manifest for 1 games
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1installer4) -> body retrieval error: stream error: stream ID 9; PROTOCOL_ERROR; received from peer. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1installer4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1installer4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1installer4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1installer4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] Bad metadata for /downloads/encased_a_scifi_postapocalyptic_rpg/en1installer4: File metadata was still fetched using much longer workaround method.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] Bad metadata for /downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4: File metadata was still fetched using much longer workaround method.

There are no dead links that I can discern.

However, there are some files with bad metadata (thankfully, there are fewer of those nowadays then there used to be) and the metadata is getting fetched using the long method (ie, mock downloading the file, computing the checksum and comparing it against the checksum once the file is actually downloaded).

This method has been implemented relatively recently in the tool so make sure you are using the latest version.

fyi, as previously mentioned, my long term plan is to try to switch to the gog galaxy api (while still downloading the offline installers from that) as GOG doesn't seem as invested in maintaining the browser method of downloading games as the galaxy method. However, my understanding is that you need a login token for that (the cookie value won't do) so I'll have to solve that.

I want to do this in a cross-platform dependable way, which means I'll need a cross-platform gui engine with web support. Tauri fits that bill in a far more lightweight manner than Electron and I'll probably use that to solve the login problem, but I still have some learning to do for that and between my other obligations, it will be a couple more months before I make meaningful progress in that direction.

fyi, I just checked and the generation succeeded:

./gogcli manifest generate --title="Encased"
[manifest writer] Generating/Updating manifest for 1 games
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1installer4) -> body retrieval error: stream error: stream ID 9; PROTOCOL_ERROR; received from peer. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1installer4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1installer4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1installer4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1installer4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] Bad metadata for /downloads/encased_a_scifi_postapocalyptic_rpg/en1installer4: File metadata was still fetched using much longer workaround method.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4) -> body retrieval error: unexpected EOF. Will retry.
[sdk] Bad metadata for /downloads/encased_a_scifi_postapocalyptic_rpg/en1patch4: File metadata was still fetched using much longer workaround method.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch5) -> body retrieval error: stream error: stream ID 35; PROTOCOL_ERROR; received from peer. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch5) -> body retrieval error: stream error: stream ID 1; PROTOCOL_ERROR; received from peer. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch5) -> body retrieval error: stream error: stream ID 1; PROTOCOL_ERROR; received from peer. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch5) -> body retrieval error: stream error: stream ID 1; PROTOCOL_ERROR; received from peer. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en1patch5) -> body retrieval error: stream error: stream ID 1; PROTOCOL_ERROR; received from peer. Will retry.
[sdk] Bad metadata for /downloads/encased_a_scifi_postapocalyptic_rpg/en1patch5: File metadata was still fetched using much longer workaround method.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en3installer0) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en3installer0) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en3installer0) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en3installer0) -> body retrieval error: unexpected EOF. Will retry.
[sdk] getDownloadFileInfo(downloadPath=/downloads/encased_a_scifi_postapocalyptic_rpg/en3installer0) -> body retrieval error: unexpected EOF. Will retry.
[sdk] Bad metadata for /downloads/encased_a_scifi_postapocalyptic_rpg/en3installer0: File metadata was still fetched using much longer workaround method.
[manifest writer] Got all info on game with id 1988363275. 0 games left to process

So beyond having to wait for the mock downloads to compute checksums, there wasn't an issue.

Thanks for that, and it seems I did not wait long enough then, though I did wait a dog awful amount of time, way beyond what I have had to wait before for these checksum issues, and looking at the console it seemed it had stopped retrying and was stuck.

I'll try again now. No doubt all slower on my current PC as well.

EDIT
Yep I did not wait long enough. It took around 25 minutes to complete, which is way longer than I have experienced before.