Great catch there! Always the simple things, eh?
Now, where else did you happen to post, for a few more +1s? :p

You're welcome!

And thanks, but no thanks. My wishlist is actually empty; I can remember the few games that I could have bought and played if I had the time just fine ;) Sides, since the Great Russian Revolution of GoG, my purchases here are limited to what is in this thread anyway (that's why I care about MaGog working!)

Thanks again!
And +1 again!

Just in case anyone's interested...

As a result of the move to HTTPS, accessing the GOG website has slowed down significantly, which means MaGog's regular database update now takes longer (about 90-110 minutes instead of about 50-60 minutes before). To compensate for this, I am changing the MaGog update schedule to begin 45 minutes earlier than before (i.e. at 04:30, 10:30, 16:30, 22:30 PDT), so that update runs will end more or less around the same time as before (i.e. 06:00, 12:00, 18:00, 24:00 PDT).

Shouldn't have too much of an effect. And as usual, I'll be temporarily modifying the schedule to adapt to sale schedules on large sales (perhaps even tomorrow?).

Btw, how many connections does Magog need to open for each game?

I am not sure what you mean by "for each game". MaGog collects information by type, not by game.
First it collects all the information it can get from the catalogue (id, name, price, discount, etc.) for all games.
Then it collects regional pricing for all games.
Then it collects wishlist information for all games.
Then it collects game page information for all games.
Etc.

Anyhow, MaGog has only one open connection at a time. It is a single threaded application.

Now I understand the question. My bad.

To collect file information from my library, I access each file in my library. If I access too many such files in one run, GOG blocks MaGog's IP for 18 hours (or until I ask Firek and he unblocks it). These are no simultaneous accesses.

When I began collecting this information about two years ago, I was able to collect information about ~150 games without being blocked (roughly 2500 accesses). Which means that to cycle through my entire library (~1000 games back then) would take about 7 runs (or 42 hours, given runs are 6 hours apart).

This number has shrinked over the years. Nowadays I am able to safely collect information about ~30 games in one run. Which means that to cycle through my entire library (~1200 games) takes about 40 runs (or 240 hours, given runs are 6 hours apart).

However, this has become less of a problem, since I now have a second source of information about game files (for all games, even those I do not own), so I collect that information first and then I know which games have had their files changed in the last 6 hours and I specifically target these games among the 30 scanned. As long as no more than 30 games are changed in 6 hours, we're pretty good.

Still the access limit killed Demagogue, because the limit also kicks in for forum accesses.

Thanks for the explanation, even if now.. the number is above what I expected.
I imagined something like 1 or 5 max, not an average of 16 O_o
I thought that Magog downloaded each game page and then processed the data offline.
Does it collect such information "by type" directly online? (through those various passes)

The information I quoted is just for downloaded file information (installer title/name/date/size, patch title/name/date/size, bonus title/name/date/size, etc.). These (and forum) are the limited accesses. Accessing the catalogue, game pages, etc. is not limited AFAIK, or I haven't hit the limit.

To collect all the information about one game, MaGog needs to:
- read and process the catalogue page containing that game (sorted by title; 50 games per page)
- read and process the catalogue page containing that game (filtered by indie; 50 games per page)
- read and process the catalogue page containing that game (sorted by bestselling; 50 games per page)
- read and process a cart with all the GOG games in it to deduce bundle discounts (all games at once, but requires up to 40 attempts to get past GOG's server errors)
- read and process the catalogue page containing that game for each of the 25 different regions (50 games per page, repeated 25 times)
- read and process the movie catalogue so these could be filtered out later on from wishlists (50 movies per page)
- read and process 100+ wishlists, some with multiple pages (100 games per page)
- read and process the game page (1 game per page)
- read and verify the game forum link (1 game per link)
- read and process the catalogue page containing that game for each of the 3 different languages (50 games per page, repeated 3 times)
- read and process 8 GOG mixes
- read and process local information about series and packs (manually maintained by me)
- read and process public file information (1 game per page)
- read and process files in my library (1-2 accesses for every link in the library, dependent on the number of files per game; note that every OS and every language is a different link; so there are games (e.g. Witcher 2/3) that require many hundreds of accesses)

Once a week, also:
- read and process the catalogue page containing that game for each of the 200+ different countries (50 games per page, repeated 200+ times)
- read and process game length information from HowLongToBeat.com (one or more pages per game)

Great Gog!
This process for each game shows how damn complex is Magog =:O

No kidding!

I used to be able to search by ID (I'm sure the field was just set to Title and contains) and it would find both the main ID and the bundled IDs but it doesn't seem to work now... I can use the link you gave me in the AF thread to find a game by ID, but it doesn't work for bundled games.

I'm not sure if you've changed something or I've just forgotten what settings I was using, but it would certainly help for finding missing box art if I could search by IDs again.

Thanks mrgknao :)

No. There was never an option to search by ID via the GUI. I didn't think it useful. That's why I gave you that link.

I can add it if you wish, but wouldn't it be simpler to search by title, via title contains. And that one already supports bundled games (e.g. you can search for "Pool" (of Radiance) to find "Forgotten Realms Archives Collection Two").

Let me know.

Odd, I know there was never an option for it, but searching for the ID in the title field did use to work... Maybe it only actually worked for the bundled IDs (as they are contained in the bundle string usually)
Don't worry about adding it on my account, I just wasn't sure why it didn't seem to work now.