It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Community
General discussionGOG Database - A website that collects data on GOG games(775 posts)(775 posts)
(775 posts)
Pages:
1
10
20
30
37
38
39
40
41
52
This is my favourite topic
Posted April 09, 2021
avatar
Yepoleb: I also would rather not show data than show incorrect or misleading data.
avatar
WinterSnowfall: If only GOG held itself to such high standards.
I don't think a website with only blank pages would be very successful.
Posted April 10, 2021
avatar
Yepoleb: Something I've talked about before is my lack of interest in GOG.
That is the main issue, ain't it? GOG, CDP, the community, none offer incentives to put in anywhere near the amount of time and effort such an endeavor requires, and haven't for a long time, quite the contrary...
Posted April 10, 2021
avatar
Yepoleb: Something I've talked about before is my lack of interest in GOG.
avatar
Cavalary: That is the main issue, ain't it? GOG, CDP, the community, none offer incentives to put in anywhere near the amount of time and effort such an endeavor requires, and haven't for a long time, quite the contrary...
You can say that again.
Posted April 10, 2021
avatar
Alexim: I know you consider the project almost complete, but if I may ask a question, would it be possible to add tracking for achievements if they are available for a game, or is there an alternative way to extrapolate the data?
avatar
Yepoleb: I don't consider the project complete by any means, there's still a lot that would be nice to add, I just want to take a break. Tracking achievements would probably be possible by requesting galaxy APIs documented here:
https://gogapidocs.readthedocs.io/en/latest/galaxy.html#get--clients-(int-product_id)-users-(int-user_id)-achievements
There's a good chance Galaxy 2 introduced even newer better APIs, but I have not looked into that.

Thanks everyone for the feedback so far!

Edit: I just noticed using the profiles it's possible to list the achievements of any game, even those not owned by the user, by changing the product ID in the URL like this: https://www.gog.com/u/Yepoleb/game/1721001406
Thanks, I think it's the same method that mrkgnao also suggested in the message just before and it works very well.

With the game "The Legend of Heroes: Trails of Cold Steel IV" instead I can't find anything, I'm using the wrong ID or maybe it's a strange case due to the fact that the versions now available are special pre-orders?
Posted April 10, 2021
avatar
Yepoleb: I don't consider the project complete by any means, there's still a lot that would be nice to add, I just want to take a break. Tracking achievements would probably be possible by requesting galaxy APIs documented here:
https://gogapidocs.readthedocs.io/en/latest/galaxy.html#get--clients-(int-product_id)-users-(int-user_id)-achievements
There's a good chance Galaxy 2 introduced even newer better APIs, but I have not looked into that.

Thanks everyone for the feedback so far!

Edit: I just noticed using the profiles it's possible to list the achievements of any game, even those not owned by the user, by changing the product ID in the URL like this: https://www.gog.com/u/Yepoleb/game/1721001406
avatar
Alexim: Thanks, I think it's the same method that mrkgnao also suggested in the message just before and it works very well.

With the game "The Legend of Heroes: Trails of Cold Steel IV" instead I can't find anything, I'm using the wrong ID or maybe it's a strange case due to the fact that the versions now available are special pre-orders?
Yes. That's always the case with games that have special short-term exclusives. They are packages rather than games. You need someone who owns the game to identify the game ID.
Posted April 10, 2021
avatar
Alexim: Thanks, I think it's the same method that mrkgnao also suggested in the message just before and it works very well.

With the game "The Legend of Heroes: Trails of Cold Steel IV" instead I can't find anything, I'm using the wrong ID or maybe it's a strange case due to the fact that the versions now available are special pre-orders?
avatar
mrkgnao: Yes. That's always the case with games that have special short-term exclusives. They are packages rather than games. You need someone who owns the game to identify the game ID.
Thanks for the validation!
Posted April 11, 2021
I just disabled the changelog entries for API access and the developer provided text changelog and purged all existing entries from the database. Unfortunately they can't be tracked meaningfully because i'm getting inconsistent data for some games. This is quite concerning, because servers randomly returning different data can lead to some nasty bugs that are near impossible to debug, but luckily I'm not the one responsible for GOG infrastructure.
Posted April 11, 2021
avatar
Yepoleb: Unfortunately they can't be tracked meaningfully because i'm getting inconsistent data for some games.
Which games are causing you issues? Can't say I've run across anything this bad. Don't get me wrong, the APIs have their fair share of black holes, but there's nothing some error handling can't fix as far as I'm aware.

Developer provided changelogs should be fine if you get them from the original products endpoint (you called it v0, if memory serves).

avatar
Yepoleb: This is quite concerning, because servers randomly returning different data can lead to some nasty bugs that are near impossible to debug, but luckily I'm not the one responsible for GOG infrastructure.
*Looks at an ever present purple dot hanging about* You don't say :P.
Post edited April 11, 2021 by WinterSnowfall
Posted April 11, 2021
avatar
Yepoleb: Unfortunately they can't be tracked meaningfully because i'm getting inconsistent data for some games.
avatar
WinterSnowfall: Which games are causing you issues? Can't say I've run across anything this bad. Don't get me wrong, the APIs have their fair share of black holes, but there's nothing some error handling can't fix as far as I'm aware.

Developer provided changelogs should be fine if you get them from the original products endpoint (you called it v0, if memory serves).
I don't know what "Developer provided changelogs" are, but GOG's changelogs have been inconsistent for most of 2020 and were still inconsistent last time MaGog checked (31 December 2020), not to mention the notorious "|" bug (any changelog containing the "|" character was corrupt as of December 2020 --- at the time, around 18 such changelogs).
Post edited April 11, 2021 by mrkgnao
Posted April 11, 2021
avatar
mrkgnao: [...].
Interesting. I've been collecting changelogs since times immemorial, but haven't really kept an eye on them lately. Do you happen to remember any of those 18 IDs that were affected by changelog corruption?

Also, what exactly do you mean by "corrupt"? How does the corruption manifest?
Post edited April 11, 2021 by WinterSnowfall
Posted April 11, 2021
high rated
avatar
mrkgnao: [...].
avatar
WinterSnowfall: Interesting. I've been collecting changelogs since times immemorial, but haven't really kept an eye on them lately. Do you happen to remember any of those 18 IDs that were affected by changelog corruption?

Also, what exactly do you mean by "corrupt"? How does the corruption manifest?
First report of the bug: https://www.gog.com/forum/general/the_what_did_just_update_thread/post24372

Here are the changelogs that MaGog identified as corrupt on its last run on 31/12/20:
Barony: Blessed Addition
Children of Morta
Dead In Vinland
Duskers
Fell Seal: Arbiter's Mark
Field of Glory: Empires
Frostpunk
Gremlins, Inc. [removed on Nov 2, 2017]
Imperator: Rome
Kona
Original War
Orphan
Prison Architect
Sins of a Solar Empire: Rebellion Ultimate Edition
Staxel
This War of Mine
Tyranny - Standard Edition
but like I said, changelogs are mildly inconsistent, so there might be a couple more.

P.S. Just checked one game manually (Fell Seal) and the bug is still there. Eight months after I had opened a ticket about it and seven months after they acknowledged the bug and said they would fix it. Amazing.
Post edited April 11, 2021 by mrkgnao
Posted April 11, 2021
avatar
mrkgnao: First report of the bug: https://www.gog.com/forum/general/the_what_did_just_update_thread/post24372
Oh, wow... I can not believe my eyes (but yes, my scripts have scraped the same corrupted changelogs). Thank you for pointing this out, I was quite unaware of it.

Still, not sure what Yepoleb meant by "servers randomly returning different data" - for me these corrupted changelogs have not changed for a while now. They are consistent in their inconsistency, one could say.
Posted April 11, 2021
avatar
WinterSnowfall: Still, not sure what Yepoleb meant by "servers randomly returning different data" - for me these corrupted changelogs have not changed for a while now. They are consistent in their inconsistency, one could say.
I've attached an example of what my script picked up, it's for Vagrus - The Riven Realms (1555142367)
Another one for Children of Morta (1774901766)

Edit: Download https://new.gogdb.org/backups_v3/products/2021-04/gogdb_2021-04-09.tar.xz if you want to take a look at the raw data before the purge.
Attachments:
screenshot_2021-.png (64 Kb)
screenshot_2021-.png (35 Kb)
Post edited April 11, 2021 by Yepoleb
Posted April 11, 2021
high rated
avatar
mrkgnao: First report of the bug: https://www.gog.com/forum/general/the_what_did_just_update_thread/post24372
avatar
WinterSnowfall: Oh, wow... I can not believe my eyes (but yes, my scripts have scraped the same corrupted changelogs). Thank you for pointing this out, I was quite unaware of it.

Still, not sure what Yepoleb meant by "servers randomly returning different data" - for me these corrupted changelogs have not changed for a while now. They are consistent in their inconsistency, one could say.
For MaGog, these changelogs would fluctuate back and forth between being corrupted and being fine on an irregular basis, sometimes once a week, sometimes several times a day. My assumption was that GOG had multiple servers with the same information and that most of them were corrupt, but a few still fine, and the data depended on which server happened to reply to MaGog's query.

I had the same inconsistent behaviour for years with other info as well (most noticeably the compatibility notes under the system requirements table on the game page).

How often do you scrape for changelogs?
Posted April 11, 2021
avatar
mrkgnao: For MaGog, these changelogs would fluctuate back and forth between being corrupted and being fine on an irregular basis, sometimes once a week, sometimes several times a day.
I knew the GOGBear gets drunk at times, but this is...

avatar
mrkgnao: I had the same inconsistent behaviour for years with other info as well (most noticeably the compatibility notes under the system requirements table on the game page).
Yes, I did notice some of those to be honest.

avatar
mrkgnao: How often do you scrape for changelogs?
Yeah, this is most likely it. I've been doing it weekly for some years now and just recently switched back to daily. I might not be seeing anything that dramatic simply because my scan frequency is nothing to write home about.

avatar
Yepoleb: I've attached an example of what my script picked up, it's for Vagrus - The Riven Realms (1555142367)
Another one for Children of Morta (1774901766)
It kind of sucks to lose changelog support in the GOG DB over a stupid bug like this, but honestly I get where you are coming from. I would stay away from it too until GOG gets its act together.
Post edited April 11, 2021 by WinterSnowfall
Pages:
1
10
20
30
37
38
39
40
41
52
This is my favourite topic
Community
General discussionGOG Database - A website that collects data on GOG games(775 posts)(775 posts)
(775 posts)