It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Community
General discussiongogcli - Gog Client(295 posts)(295 posts)
(295 posts)
Pages:
1
5
10
12
13
14
15
16
20
This is my favourite topic
Posted October 30, 2021
avatar
Geralt_of_Rivia: GOG's content delivery servers for your part of the world must suck really hard or there is some problem with their hardware or in their configuration.
It didn't always used to be this way, until the last year maybe. Before that we had relatively good speeds.

avatar
Geralt_of_Rivia: I just downloaded updated offline installers for 9 games and had 60 MB/s pretty much the whole time (for plain old single threaded browser downloads) so it's obviously not something everyone is seeing. Have you asked fellow Australians about their download experience?
I'm aware that in many other parts of the world, GOG customers get amazing speeds. So it seems pretty clear some kind of bias or lack of care must be happening at the very least. It is hard to credit a faulty server ... seems more like a deliberate act to me.

I don't know any other Aussies locally who get games from GOG. Most of my friends aren't really gamers. Both my boys have GOG accounts, but rarely use them, and one of them lives at one end of the country near the big smoke of Sydney, so a long way from me. As for any Aussies here at GOG, I am relying on them to respond to the posts I make at the forum, and I've made quite a few now in the last year about speed issues. Internet services and speeds vary quite a bit around the country, and I have not changed anything for a few years in regard to my Internet connection, so I know it ain't me, plus there is the good speeds I get from Epic and Steam and many other places, so my ISP cannot be blamed.

While this speed issue continues to happen for me, I will not be buying large file games. My purchase model is built around buy and immediately download. If I cannot do that, I'm not buying.

I've been a great supporter of GOG, buying lots of games from them ... though they don't get an automatic pass from me, and need to improve in many ways. I never turn a blind eye to their failings, and this speed issue is impacting me badly and damaging our relationship.

It's not good when a business loves to take your money, and then makes it hard to get the goods once you've paid.

@Magnitus - My apologies for taking things off subject, though I certainly hope a new version of gogcli.exe can help improve the download speeds I get.
Posted October 30, 2021
avatar
Timboli: I'm aware that in many other parts of the world, GOG customers get amazing speeds. So it seems pretty clear some kind of bias or lack of care must be happening at the very least. It is hard to credit a faulty server ... seems more like a deliberate act to me.
I usually get better download speeds then you do (possibly a perk of being less than 100km North of the US border, though I would posit that the quality of the regional networks also makes a huge difference), but on the day you mentioned, I also had poor performance from GOG.

Since I've implemented retries, I keep my download retries at 2 and experience practically no download interruptions, but on that day, there were 2 interruptions where I had to manually resume the downloads (and it wasn't because my manifest was out of sync due to some update on the installers, the download performance coming from GOG was just not good). I think it wasn't a good day for them.

I guess I could have put the retries at an absurdly high value like 50, but if you need to put it that high, it probably means that the server is struggling quite a bit and you're essentially just helping kicking it further down the curb by saturating it with even more traffic. At some point, it's best to just desist and try again when things are better.

avatar
Timboli: While this speed issue continues to happen for me, I will not be buying large file games. My purchase model is built around buy and immediately download. If I cannot do that, I'm not buying.
Agreed. The main differentiator of GOG is that you can download your game and play them offline. If that is gone, I don't see much of a point.

avatar
Timboli: I've been a great supporter of GOG, buying lots of games from them ... though they don't get an automatic pass from me, and need to improve in many ways. I never turn a blind eye to their failings, and this speed issue is impacting me badly and damaging our relationship.
Same boat as you, in my case more in relation to their increasing disregard for the values of the user base they purportedly built their brand around and that helped make the website relevant in the first place.

avatar
Timboli: It's not good when a business loves to take your money, and then makes it hard to get the goods once you've paid.
Unfortunately, wouldn't be the first time and probably won't be the last.

Referring more broadly in my case to getting what is advertised: drm-free games.

avatar
Timboli: @Magnitus - My apologies for taking things off subject, though I certainly hope a new version of gogcli.exe can help improve the download speeds I get.
No worries. I hope so too.
Post edited October 30, 2021 by Magnitus
Posted October 30, 2021
avatar
Geralt_of_Rivia: GOG's content delivery servers for your part of the world must suck really hard or there is some problem with their hardware or in their configuration.
avatar
Timboli: It didn't always used to be this way, until the last year maybe. Before that we had relatively good speeds.

avatar
Geralt_of_Rivia: I just downloaded updated offline installers for 9 games and had 60 MB/s pretty much the whole time (for plain old single threaded browser downloads) so it's obviously not something everyone is seeing. Have you asked fellow Australians about their download experience?
avatar
Timboli: I'm aware that in many other parts of the world, GOG customers get amazing speeds. So it seems pretty clear some kind of bias or lack of care must be happening at the very least. It is hard to credit a faulty server ... seems more like a deliberate act to me.

I don't know any other Aussies locally who get games from GOG. Most of my friends aren't really gamers. Both my boys have GOG accounts, but rarely use them, and one of them lives at one end of the country near the big smoke of Sydney, so a long way from me. As for any Aussies here at GOG, I am relying on them to respond to the posts I make at the forum, and I've made quite a few now in the last year about speed issues. Internet services and speeds vary quite a bit around the country, and I have not changed anything for a few years in regard to my Internet connection, so I know it ain't me, plus there is the good speeds I get from Epic and Steam and many other places, so my ISP cannot be blamed.

While this speed issue continues to happen for me, I will not be buying large file games. My purchase model is built around buy and immediately download. If I cannot do that, I'm not buying.

I've been a great supporter of GOG, buying lots of games from them ... though they don't get an automatic pass from me, and need to improve in many ways. I never turn a blind eye to their failings, and this speed issue is impacting me badly and damaging our relationship.

It's not good when a business loves to take your money, and then makes it hard to get the goods once you've paid.

@Magnitus - My apologies for taking things off subject, though I certainly hope a new version of gogcli.exe can help improve the download speeds I get.
Like you, lately I've often had low download speeds from gog, while all tests (other sites, speedtest, etc.) show my internet connection (80Mb/s) is fine. Speeds on the order of a few hundred kilobits per second. Unlike you, I'm in the northern central USA. For a while, I found that downloading at 1am UTC-5 would have faster speeds, but that seems to be rare anymore.

I'm not familiar with the details of modern distribution methods. I know some sites use distribution networks for supplying files, but I'd suspect that gog uses their own servers. It's beginning to seem like they use a single 386 machine with a 1 Mb/s connection.

One problem probably comes from galaxy. Before galaxy, one had to manually select a download, although the old downloader could queue up a few downloads (up to 4 at once, if I recall). Now, galaxy updates things automatically and there are a lot of updates each day.
Posted October 31, 2021
avatar
Magnitus: I usually get better download speeds then you do (possibly a perk of being less than 100km North of the US border, though I would posit that the quality of the regional networks also makes a huge difference), but on the day you mentioned, I also had poor performance from GOG.
Alas, I had exactly the same poor speeds yesterday, and while generally I have much better than that, they are no longer now around what they should be ... or it has become a rare event when they are.

One could for instance wait until a sale is over, though GOG are always having sales, and in reality for me it is often just a case of bad instead of worse much of the time, if I do wait. I'm not inclined to wait, why should I ... so if that further taxes their servers then so be it ... they need to improve the server situation, which probably won't happen unless more complaints occur.

It is simply not right, that a good number can download at what I would call fantastic speeds, and I cannot download at even just a decent one.

I am about try try again, for a game I bought last night (The Shore), but no doubt I will be out of luck and be forced to use Free Download Manager 5, which is marginally better with download speed in the last few days. Wish me luck.

EDIT
Nope, precisely the same lousy download speed again for the third day in a row - 204 KB/s.
Free Download Manager 5 is marginally better at around 800 KB/s.
This is for two 4 Gb BIN files.
Dinner time here on a Sunday.
EDIT 2
Took me 1 hour and 23 minutes to download the first BIN file, and just under 5 minutes to validate its MD5.
Luckily it is only two large BIN files all up.
Free Download Manager 5 is set to use 5 threads for one download.

avatar
drrhodes: Like you, lately I've often had low download speeds from gog, while all tests (other sites, speedtest, etc.) show my internet connection (80Mb/s) is fine. Speeds on the order of a few hundred kilobits per second. Unlike you, I'm in the northern central USA. For a while, I found that downloading at 1am UTC-5 would have faster speeds, but that seems to be rare anymore.
I download at all sorts of times around the clock, and like for you, it rarely seems to make much difference now, certainly in the last few days.

I don't know if Galaxy Users fare better, but I wouldn't be surprised if they did ... bias and all ... at least many things seem to point that way. Us 'Offline Installer' folk seem to be an endangered species.
Post edited October 31, 2021 by Timboli
Posted November 03, 2021
For those not aware, I have now released GOGcli GUI v2.0 a Windows frontend for gogcli.exe.

https://www.gog.com/forum/general/gogcli_gui_frontend_downloader_and_validator/post76

As noted there, I have only just started using gogcli.exe v0.12.0 yesterday.

I did not get to use that version much yesterday, as my download speed from GOG was an appalling 102 KB/s ... only slightly improved upon (maybe a bit over double, but constantly fluctuating) using Free Download Manager 5 with its multiple threads. So all I did was a manifest update for a new game (latest demo at GOG), and then attempt to download that game, which I cancelled due to such pathetic download speed. Luckily the demo was only a couple of hundred Mb, so FDM5 sufficed.

Five or more days of absolutely appalling download speeds from GOG ... what more can I say ... I feel a boycott coming on. In that same time I have managed to do quite a few large game downloads from Epic at great speeds, while testing another program of mine (ESC), which I have been developing. So the fault is all GOGs ... clearly.
Post edited November 03, 2021 by Timboli
Posted November 20, 2021
A bit more feedback about download speed when a big sale is not on.

If I download a game file that is less than 1.5 Gb, I can often get a good speed approaching 4 MB/s. File sizes over that unfortunately, the speed drops by about half. I have been taking note of this for many months now.

This has caused me to download the small files using my GOGcli GUI, just to get the folders created and download cover image and description and changelog files. And then I use Free Download Manager 5 to get the larger files, which I can download at speeds approaching 5 MB/s (no doubt because of multi-thread). It is all rather painful though, as I then have to relocate and separately checksum those large files afterwards. So clearly GOG are still deliberately limiting speed ... probably to force use of Galaxy.
Post edited November 20, 2021 by Timboli
Posted November 20, 2021
avatar
Timboli: A bit more feedback about download speed when a big sale is not on.

If I download a game file that is less than 1.5 Gb, I can often get a good speed approaching 4 MB/s. File sizes over that unfortunately, the speed drops by about half. I have been taking note of this for many months now.

This has caused me to download the small files using my GOGcli GUI, just to get the folders created and download cover image and description and changelog files. And then I use Free Download Manager 5 to get the larger files, which I can download at speeds approaching 5 MB/s (no doubt because of multi-thread). It is all rather painful though, as I then have to relocate and separately checksum those large files afterwards. So clearly GOG are still deliberately limiting speed ... probably to force use of Galaxy.
I'm getting worked to the bone right now (I'm pulling 60-70 hours work weeks to meet crushing deadlines atm and I'm pretty much at my limit psychologically in terms of how much I can do) so I can't do much atm to try address your predicament.

Trying to leverage a cloud provider to resolve your issue is something that is of great interest to me though, both from a technical perspective, but also in terms of social agenda (to empower non-technical end-users to leverage cloud providers more without having to rely on the centralised control of middlemen corporate operators) so I'll definitely be interested to work with you on this once I have more time to call my own.
Post edited November 20, 2021 by Magnitus
Posted November 21, 2021
avatar
Magnitus: I'm getting worked to the bone right now (I'm pulling 60-70 hours work weeks to meet crushing deadlines atm and I'm pretty much at my limit psychologically in terms of how much I can do) so I can't do much atm to try address your predicament.
No worries, please don't feel I am trying to put pressure on you or attempting to guilt trip you, that wasn't the purpose of my post. I just needed to tell someone, and you were the obvious one going by previous posts etc ... and you won't be the only one to read my comments. You are also not responsible for the speed issue, GOG is.

In any case, there is no urgency my end, as I am dealing with the situation, if in a less than desirable way.

avatar
Magnitus: Trying to leverage a cloud provider to resolve your issue is something that is of great interest to me though, both from a technical perspective, but also in terms of social agenda (to empower non-technical end-users to leverage cloud providers more without having to rely on the centralised control of middlemen corporate operators) so I'll definitely be interested to work with you on this once I have more time to call my own.
I remain interested in anything that might improve things, and this certainly sounds a possibility.

Hope you get a break from all that work soon, sounds like you will need a good rest, perhaps just to chill out and do nothing for a while.
Posted February 21, 2022
All the problematic files during the sale (those that returned length 1 for the file metadata link and others) seem to now download properly with my client, except for Northgard (clan of the kraken dlc in Windows).

Given that curl also gives me the same error when I add ".xml" to the final download url, I think it is really a problem with the source.

Somehow, I don't see GOG fixing that anytime soon, so I guess I'll just refrain from updating the game for now and implement a workaround when I have more time in March.
Posted February 21, 2022
Thanks for the info.
Posted February 21, 2022
avatar
Magnitus: All the problematic files during the sale (those that returned length 1 for the file metadata link and others) seem to now download properly with my client, except for Northgard (clan of the kraken dlc in Windows).
For me it's the XML for /downloads/arma_gold_edition/en1installer3 that is still empty. :-(

I hope GOG fixes this soon, otherweise there is no way to properly verify that the files have downloaded without error.
Posted February 21, 2022
avatar
Timboli: Thanks for the info.
You're welcome :).

avatar
Geralt_of_Rivia: For me it's the XML for /downloads/arma_gold_edition/en1installer3 that is still empty. :-(

I hope GOG fixes this soon, otherweise there is no way to properly verify that the files have downloaded without error.
There are two ways:
- You maximize the number of formats that are integrity checked like zip files are (I don't think it is possible with the Linux bash installers, might be doable with the Windows and MacOS installers... it would be an incredible amount of work though)
- You download the file separately before you do your real download with the sole purpose being to get a checksum

My reasoning being:

Its very unlikely that transient download content errors that don't give any other indication (bad http status code or download size that doesn't match the http content length header) would result in the exact same download malformation occuring twice.

Granted, gog may be holding a bad file, but at that point, my guess is that the xml format would match the info of the bad file so it wouldn't be caught either way (unless you're applying some REALLY fancy pattern recognition and expert system logic in there, automation is not magic... its unlikely that whatever automaton they are using to generate the xml content would catch a bad file on their end).

The solution I'm most likely to implement is:
- The user pass a flag to indicate whether he/she is willing to tolerate downloads that have no xml metadata
- If the xml metadata is available, good
- If the xml metadata is bad and the user didn't pass the flag, everything halts there
- If the xml metadata is bad and the user pass the flag, download the file separately before the real download to get a checksum to match the download against, give the user feedback about what just happened and proceed normally. At that point, the user can decide, based on the feedback, if he/she wants to manually verify the installer or otherwise redownload the game at a later time to see if something changed.
Post edited February 21, 2022 by Magnitus
Posted February 21, 2022
avatar
Timboli: Thanks for the info.
avatar
Magnitus: You're welcome :).

avatar
Geralt_of_Rivia: For me it's the XML for /downloads/arma_gold_edition/en1installer3 that is still empty. :-(

I hope GOG fixes this soon, otherweise there is no way to properly verify that the files have downloaded without error.
avatar
Magnitus: There are two ways:
- You maximize the number of formats that are integrity checked like zip files are (I don't think it is possible with the Linux bash installers, might be doable with the Windows and MacOS installers... it would be an incredible amount of work though)
- You download the file separately before you do your real download with the sole purpose being to get a checksum

My reasoning being:

Its very unlikely that transient download content errors that don't give any other indication (bad http status code or download size that doesn't match the http content length header) would result in the exact same download malformation occuring twice.

Granted, gog may be holding a bad file, but at that point, my guess is that the xml format would match the info of the bad file so it wouldn't be caught either way (unless you're applying some REALLY fancy pattern recognition and expert system logic in there, automation is not magic... its unlikely that whatever automaton they are using to generate the xml content would catch a bad file on their end).

The solution I'm most likely to implement is:
- The user pass a flag to indicate whether he/she is willing to tolerate downloads that have no xml metadata
- If the xml metadata is available, good
- If the xml metadata is bad and the user didn't pass the flag, everything halts there
- If the xml metadata is bad and the user pass the flag, download the file separately before the real download to get a checksum to match the download against, give the user feedback about what just happened and proceed normally. At that point, the user can decide, based on the feedback, if he/she wants to manually verify the installer or otherwise redownload the game at a later time to see if something changed.
I'm already maximizing the amount of files that are integrity checked and use different methods for checking according to file type, that is not the problem. The problem is that .bin files can only be checked with the md5 from the XML or by using innoextract. I'm NOT going to use innoextract because that is slower than a snail and if the XML is defective I'm out of options.

At this point I'm also only logging which files can not be checked and leave it to manual checking.

I have also found that the rabbit hole called XML problem goes deeper than just some files being empty or just 1 character long. Some XML files (the one for /downloads/aven_colony/en1installer3 for example) don't download at all. I also tested it in a browser. It takes about 5-15 secornds until it times out with a 'Secure connection failed' message in the browser. Strangely enough when you try to download the same URL with http directly afterwards it will redirect to https and then the XML will download. Trying http right from the start doesn't work either.

GOG still has quite a bit of fixing to do...
Posted February 21, 2022
avatar
Geralt_of_Rivia: I'm already maximizing the amount of files that are integrity checked and use different methods for checking according to file type, that is not the problem. The problem is that .bin files can only be checked with the md5 from the XML or by using innoextract. I'm NOT going to use innoextract because that is slower than a snail and if the XML is defective I'm out of options.
I verify the integrity of the zip files already. Not sure of other formats to check.

For the speed of checking innoextract, I don't mind doing it as a separate step from the download if necessary. The real killer for me is that if I want to support this is a platform-agnostic way with golang, I might have to either create some bingings for it or re-write the thing in golang. Would be a fun little project if I was between jobs, but with the limited time I have to work on the tool right now, I'd rather focus on something else.

avatar
Geralt_of_Rivia: I have also found that the rabbit hole called XML problem goes deeper than just some files being empty or just 1 character long. Some XML files (the one for /downloads/aven_colony/en1installer3 for example) don't download at all. I also tested it in a browser. It takes about 5-15 secornds until it times out with a 'Secure connection failed' message in the browser. Strangely enough when you try to download the same URL with http directly afterwards it will redirect to https and then the XML will download. Trying http right from the start doesn't work either.

GOG still has quite a bit of fixing to do...
Yes, Northgard gives me some weird protocol error:

"GetDownloadFileInfo(downloadPath=/downloads/northgard_lyngbakr_clan_of_the_kraken/en1installer0) -> retrieval body error: stream error: stream ID 445; PROTOCOL_ERROR"

curl reports similar errors. The 1 length content problems seem to be getting fixed, but those protocol errors are not getting addressed and I get the feeling they might be here to stay.

Honestly, I've been slow to invest a lot of time to deeply leverage everything offered in the xml format (partial downloads for example), because I had a feeling something like this might happen.

My understanding is that the previous downloader that gog officially supported made use of it, but now that downloader is obselete and I don't think there is anything GOG officially provides that uses this xml format anymore.

And my experience with things that are not openly relied on is that they end up getting broken (software is a moving target and things that are important get maintained properly... everything else tends to fall by the wayside).
Post edited February 21, 2022 by Magnitus
Posted February 22, 2022
avatar
Magnitus: My understanding is that the previous downloader that gog officially supported made use of it, but now that downloader is obselete and I don't think there is anything GOG officially provides that uses this xml format anymore.

And my experience with things that are not openly relied on is that they end up getting broken (software is a moving target and things that are important get maintained properly... everything else tends to fall by the wayside).
The prospect of where that could lead is troubling, and could well indicate that third party downloaders for GOG now have a useby date. No doubt it is likely a slow slide, for now anyway, but when we may increasingly need to use Galaxy to download a particular game, that's just an indicator of the coming end.
Pages:
1
5
10
12
13
14
15
16
20
This is my favourite topic
Community
General discussiongogcli - Gog Client(295 posts)(295 posts)
(295 posts)