It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
avatar
timppu: I'm a long-time gogrepo,py user (which is the other active user-made tool for the same purpose), and just out of curiosity I finally now decided to try out lgogdownloader as well, as I've lately become more active in Linux too. The main reason I didn't try lgogdownloader before was because I was not sure how to set it up on Windows (has someone already tried it successfully?).

For now I tried it in a virtualboxed Linux Mint, but I'll probably try it also on my real Mint installation. Seems to work fine as far as I can see, no hiccups so far. I can't really make a complete comparison of features as I tried lgogdownloader only barely (login, get list of all games, start downloading all games).

One positive finding though was that the whole setup part (where the tool prepares for downloading by getting game details) seems to go much faster on lgogdownloader than it goes in gogrepo. For my 1554 games (English/Windows versions only), the setup part currently takes like 3 hours or more on gogrepo (it used to be much worse earlier like 11 hours or so, but the setup process was later optimized and become quicker), but on lgogdownloader it took like 10 minutes or so before it could start downloading the games.

Is lgogdownloader using some completely different method for that setup process, preparing for downloads? Or is it because of the different API?
I'm not sure but I think gogrepo gets more details during the manifest update.
I haven't checked the code of gogrepo in a while so I don't know if it gets the details in parallel using multiple threads.

lgogdownloader uses the account page api (https://www.gog.com/account/getFilteredProducts) to get initial product info (product names and ids, dlc names, etc)
Then it uses multiple threads to get more details using galaxy api (https://gogapidocs.readthedocs.io/en/latest/galaxy.html#api-gog-com)
Just before starting to download a file it tries to get xml data for the file

avatar
timppu: Too bad lgogdownloader and gogrepo directory structures are not 1:1, ie. the former puts extras to a separate subdirectory while gogrepo doesn't, so I can't freely use both interchangeably. I kinda like the idea of extras going to a subdirectory, I need to check if it would be possible to make it an option in gogrepo to put extras to a similar subdirectory so then both could be used for the same collection.
You can change the directory structure of lgogdownloader by using --subdir-* options

Set subdirectory for installers
--subdir-installers arg

Set subdirectory for extras
--subdir-extras arg (=extras)

Set subdirectory for patches
--subdir-patches arg (=patches)

Set subdirectory for language packs
--subdir-language-packs arg (=languagepacks)

Set subdirectory for dlc
--subdir-dlc arg (=dlc/%dlcname%)

Set subdirectory for game
--subdir-game arg (=%gamename%)

Supported templates:
%platform% --> windows/mac/linux
%gamename% --> name of the game
%dlcname% --> name of the dlc
There is also --no-subdirectories option which disables subdirectories for extras, patches and language packs
These can also be overriden using game specific config files (see man page for details)

For example setting --subdir-extras to empty puts downloaded extras in same directory with installers
--subdir-extras ""

avatar
timppu: I'd also like to check if it is possible to easily set lgogdownloader to run on Windows...
It was possible to compile lgogdownloader on Windows using cygwin or msys but I'm not sure if it still works because I haven't tried it in a long time.
avatar
Sude: There is also --no-subdirectories option which disables subdirectories for extras, patches and language packs
So it puts all game files (including extras) to the same game subdirectory? I guess that should be then pretty much the same as the gogrepo directory structure, maybe I'll check it.

A couple of clarifying questions which I am sure have been answered earlier in the thread:

1. What are "languagepacks"? Do they refer to the game installer in different languages, or something else?

2. What are the common commands to run in order to keep a clean and up-to-date collection? Something like:

lgogdownloader --login (is this needed only the first time, or every time you want to use lgogdownloader?)

lgogdownloader --download --include-all

lgogdownloader --check-orphans (does this delete all obsolete files, or move them somewhere, or just report which such files there are?)

lgogdownloader --repair (what does this mean in practice? Check the file integrity against the md5 checksums, and redownload those files which don't pass the check? How does it check e.g. the extras (as they don't have md5 checksums, I believe)?
Post edited August 23, 2018 by timppu
avatar
timppu: ...
I use them both and I'm pretty sure its no-subdirectories that is all thats needed to get it all working on the same data.
avatar
timppu: 1. What are "languagepacks"? Do they refer to the game installer in different languages, or something else?
Some games have separate language pack installers which you can download and install to get extra language options.
For example Witcher 3 has multiple language packs (German, French, Portuguese-Brazilian, Polish, Russian and Japanese). To list all language packs for Witcher 3 you can use:
lgogdownloader --list-details --language all --include languagepacks --game witcher_3

avatar
timppu: 2. What are the common commands to run in order to keep a clean and up-to-date collection?
To keep clean and up-to-date collection you can use
lgogdownloader --download --check-orphans
to download everything and list orphaned files and then do what you want to orphaned files.
Orphan check outputs the file paths for orphaned files so you can just copy/paste those file paths to a document to keep track of them, paste them to a program to mass delete or move them, etc.

You can also use "lgogdownloader --status" to check status of currently existing files.

--status Show status of files
Output format:
statuscode gamename filename filesize filehash

Status codes:
OK - File is OK
ND - File is not downloaded
MD5 - MD5 mismatch, different version
FS - File size mismatch, incomplete download
As an example here's what it outputs for me when I check status for Beneath a Steel Sky

$ lgogdownloader --status --platform all --language all --game beneath_a_steel_sky
Getting game names (2/2) 38 / 38
Getting game info 1 / 1
OK beneath_a_steel_sky setup_beneath_a_steel_sky_1.0_(20270).exe 91327544 9c136a1b34d759bc8fb748eca732f092
ND beneath_a_steel_sky beneath_a_steel_sky_2.0.0.16.dmg
OK beneath_a_steel_sky beneath_a_steel_sky_en_gog_2_20150.sh 105864419 5cc68247b61ba31e37e842fd04409d98
ND beneath_a_steel_sky beneath_a_steel_sky_french_2.0.0.16.dmg
ND beneath_a_steel_sky beneath_a_steel_sky_italian_2.0.0.16.dmg
ND beneath_a_steel_sky beneath_a_steel_sky_german_2.0.0.16.dmg
ND beneath_a_steel_sky beneath_a_steel_sky_spanish_2.0.0.16.dmg
OK beneath_a_steel_sky bass_manual.zip 125311 20380c6ee39791ef126e8f82338b6d85
OK beneath_a_steel_sky bass_security_manual.zip 6256215 1fe6133108ec0efc548594bd6ae9b25c
OK beneath_a_steel_sky bass_wallpapers.zip 5331651 020d9e45974c90f7a980dec12f5e405f
OK beneath_a_steel_sky bass_avatars.zip 264808 543a32578a2c7d0b64b9da2f9206cc83
OK beneath_a_steel_sky bass_comic_book.zip 12407867 1364b8d8d7051eed1b7aeb87741aecc3
avatar
timppu: lgogdownloader --login (is this needed only the first time, or every time you want to use lgogdownloader?)
--login is only needed the first time or if your login has expired

avatar
timppu: lgogdownloader --check-orphans (does this delete all obsolete files, or move them somewhere, or just report which such files there are?)
--check-orphans just lists all files the downloader thinks are orphaned.
It uses regex to determine which files it is interested in.
The default regex is '.*\.(zip|exe|bin|dmg|old|deb|tar\.gz|pkg|sh)$' which should include all file extensions that GOG uses (zip, exe, bin, sh, dmg) or used previously (deb, tar.gz, pkg) and "old".
".old" file extension is used by lgogdownloader when it moves a file because new version with same filename is encountered.

I wrote a patch long time ago that made --check-orphans delete orphaned files but I never merged it and I'm not sure if it still applies correctly to current version of downloader.

avatar
timppu: lgogdownloader --repair (what does this mean in practice? Check the file integrity against the md5 checksums, and redownload those files which don't pass the check? How does it check e.g. the extras (as they don't have md5 checksums, I believe)?
--repair checks the file integrity against the md5 checksums in xml data.
If a file chunk fails integrity check then it tries to repair it by downloading that specific file chunk.

For extras it relies on checksum data that the downloader creates for them after it finishes downloading them if automatic xml creation is enabled (--create-xml "automatic" or --automatic-xml-creation)
It's a hack to get around the issue of GOG not providing checksum data for extras.
Because the data isn't provided by GOG --repair is not 100% accurate for extras but gets the job done.
For example if GOG updates the extra and it has same file name and size then it's possible that lgogdownloader fails to download the new file or uses old xml data on new file and thinks that the new file is corrupted when actually the xml data is wrong.

There are some differences to how automatic xml creation works depending on whether the downloader is working in multithreaded mode (default) or single threaded mode (forced by using --repair --download, because repair mode is single threaded)
When running the downloader in multithreaded mode it adds all successfully downloaded extras (libcurl returns CURLE_OK) to queue to wait until all downloads have finished and then creates xml data for all files in the queue.
When running the downloader in single threaded mode it creates the xml data right after successfully downloading the file.
Hi all,

Something strange and annoying happened to me 2 days ago: Alongside downloading the updated installers for War for the underworld (war_for_the_overworld_en_2_0_4_23102.sh) and Icewind Dale Enhanced Edition (icewind_dale_enhanced_edition_en_2_5_17_23121.sh), lgogdownloader found and downloaded updates for the previous installers (war_for_the_overworld_en_2_0_3f1_22287.sh and icewind_dale_enhanced_edition_en_2_5_16_3_20626.sh, respectively).
It was suspicious enough for me to investigate what those updates were, and I noticed:
- that those updated previous installers selfcheck (--check) correctly;
- are significantly smaller than the files they replaced;
- fail to run correctly on my box:

$ ./war_for_the_overworld/war_for_the_overworld_en_2_0_3f1_22287.sh.broken
Verifying archive integrity... All good.
Uncompressing War for the Overworld (GOG.com) 100%
Collecting info for this system...
Operating system: linux
CPU Arch: x86_64
trying mojosetup in bin/linux/x86_64
USING fr_FR

PANIC
Initial setup failed. Cannot continue.

Error: Couldn't run mojosetup

Those previous installers are no longer available for download anymore, so I understand this issue will be difficult to debug now. I assume some broken installers have been generated by GOG, and were available for a while thru the API used by lgogdownloader; I reported the issue to the support, but we may never know for sure.

The good thing is, lgogdownloader renamed the old installers, so you shouldn't lose anything automatically; But you should be extra careful if you see installers being updated "in place" (with no change of version). Maybe it's a minor change, maybe that's a broken installer overwriting your previous installer version.

Regards,
Pierre.
Post edited August 24, 2018 by petchema
35b178e Galaxy: Add support for downloading dependencies
Adds support for downloading dependencies for a game
Added --galaxy-no-dependencies option to disable downloading dependencies during --galaxy-install

05e9409 Add separate option to set libcurl to verbose mode
This makes --verbose option much more useful because the downloader no longer prints lots of info from libcurl which is mainly useful for debugging and not very useful for most users.
The new option to set libcurl to verbose mode is --curl-verbose
I used the Linux Galaxy install for Deadfire for its last update due to the patch file being rather slow to show up. I ran into a tiny issue with it as it didn't use the same directory name as the offline installer. It stuffed a : after the pillars 2 part. Would it be possible to add the means to modify the game directory when using the Galaxy install. Its probably more work than a renamed directory is worth but as its something you don't spot till lgogdownloader is pulling files its easy to miss or as with me realise whats happening.

Other than that its really a very handy option to have so thanks for the work.
avatar
Cusith: Would it be possible to add the means to modify the game directory when using the Galaxy install.
1b3b04b Galaxy: Add option to set subdirectory for installation

8e55b81 Galaxy: Extend subdir option to work with MojoSetup hack

--subdir-galaxy-install arg (=%install_dir%)
Set subdirectory for galaxy install

Templates:
- %install_dir% = Installation directory from Galaxy API response
- %gamename% = Game name
- %title% = Title of the game
- %product_id% = Product id of the game
- %install_dir_stripped% = %install_dir% with some characters stripped
- %title_stripped% = %title% with some characters stripped

"stripped" means that every character that doesn't match the following list is removed:
> alphanumeric
> space
> - _ . ( ) [ ] { }
The new option also works with --no-subdirectories option so keep that in mind if you have been using --no-subdirectories option or have it set to true in config file
avatar
Sude: 1b3b04b Galaxy: Add option to set subdirectory for installation
8e55b81 Galaxy: Extend subdir option to work with MojoSetup hack
Thats great thanks
Post edited September 12, 2018 by Cusith
Since a while I get a lot of download errors (404) for the CRC files on nearly every run of lgogdownloader. Does anybody else also see these errors or is it just my GOG mirror causing the problem?

I'm still using the last tagged version of lgogdownloader, v3.3, in case that matters.

Edit: I also get the errors with the current development version (ab84d6f). Not that I think it's a problem of lgogdownloader, I'm rather sure that's just another GOG problem. But I wanted to update anyway.
Post edited September 14, 2018 by eiii
LGOGDownloader 3.4
- Added options to set low speed timeout values for libcurl
* --lowspeed-timeout
* --lowspeed-rate
- Fixed usage of deprecated jsoncpp API (Patch by: Hans Philipp Annen)
- Made --galaxy-install multithreaded
- Added support for generation 1 builds to --galaxy-show-builds
- Show build generation info in --galaxy-show-builds output
- Removed cover download support
- Removed numeric option values from help text
- Updated help text for --galaxy-install and --galaxy-show-builds
- Skip complete files instead of trying to resume
- Fixed -Wcatch-value for GCC 8
- Added hack for installing Linux games using --galaxy-install
* Downloader tries to use Linux installers as replacement for Galaxy repository
- Use regex for language detection in --galaxy-install
* Some builds used different language code format
- Added operating system and architecture to default user agent
* New default user agent is LGOGDownloader/{version_number} ({operating_system} {architecture})
- Removed aliases for --game option
- Removed --update-check option
- Added new option --updated
* Restricts downloader to operate only on games that have update flag set in account page
- Added new option --clear-update-flags
* Clears update notification flags for all games
- Added new option --notifications
* Shows the number of new forum replies, updates games, unread chat messages and pending friend requests
- Added support for downloading dependencies when using --galaxy-install
* Use the new --galaxy-no-dependencies option to disable downloading dependencies
- Added separate option to set libcurl to verbose mode
* --curl-verbose
* This makes --verbose option more useful because it no longer sets libcurl to verbose mode
- Added option to set subdirectory for --galaxy-install
* --subdir-galaxy-install
* Default: %install_dir%
* See help text for possible template values
- Added option to set priority for content delivery network used with --galaxy-install
* --galaxy-cdn-priority
* Default: "edgecast,highwinds,gog_cdn"

https://sites.google.com/site/gogdownloader/lgogdownloader-3.4.tar.gz

sha256: 1ac33dc6678d2d40ecd3c042f0514ca905d2949c91240bda4d9e17d3ff433903
md5: 69fdd40ab8ca056b2c82bc351d0f6010

The alises "all" and "free" for --game option had become useless and thus removed.
"free" alias could no longer be used as originally intended and "all" was just unnecessary because leaving regex empty has the same effect.
Cover download support was removed because the cover list hadn't been updated in a long time and no one was willing to maintain it.
Checking for updates was rewritten to expect GOG update notifications to be accurate. Trying to work around issues with GOG's update flag problems just isn't worth the effort. Getting GOG to have consistent update notification flags is much better solution in the long term.
The numeric option values for include/exclude options have changed and have been removed from help text because users shouldn't be using them anymore. If you are seeing some weird behavior with include/exclude then make sure you are not using numeric option values for them.
Post edited September 15, 2018 by Sude
just had to reinstall Ubuntu and didn't get my lgogdownloader config folder saved. I exported the gog.com cookie like I was supposed to, but lgogdownloader keeps deleting it and immediately complains about reCAPTCHA. how do I make lgogdownloader stop deleting the cookie it needs to log into gog and youknow, log in to gog?
Randall, log off GOG in your web browser and log in again (filling out the captcha); then export the cookie. It seems that GOG stores some information about your installed OS which did not quite match with the stored information in your cookie.

Regarding the --covers switch: the wiki on http://www.gogcovers.com/index.php?title=Main_Page seems to be quite active and maintained? Would it be possible to draw the covers directly from there?
Post edited September 22, 2018 by coffeecup
New step-by-step building instructions for CentOS 7:
https://gist.github.com/shakeyourbunny/33e228844bd04e5ca6ce750cc287c24d

Short / condensed version:
- install (and activate) epel and city-fan.org repository for newer curl version (lgogdownloader will choke on it otherwise)
- install these packages: git cmake3 gcc gcc-c++ boost boost-devel libcurl-devel liboauth jsoncpp-devel rhash-devel help2man tinyxml2-devel
- download, compile and install htmlcxx from source (sourceforge.net), quirks on pkconfig beware, edit html/tree.h, add '#include <stddef>' there before compiling.
- use cmake3 instead of cmake (on CentOS 7, cmake is 2.x)
- install to /usr/local or /opt/lgogdownloader, /usr is only for stuff directly managed by the operating system (this is also true for every linux/unix system; don't put self compiled stuff there)

Old instructions: https://www.gog.com/forum/general/lgogdownloader_gogdownloader_for_linux/post763

I would be grateful if a link to it would be included in the regular building instructions, they seem to be tailored for Debian / Ubuntu distributions.
Post edited September 23, 2018 by coffeecup
avatar
coffeecup: Randall, log off GOG in your web browser and log in again (filling out the captcha); then export the cookie. It seems that GOG stores some information about your installed OS which did not quite match with the stored information in your cookie.
that seems to have worked, thank you