It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Community
General discussion (archive)gog-backup, an unofficial GOG downloader (Cross-platform, CLI)(31 posts)(31 posts)
(31 posts)
Pages:
1
2
3
This is my favourite topic
Posted February 17, 2012
Neat program! This looks much more powerful than GOG's downloader. I might finally be able to download my whole GOG library. :D



Unfortunately I've run into some trouble. I've successfully done a login, manifest, & compare but my fetch fails. :(

fd = os.open(name, os.O_WRONLY | os.O_CREAT | os.O_BINARY, 0666)
AttributeError: 'module' object has no attribute 'O_BINARY'

http://pastebin.com/d3nrgUjt


I'm running Debian Sid i386 and Python 2.7.


Any idea how I can work around this? Am I doing something wrong?

Any help would be appreciated. Thanks.


Edit:

I did a little googling and found this:

https://bitbucket.org/mchaput/whoosh/issue/100/oso_binary-is-only-available-on-windows

"filedb/filestore.py:64 contains os.O_BINARY which is only available on Windows. On other platforms fails with

AttributeError: 'module' object has no attribute 'O_BINARY'"


so I went to line #222 of gog-backup and changed

fd = os.open(name, os.O_WRONLY | os.O_CREAT | os.O_BINARY, 0666)

to

fd = os.open(name, os.O_WRONLY | os.O_CREAT)


I then tried another fetch and it seems to be working now. I have about 70GB to go. I'll let you know how it turns out. :)

Edit: Edit:

It seems to have worked! Thank you for this excellent program epowers! :D
Post edited February 18, 2012 by Snickersnack
Posted February 19, 2012
Hi. I tried your application but it doesn't want to log me in.

I downloaded a zip from your page and extracted it. Then, in Ubuntu, I installed html5lib from the package manager.

Here is what I get when I run the login command:
> https://www.gog.com/en/login [200]
Traceback (most recent call last):
File "./gog-backup", line 495, in <module>
sys.exit(main(sys.argv[1:]))
File "./gog-backup", line 477, in main
globals()['cmd_' + args[0]](*args[1:])
File "./gog-backup", line 229, in cmd_login
if etree.find(".//div[@id='register_holder']") is not None:
File "/usr/lib/python2.6/xml/etree/ElementPath.py", line 186, in find
return _compile(path).find(element)
File "/usr/lib/python2.6/xml/etree/ElementPath.py", line 176, in _compile
p = Path(path)
File "/usr/lib/python2.6/xml/etree/ElementPath.py", line 93, in __init__
"expected path separator (%s)" % (op or tag)
SyntaxError: expected path separator ([)

Note that I have several special characters in my password and I had to retry a few times before it accepts to even run. I finally surrounded my password with simple quotes.

Any idea ?
Post edited February 19, 2012 by sebarnolds
Posted March 07, 2012
Those of you having problems may want to try the latest version on the branch "debug". Changes include:

* UTF-8 issues on Windows are finally fixed, I think--such as with games like Interstate '76, which has a right-single-quote in its name
* prompts for a password if you don't include it on the command line
* that dumb O_BINARY bug on Linux
* log recording & playback modes, for reproducing problems on my machine--not sure I'll keep this feature, it's big and ugly

Anyway, feel free to give it a try and let me know how things go.
Posted March 24, 2012
I've just set this up and have been trying to manifest but I keep running into this:


> http://www.gog.com/en/download/game/dragonsphere/0 [206]
> http://www.gog.com/en/download/file/dragonsphere/1066 [206]
Traceback (most recent call last):
File "C:\evanpowers-gog-backup-b604b97\gog-backup debug.py", line 649, in <mod
ule>
sys.exit(main(sys.argv[1:]))
File "C:\evanpowers-gog-backup-b604b97\gog-backup debug.py", line 643, in main

return dispatch(arg0, args, opts)
File "C:\evanpowers-gog-backup-b604b97\gog-backup debug.py", line 605, in disp
atch
fn(*args[1:])
File "C:\evanpowers-gog-backup-b604b97\gog-backup debug.py", line 402, in cmd_
manifest
etree = request(f.url.replace('?', '.xml?'), html=False, hide=True)
File "C:\evanpowers-gog-backup-b604b97\gog-backup debug.py", line 207, in requ
est
pg = _request(url, args, None, True, hide)
File "C:\evanpowers-gog-backup-b604b97\gog-backup debug.py", line 187, in _req
uest
pg.page = opener.open(req)
File "C:\Python27\lib\urllib2.py", line 400, in open
response = meth(req, response)
File "C:\Python27\lib\urllib2.py", line 513, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python27\lib\urllib2.py", line 438, in error
return self._call_chain(*args)
File "C:\Python27\lib\urllib2.py", line 372, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 521, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 403: Forbidden
Twice on the master branch and just now on the debug branch (which is what generated the traceback in the quote).

I haven't done anything stupid have I? It got through quite a few games fine before running into this problem either just after or during Dragonsphere.
Posted March 26, 2012
avatar
SirPrimalform: I've just set this up and have been trying to manifest but I keep running into this:
avatar
Duke22: I'm getting a similar error.
Huh. I get the error too, even with v1.0; that seems to indicate (since it worked before) that GOG.com has changed something about their server behavior or how they serve XML to the official downloader. I'll probably need some time to reverse engineer the differences.
Posted March 26, 2012
Hmm. Looks like they now sometimes serve content from Amazon S3 in addition to the CDN they were using before. Not sure why that matters yet, but it'll have to wait until tomorrow.
Posted March 26, 2012
avatar
epowers: Hmm. Looks like they now sometimes serve content from Amazon S3 in addition to the CDN they were using before. Not sure why that matters yet, but it'll have to wait until tomorrow.
Amazon requires valid signature to download (that is why it's getting HTTP Error 403: Forbidden)
The signature is generated based on the URL and secret key
We don't have the key so it's impossible to generate valid signature
Posted March 26, 2012
avatar
epowers: Hmm. Looks like they now sometimes serve content from Amazon S3 in addition to the CDN they were using before. Not sure why that matters yet, but it'll have to wait until tomorrow.
avatar
Sude: Amazon requires valid signature to download (that is why it's getting HTTP Error 403: Forbidden)
The signature is generated based on the URL and secret key
We don't have the key so it's impossible to generate valid signature
Uh oh, does this mean they've scuppered unofficial downloaders? =/
Posted March 26, 2012
avatar
SirPrimalform: Uh oh, does this mean they've scuppered unofficial downloaders? =/
Only the free games seem to be hosted on Amazon so we still get XML data for most of the games.
We could start using the API that official downloader is using if someone has the time to figure out how it works. I know it uses JSON for data and OAuth for authorization but haven't spent more time on it so far.
If someone knows C#/.NET and wants to take a look at how the new official downloader works then send me a PM
Posted March 30, 2012
Really like the concept. Unfortunately it's not working for me:
$ gog-backup.py login EMAIL
Password:
> https://www.gog.com/en/login [200]
$ gog-backup.py manifest
> https://www.gog.com/en/myaccount/list [200]
> https://www.gog.com/en/myaccount/shelf [200]
$ gog-backup.py list
0 games, a total of 0.0M

This is from the debug branch, similar behaviour on master. The log file created with --debug has a 3.txt and a 4.txt, both nearly identical except for some javascript references. They appear to most, but not all, of my games listed. Ubuntu 11.10, Python 2.7.2, html5lib 0.90

EDIT:
Apparently I can't reply to my own post...
there's no results from:
for game in list_et.findall(".//div[@class='tab_1_row']"):
and tab_1_row isn't anywhere on the page (in 3.txt or 4.txt or on my actual account page if I view source)...has anyone gotten this to work since the site changes on Tuesday?
Post edited March 31, 2012 by zorach
Posted April 17, 2012
avatar
epowers: Hmm. Looks like they now sometimes serve content from Amazon S3 in addition to the CDN they were using before. Not sure why that matters yet, but it'll have to wait until tomorrow.
I don't mean to be rude, but are you still working on this? Just so I know whether to move on and try and get a different downloader working or not.
Posted April 18, 2012
Not a fix, but a dirty workaround to just get it to skip the free games:

@@ -399,7 +399,12 @@ def cmd_manifest():
if i < len(g.setup):
f.url = page.url_after
for f in g.setup:
- etree = request(f.url.replace('?', '.xml?'), html=False, hide=True)
+ try:
+ etree = request(f.url.replace('?', '.xml?'), html=False, hide=True)
+ except IOError:
+ err(f.url.replace('?', '.xml?'))
+ msg('WARNING: This is probably a free game ')
+ continue
del f['url']
assert f.name == etree.attrib['name']
assert f.size == int(etree.attrib['total_size'])

This way, at least your paid games can be backed up correctly.

Edit: Argh, can't get the code to format correctly. You need to indent the line after "try" and the new lines after "except" for this to function.
Post edited April 18, 2012 by gthornblad
Posted April 20, 2012
avatar
SirPrimalform: I don't mean to be rude, but are you still working on this? Just so I know whether to move on and try and get a different downloader working or not.
My job has consumed my time the last few months; were it not for that.... That said, my downloader is open source and on GitHub--if you like it, don't want to wait for me, and are willing to write some code, send me a pull request or PM me a diff. I very much encourage contributions.
Posted April 20, 2012
avatar
epowers: My job has consumed my time the last few months; were it not for that.... That said, my downloader is open source and on GitHub--if you like it, don't want to wait for me, and are willing to write some code, send me a pull request or PM me a diff. I very much encourage contributions.
I was just curious... Unfortunately I lack the necessary understanding to fix this myself, so unless someone else submits a patch, I'll have to look into other downloaders with backup functions. Thanks anyway!
Posted January 22, 2013
Hi,

I just tried the script and can't even login (Error 104, on the debug branch). I believe it is totally unmaintaned since the author is busy with real life? ;-)
Pages:
1
2
3
This is my favourite topic
Community
General discussion (archive)gog-backup, an unofficial GOG downloader (Cross-platform, CLI)(31 posts)(31 posts)
(31 posts)