Posted October 06, 2021
triggered by https://www.gog.com/forum/factorio/cloud_saves_not_saved/post7
Since the handling of the cloud saves seems to still have room for optimization i'd like to propose the following.
Cloud sync Client to Server in steps.
1. Get restraints (e.g. file limits, size) from the server and check against local files. Don't even bother uploading everything if it won't work anyway.
2. Check what needs uploading. Be it the last modified timestamp or checksums.
purely optional thoughts: Maybe at least timestamp and a simple checksum since the client may have wrong local time. In case of checksums it seems to be a doable thing to keep a checksum history per file or directory to recognize if for example my laptops clock is a year ahead of now and tries to upload yesterdays savegames again. But that's probably a rare corner case where the user is at fault.
3. Upload the stuff. Be it all at once, file by file, just delta updates of files that's up to you. An efficient way for both sides would be prefered. Since steam for example seems to have problems with huge numbers of small files there may be a good reason to offer at least two paths. Uploading file by file for low numbers and uploading solid blocks of the whole stuff for large numbers of small files.
The important thing is to make sure that neither the server gets too much load nor the process gets too slow while making sure that the stuff does not get blocked by a slightly disturbed connection.
4. The replacement should then happen safely without bothering the user about internal processes like the old + new files getting momentarily over the limit. This should be transparent to the client and also the user usually just doesn't care how you do that. Of course it is correct to replace the files safely and therefore only delete the old files if the new ones are verified. But then please either allocate enough temporary space (plainly maximum usable space * 2 in case everything is at max and gets replaced) or use more space efficient methods like doing the files one after another instead of all at once. It only becomes the same if there is a single file to sync.
It's only temporary so it shouldn't be too hard.
Additional wishlist:
Give us the ability to manage our cloud stuff if space is that precious. Same with bandwith.
I certainly have old stuff that i will never need again and don't care for. Thing is that only i know for sure what can be cleaned safely and what i will be very happy to get out of the cloud if i get stranded with my old laptop in 5 years. Thinking of dosbox games and the like.
To make it short
1. I would like to be able to delete saves that i no longer need. If you can't tell for sure what makes a save just give me file wise access with a warning that i may break my own stuff. Make the warning big and scary if people tend to blame you afterwards. Of course this is only important if file space is a problem. Otherwise just give us more ;)
2. I would like to give hints to the client in an easier way than just disabling sync at all. It takes just 3 clicks to do that but an option to toggle the cloud for a game right in the menu line - where the overview, achievements, extras and play button reside - would be nice. The menu nesting is deep and the UI just not that responsive.
Just because i don't need the sync every time i restart the game for testing or debugging purposes. Maybe an option to only sync manually so that if i want to jump over to a friend i just hit the 'sync' button before while saving the bandwith and server load for the rest of the time. As in i don't need to resync hundreds of MiB several times a day if i know that i will play only at the same computer for the next time.
3. For Cloud management i really would like to see an overview where i get a list of all games that can use the cloud, if it's activated and how much space is in use with shortcuts to toggle that, manage, delete, resync. Sortable by name, last time played/synced, used space and filter by tags - for example 'want to sync' to easily exclude those from deletion.
4. Bonus would be to get a dynamic cloud space where i can use as much as i need to the current game. Currently there seems to be just a fixed amount of space per game. So some will have lots of free space while others are unable to sync due to the saves being too large. Thats nice for formality and static planning but highly inflexible and just not what at least i would need. A few GiB of space that i can use as my current games need it would be well worth the time to skim through the list in point 3 every now and then to clear the clutter.
Since the handling of the cloud saves seems to still have room for optimization i'd like to propose the following.
Cloud sync Client to Server in steps.
1. Get restraints (e.g. file limits, size) from the server and check against local files. Don't even bother uploading everything if it won't work anyway.
2. Check what needs uploading. Be it the last modified timestamp or checksums.
purely optional thoughts: Maybe at least timestamp and a simple checksum since the client may have wrong local time. In case of checksums it seems to be a doable thing to keep a checksum history per file or directory to recognize if for example my laptops clock is a year ahead of now and tries to upload yesterdays savegames again. But that's probably a rare corner case where the user is at fault.
3. Upload the stuff. Be it all at once, file by file, just delta updates of files that's up to you. An efficient way for both sides would be prefered. Since steam for example seems to have problems with huge numbers of small files there may be a good reason to offer at least two paths. Uploading file by file for low numbers and uploading solid blocks of the whole stuff for large numbers of small files.
The important thing is to make sure that neither the server gets too much load nor the process gets too slow while making sure that the stuff does not get blocked by a slightly disturbed connection.
4. The replacement should then happen safely without bothering the user about internal processes like the old + new files getting momentarily over the limit. This should be transparent to the client and also the user usually just doesn't care how you do that. Of course it is correct to replace the files safely and therefore only delete the old files if the new ones are verified. But then please either allocate enough temporary space (plainly maximum usable space * 2 in case everything is at max and gets replaced) or use more space efficient methods like doing the files one after another instead of all at once. It only becomes the same if there is a single file to sync.
It's only temporary so it shouldn't be too hard.
Additional wishlist:
Give us the ability to manage our cloud stuff if space is that precious. Same with bandwith.
I certainly have old stuff that i will never need again and don't care for. Thing is that only i know for sure what can be cleaned safely and what i will be very happy to get out of the cloud if i get stranded with my old laptop in 5 years. Thinking of dosbox games and the like.
To make it short
1. I would like to be able to delete saves that i no longer need. If you can't tell for sure what makes a save just give me file wise access with a warning that i may break my own stuff. Make the warning big and scary if people tend to blame you afterwards. Of course this is only important if file space is a problem. Otherwise just give us more ;)
2. I would like to give hints to the client in an easier way than just disabling sync at all. It takes just 3 clicks to do that but an option to toggle the cloud for a game right in the menu line - where the overview, achievements, extras and play button reside - would be nice. The menu nesting is deep and the UI just not that responsive.
Just because i don't need the sync every time i restart the game for testing or debugging purposes. Maybe an option to only sync manually so that if i want to jump over to a friend i just hit the 'sync' button before while saving the bandwith and server load for the rest of the time. As in i don't need to resync hundreds of MiB several times a day if i know that i will play only at the same computer for the next time.
3. For Cloud management i really would like to see an overview where i get a list of all games that can use the cloud, if it's activated and how much space is in use with shortcuts to toggle that, manage, delete, resync. Sortable by name, last time played/synced, used space and filter by tags - for example 'want to sync' to easily exclude those from deletion.
4. Bonus would be to get a dynamic cloud space where i can use as much as i need to the current game. Currently there seems to be just a fixed amount of space per game. So some will have lots of free space while others are unable to sync due to the saves being too large. Thats nice for formality and static planning but highly inflexible and just not what at least i would need. A few GiB of space that i can use as my current games need it would be well worth the time to skim through the list in point 3 every now and then to clear the clutter.