Wiffer: Same thing happens to me. No answer found yet.
I contacted the devs. I have set the exe to "run as an administrator" (I am not sure if it's essential) and deleted the two folders in Appdata/local named "Bilkins Folly" and "BilkinsFolly". The game recreated the folders once it saw they weren't there, and I was able to save and load normally afterward.
I lost the progress I made before this (although it's more accurate to say that the game didn't save before), but I think that's unavoidable.
Edit: I wish I could mark my own replies as the solution. XD