In the game directory - Sierra - Arcanum - Modules - Arcanum - Save

Edit : Here . . .=)

Interestingly, there is no Save directory at that location, but the application still lets me save and load games without problems. I suspect this is related to not being an administrator user, of course, browsing the Program Files (x86) directory as an administrator does not yield results.

Look under C:\Users\[your user name]\Saved Games or C:\Users\[your user name]\Documents\My Games
It should save the game save files in the GOG\[game name] directory, this may be a good question to ask GOG support as well.
Edit: Don't know why a GOG game would save here but . . . Start, Run, type in %AppData% and hit enter, lots of apps save data here under the app name.

Found it!
It actually stores the files in
C:\Users\<user name>\AppData\Local\VirtualStore\Program Files (x86)\GOG.com\Arcanum\modules\Arcanum\Save

UAC is the cause behind this one. Older games installed to Program Files (or other protected areas) have their saves redirected to the Virtual Store. This redirection is seamless and is done to keep compatibility with older programs/games without UAC elevation required at each run (and without compromising the UAC security).
Look in C:\Users\[User Name]\AppData\Local\VirtualStore\Program Files\GoG.com\Arcanum
Edit - a second too late :p

GOGs are usually patched up to the latest official patch version which is what I'd describe as necessary. All the other stuff is just a bonus. Arcanum saves that you created after the unofficial patch was applied will not work with the version you get at GOG. I tested that. Whether a save that was created before the UOP was applied will work, I do not know.

Thanks

Is this the same for WIndows 7 as it is for Vista? I've tried everything suggested here and then some, and I still can't seem to find my saved games...

Search your entire hard drive for files with the *.gsi (or *.tfai) extension

OK, I had the same problem with Win7. The problem is that Win7 hides the files. You have to klick on "Organize" in the Folder-window's upper left corner, chose "Folder and search options", the "View" tab and mark "Show hidden files".
Then you can find the "AppData" folder and the savefiles. :)

Not sure if this helps but I'm on Windows 10 using the GoG Galaxy launcher. By right clicking on Arcanum from the games list, it gave me the option to "Manage Installation > show folder" which by default for me is This PC > harddrive (C:) > Program Files (x86) > GOG Galaxy > Games > Arcanum

from there as with the original path from the store bought CD

modules > Arcanum > Save

placing my old saves including one from an original CD install in the Arcanum > Save folder listed above makes them all appear in the save list and in testing them, they have all been loaded and work, at least with version "1.0.7.4 HOTFIX" which as of the time of this post is the most current

I hope this helps