I've looked at the files, and it's not a config setting.

I assume it's in the savegames, but want to get confirmation on that. If I simply delete all of the savegames, will that erase the achievements?

In the save game directory there is a file called "profile.dat", deleting it would reset the achievements.

However take note that loading an old save game would restore the achievements to the time of the saved game.