I left a teleporter pyramid in the Elven Village hoping I'd get to teleport out of the wastelands and do a few things before the end.

Well that didn't work.

Is it possible to use the pyramids to leave the wastelands and complete tasks in the regular maps of the game? I could have swore someone was saying they did this.

It is possible to get back to the previous areas by hex editing a file in the save folder, though I do not know if you can complete quests after than (the quest log gets cleared when you enter the wastelands).

As you found, if you leave one or both teleporter stones behind they get replaced by ancient pyramids near the start of the wastelands (screenshot).

Thanks, I'll just go and finish the game then.