As I understand things, each city you have of a certain size grants you a certain amount of fame. I don't know the exact the numbers, so for my questions, assume that a city of size 15 grants you 3 fame just for possessing it. For the purposes of my examples, I'm going to ignore population loss because of battle, and I'm going to ignore the fame lost/gained from losing/winning large battles and battles with strong units.

Q1) You lose some fame when a city is taken from you. Is this fame loss based solely on the size of the city? For example, if I were to lose my size 15 city (losing 3 fame because I no longer control the city), then take the city back (gaining 3 fame because I now have another size 15 city), would I then have a net change of zero fame? Or is there an extra loss associated with losing such a city (ex. I lose 4 fame for losing that size 15 city)?

Q2) In the same vein, if you take a city and raze it, you typically lose some fame. Is the fame lost just what you gained from taking a city of that size, meaning that if I conquer a size 15 city and raze it, I only lose the 3 fame I gained from taking control of a city that size?

I'll have to see if I can find the source that talked about having cities of a certain size give fame. Maybe that was only for conquering. I knew about the fame gain/loss from winning/losing a battle against/with 4+ units (and summoned units do increase the total count, so if you call a Fire Elemental to help your 3 spearmen, you now have a point of fame on the line). I also knew about the fame gain/loss from winning/losing a battle against/with powerful units; I don't know the cutoff, but when the loser has at least one unit above a certain power level, then the winner gets a point of fame for winning. The 'size' and 'power' fame points are cumulative (2 fame for winning a large battle that includes a powerful unit).

I haven't looked into the answer for my first question (losing and then retaking a city).

I tested the answer to my second question last night: your fame gain or loss is decided when you decide whether to keep or raze a captured city. If you keep it, your fame goes up the appropriate amount. If you raze it, your fame goes down the appropriate amount. The difference between the two options is 4x the fame value of the city.
For example, if your fame is currently 50, and you take a city worth 2 fame, then you end up with 52 fame if you keep it and 48 fame if you raze it. This makes me a little sad, though measures can be taken to reduce the city size before you capture the city for real.

Edit: That should be "2x the fame value of the city." Thought about just leaving it along, but the thought bothered me too much. >.>