If you are using the Co8 mod, there is a magic map item you can get in your inventory when you enter the temple that allows you to choose any location in the temple already visited and automatically go there.

If you are playing vanilla, you can stash excess loot in the barrels found lying around, so you can establish supply caches/storage and don't need to go back to town as often.




SPOILER ALERT
You can safely rest in Wonilon's hideout once you have freed him on level 1 so you don't need to leave the dungeon as often to replenish spells. Supposedly you can also rest safely in the large secret circular stairwell found on the temple entrance level, but that doesn't do you much good anyways for saving travel time.

Also, there are 3 alert states for the temple: unalerted, alerted and lockdown. The "unalerted" one still has all the large bronze doors found throughout the levels unlocked so it is faster to get around between levels. You only get the unalerted status though if you haven't triggered certain plot events yet, and once the temple is alerted it never goes back to unalerted.

There is also the secret entrance/exit to the third level through the hidden trapdoor in the ruined bandit tower.