After the rat caves , I think we need to find the hermit. Where is he and how do I get to him from the starting town?

He's on a different map. If you follow the road northwest, you'll find an ogre (or troll? can't remember) that will crush you at this level. You will need to go all the way across the forest with the giant mushrooms to get past him. Once you get to the map, he is in the golden grove (can't remember the name specifically, it's close to the entrance). You'll need to avoid the flowers so you don't fall asleep. I think if you are avoiding the flowers there is only one path that will lead you to him.

If I remember correctly, there is also a note in the woods regarding avoiding the mushrooms.