You don't necessarily have to fight every group of monsters. If there's enough room to move around, you can spend your entire turn running for 3-4 turns, which will generally cause the monsters to stop pursuing you and retreat. You may incur some damage during this time and your stamina will take a large hit, but you will very likely survive. You can also exit the zone during a fight, which will automatically stop combat (though it may resume when you reenter if the monsters haven't wandered off again). Just be careful not to get too close to any new wandering groups.
If you want or need to fight a group, consider wedging yourself into a corner. It's a somewhat counter-intuitive tactic, but it prevents the swarms of enemies from flanking your group and allows you to concentrate incoming hits on your high-AC characters. A regenerating Lord with a good shield as the only character in the front slot works wonders if you have several polearm users and spellcasters to support him. As a bonus, this strategy allows you to maximize the effectiveness of cone-shaped spells. Some of the more difficult areas in the early game like the Arnika Road are absolutely full of nice tight corners built specifically for this purpose.