Yes but in varying degrees. If you're playing on easy or normal mode and you exceed a units leadership you lose control of the unit and it may or may not attack the enemy but usually the unit will just move and/or defend. The higher you exceed the percentage though the more chance your troops will go into a frenzy and then will either attack you or the enemy. If you're playing on hard then your units will attack anyone on the battlefield if you go over your leadership score at all.