The following observation may or may not be interesting for others, but I've seen a couple of posts dealing with resolution problems of the two AOW2 games, and the only solution I have seen in these posts is to tamper with the registry, which is not really desirable.
At least in my case, it was sufficient to run the setup program in XP compatibility mode, and perhaps this works for others, too.

Here is what happened to me (on a Win10 PC):

At first I installed AOWSM, set the resolution to my monitor resolution 1680x1050, and played it for a while. Everything ok. But when I restarted the game later, it came up in a window. I could ALT+ENTER for full screen, but the resolution was wrong, too. When I opened AOWSMsetup, I couldn't change the resolution anymore, there was only a "Display Mode" entry, with "Window" as the only option to select.
Then I installed AOW2, and found exactly the same behaviour (was to be expected, I guess).
Now I un- and re-installed the two games, but nothing changed: no option to change the resolution.
After some reading and experimenting I tried to run the setup programs aow2setup and aowsmsetup in XP compatibility mode, and this worked.

I also had some horrible Windows 10 display issues for Shadow Magic until I randomly decided to run the setup programs in compatibility mode. Good to ave this here so that people with the problem who google it will find the answer

Thankyou Greywolf! I just had this problem and running it in Compat for XP sp 3 worked perfectly for me too! Saved me a lot of time! p.s.Can I have that funky Long Sword +2 you are carrying? ;-)

Although the resolution I picked isn't bad, I'm having this same problem: I managed to pick a fuller resolution, and then the option seemed to disappear. XP Compatibility doesn't seem to have changed anything.

However, it seems able to change in game just fine if I click "Ok" rather than "apply."