It's not possible to view monsters' stats without using an editor, no. Although you can use CTRL-Q to force them to join the party so you can look at their records screen, that gives very limited information and can even crash your game. Many of their unique abilities are from their scripts or from undroppable items with no ingame description.
However, there are lots of game guides and walkthroughs online that give extensive info about how to beat various monsters, so you might want to consult those. In general, things like dragons and mind flayers and beholders either have high magic resistance, or very good saves, or both. So you may need to use Lower Reistance/Pierce Shield or Greater Malison to give you a better chance of affecting them. Other things like demons are often resistant or immune to things like fire or electricity, but are otherwise affected by spells normally. There are only a couple of spells (like Bolt of Glory) that have different effects against different types of monsters. Bosses are also usually immune to instant-kill spells, so don't bother.
The biggest thing is to read the spell descriptions thoroughly, and to experiment.
