NoNewTaleToTell: will a summoner/mage build still be effective mid-game and late game or am I basically just wasting skill points?
Yes, a mage/summoner build is quite viable. In fact, in the Larian forum when DKS first came out someone described their playthrough focusing on using summons, the creature and Charm (they played on nightmare, did not use a breastplate or helmet, and stopped distributing stat points after about level 12) .
Since I mentioned it, the creature can be gotten from about level 10-12, depending on character build and difficulty, if you know where to go. You should have gotten the quest already, from the necromancer in the starting section of Broken Valley.
For summons, the demon is best for melee combat, though isn't available until level 20. The ghost can remain useful the entire game if you keep it leveled.
Charm can turn an opponent to your side. The experience you get killing an opponent that was charmed depends on how much damage you (or your summon/creature) did vs your opponents while it was charmed. If you Charm an opponent's summon, which you don't get experience for anyway, you don't 'loose' any experience. Charm can make fights easier, though, so it doesn't matter if you level up slightly slower as a result of using it.
You can not really waste skill points. Once you get to a certain part in the plot, at a character level in the high teens to early twenties, you will be able to unlearn your skills. There is a cost to do so, which doubles each time, but you should easily be able to afford to do so 2 or 3 times.