There is no need to start again, unless you started the game at a difficulty other than hard.

The amount of points you have put in your attributes is not a significant factor in whether you will be able to win any battle. Rather, your choice of weapons/spells and availability of potions and companions will be the major deciding factor in winning battles. As such, if you play the game at hard difficulty, all rare items (items in blue coloured font) and unique items (items in yellow coloured font) will occasionally have this rare attribute to give you bonuses in all attributes (Of divine power) and by saving and loading game many times, you can generate sets of equipment that have this rare attribute that gives you considerably more attribute points than leveling your character ever would.

Alternatively, there is always the console command to add infinite amount of attribute points to max out all your attributes if you know how to do it.

At your current level you described, adding points into strength and dexterity is not a problem, as you will need the strength and dexterity initially to reach the Disciple rank in Armour Use, Melee Combat and Ranged Combat to use chain mails, flails and longbows which are the highest tier of equipment you can use as a priest. Your current attribute point distribution is fine. Once you reach the Disciple rank in the listed skills, put points into Speed, Constitution and Intelligence until maxed out before going to max either Strength or Dexterity. Through leveling, you should be able to max up to four attributes by the time the game ends.