I started a new game with the build you provided with one small adjustment. I had to drop intelligence down by another point get the rest of the numbers. I chose the background living legends and Mercenary.

I'm playing on normal this time and I didn't run into any trouble until I hit the stronghold. I got schooled by several phantoms and their entourage.

My plan was to head straight to Dryford Village and pick up Grieving Mother but apparently the bridge is out to Defiance Bay.

I'll have to do some leveling before making another run at the stronghold. It would be easier if Aloth wasn't made of glass. If he gets hit once he is down for the count.