After two more hours of trying and then watching a couple youtube videos it is starting to seem like I don't have near enough armour. I have the best things I found along the way, Armour of Loc Muinne fully upgraded, but the people in the video's take a lot less damage than me. I am using a Super Duper Robust something or other Red Metorite Silver sword with both slots filled with the highest damaging runes but to be honest, he seems to take an appropriate amount of damage "when" I can actually get a swipe on him.
1 swipe from him is about 25% of my health and even when I last a long time with the guy I can't avoid getting hit at least 4 times. Trebuchet does about the same damage.

To the people mentioning getting behind him when he calls the orders, in my game, in mid order, when other people say you can get behind him, he will just sit there and swipe me down while he continues to order the trebuchet.

Only bright spot seems to be that you can dodge the tornado just rolling around like normal. Don't try hiding behind a rock cause it will just hit you anyway.

How do you adjust difficulty in game? I see people say you can do that but...I don't see where. I'm gonna keep trying in Normal but getting close to giving in. :(

the first time i fought him it was a long fight and i ended up dead with arrows,trebuchet and me getting stuck on terrain and could not move out, second time i killed him really quick

Wow. You people are having a very different experience than me. After the first five deaths, reloads, wading though the ten minutes of cut-scenes and minor fights to get back to Draug, I switched to easy. I have yet to see his back. Even when he charges to the edge of the field, his shield is toward me. I have yet to get close enough to hit him with the sword. Signs don't do anything. Only time I can hit him is in the middle of the tornado. Once, I managed to bomb through his blue bar, but another hour of this and it will be time to throw the character away.

When I fought him (on normal) he was just so slow and clumsy that I just stayed behind him and whaled on him with strong attacks, without even needing to use a Sign.

Think it's really a build issue... He took on multiple strong hits from me and was still going strong, walked right through my Yrdens like others have mentioned. Eventually I dropped difficulty down to easy and still it took many many hits before it went down. (although come to think of it I can't be sure if I was actually using the right sword...)
Press Esc while in game, it's under Options or Gameplay.

Not sure how you get behind him when he issues order. He turns to face me whenever I roll near me and whoops my ass.

Yrden works sometimes. For me it was probably 1 out of 4 or 5 tries. Sometimes even if it "ensnares" according to combat log it seems like he regains free movement instantly (same for many enemies). So in the end I didn't even bother with Yrden.

Rolling behind him when he charges and strong hit him works pretty well. Though I was using Negotiator (crappy sword) so I had to repeat 4 times or so. What makes dodging tricky through is his charge attack cover quite a large area.

And for some reason, some of my attacks miss so I don't know what's the deal with that.

I killed him easily in 2 playthroughs before the patch but it was hard after the patch. He's calling trebs and arrows more often. maybe it's just my luck.

For me it was just like Tornado/Fireballs/Arrows/Attack-while-the-two-previous-one-are-constantly-hitting-the-ground all the time second per second in Hard and even in Normal, so I just got very tired and beat him stupidly in Easy.

No kidding. What's more, every single one of the bosses that I fought did pretty similar things, only constant attacks all the time that I only could hit one time per five seconds as much. Ah, and Quen was useless, hitting me, taking all the Quen spell in one strike and then striking me five times in a row...

Why me!?

When I played against him on Normal, he swung his sword at my face with a fast strike (that went through my upgraded Quen, threw me back and did some damage) the moment I got close to him if he wasn't in the middle of an earlier swing. I beat him on my third try, by waiting for him to try and smack me, rolling behind him and smacking him once to the back, then repeating over and over again. Got to see the tornado attack, the trebuchets, volley, and charges. Because I couldn't get close to him on my first try, I ran around the whole battlefield trying to find some tricks on my second try.

use Quen level III. I dont even need to kick hit ass. Quen shocks him for me.

One more thing: when he calls in fireballs. Stay somewhere near him with yr Quen on. I love see the fireballs hits him as well :))

Are you sure you are in the right menu? Other than when you start a new game, the "Difficulty" setting is only available in the Options > Gameplay menu while you are IN-GAME. It is not available at the main menu when you launch the game; you have to have started a game or loaded a save, then press Esc for the menu (do not choose Exit to Main Menu), go to Options > Gameplay, and it should be above the Combat Log option. If you're at the main menu, the option does not show.

OK, here is a solution for those who cannot kill it.

******SPOILERS********


Right after the Selkirk's ghost leaves your body there is a draugir that you will fight but it doesn't attack till you come closer. So, have ~20 claw traps (or other traps in large quantities) in your backpack (collect them at chapter 1 very easy) and set them all on the spot where you are standing.

Then attack draugir, defeat him. As soon as draug appear cast quen and run to your traps. Position yourself so that the traps are between draug and yourself. Wait for him to charge. He charges into the traps... instant kill :)

If you are trying yrden + attack from the back there is a bug. It seems like draug is always turning to face you but it is just an animation, which doesn't correspond to the actual situation. If you attack the spot where its back side must be (even though it looks like a frontal attack) you will deal double damage bypassing a shield.

To prevent draug from calling catapults and archers stay close to him and attack. It never calls support when you are slashing it with your sword.

Cheapskate way:

throw 10 poisoned harpy claw daggers in quick succession.

daggers are OP but if you're playing on a harder difficulty you might as well abuse the mechanic.

I wasn't disappointed. I wasn't sure if I'd make it or not and it was a little intense. Those archers almost got me. Don't let him corner you!

Playing on normal here, using the yellow meteorite sword that the mysterious merchant sells, with armor of 20 and with 60% concentration on sword + 40% on magic, I got Draug with the first try. Didn't use Yrden as it doesn't seem to hold enemies long enough. Used the sword and Igni as much as I could. Dodged and jumped around Draug when I couldn't.