I've found all the perfect witcher gear schemes and I want to craft them, but finding enough dimeritium is driving me crazy.
Is there a place or some other source where I can get LOADS of it?

A merchant (named: Merchant) in Kaer Trolde on Skellige at the harbor area always (at least for me) has 5 of them. They are rather expensive though.

I found a lot of dimeritium handcuffs on witch hunters that can be dismantled into dimeritium.

Sadly upon finishing the main quest I can only encounter lvl 47 Witch Hunters, which are not easy to kill. So I went to said merchant and spend a fortune there.

I know, that's what I've been doing so far, but it's too expensive and too long/boring...

Damn that merchant is sure expansive XD
I think I'll just keep on crafting it... it's boring, but it's cheaper XD

That is why you have a lot of "?" on Skelige :)
And you can dismantle thing you know. Some steel and silver swords actually not a steel and not a silver :)

Yay! I did it! I've crafted all perfect witcher gear :)
Thank you all for your suggestions: I've explored lots of points of interest across Velen and I've found several weapons and armors that, when dismantled, provided me with some refined dimeritium; in some places I've even found dimeritium ingots, some more I've crafted and bought, and I've managed at last :)