Haven't seen anything anywhere about it yet, do I have to beat the main game first ?

There will be gates that communicate with a kind of hell popping in some places in mellow hills, I think they start appearing after you get the food from the halflings, Im not sure

The hell gates open up after you have completed certain quest in the game. After you finish meadow hill the first one pops up. You have to complete that one and other quest for the next one to appear and so forth. If I recall there are 5 of them. meadow hills, golden hills, elves area, Dwarf stronghold, and desert. Been awhile but I thinks that's all of them.

As already said, the abysses appear one after another while playing the game.
Story-wise it's better to finish the main game first, in my opinion. The more puzzle like approach of the abysses doesn't fit seamlessly in the main game and it's more fun to see stuff from the levels again, when you've been away for a while. If you decide to play the abysses right after finishing the corresponding area, the levels tend to feel a bit lengthy. Go on to the next area and come back after the main game.