Yeah, but by using a scout and choosing right abilities you can get this down by a lot (scout has options which decrease cost by 50% from the start! No investment needed)
The ability you mean is Diplomacy which you can get on the Scout/Ranger guy.
This is a good option when the defenders are just a bit to tough to take on so early as ideally you could buy the Ranger Hero turn 1 and get the diplomacy on him and take over a neighboring "Free Province" that has say a 10-20 income because that province will probably be beyond your conquering ability for a while.
TIP Buying your starting hero can be a bit tricky as the one you want may not have the ability you want (I hate this part not being viewable before you buy :( ), so the best way to get the "right" hero is first do your first day build, I usually build tavern, then hit end turn. The game at this point autosaves for you so the first thing you do the next turn is go try to buy the hero you want, if it does not have the starting skill you want then click the bottom right menu button and choose "into the past" to reload. Now Day 2 will always give you the same hero it did before so this time you build again and try the same thing on day 3. You can keep this up until you get the hero you want with the ability you want.
That makes sense, but it's pretty impractical as the price point to have a province switch side is pretty high, at least in my experimentation.
The price is usually 360 Gold I believe for "Free Provinces". With the Ranger's Diplomacy ability it opens the option to pay the company leader half that right to him so only costs 180 then or so.
If the province is one of those that generates close to or over 10 gold per turn this will be quickly returned and then will just be bonus cash.