So, this isn't the first time with her. The same issue was with "A Deadly plot" quest, but back then I figured out, that I need to go in back room where she let you in secret room. Once I've done it, I never came back there untill I started complete gwent quests. After I beat Vivaldi, he said I need to play with her, and once I get to her, there was the same issue as before, I can't talk to her, she just standing there and waving her fan. Will it be fixed in the upcoming 1.07 patch? Cuz it's look like it starded from 1.05 patch

There is no known solution for this issue. It is also uknown if it will be fixed in 1.07 (we don't have a detailed changelog yet). I suggest putting this quest on hold right now. And one more thing, when you reach the main story line quest titled "Final Preparations" (or something like that), put it on hold too or you will probably not be able to finish "Big City Players" at all (that depends on your decisions) for reasons I'm not going to reveal here... spoilers and such.

Yeah, I'm already at that. I'm curious by the way, after that huge quest at Kaer Morhen, some side quest can't be done after that? I'm didn't complete everything on Skellige and probably something in Novigrad (probably with that King of the poor (orr what was his name?))

Yes. I don't know what missions are affected, but you can safely assume that all missions involving characters you suspect to find at Kaer Morhen should be completed prior to taking Uma. For example, a side quest with Lambert is one of those missions. Leaving contracts, witcher gear hunt and POI for later should be fine. Some of the random encounter quests can also no longer show up after relocation to Kaer Morhen.

You will come back to Velen and have a chance to roam freely later on. First journey to Kaer Morhen is not the end of the main story line yet.

From official 1.07 changlog
* Fixes a rare issue where the player could not play gwent with Madame Serenity at the Passiflora.