Will Vollinger still leave your party at a certain point of the game if you have 20pts in Charisma?

Yes. It has do do with how you handle a special scene in the game. You have to read the book about Molochean Hand you get at an INN in the main quest and then chose to talk about it when you talk to Gideon Laier later in the game.

A warning! There is a bug that happened to me where Vollinger still leave your party even if you do all the right things. You will notice if something is wrong if Vollinger does not say anything during the talk with Gideon.
The bug comes from having to many followers. Just tell some of them to wait before talking to Gideon and it should work.

If you ever want to keep vollinger or virgil for the whole game, just ask them to wait outside of Tsen Ang. Pick them back up when you come out.

Virgil stays regular if you do this, and will not transform into surly/good virgil.