I see it is pointless to post here, and I found it impossible to create an account on the Discord server because I do not own a mobile phone. So any response to my bug report there will be forever lost to me. However, I shall repeat it here for the users who might be interested.

Playing a regular level sandbox game with the Hooves & Wagons DLC installed will result in all social classes arriving at your inn, even from the very beginning when you ought to only be getting Distressed. Un-installing that DLC made it possible to play normally again.

Edit: had wrong name for DLC