I don't recall if I've seen the movie (I've seen lots of movies with Joe Pesci and Robert De Niro in them so it is impossible to remember them all), but googling found this:

https://www.youtube.com/watch?v=d6up8RHNU-o Does that explanation sound plausible? There was also some other explanation in those Youtube comments about waiting for muffins and shit, I have no idea what that was all about, just someone joking?

I haven't watched the film in many years, but going by the clip that timppu posted, I can infer the answers to your questions. The guy timppu quoted has the right idea. But to give some more depth in answer to your questions:

1. Carmine is either a fictional made up person, or a well-known gambler with underworld connections who exists in the film's story (this is still possible even though Carmine is never shown). There is no way to determine for sure which one of those possibilities is true, because the film does not provide enough information to make that determination. Either way, it doesn't matter. The film doesn't say for sure if Carmine really exists or not, because that point is not important to the film's story.

2. Nicky knew they would rob the casino because Nicky is part of the criminal underworld, and so are those guys. All three men were obviously known to each other before Nicky approached them. In other words, the casino visitors' reputations proceeded them.

3. Yes, they definitely recognized Nicky as a mafioso, and that made them become scared (because Nicky's reputation for brutality also proceeded him). Therefore, after Nicky told them that he is protecting the casino, that caused them to abandon their plan to rob it, and so they decided to leave. Because they knew that if they didn't leave, then Nicky was going to punish them brutally.

4. They definitely didn't believe Nicky about Carmine leaving, because they didn't expect that Carmine was ever there in the first place. They had never intended to meet Carmine there.

They just said they were waiting for Carmine as a polite way superficially to justify their presence at the casino (even though they knew that Nicky wouldn't believe them).

Likewise, Nicky said that Carmine left because Nicky wanted to give them a polite way to leave the casino (even though Nicky knew that they knew he was making up his story that he just saw Carmine leave).

All of the talk about Carmine was simply a masquerade designed to allow the conversation to take place civilly, as opposed to, say, Nicky threatening them, and as opposed to them admitting that they were there to rob the casino. All three men understood what was happening during that masquerade.

The point of the masquerade talk about Carmine is: gangsters prefer to deal with each other respectfully when possible, because when disrespect happens, that's when people get hurt. Since everyone maintained respect for each other - via making up baloney about "Carmine" - that is the only thing that allowed the situation to be resolved peacefully.

In contrast, if instead of saying "We are waiting for Carmine," the guests had said, "You caught us, we were here to rob your casino, but we will leave now," then Nicky would have been obligated by his gangster code to respond violently, because the overt admission that they were going to rob his casino would have been disrespectful.

Is this your pen?

Looks like an interesting movie. Going to see if I can rent it

Edit: Oh my. that opening scene was funny as a dummy. (too bad no spoiler tags so i won't spoil it)

Thanks for clearing that up guys!

I don't believe that the Carmine story is bullshit, and that they (Nicky and the other guys) bullshit each other to the point, that the guys simply realize they have to leave.....and if they don't something happens immidiately.

At first they are firm that they are waiting on Carmine, and so if Nicky doesn't give them the right answer, they don't seem to be afraid of Nicky, even though they know that Nicky is with the Casino owners. So Nicky tells them "Carmine?, he had a suitcase and all and then he left ", and this statement at first is a shock to the other guys, so that they keep asking "is he really gone", " he is not here".....But then the guy with the beard (Jerry) says something: "...Carmine is out....".

This is the key statement in this entire conversation. Because it is after this statement that the situation changes and that the guys decide to leave. So the question is what is the meaning of "....Carmine is out....". I believe it means that Carmine has realized something, and that he is out of the action (robbery), namely he has opted out, and it also implies that because Carmine knows something, that he has decided to do so.....Namely the relationship between the Casion and the Bosses and the whole Mafia-structure behind the Casino. Because right after this scence Ace says "...but the other yocals who had never heard of Nicky and the Bosses....those are the ones giving you the most trouble"

So the fear that drives the guys out of the Casino and makes them change their mind regarding robbing the joint, isn't merely due to the presence and the affiliation of Nicky, it is because of something bigger, that which Carmine has realized, and has made him want to quit and leave.

Ans since the baloon-head mobsters trust Carmine more than their own senses, they follow him on his action and decide to quit, and go some place else or across the street or something.....

I believe this analysis holds better that a fictional character of Carmine, and a bullshit conversation about him.