Only elite units go into your inventory. Once a unit reaches veteran status, you can select it and click on the "level up" icon on the interface on the right side of the screen and then you can choose an upgrade, for example increased armor or increased range. When a veteran units gains even more experience and reaches elite status, you can upgrade the unit again.

If you did not level up any units to elite, you start off mission 2 with all rookie units as given by the scenario.

If you did make elite units in mission 1, you'll get a screen prior to starting mission 2 where you can select which elite units from your inventory to take along. But you can't take along every elite unit in your inventory, there's a limit set by a certain amount of points, in mission 2 that's 70 points so that allows you to take along e.g. 2 Bandits (at 35 points each).

Important: The total number of units you start mission 2 with doesn't change, regardless of whether you bring Elite units along or not!

In mission 2 you have 12 units by definition. If you don't have any elite units to take along, you have 2 Bandits, an Explorer, a Hunter, 2 Cerberus tanks, 2 Hydra artillery and 4 Transporters. Since I chose to bring 2 elite Bandits from mission 1, I started mission 2 with 4 Bandits and the game didn't give me an Explorer or Hunter, the rest of the outfit (Cerberus/Hydra/Explorer) stayed the same. 2 elite Bandits are more valuable in combat than the rookie Hunter and Explorer you get normally but not being able to pick which units get replaced with your elite units is a bit unfortunate. I'd rather have only one extra Elite Bandit and keep the Explorer you normally get due to its wide range of sight.