tinyE: I always thought 4 had far and away the biggest, easiest to get lost in levels. Add to that the fact that you had to go back and forth from one to the other over and over and it made TR 2 seem easy. I don't think I ever would have completed the whole city thing without a walkthrough; just getting the nitro for the bike took you across three or four different levels.
TR2 had a lot of tricky jumping. Tricky as hell, because for example you had to jump on a gliding wall and do a backflip to area that wasn't even visible in the camera.
It was all trial and error.
TR4 had some tricky parts too, like *shoot that sign on a wall and door opens*, plus immortal skeletons, but Tr2 was imho the hardest.
TR3 was much easier, or had better level design, depending how you define stuff