Some good moments yeah, the atmosphere is very well done, I remember the first time I saw the map where you get the sniper rifle and you have to infiltrate a big building ahead, I was astonished by the beauty of the map.
Sad thing they didn't keep the ameliorations of the Unreal Engine 2 present in U2 to implement them in UT2K3/4. (I'm thinking of fire effects, advanced GUI... would have been useful for the modders.)

A friend of mine took all the unreal 3 models of unreal 2 and made them work in ut2004 and I think he converted a few to ut3 and tweaked them a lot. Also most of the unreal 2 assets were brought over to ut2004 and work in there now (not the single player, just the models.) XMP was recreated in ut2004, but the people who did it had some different ideas about balance, and I liked the original xmp a lot better. It's still worth checking out. Almost every Unreal 2 model you encounter is in Ut2004 now.