I'm particularly proud of the four CE boxes I have which are big or pretty enough to place on a shelf and look imposing. Especially the Fallout 3 one - the lunch box is just awesome.
Other than that, well, I'm a proud owner of several classic games, but none of them can really be considered as collectible since I threw away boxes and sometimes manuals as well.
I've only actually been "collecting" games for the last year and a half - that is to say, I didn't really know about Collector's Editions and how nice they can look on a shelf until then.
I've always been a gamer, and occasionally I've saved the boxes for a while, but it wasn't until I saw the World of WarCraft CE box that I realised that I might want my boxes to show on my shelves - to show my passion for gaming to the world.
Before that I took pleasure in occasionally hunting down a rare game I'd heard good things about so I could try it, but that was mostly for the sake of the game. It still is, of course - my CE boxes are there to show I own a copy of the game, nothing else, so I open the boxes and toy with the stuff inside, thus pretty much voiding the value of the box, but it feels good to know that I have something physical to show for my hobby.
Other than CE stuff, I'm pretty proud of owning an almost-new copy of Vagrant Story, my 6 old SNES games (alas, the SNES itself died a few years back... thank the gods for Virtual Console), a large collection of N64 games, as well my library of Game Boy games, and my CD version of Planescape: Torment.
It's difficult to say whether I'm really proud of actually owning a particular game - indeed, I would say that it's more important for me to have experienced the game in addition to owning it. I value Steam and GOG very much because of that: I might not be able to hold the games in my hands, but I can play these classics which were given up for lost to time.
I do kind of regret not getting Banjo-Tooie for the N64 back in the days, and if I recall correctly I had a yellow DK64 cartridge I lost , but I'm not overly bothered: next month I can afford a 360 which will allow access to the former, and the latter was solved by buying a new (standard grey) cartridge five or so years back.
Another thought occurs to me: I feel especially proud about the games I bought myself. My "big box" CEs cost a great deal, and my PSP-2004, while not exactly a great machine, was still worth scrounging for - and then I had to wait another month before I could afford a game and a memory card. I think that's still my single largest purchase, although I do spend more overall now.