Luckmann: I'm a total packrat, so I have a tendency to save stuff. But no, I don't think there's much point to it. There's been plenty of games like this that have some NPCs that you might want to equip later and so on, but I think we'd heard about it by now if there were any.
Save your rocks and pilfer all torches within reach and you'll never be short on things to use for pressure plates and so on. Oh, also, save your throwing weapons, even your throwing knives if you use Shurikens, etc.
Throwing weapons can be thrown much farther than rocks. This is necessary for some puzzles.
Items don't disappear when you drop them, so I suggest you to select a niche near some stairs or a room you can reach easily and leave all the crap there every few levels. You don't really need to do it with weapons though.