I have just begun, and found a lot of books lying about. Should I take the time to read them? Is it worth it? My time is limited so I try to move through the game quickly, unfortunately...

I do sometimes, but its really not necessary unless you want to be drawn into the world more and learn about the lore, etc.

Almost all of the books only provide a bit of lore about the Forgotten Realms setting, so unless you're particularly interested in that lore you can safely skip nearly all of the books and not miss out on anything. There are a few books that provide a little more pertinent information or are involved in quests, but these books almost always have a different appearance from the lore books so they're easy to spot.

Journals of people and notes are generally good to read, but everything else is just supplementary.

As said above you may safely skip most of the books.

The cool thing about them is that they describe the parts of the world you won't get to know in game (or at least not in detail). The sad thing is that this knowledge won't help you at all in your quest.
It would be awesome if reading books remained optional, but allowed you to, for example, find a shorter way of getting somewhere or unlocked additional dialogue options. Do you folks know any game which features this kind of stuff?

"my time is limited"? I thought that people play for fun :-). Like a reading a book. You are playing when you have a time... OK. That books in NWN are pretty short. But I always think that it is important because of story addon... But they do not have such interesting stories as books in IWD or BG or Morrowind in my opinion...

Every now and then, module designers will use a book to provide puzzle or plot hints. Chapter 2 of SoU for example has a few books lying around that provide some useful clues for the primary quest.