Does the GOG edition of Riven install or require the program QuickTime?
(I seem to remember the Myst series using it, but I may be wrong...)

I don't recall needing Quicktime to run Riven, but I also simply have VLC.

Yes and you can get the newest version of Quicktime here:
http://www.apple.com/quicktime/download/

If you're still experiencing video problems, try this:

In QuickTimes control panel change the video mode to "Safe mode (GDI only)"
You find QuickTime's control panel in your system's Control Panel => QuickTime (on 64-bit Windows you must click the "View x86 Control Panel Icons" button on the left side of the Control Panel window before the QuickTime icon appears)

If this doesn't fix your problem, you can try using an older version of QuickTime (uninstall the current one before)
GOG provides a download for QuickTime 6.52.
To set "Safe Mode" in QuickTime 6 look in the Video Settings section in the drop-down menu of QT.

It's theoretically possible that the game plays its videos without Quicktime installed, as long as your system has got an alternative application registered for those files. VLC can do it, but I think it also uses copies of the original Quicktime libraries for playback (as long as they haven't been rebuilt in an open source project, yet). Though, I haven't tried it myself.

Thank you for the info! ^_^