I'm trying to figure out what exactly the F1, F3 and F4 keys do when pressed in-game, they don't seem to do what it says in the manual.

F1 seems to do nothing, I know F2 shows the framerate in the corner of the screen while playing, F3 seems to do something to the graphics, but not sure what, and F4 seems to pause the game when held down but for what purpose?

F3 significantly reduces the occurrence of the black polygon "seams" between the angles of the scenery as you move by, likely by enabling a higher precision rendering mode. In the early days of 3D graphics, especially with PSone and Saturn titles, the low precision calculations used to render the graphics led to all sorts of ugly artifacts and glitches. I find that toggling this mode "ON" does a good job of eliminating the distraction these glitches can cause.

With that being said, at least on my setup, having the feature "ON" can sometimes manifest other glitches around the sprite objects (the muzzle flash from the pistols is the most noteworthy), but I find these to be minor compared to the alternative.

No idea about F4... it might just be a feature to help grab screenshots.