They can also be found in some mines - a few of these are put on the map by Wheel Clan NPCs (so you probably haven't explored them yet), but can be found by fast-traveling along the mountains as closely as you can.
There are a few 'secret areas' scattered about that no-one maps (although some townspeople will tell you rumors hinting at their locations) that have [at least some times] Kathorn crystals. These areas have pretty large detection radii (so you can search for them in a pretty wide pattern). Some of them are beyond one of the passes marked later by the storyline, but again, that *can* be found (with some difficulty) by careful searching
[Actually, most places can be discovered this way, and a few of them have goodies that are extra-useful early on.]
Finally, if you only just left the island and have a save there, Thorvald has a pair of glasses that you can steal or mug him for.