Is this normal? Or do I finally have to do some optimizing for the game to be able to take advantage of my hardware? Running on a Skylake i5, everything spec-wise shouldn't be a concern, or even noteworthy for that matter. The game runs flawlessly and I never before saw a performance drop until I got to the Marketplace in Chapter III just now.

Goes to 40s and 30s. Is this sort of like The Witcher's Bethesda game area? Where everyone simply gets bad FPS?

Yeah, it's pretty normal. FPS drops occur in the third chapter. It's an engine problem rather than a hardware one.