After getting the last shade, Apollo made a remark having another duel now that we have level 3 Combat Arts. Then he said that we would duel later, when I was used to using level 3 Combat Arts. I've beaten the game now, and I haven't had a duel with him. There was a fourth duel, in a sense, with Shizuka, but there's a great opportunity here: There's already a reason for it in the game, and a secret end boss that's tough as nails would be great. It would give a pinnacle to reach for. A reason to get the most powerful equipment other than for reaching 100% completion. My favorite parts of the game have been the Apollo Duels. The third one taught me the value of Guard Arts, and the battle with Shizuka taught me the value of Dash Arts. I hope the reason it's not in the game yet is that the developers are making it even harder than Shizuka.

IIRC, Apollo said something about heading for the arena so I'm fairly sure they intend for the duel to happen there once it is finished. Currently if you level up with boosted enemies at the end without Apollo in your party he stays at a lower level, so I suspect he won't be all that tough. I also found the duels to be helpful.