One issue that a topic in the general forum reminded me of:
* When a party member is under fear status, the character will run away uncontrollably. This means that the character can't participate in the battle, and more obnoxiously, you need to waste time finding the character and ordering them to move back to the party. The possibility of a trap or encounter being triggered makes this issue even worse. (I actually think traps are not a good fit for a game like this without manual movement and are actually bad game design in this game; when you factor in fear, the issue is even worse.)
* When an enemy is under the effect of fear, the enemy will run away. This is annoying, as you now need to chase that enemy down, which can be quite frustrating and time consuming.
* Confusion has similar issues.
* (Charm actually doesn't have these sort of issues; a charmed character will be attacking their own companions, and will therefore remain relatively close to them, so the particular issue of having to find them afterwords doesn't apply there.)

So, are there any mods that tone down these effects, or at least remove/mitigate the factors that make these effects annoying regardless of which side gets affected?

(This issue also applies to other Infinity Engine games, and many other WRPGs (particularly those with tactical combat, whether turn based or real time with pause) have these sorts of issues.)

There is a mod that negates mind afflicting abilities. It's called the greenstone amulet.

https://baldursgate.fandom.com/wiki/Greenstone_Amulet

You can find this mod at Ulgoth's Beard in (BG1) and the beholder cave (BG 2) :)

You don't need mods to get around these issues. The resist fear spell is a life saver. Having an enemy run off into the wilderness is a risk you take with using fear/confusion. You should restrict these spells to tight spaces. Alternatively, you can keep them in place with web, grease, hold person, or stinking cloud.

It's not that there aren't ways to get around them; it's just that I see them as bad game design as implemented.

Also, the problem with those spells is that they don't go away when the battle is over (so it's still necessary to wait around, which wastes real time), not to mention that the characters may move out of the area before the spell finishes casting. (Hold Person will go away if cast on an enemy that's killed, but not if it's cast on a party member.)

The problem with the Greenstone Amulet is that, aside from the issue of finding them and the fact that they use up the amulet slot. and that they need to be actively used according to the wiki, is that they protect against too much, including things that don't have the same issues (like charm).

In summary, I'm interested in a mod that fixes it because I see it as a game design issue, not because I want to be able to avoid these effects myself.

so running away from a dragon is bad design? :)

Have you tried the usual places? Gibberlings, Sorcerers.net, etc.

If it's caused by something other than player input, then yes it is bad design. (Having the player willingly run from a dragon, on the other hand, is not, even if it's something like Final Fantasy 3's first Bahamut encounter, where the only way to progress is to run (or if it is bad design in the FF3 case, it's for an entirely different reason).)

Fear effects really shouldn't be implemented by having the targets run away, as that gets rather frustrating for the player, regardless of whether it's the player or the enemy that's affected.

If fear simply prevented actions other than movement (and perhaps certain spells like Dimension Door, though that would probably be hard to do in the engine), it wouldn't be nearly as much of an issue, at least when it's the party that's targeted. (Still would be an issue when an enemy is targeted, however.)

By the way, does anyone know how Baldur's Gate 3 handles fear effects?

Thing is, this does not match the behavior in the Baldur's Gate games, where the characters just run uncontrollably.

In the 5e rule you mention, things are more orderly, happening in turns. Furthermore, if it hits a player, the player may have some choice about where the move occurs, particularly if it's a tie in terms of what the safest available route is. There's also the fact that, in a table top game, the DM can just handwave away the annoying parts of the mechanic; if the encounter is over, for example, the DM can just say "the spell effects end and the party regroups" (or something to that effect), unlike in BG1/2 where the player has to wait for the effect to end, then find the run-away characters, then have them rejoin the party, hoping that the pathfinding cooperates.

In any case, this is one mechanic that may work in a TTRPG, but which has serious issues in a CRPG.

There is, however, another thing: In a TTRPG, every player needs to be having fun, or the player(s) who isn't having fun will get frustrated, and may end up leaving the game or worse if it continues.

(I still don't think I would include fear effects like this if designing a TTRPG, but then again I likely would not include tactical combat rules, instead handling combat in a manner similar to games like early Might and Magic.)

Anyway, you described what the 5e rule is (for one spell), but the question is how it is actually implemented in Baldur's Gate 3.

(OK, maybe this post might not all agree with the same conclusion, as I was writing it as the responses came to mind, rather than trying to make a coherent argument here.)

bg3 teats fear as a will save each turn but so far I have not actually come across the effect in game play

Although a minor bit, finding the characters after the effect ends is easy. Select the character (via keyboard or sidebar), then select it again. The view will center on that character. Alternately, select the entire party (via keyboard) and order them to move to a point you can see. This doesn't help with the other problems you mentioned (particularly the panicked characters triggering more traps/encounters), but at least cleanup is less bad.

there is an item in Sod called "Kiel's Helmet" which Minsc has as default armour i.e, he only has this gear if you didn't collect him in BG1... anyway this helmat blocks fear, panic etc and I offen give it to the goblin

put this helmat on the character you don't want to have running away or edit the item you do want to wear to include the spell effect from this helmat are both simple fixes

Kiel's Helmet first appears in Durlag's Tower, an area available in Baldur's Gate: Enhanced Edition and in Baldur's Gate: Tales of the Sword Coast. This is a somewhat late-game area, but parties can acquire it before Siege of Dragonspear.

Just get a bard and make him Sing Bard Song

Easier said than done, especially in BG2.

(The game does not have a good distribution of classes when it comes to recruitable characters; there's too many thief/mage characters, and not enough characters who can actually level up the base thief class; also, there's only one bard, and he's not exactly the easiest character to recruit and keep.)