1) Different golems are beaten by different things. So which golems? (You could more efficiently find out how to kill your particular golems by looking it up in a walkthrough.)

2) Magic weapons are all over the place.

You need +2 or better weapons (or ammo in case of ranged attacks) to hurt the stone golems.
In BG2 that shouldn't be too difficult. Like there's a +2 sword in Irenicus' Dungeon at the genie. Or the +3 sword Lilarcor in the sewers under the Copper Coronet. A +2 short sword from a special random encounter. The Falial of Ages can be made right in its keep to +3.
And most stuff in shops start at +2 too.

Obviously, the game is swimming in the necessary weaponry.

What are some shops with magic weapons

Ok i did it thanks