As mentioned, basically, the exe files are the installer and the .bin files are archives containing the game file.
As to the "why".
PE files, Portable Executable, or .exe files have limitations. One of those limits is a maximum size, that is less than the size of the archive data for the game. So if that data were put in the .exe installer itself the file would no longer function at all, in that it would no longer be a valid .exe/program.
Another reason for the why, is that when you run an executable it gets loaded into memory, can you imaging opening an exe that is many gigabytes in size. Now sure not all of it would need to be loaded into memory but even so it would cause a very large and unnecessary delay before it opens. Then there's things like virus scanners, which would only add to that delay because they would scan the entire multi-gigabyte file before it opens.
Also imagine a user waiting for such a massive thing to load without any feedback, so the user tries to open it again causing it to be opened twice at the same time.
All such problems, and more, are avoided by having the .exe file small and separate from the large archived install data. When you open the installer it only has to deal with a small file that opens quickly and will only access the larger data files when appropriate.
There are other reasons. Such as ones related to GOG packaging the games. It wouldn't make much sense to build each game installer from scratch because this would mean doing a lot of work many times over. They'll instead have a template installer which has it's values changed for each game, such as "Game Name" for example and create an archive from which it installs the files.
That way it would be easier and quicker to create new installers, test them, and then make any corrections. Can you imagine having to rebuilt a 4GB or even a 20GB+ installer to fix a simple spelling error, keeping the installer separate means they would only have to rebuild the 2MB file, quick and easy.
GOG's installers (using Inno if I remember right) have had some ups and downs over the years, unfortunately they're far from perfect and haven't even recovered from some of those downs. Still worlds better than the typical InstallShield etc nonsense games and programs are often installed via. Though don't get me started on the GOG version nightmare as there is nothing nice to say there and will likely be appropriately accompanied by profanity.
