Sarisio: Actually I think we are still in Golden Age of PC and we entered it somewhat recently (post-2000). Pre-2000 good PC would cost you an arm and a leg. PCs are still quite expensive now, but much more affordable.
dtgreene: Windows doesn't come with a compiler; you need to find and download it. Furthermore, the environment doesn't lend itself to casual programming.
Sarisio: Depends. Cornerstone of all programming is C++ and Assembler. MS Windows doesn't really have anything to do with programming languages. It, however, provides framework (XNA) to make games, as well as DirectX APIs.
Compilers are purely part of programming languages, not OS. If some programming language doesn't have ready compiler (issue with many high-level languages), it is the problem of that programming language and its makers, not OS.
There are free available C++ packages, which have everything one needs for programming (including compiler, ofc). There are probably still some working variants of Delphi and other less convoluted programming languages which come with true compilers (instead of built-in interpretation of byte-code).
There are also some interesting free engines, like Panda 3D, there are some nice Java libraries for easy cross-platform programming, and so on. So I'd say Windows offers quite a lot in terms of casual programming. Not that I am fan of MS.
Yes, but you really need to seek those things out yourself. In particular, a child playing around with the system is much less likely to stumble across the means to write her own computer programs than on GNU/Linux.
On GNU/Linux, however, all you need to do is something like "apt-get install build-essential" and you have a complete toolchain to build binaries from source code for your OS. Gentoo Linux is even better because, being a source-based distribution, it necessarily has a C compiler installed.
Also, high level languages like Python are perfect for casual programming, especially for beginners. The thing is, while Python code runs more slowly, it takes much less time to write. Also, if performance is an issue, you can use libraries like numpy or programs like cython and pypy to speed it up. (Also, Python is often pre-installed on GNU/Linux systems because distribution-specific tools (like Gentoo's portage) are written in it.) Once a new programmer has learned Python, it is then much easier for her to learn other, lower level languages like C.