Just came across this set of videos.
Basically it's a thesis regarding how to teach an AI how to play NES video games, and the examples include Contra, Megaman, Mario brothers, etc. And surprisingly the AI does pretty well... and then it barfs at various areas and then does better...
A large portion of it is the AI has access to the 2k of memory that the game uses and based on certain prerequisites of it's goal (
usually score going up) it will play and improve on it's play to do that. Although it works better on side-scrollers, other games like the stock market or karate kid or something it does really bad at because it doesn't understand penalties...
Part 1 -
Part 2 -
Part 3