More complex environment

What happens when Ernest is put in a more complex environment? He still starts learning low-level regularities, then he tries finding higher-level regularities that would continue increase his satisfaction.

In this video, Ernest 7.1 drives a little tank. Ernest 7.1 has anticipations associated with his actions. When the result of his action does not meet his anticipation, he says "Oh!". We can see that he first learns the strategy consisting of sensing before moving forward (although the yellow flashing is sometimes too fast to bee seen in this video).

Once this strategy is learned, Ernest stops bumping into walls and becomes more confident in exploring his environment. We can see him going up the left border by being surprised of sensing empty squares and running into them, because he enjoys it.

In this example, Ernest's second-level strategy consists of turning to the right after facing a wall. At the end of the video, this habit gets him trapped in a little loop on two squares. After a few cycles in this loop, he stops being surprised of the result of his acts and stops saying "Ho!".

Two insights from this experiment:

- Ernest 7.1 now has a forgetting mechanism. At the end of each cycle, the weight of all the schemas that have not been enacted during this cycle is decremented of 0.2. When a schema reaches a weight of 0 it is deleted from memory. Forgetting useless schemas helps prevent the continual accumulation of schemas and rather keeps their number stable. This forgetting mechanism actually accelerates the learning because it helps get rid of useless schemas.

- Ernest is tailored for sequential regularity learning but not for spatial regularity learning. Therefore, he has troubles exploiting the possibilities offered by a freedom of movement in a two-dimensional environment. Future developments should provide him with mechanisms to capture spatial regularities, through constructing an internal representation of his surrounding environment.