Figure 11: Learning Two Things, rho at .2
Please notice the horizontal regions of the graphs in Figures 8-11. The
frequency of occurrence and length of these regions increases with the step
value. What this demonstrates is that the network is learning slower as the
step value increases. The horizontal regions represent situations where, after
training the net to perform a certain function, it returns to that same
internal state and the number of iterations does not change. Thus these are
regions where the network did not learn. We conclude that a step value any
higher than .1 will interfere with learning more than it will expedite it. It
is not reasonable to trade speed for accuracy in the network's thought
processes.
The
first tests of our genetic algorithm program generated surprising data. We
found that no matter what we did, the worst rat would come up to the level of
the best rat in about ten generations, with the top rat never improving. At
this point the best code remained the same from generation to generation. (See
Figure 12.)
Figure 12: Health vs. Generations Using a Defective Breeding
Technique
To ward off this stagnation, we rewrote the entire genetic mating algorithm so
that the codes would cross over once in every rule, not just once per rat like
the previous algorithm. The new algorithm proved to be successful, and the top
rat gradually increased in fitness through several generations. Figures 13, 14
and 15 on the following pages are graphs that examine the relationship between
health and generation when changes are made to the number of mazes to be run
and/or the mutation rate. Figure 13 is a graph of the rat population's
progress over many generations where it is confronted with no change of
environments and no mutations.
The results in Figure 13 show the genetic algorithm in action and breeding good
rats. In this example, the best health value rises from 14 to 42 while the
average continues to slowly climb. The worst rat fluctuates in a near random
manner. Once we saw that the algorithm was working, we began to challenge the
rats by changing mazes after a certain number of generations. In the
experiment shown in Figure 14, the maze was changed every 25 generations.
Figure 13: Health vs. Generations in One Maze and without Mutations
Figure 14: Health vs. Generations in a Changing Environment with No
Mutations
Notice that, in the experiment described by Figure 14, the fitness values of
the rats converge and stagnate at generation 27. These results led us to
explore the effects of increasing the mutation rate. Figure 15 shows the
results of our investigations into the effects of enhanced mutation.
Figure 15: Fitness vs. Generation Graph in a Changing
Environment
with a Mutation Rate of 1 out of 500 Bits
In Figure 15, the best rat continues to rise in health. The average and worst
health values fluctuate throughout the graph. Also note that the health of the
best rat drops slightly at the first two maze changes (generations 25 and 50)
and then improves. However, after the third maze change, the rat suffers no
loss of health.