Evolving Neural Networks for Online Reinforcement Learning
Jan Hendrik Metzen, Mark Edgington, Yohannes Kassahun, Frank Kirchner
Editors: G. Rudolph, T. Jansen, S.M. Lucas, et al.
In Proceedings of the 10th International Conference on Parallel Problem Solving From Nature, (PPSN-2008), 13.9.-17.9.2008, Dortmund, Springer, series Lecture Notes in Computer Science, volume 5199, pages 518-527, 2008.
For many complex Reinforcement Learning problems with large and continuous state spaces, neuroevolution (the evolution of artificial neural networks) has achieved promising results. This is especially true when there is noise in sensor and/or actuator signals. These results have mainly been obtained in offline learning settings, where the training and evaluation phase of the system are separated. In contrast, in online Reinforcement Learning tasks where the actual performance of the systems during its learning phase matters, the results of neuroevolution are significantly impaired by its purely exploratory nature, meaning that it does not use (i. e. exploit) its knowledge of the performance of single individuals in order to improve its performance during learning. In this paper we describe modifications which significantly improve the online performance of the neuroevolutionary method Evolutionary Acquisition of Neural Topologies (EANT) and discuss the results obtained on two benchmark problems.