Empirical Evaluation of Contextual Policy Search with a Comparison-based Surrogate Model and Active Covariance Matrix Adaptation
Alexander Fabisch
In Proceedings of the Genetic and Evolutionary Computation Conference Companion, (GECCO-2019), 13.7.-17.7.2019, Prag, o.A., 2019.
Abstract
:
Contextual policy search (CPS) is a class of multi-task reinforcement
learning algorithms that is particularly useful for robotic applications.
A recent state-of-the-art method is Contextual Covariance
Matrix Adaptation Evolution Strategies (C-CMA-ES). It is based on
the standard black-box optimization algorithm CMA-ES. There are
two useful extensions ofCMA-ES thatwe will transfer to C-CMA-ES
and evaluate empirically: ACM-ES, which uses a comparison-based
surrogate model, and aCMA-ES, which uses an active update of
the covariance matrix. We will show that improvements with these
methods can be impressive in terms of sample-efficiency, although
this is not relevant any more for the robotic domain.