Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Tasks

Josip Josifovski, Mohammadhossein Malmir, Noah Klarmann, Bare Luka Zagar, Nicolás Navarro-Guerrero, Alois Knoll
In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), (IROS), 23.10.-27.10.2022, Kyoto, IEEE, pages 10193-10200, Oct/2022. ISBN: 978-1-66547-927-1.

Abstract

Current state-of-the-art approaches for transferring deep-learning models trained in simulation either rely on highly realistic simulations or employ randomization techniques to bridge the reality gap. However, such strategies do not scale well for complex robotic tasks. Highly-realistic simulations are computationally expensive and challenging to implement, while randomization techniques become sample-inefficient as the complexity of the task increases. This paper proposes a procedure for training on incremental simulations in a continual learning setup. We develop a simulation platform for the experimental analysis that can serve as a training environment and as a benchmark for continual and reinforcement learning sim2real approaches. The results show that training time for complex tasks can be reduced. Thus, we argue that Sequentially-Randomized Simulations improve the sim2real transfer.

Files

Cite and export

Choose a format, then copy or download.

@inproceedings{Josifovski2022Analysis,
  author    = {Josifovski, Josip and Malmir, Mohammadhossein and Klarmann, Noah and Zagar, Bare Luka
               and Navarro-Guerrero, Nicolás and Knoll, Alois},
  title     = {Analysis of Randomization Effects on {Sim2Real} Transfer in Reinforcement Learning
               for Robotic Manipulation Tasks},
  booktitle = {{IEEE}/{RSJ} International Conference on Intelligent Robots and Systems ({IROS})},
  pages     = {10193--10200},
  month     = oct,
  year      = {2022},
  publisher = {IEEE},
  address   = {Kyoto},
  isbn      = {978-1-66547-927-1},
  doi       = {10.1109/IROS47612.2022.9981951},
  url       = {https://robotik.dfki-bremen.de/en/research/publications/12962}
}
TY  - CPAPER
AU  - Josifovski, Josip
AU  - Malmir, Mohammadhossein
AU  - Klarmann, Noah
AU  - Zagar, Bare Luka
AU  - Navarro-Guerrero, Nicolás
AU  - Knoll, Alois
TI  - Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Tasks
T2  - IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
SP  - 10193
EP  - 10200
PY  - 2022
DA  - 2022/10//
PB  - IEEE
CY  - Kyoto
SN  - 978-1-66547-927-1
DO  - 10.1109/IROS47612.2022.9981951
AB  - Current state-of-the-art approaches for transferring deep-learning models trained in simulation either rely on highly realistic simulations or employ randomization techniques to bridge the reality gap. However, such strategies do not scale well for complex robotic tasks. Highly-realistic simulations are computationally expensive and challenging to implement, while randomization techniques become sample-inefficient as the complexity of the task increases. This paper proposes a procedure for training on incremental simulations in a continual learning setup. We develop a simulation platform for the experimental analysis that can serve as a training environment and as a benchmark for continual and reinforcement learning sim2real approaches. The results show that training time for complex tasks can be reduced. Thus, we argue that Sequentially-Randomized Simulations improve the sim2real transfer.
UR  - https://robotik.dfki-bremen.de/en/research/publications/12962
LA  - eng
ER  - 
Josifovski, J., Malmir, M., Klarmann, N., Zagar, B. L., Navarro-Guerrero, N., & Knoll, A. (2022). Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Tasks. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 10193–10200). IEEE. https://doi.org/10.1109/IROS47612.2022.9981951


Back to the list
to top