Dynamische Bewegungen wie das Hüpfen erweitern die Fortbewegungsmöglichkeiten von Laufrobotern und ermöglichen es ihnen flexibel auf ihre Umgebung zu reagieren. In ihrem Beitrag mit dem Titel "End-to-End Reinforcement Learning for Torque-Based Variable Height Hopping" stellen die DFKI-Forschenden einen innovativen Ansatz vor, um die komplexe Steuerung des robotischen Hüpfens zu meistern. Dieser Ansatz verwendet bestärkendes Lernen, um Sprungphasen präzise zu erkennen und in eine drehmomentbasierte Steuerung zu übertragen.
Die Trajektorienplanung ist in der Bewegungssteuerung von unteraktuierten Robotern weit verbreitet. Bei seriell-parallelen Hybridrobotern wie Humanoiden wird jedoch oft die Steuerung im geschlossenen Regelkreis vernachlässigt, was dazu führen kann, dass das volle Potenzial des Systems nicht vollständig ausgeschöpft wird. In ihrer Arbeit "Exploiting a Series-Parallel Hybrid Humanoid's Full Capabilities via Whole Body Trajectory Optimization" demonstrieren die Forschenden die Optimierung von Trajektorien für einen seriell-parallelen Hybridroboter. Dabei werden holonome Beschränkungen berücksichtigt, die aus geschlossenen kinematischen Schleifen während einer Gewichthebeaufgabe resultieren.
Der Beitrag "AcroMonk: A Minimalist Underactuated Brachiating Robot" stellt einen einfachen Brachialroboter-Prototyp vor, der lediglich über einen Aktor und passive Greifer verfügt, um affenähnliche Bewegungen auszuführen. Das Design verwendet verschiedene Steuermethoden, wie zum Beispiel Trajektorienoptimierung und Verstärkendes Lernen, und vergleicht ihre Effektivität und Energieeffizienz. Das quelloffene Design bietet eine wertvolle Ressource für Bildung und Forschung im Bereich der unteraktuierten Robotik.
Serienelastische Aktoren (SEA), die ursprünglich für serielle Roboterarme entwickelt wurden, erfordern den Einsatz von Algorithmen zur modellbasierten Bahnverfolgung, die zeitliche Änderungen berücksichtigen. Bisher wurde dieser Ansatz jedoch nicht für SEAs in parallelen kinematischen Manipulatoren (PKM) untersucht. Die Arbeit "A Recursive Lie-Group Formulation for the Second-Order Time Derivatives of the Inverse Dynamics of Parallel Kinematic Manipulators" schließt erstmals diese Lücke, indem sie die spezielle Struktur von PKMs nutzt und die Algorithmen, die für die inverse Dynamik von seriellen Robotern verwendet werden, mithilfe einer sogenannten Lie-Gruppen-Formulierung adaptiert.
Die Paper im Detail (englisch):
Title: End-to-End Reinforcement Learning for Torque Based Variable Height Hopping
Authors: Raghav Soni, Daniel Harnack, Hannah Isermann, Sotaro Fushimi, Shivesh Kumar, Frank Kirchner
Abstract:
Legged locomotion is arguably the most suited and versatile mode to deal with natural or unstructured terrains. Intensive research into dynamic walking and running controllers has recently yielded great advances, both in the optimal control and reinforcement learning (RL) literature. Hopping is a challenging dynamic task involving a flight phase and has the potential to increase the traversability of legged robots. Model based control for hopping typically relies on accurate detection of different jump phases, such as lift-off or touch down, and using different controllers for each phase. In this paper, we present an end-to-end RL based torque controller that learns to implicitly detect the relevant jump phases, removing the need to provide manual heuristics for state detection. We also extend a method for simulation to reality transfer of the learned controller to contact rich dynamic tasks, resulting in successful deployment on the robot after training without parameter tuning.
Paper Link: https://arxiv.org/pdf/2307.16676.pdf
Video Link: End-to-End Reinforcement Learning for Torque Based Variable Height Hopping
Title: Investigations into Exploiting the Full Capabilities of a Series-Parallel Hybrid Humanoid Using Whole Body Trajectory Optimization
Authors: Melya Boukheddimi, Rohit Kumar, Shivesh Kumar, Justin Carpentier, Frank Kirchner
Abstract:
Trajectory optimization methods have become ubiquitous for the motion planning and control of under-actuated robots for e.g., quadrupeds, humanoids etc. While they have been extensively used in the case of serial or tree type robots, they are seldomly used for planning and control of robots with closed loops. Series-parallel hybrid topology is quite commonly used in the design of humanoid robots, but they are often neglected during trajectory optimization and the movements are computed for a serial abstraction of the system and then the solution is mapped to the actuator coordinates. As a consequence, the full capability of the robot cannot be exploited. This paper presents a case study of trajectory optimization for series-parallel hybrid robot by taking into account all the holonomic constraints imposed by the closed kinematic loops present in the system. We demonstrate the advantages of this consideration with a weightlifting task on RH5 Manus humanoid in both simulation and experiments.
Paper Link: https://www.researchgate.net/publication/372677906_Investigations_into_Exploiting_the_Full_Capabilities_of_a_Series-Parallel_Hybrid_Humanoid_using_Whole_Body_Trajectory_Optimization
Video Link: https://www.youtube.com/watch?v=qcJiLLTbDmk
Title: AcroMonk: A Minimalist Underactuated Brachiating Robot
Authors: Mahdi Javadi, Daniel Harnack, Paula Stocco, Shivesh Kumar, Shubham Vyas, Daniel Pizzutilo, Frank Kirchner
Abstract:
Brachiation is a dynamic, coordinated swinging maneuver of body and arms used by monkeys and apes to move between branches. As a unique underactuated mode of locomotion, it is interesting to study from a robotics perspective since it can broaden the deployment scenarios for humanoids and animaloids. While several brachiating robots of varying complexity have been proposed in the past, this paper presents the simplest possible prototype of a brachiation robot, using only a single actuator and unactuated grippers. The novel passive gripper design allows it to snap on and release from monkey bars, while guaranteeing well defined start and end poses of the swing. The brachiation behavior is realized in three different ways, using trajectory optimization via direct collocation and stabilization by a model-based time-varying linear quadratic regulator (TVLQR) or model-free proportional derivative (PD) control, as well as by a reinforcement learning (RL) based control policy. The three control schemes are compared in terms of robustness to disturbances, mass uncertainty, and energy consumption. The system design and controllers have been open-sourced. Due to its minimal and open design, the system can serve as a canonical underactuated platform for education and research.
Paper Link: https://arxiv.org/pdf/2305.08373.pdf
Video Link: https://www.youtube.com/watch?v=FIcDNtJo9Jc
GitHub Link: https://github.com/dfki-ric-underactuated-lab/acromonk
Title: A Recursive Lie-Group Formulation for the Second-Order Time Derivatives of the Inverse Dynamics of Parallel Kinematic Manipulators
Authors: Andreas Mueller, Shivesh Kumar, Thomas Kordik
Abstract:
Series elastic actuators (SEA) were introduced for serial robotic arms. Their model-based trajectory tracking control requires the second time derivatives of the inverse dynamics' solution, for which algorithms were proposed. Trajectory control of parallel kinematics manipulators (PKM) equipped with SEAs has not yet been pursued. Key element for this is the computationally efficient evaluation of the second time derivative of the inverse dynamics' solution. This has not been presented in the literature and is addressed in the present paper for the first time. The special topology of PKM is exploited reusing the recursive algorithms for evaluating the inverse dynamics of serial robots. A Lie group formulation is used, and all relations are derived within this framework. Numerical results are presented for a 6-DOF Gough-Stewart platform (as part of an exoskeleton), and for a planar PKM when a flatness-based control scheme is applied.
Paper Link (open access): https://ieeexplore.ieee.org/document/10101821
Kontakt:
Dr. rer. nat. Shivesh Kumar
Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI)
Robotics Innovation Center
Teamleiter Mechanics & Control
Mail: shivesh.kumar@dfki.de