Towards Autonomous Robotic Catheter Navigation Using Reinforcement Learning

Zusammenfassung (Abstract) :

Autonomous navigation with robotic catheter, which can be seen as a specialization of continuum robots, inside the aorta is a challenging task due to the deformable and dynamic nature of the aorta and due to the fact that the geometry varies considerably from patient to patient. A reinforcement learning (RL) method, Q-learning, is applied to navigate inside the aorta. The knowledge learned by the robotic catheter in a given 2D aorta mesh is transferred by means of the action value function to a modified 2D aorta mesh. In the first mesh, for testing with the knowledge from training on the first mesh, the action value function is initialized using the knowledge from the first mesh. In the second mesh, for testing with the knowledge from training on the second mesh, the action value function is initialized using the knowledge from the second mesh. In the second mesh, for testing transferability of training knowledge from the first mesh to the second mesh, the action value function is initialized using the knowledge from training on the first mesh. The initial result shows that the knowledge transferred from the first mesh to the second mesh by using action value function enables autonomous navigation in the second mesh. The initial result shows also that the RL algorithm does not depend on the 2D aorta mesh but rather depends on the robotic catheter which learns the knowledge.