Vision-Based Online Key Point Estimation of Deformable Robots
We present a markerless vision-based system for estimating key points in deformable robots. Using a convolutional neural network (CNN), the system accurately reconstructs soft robotic shapes in real-time from multiple perspectives, outperforming existing methods. This work provides a robust solution for soft robots in dynamic and occlusive environments. The work is published in Advanced Intelligent Systems by Wiley.

The precise control of deformable robots requires accurate shape estimation, which current proprioceptive and exteroceptive methods fail to provide reliably. This paper introduces a CNN-based system for markerless key point estimation, using two-camera setups to infer shape representations in real-time. The method surpasses current markerless techniques, showing a maximum accuracy improvement of 4.5%. Evaluations on soft robotic arms and fish demonstrate its versatility.
external page Link to the open access paper on Advanced Intelligent Systems