Manipulating Elasto-Plastic Objects With 3D Occupancy and Learning-Based Predictive Control
Robot manipulating elasto-plastic materials using 3D occupancy prediction and model predictive control.
Research Question
How can robots perceive, model, and manipulate elasto-plastic objects (like clay) that undergo complex, history-dependent plastic deformation and heavy self-occlusion? Traditional point-cloud or mesh tracking methods struggle when objects change shape irreversibly and conceal internal or rear contours.
What We Built
We introduced a learning-based predictive control framework leveraging a volumetric 3D occupancy state representation. By inferring a complete 3D occupancy grid from multi-view RGB cameras, our model represents both visible surfaces and occluded volume. Coupled with a deformation prediction model and shape-aware action initialization, the robot plans manipulation actions efficiently toward target shapes.
My Contribution
Contributed to the formulation of the learning-based model predictive control framework, experimental validation on the physical robot platform, and analysis of multi-view representation accuracy during deformation tasks.
Collaborators & Credits
Authors: Zhen Zhang, Xiangyu Chu, Yunxi Tang, Lulu Zhao, Jing Huang, Zhongliang Jiang, and K. W. Samuel Au
Published in: IEEE Robotics and Automation Letters (RA-L), 2025 · ICRA 2026 Transfer (Vol. 10, No. 7, pp. 7222–7229).