Built a zero-shot perception-reasoning-planning pipeline that converts natural-language instructions and raw visual inputs into executable multi-robot plans, integrating visual foundation models, VLM scene understanding, LLM code generation, and ROS 2/MoveIt 2 execution for ABB robot validation.
Diffusion Model-based Policy Learning for Human-to-Robot Object Handover
Developed a two-stage imitation learning framework that combines offline demonstration synthesis with diffusion-based handover policy learning, using 4D spatiotemporal flow to represent human-object motion and validating the policy on an ABB IRB1600 robot.
Digital Twin-driven Multi-robot Intelligent Construction System
Developed ConstrucTwin, a digital twin-driven multi-robot framework linking physical ABB robots, Unity3D twins, communication networks, RGB-D visual servoing, real-time monitoring, 6D pose estimation, and MoveIt 2-based planning for robotic construction scenarios.
Diverse Human Motion Prediction for Human-Robot Collaboration
Developed FineMLD, a fine-grained motion latent diffusion framework for diverse human motion prediction, supported by 75,541 synchronized human-robot motion frames and integrated into closed-loop collision avoidance for HRC workspaces.
Multimodal Human Digital Twin Modeling for Human-Robot Collaboration
Built a multimodal human digital twin framework for motion pattern recognition and handover intention prediction, combining IMU, EMG, RGB-D, and pressure sensing with BiLSTM/ST-GCN fusion and domain adaptation.