PhD Dissertation Proposal by YANG Yifei | Towards Embodied Human Motion Generation

ABOUT THE TALK

Generative models now produce human motion that faithfully follows text, music, or user intent, yet such motion is optimized for how it looks against captured data, not for whether a body could perform it in an environment. This dissertation calls this distance the embodiment gap and pursues embodied human motion generation by decomposing the gap into four requirements: coherence over long horizons at tractable cost (R1), a body free of self-collision (R2), closed-loop response to streaming user input (R3), and physical executability on a humanoid robot (R4). Three completed works address the first three. Lagrangian Motion Fields addresses R1 with a compact, training-free motion abstraction that improves long-term generation quality while reducing inference cost. FreeMo addresses R2 with a differentiable, trajectory-level collision energy that removes self-collision from pretrained generators without retraining. TINMO addresses R3 by discovering unsupervised latent actions over motion primitives, recasting interactive generation as closed-loop control on a learned world model. The proposed work addresses R4. Generated motion specifies where joints should be, not the forces a robot can supply, and routinely demands effort that fights the robot’s own inertia without serving the motion. The proposed parasitic-effort projection detects and removes this effort, repairing physically impossible segments and lowering the cost of feasible ones while preserving the motion’s task and meaning. Preliminary results on a simulated humanoid support the approach, and the remaining work will contribute an executability benchmark for humanoid reference motion and a full evaluation toward executable, semantically faithful motion generation.

ABOUT THE SPEAKER

Yifei YANG is a PhD candidate in Computer Science at the School of Computing and Information Systems, Singapore Management University, supervised by Prof. Shengfeng He. His research focuses on human motion generation, with the goal of closing the gap between motion that looks right and motion that a physical body can execute. His work on Lagrangian Motion Fields is published in IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), and his subsequent works on collision-free and interactive motion generation are under review at IEEE Transactions on Multimedia and NeurIPS, respectively.