Mocap as a Service: Video Motion Capture Makes Human Motion Analysis for Everyone

In this plenary talk, Professor Yoshihiko Nakamura from the University of Tokyo presents his vision of making motion capture technology accessible to everyone through video-based systems and cloud computing. He begins by introducing his lab’s work on humanoid robotics, emphasizing the importance of understanding human motion for developing natural robot behaviors. He then describes traditional marker-based and IMU-based motion capture systems, highlighting their limitations in terms of setup time and cost. The core of the talk focuses on their recent development of a video-based motion capture system that uses multiple cameras and deep learning (specifically OpenPose) to reconstruct 3D skeletal movements without markers. This system is demonstrated in various applications, including analyzing the muscle activities of elite athletes such as Olympic medalists and professional football players. The ultimate goal is to provide ‘Mocap as a Service’ over high-speed academic networks, enabling widespread use in sports training, healthcare, and other fields. The talk concludes with a Q&A session addressing cost, integration with robotics, and potential for energy expenditure estimation.