Human Motion Capture

Driver 360
Driver 360 is one of the tools used in the Visual Intelligence Laboratory.

What is Human Motion Capture?

The Visual Intelligence Laboratory pushes the capability of computer vision algorithms to track various human motions and activities without using rather cumbersome, expensive sensors. A deep-learning-based computer vision algorithm developed here can detect and track joint-level kinematics of people through ordinary cameras, such as smartphones, dashcams, etc. Our algorithm allows marker- and sensor-less tracking of human body motion, which eliminates the practical limitations of human motion capture, permitting the use of human motion analysis in much broader application areas. 

As an active member of the University of Iowa’s Virtual Soldier Research program, the Visual Intelligence Laboratory is currently collaborating with a number of researchers across campus, including UI Athletics, Biomedical Engineering, and the University of Iowa Hospitals and Clinics, to bridge the Santos human physics simulation capability to diverse domain-specific applications. Our system allows easy and hassle-free acquisition of human motion data, which can then be fed into the Santos simulation engine, where a variety of biomechanical and physiological measures can be estimated. Using such technology, we help the U.S. military reduce musculoskeletal injuries during training, assist sports scientists in analyzing player performance, and mitigate work-related injuries of manufacturing workers.  

In addition, we are collaborating with the National Advanced Driving Simulator to develop a vision-based vehicle occupant monitoring system for automated/autonomous vehicles. As the level of automation of driver assistance systems advances, the vehicle’s ability to monitor and understand the occupants’ physical state becomes crucial. However, such an application does not enjoy the luxury of attaching sensors to the human body, which results in a significant bottleneck for current human sensing technology. Our method provides an effective solution to mitigate this problem by allowing marker-less tracking of drivers and passengers. From such tracking data, a car, for example, can discern if the driver is distracted and, if so, intelligently adapt the level of automation to enhance safety. 

Click here to learn more about our facilities and equipment


Prof. Steve Baek, Ph.D.