Our goal is to design computational algorithms to represent people ditially, and to train machines to perceive humans from visual data.
We develop techniques in machine learning, computer graphics and computer vision to build generative 3D models of people.
We leverage visual data (3D/4D scans, videos, images) of real humans. In particular, we develop new visual computing methods to efficiently digitize the data, and new machine learning methods to create virtual people models. Such models should be compact, easy to use and should look and move like real people.
One of the fundamental problems in computer vision is to extract information about the 3D world from visual data, coming from a single camera for example.
In our group, we focus on the 3D recovery of people from visual input. To address this challenging problem, we investigate ways to combine deep learning methods with statistical body models and optimization.
In essence, we are teaching machines to see and perceive people.
IMUs or Inertial Measurement Units are small devices that can measure 3D orientation and acceleration. IMUs are becoming available on a large number of devices like phones and fitness watches.
Our research is focused on methods to infer 3D human motion using IMUs attached at different body parts. To make it practical we use as few sensors as possible -- sometimes as few as four. This makes the problem very hard and underconstrained. Furthermore, IMUs are noisy and suffer from drift. Therefore, we work on robust optimization and learning algorithms that leverage prior knowledge about humans via statistical human body models. We develop sensor fusion algorithms to combine IMU with video input.