Our research is at the intersection of computer vision, computer graphics and machine learning--we develop computational algorithms to efficiently digitize people and train machines to perceive people from visual data.
Current computer vision algorithms can detect people in images or estimate 2D keypoints to a remarkable accuracy. However, people are far more complex–-we effortlessly sense other people's emotional state based on facial expressions and body movements, or we make guesses about people's preferences based on what clothing they wear. Our goal is to build virtual humans that look, move and eventually think like real ones.
Big News! The group is moving to Tuebingen
Gerard Pons-Moll accepted a W3 professorship at the University of Tübingen at the department of Computer Science. The group will focus on learning, vision and graphics, with focus on capturing and synthesizing human and object shape apperance, as well as learning digital humans which can move and interact with the 3D world. More infos here.
4 Papers (1 oral) accepted at #CVPR2021!
Pdfs, data and code will be available soon! here!
1) HPS: Capturing and self-localizing humans in large 3D scenes with IMUs and a head mounted camera (Oral).
2) Stereo Radiance Fields: generalizing NeRF to multiple scenes using classical stereo principles.
3) SMPLicit: An implicit based representation of people in layered clothing.
4) D-Nerf: generalizing NeRF to dynamic scenes.Congratulations students and collaborators!
2 Papers accepted at NeurIPS (1 Oral, 1 Poster)
-1) LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration
-2) Neural Unsigned Distance Fields for Implicit Function LearningCongrats to the team, and thanks to the reviewers for helping us in improving our papers!
Code, data and papers will be available here.
Winners of all ECCV SHARP'20 Challenges
Congratulations to Julian Chibane and Gerard Pons-Moll for winning all tracks of the ECCV'20 SHARP challenge on 3D shape recovery from partial textured 3D scans.
We extended IF-Nets (Chibane et al. CVPR'20) to complete geometry and texture (Chibane et al. ECCV-Workshop'20). In our experience, the model is easy to use, and works really well for a wide variety of 3D completion and reconstruction tasks. Code available here.
5 Papers (2 orals, 3 posters) accepted at #ECCV2020!
Pdfs, data and code will be available soon! here!
Topics are: 1) Combining implicit functions and meshes for reconstruction, 2) A model of cloth sizing, 3) Unsupervised disentanglement of shape and pose from meshes, 4) A human implicit function parameterized by pose (NASA) and 5) Monocular 3D object detection in driving scenes. Congratulations students and collaborators!
CVPR20 Best Student Paper Honorable Mention!
Congrats to collaborators, and thanks to the reviewers for useful feedback, and the awards committee for selecting our paper among many others worthy of the prize--we are honored.
3DPW Challenge and Workshop at ECCV 2020 featured on the front-page of computer vision news
Gerard Pons-Moll, Angjoo Kanazawa, Michael Black and Aymen Mir talk about challenges in perceiving people in 3D, see the interview. Thanks Ralph Ansarouth!
Deadline is approaching, participate!: 3DPW Challenge.
5 Papers (2 orals, 3 posters) accepted at #CVPR2020!
Pdfs, data and code are available here!Congratulations to all collaborators!
3DPW Challenge and Workshop at ECCV 2020.
Gerard Pons-Moll will organize the first 3DPW Challenge and Workshop at ECCV 2020 together with Angjoo Kanazawa, Michael Black and Aymen Mir.
The aim of the workshop and challenge is to establish a benchmark to quantiatively evaluate 3D pose and shape human reconstruction methods in the wild using the 3DPW dataset.
Area Chair CVPR 2021 and 3DV 2020.
Gerard Pons-Moll will serve as Area Chair for CVPR 2021
He will also serve as Area Chair for 3DV 2020
German Pattern Recognition Award
Gerard Pons-Moll has been awarded the 2019 German Pattern Recognition Award -- the highest prize awarded annualy to one researcher by the German Society of Computer Vision and Machine Learning.
Congrats to the group and collaborators!
3 papers accepted at ICCV 2019! 1 paper at 3DV'19
Pdfs, data and code here!
-1) Multi-Garment Net: Learning to Dress 3D People from Images
-2) Tex2Shape: Detailed Full Human Body Geometry from a Single Image
-3) AMASS: Archive of Motion Capture as Surface Shapes
-4) 360-Degree Textures of People in Clothing from a Single ImageCongratulations to all co-authors!
3 papers accepted to CVPR 2019!
February 2019Paper pdfs, videos and code coming soon!
1) Learning to Reconstruct People in Clothing from a Single RGB Camera
2) SimulCap : Single-View Human Performance Capture with Cloth Simulation
3) In the Wild Human Pose Estimation using Explicit 2D Features and Intermediate 3D Representations
Congratulations to all co-authors!
Emmy Noether starting grant!
Gerard Pons-Moll has been awarded an Emmy Noether grant. The grant, called like the group "Real Virtual Humans" (RVHu), conists of 1.6 Million euros to conduct research at the interesction of vision, graphics and learning with special focus on analyzing and digitizing humans.
3 papers accepted at 3DV 2018!
1 Paper won the best student paper award!
Pdfs and videos available!
-Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation
3DV Best Student Paper Award
-Detailed Human Avatars from Monocular Video
-Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB
1 paper at ECCV'18
Recovering Accurate 3D Human Pose in The Wild Using IMUs and a Moving Camera
New dataset with ground truth 3D poses in the wild! Download
We have released the first (and most challenging) dataset of natural scenes with multiple people with accurate 3D pose and shape! The RGB video includes scenes like taking the bus, walking on the city, shoping, sports, etc.
Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors
in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
(First two authors contributed equally)