Data & Software | Real Virtual Humans | University of Tübingen

Almost all software and data is for research purposes only. Please, check the corresponding licenses.
NOTE: Some code and data can also be found under the link Code/Data in publications.

NRDF: Neural Riemannian Distance Fields for Learning Articulated Pose Priors, CVPR'24

Code and pretrained models.
Inverse kinematics from partial observations.

Code/Data

Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation. CVPR'24

Code and pretrained models.
ProciGen dataset.

Code/Data

CloSe: A 3D Clothing Segmentation Dataset and Model. 3DV'24

Code and pretrained models.
3D interactive tool for segmentation labeling and refining.
3D Scans, SMPL and Clothing segmentation dataset.

Code/Data

GAN-Avatar: Controllable Personalized GAN-based Human Head Avatar. 3DV'24

Code.
Pretrained model.

Code/Data

Interaction Replica: Tracking human–object interaction and scene changes from human motion. 3DV'24

iReplica Dataset.

Code/Data

NSF: Neural Surface Fields for Human Modeling from Monocular Depth ICCV'23

Code for training and inference.
Pretrained model.

Code/Data

Object pop-up: Can we infer 3D objects and their poses from human interactions alone? CVPR'23

Code for training and inference.
Pretrained model.

Code/Data

Visibility Aware Human-Object Interaction Tracking from Single RGB Camera CVPR'23

Code for training and inference.
Pretrained model.

Code/Data

Blendify

A Python framework for creating and rendering scenes in Blender.
Provides a high-level API to make common visualization tasks easier.

Website GitHub

RVH Mesh Registration Library

Code for registering SMPL to 3D scans and point clouds.
Supports non-rigid deformations on top of SMPL.
Optional support for SMPLH.

GitHub

CHORE: Contact, Human and Object REconstruction from a single RGB image. ECCV'22

Code to reconstruct human and object from single RGB images.
Data preprocessing and model training code.
Pretrained model.

Code/Data

TOCH: Spatio-Temporal Object-to-Hand Correspondence for Motion Refinement. ECCV'22

Code for training and inference.
Pretrained model.

Code/Data

Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fields}

Neural distance field pose prior.
Pretrained model.

Code/Data

BEHAVE: Dataset and Method for Tracking Human Object Interactions. CVPR'22

Data: multiview RGBD images paired with SMPL and object registrations.
Code: tracking human and object with multiview RGBD sensors.
Pretrained models.

Code/Data

Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes

Box2Mask: 3D Semantic Instance Segmentation Through Bounding Box Supervision. ECCV'22

Code for training and generation.
Pretrained models.
Dataset.

Code/Data

Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothing. ICCV'21

Code for training and generation.
Pretrained models.
Dataset.

Code/Data

Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenes CVPR'21

Code for training and generation.
Pretrained models.
Data and experiment preparation.

Code/Data

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors. CVPR'21(Oral)

DATA: Scene files, Localization files, Motion files, Raw IMU files, Pose from IMU.
Code for visualising motion in scenes.
Code for obtaining motion files from input data
Codes for data pre-processing.

Code/Data

LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration. NeurIPS'20(Oral)

Pre-trained LoopReg models to predict dense correspondences from scan/ pointcloud to parametric body model.
Code for training and testing LoopReg.
Code to register SMPL/SMPL+D with LoopReg.
Code to diffuse mesh functions to R3.
Codes for data pre-processing.

Code/Data

Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction. ECCV'20(Oral)

Pre-trained IPNet models to predict the full dressed surface of human, body shape under clothing and SMPL part correspondences.
Code for training and testing IPNet.
Code to register SMPL/SMPL+D to IPNet predictions.
Codes for data pre-processing.

Code/Data

NASA: Neural Articulated Shape Approximation

Neural Articulated Shape Approximation. ECCV'20

Dataset.
Code for training and testing NASA.

Code/Data

SIZER: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothing. ECCV'20(Oral)

3D scans and SMPL+G registrations of 100 subjects in various garments styles and sizes in A-pose.
Code for training and testing ParserNet and SizerNet.
Pre-trained ParserNet and SizerNet models.

Code/Data

Unsupervised Shape and Pose Disentanglement for 3D Meshes. ECCV'20

Pre-trained disentangled auto-encoder on AMASS that can infer latent shape and pose codes from human bodies.
Code for preprocessing and training on AMASS.

Code/Data

TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style. CVPR'20(Oral)

Dataset of 3D garments in various pose, shape and garment styles.
Code for training and inference of TailorNet model.
Pretrained TailorNet model for multiple garments.

Code/Data

Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion. CVPR'20

Pre-trained models for: Point Cloud Completion, Voxel-Superresolution and Human Single-View Reconstruction
Code for: data processing, IF-Net model, training, generation and evaluation.
Link to the ShapeNet data used.

Code/Data

Learning to Transfer Texture from Clothing Images to 3D Humans. CVPR'20

Code and pre-trained models for mapping the front and back of clothing images - TShirts, Pants, ShortPants - downloaded from clothing stores to 3D garment meshses worn atop the SMPL body model.
Code for optimization method to obtain training data.
Code for training the neural model - Pix2Surf.

Code/Data

Multi-Garment Net: Learning to Dress 3D People from Images. ICCV'19

Released 'Digital Wardrobe' of registered garments to dress SMPL model. The dataset includes, segmented 3D scans, registrations, garments and texture maps.
Code to dress SMPL model with the released garments.
Code to retarget clothing across subjects with different body shapes and poses.
Pre-trained MGN model to infer 3D garments and underlying body shape from a few RGB images.

Code/Data

Tex2Shape: Detailed Full Human Body Geometry from a Single Image

Tex2Shape: 3D avatars from a single image. ICCV'19

Code and model for the pose-independent image-to-image translation method to reconstruct a full-body avatar from a single image. The avatars feature fine details even on unseen parts!
Reconstructions are based on the SMPL body topology so they can be re-posed and re-shaped.

Code/Data

Learning to Reconstruct People in Clothing from a Single RGB Camera

Octopus: 3D avatars from a video. CVPR'19

Code for the learning+optimization based method to reconstruct a detailed body shape including clothing and hair from a single RGB video. Given semantic segmentations as input, reconstructions are computed in 10 seconds!
Reconstructions are based on the SMPL body topology so they can be re-posed and re-shaped.

Code/Data

Detailed Human Avatars from Monocular Video

Sharp texture map from multiple views/frames. 3DV'18

Code. INPUT: Multiple images (frames, or views) and a 3D human model roughly aligned to them. OUTPUT: texture map, sharp and without stitching artefacts.

Code/Data

Video Based Reconstruction of 3D People Models

3D avatars from a video and PeopleSnapshot Dataset. CVPR'18

Code for the optimization based method to reconstruct a detailed body shape including clothing and hair from a single RGB video. Reconstructions are based on the SMPL body topology so they can be re-posed and re-shaped.
PeopleSnapshot dataset including 24 subjects rotating in front of the camera.

Code/Data

Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time. SiggraphAsia'18.

Code for real time 3D pose from 6 IMUs. Training and testing parts along the real-time demo.
DIP-IMU: large scale dataset including real and syntehtic IMU readings paired with SMPL poses. Largest IMU dataset available.

Code/Data

Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation

Neural Body Fitting. Human Body Shape and Pose from a Single Image. 3DV'18.

Code for 3D human pose and shape estimation from a single image. Better performance than SOTA when trained with the same data.

Code/Data

Recovering Accurate 3D Human Pose in The Wild Using IMUs and a Moving Camera

3DPW: 3D Poses in the Wild Dataset. ECCV'18.

Unique dataset with monocular hand-held video together with accurate 3D human poses for evaluation. Very accurate 3D poses are obtained combining video and IMU.

60 video sequences: downtown, taking the bus, having coffee, sports, etc.
2D pose annotations
3D poses reference poses
Camera poses for every frame in the sequences
3D body scans and 18 3D human models with different clothing variations.

Code/Data

Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB

Multiperson 3D pose dataset

MuPoTS-3D: Dataset of multiple people with reference 3D poses. Compared to 3DPW above, the camera is static but reference pose for more than 2 persons is provided
MuCo-3DHP: Scripts to generate a syntehtic 3D multiple people dataset by compositing pictures.

Code/Data

A Generative Model of People in Clothing

Image Based Generative Model of People in Clothing. ICCV'17

Based on conditional variational autoencoder.

Models: trained models. They allow to generate semantic segmentation images of people and to "translate" a SMPL rendering into a photo-realistic image.
Training code (based on TensorFlow v1.1).
The training dataset is built on top of Chictopia10K. We provide processed annotations as well as the SMPL body model fit to the images.

Code/Data

Detailed, accurate, human shape estimation from clothed 3D scan sequences

BUFF dataset.

BUFF a unique dataset of 4D dynamic scans of people in clothing.

High quality ~9.000 4D scans of people in clothing
Ground truth 3D shape under clothing.
5 subjects, 3 male and 2 female
2 clothing styles: a) t-shirt and long pants and b) a soccer outfit.
3 different motions i) hips ii) tilt_twist_left iii) shoulders_mill.

Code/Data

Dynamic FAUST: Registering Human Bodies in Motion

DFAUST.

This dataset is a unique resource containing over 40,000 4D scans of multiple people; 4D means 3D scans over time. Processing 4D data is challenging, so we provide aligned data in which we have registered a common template mesh to all scans. This alignment process takes into account geometry and surface texture to make it accurate. The dataset includes the raw scan data, registered template meshes, and masks that say where the template mesh is sufficiently accurate to be considered ground truth.

Code/Data

Human Pose Estimation from Video and IMUs

Multimodal Motion Capture Dataset (TNT15). PAMI'16

video data: multi-view sequences obtained from 8 calibrated RGB-cameras.
silhouettes: binary segmented images obtained by background subtraction.
IMU data: orientation and acceleration data of 10 IMUs.
projection matrices: camera parameters of all 8 cameras.
meshes: 3D laser scans and registered meshes of each actor.

Code/Data

Dyna: A Model of Dynamic Human Shape in Motion

Dyna dataset. SIGGRAPH'15

The dataset contains the 40.000 registered dynamic meshes used to train Dyna. Sequences exhibit dynamic soft-tissue deformations.

Code/Data

SMPL: A Skinned Multi-Person Linear Model

SMPL model. SIGGRAPH Asia'15

3D body model of pose and shape. Learned from thousands of scans.
Useful for computer vision. Body shape, 2D/3D pose, etc.
Compatible with most graphics packages (FBX). Scripts to run it in Python, Maya, Unity.
Regular updates: dynamic blendshapes, model improvements, etc.

Code/Data