ML4508: Virtual Humans

Course: Winter Semester 2025

This course will not be offered in WS25

It will again be offered in WS26

Continuous Learning of Multimodal Data Streams

University of Tuebingen

appearance hps

Description

A virtual human is a digital representation of a real human. Virtual humans (VH) should look, move and eventually think like real humans. Building such VH is one of the long standing goals of Artificial Intelligence. Learning them requires techniques and algorithms at the intersection of Machine Learning, Computer Vision and Computer Graphics. In this course, we will cover the key mathematical foundations and computational tools to learn VH from 3D scans, images and video of real humans. The course will cover classical representations of humans based on 3D meshes and textures, as well as modern ones where the appearance and behavior of virtual humans are encoded in neural networks.
The following topics will be covered in the course:
  • Body and Clothing models.
  • Rotations, surface paramterization etc.
  • Neural Implicits and NeRF based human models.
  • Human Behaviour capture and synthesis.

Organization

Course module number is ML4508. This course is worth 9 ECTS points. There will be one lecture and one tutorial session per week. The TAs are:

Prerequisites

Basic knowledge of linear algebra, probability theory, optimization, and programming skills in Python are required. Having experience with PyTorch, and knowledge of computer graphics concepts (geometry representation, rendering) is a plus.

Location


Assignments

There will be 3 bi-weekly assignments and a research project.

Exam

The final exams will take place on-site in Tuebingen, and you need to be physically present. There is going to be one exam at the beginning of the semester break and one at the end of the semester break. The exam format will be determined by the number of students attending the course (written in case of many people attending or oral in case of smaller attendance). The format will be announced at the start of the course.
Exam date: TBD, Second exam: TBD. Location: TBD.


Schedule for Lectures

The lecture happens every Wednesday 12-14pm. Depends on the availability, lectures will take place on Sand (Room C118a).
# Date & Time Location Title Links
01: Oct. 15, 1200-1400 MVL1 Organization and Introduction to Body Models
02: Oct. 22, 1200-1400 MVL1 Image formations and Rotations
03: Oct. 29, 1200-1400 MVL1 Surface Representations and Procrustes alignment
04: Nov. 5, 1200-1400 MVL1 ICP, Non rigid alignment & vertex based models
05: Nov. 12, 1200-1400 MVL1 Fitting SMPL to scans
06: Nov. 19, 1200-1400 MVL1 Fitting SMPL to images
07: Nov. 26, 1200-1400 MVL1 Fitting SMPL to IMUs
08: Dec. 3, 1200-1400 MVL1 Vertex based Clothing Models
09: Dec. 10, 1200-1400 MVL1 Neural Implicits and Point based Clothing Models
10: Dec. 17, 1200-1400 MVL1 NeRF, Gaussian Splatting and Humans
11: Jan. 7, 1200-1400 MVL1 Human behaviour capture
12: Jan. 14, 1200-1400 MVL1 Human behaviour synthesis
13: Jan. 21, 1200-1400 MVL1 Diffusion model and human motion, reconstruction
14: Jan. 28, 1200-1400 MVL1 Diffusion model for 3D reconstruction


Schedule for Tutorials

The tutorials will happen every Tuesday 16-18pm in Room A-501, Floor 5 / 3. OG, Maria-von-Linden Strasse 1.
# Date & Time Location Title
01: Oct. 21, 1600-1800 MvL1 Introduction, assignment 01 release
02: Oct. 28, 1600-1800 MvL1 TBD
03: Nov. 4, 1600-1800 MvL1 Assignment 02 release
04: Nov. 11, 1600-1800 MvL1 TBD
05: Nov. 18, 1600-1800 MvL1 Assignment 03 release
06: Nov. 25, 1600-1800 MvL1 TBD
07: Dec. 2, 1600-1800 MvL1 Project topics chosen
08: Dec. 9, 1600-1800 MvL1 Individual project discussion
09: Dec. 16, 1600-1800 MvL1 Project: midterm presentation
10: Jan. 13, 1600-1800 - Individual project discussion
11: Jan. 20, 1600-1800 - Individual project discussion
12: Jan. 27, 1600-1800 MvL1 Project presentations


Registration

Contact

Prof. Dr. Gerard Pons-Moll (mail).
Yannan He (mail).
Eyvaz Najafli (mail).
Chuqiao Li (mail).