Virtual Humans

Course: Winter Semester 2022

Continuous Learning of Multimodal Data Streams

University of Tuebingen

appearance hps

Description

A virtual human is a digital representation of a real human. Virtual humans (VH) should look, move and eventually think like real humans. Building such VH is one of the long standing goals of Artificial Intelligence. Learning them requires techniques and algorithms at the intersection of Machine Learning, Computer Vision and Computer Graphics. In this course, we will cover the key mathematical foundations and computational tools to learn VH from 3D scans, images and video of real humans. The course will cover classical representations of humans based on 3D meshes and textures, as well as modern ones where the appearance and behavior of virtual humans are encoded in neural networks.
Tentatively, the following topics will be covered in the course:
  • Body and Clothing models.
  • Rotations, surface paramterization etc.
  • Neural Implicits and NeRF based human models.
  • Human Behaviour capture and synthesis.

Organization

Course module number is ML4508. This course is worth 6 ECTS points. There will be one lecture and one tutorial session per week. The TAs and their office hours are:

  • Bharat Lal Bhatnagar [Friday, 15.00-16.00 (Virtual) ]
  • Ilya Petrov [Monday, 13.30-14.30 (Maria von LIndenStrasse-6, 2nd floor, 20-7/A18)]
  • Garvita Tiwari [Wednesday, 14.30-15.30 (Maria von LIndenStrasse-6, 2nd floor, 20-28/A15)]

Prerequisites

Knowledge of linear algebra, probability theory and programming skills are required.

Location


Assignments

There will be bi-weekly assignments.

Exam

The final exams will take place on-site in Tuebingen, and you need to be physically present. There is going to be one exam at the beginning of the semester break and one at the end of the semester break.


Tentative Schedule for Lecture

Lecture Date & Time Title
Lecture 01 November 02, 10-12AM Organization and Introduction to Body Models (Assignment 01)
Lecture 02* November 09, 12-14PM Image formations and Rotations
Lecture 03 November 16, 10-12AM Surface Representations and Procrustes alignment(Assignment 02)
Lecture 04 November 23, 10-12AM ICP, Non rigid alignment and vertex based body models
Lecture 05* December 05, 15-17PM Fitting SMPL to scans (Assignment 03)
Lecture 06 December 07, 10-12AM Fitting SMPL to images
Lecture 07* December 14, 12-14PM Fitting SMPL to IMUs
Lecture 08 December 21, 10-12AM Vertex based Clothing Models
Lecture 09 January 11, 12-14PM Neural Implicits and Point based Clothing Models
Lecture 10 January 18, 10-12AM Humans and NeRFs
Lecture 11* January 25, 10-12AM Human behaviour capture
Lecture 12 February 01, 10-12AM Human behaviour synthesis

Room: 20-10/A 12, Second Floor, Maria-von-Linden Strasse 6
*Tutorials/practicals: 12-14PM in 20-10/A 12, Second Floor, Maria-von-Linden Strasse 6 after the lecture, except for November 09, December 14 and January 11 in which tutorials will be from 10-12 AM before the lecture.


Registration

  • Please register at ILIAS.

Contact

Prof. Dr. Gerard Pons-Moll (mail).
Bharat Lal Bhatnagar (mail).
Ilya Petrov (mail).
Garvita Tiwari (mail).