Virtual Humans
Course: Winter Semester 2023
Continuous Learning of Multimodal Data Streams
University of Tuebingen
|
|
Description
A virtual human is a digital representation of a real human. Virtual humans (VH) should look, move and eventually think like real humans.
Building such VH is one of the long standing goals of Artificial Intelligence. Learning them requires techniques and algorithms at the
intersection of Machine Learning, Computer Vision and Computer Graphics. In this course, we will cover the key mathematical foundations
and computational tools to learn VH from 3D scans, images and video of real humans. The course will cover classical representations of
humans based on 3D meshes and textures, as well as modern ones where the appearance and behavior of virtual humans are encoded in neural networks.
The following topics will be covered in the course:
The following topics will be covered in the course:
- Body and Clothing models.
- Rotations, surface paramterization etc.
- Neural Implicits and NeRF based human models.
- Human Behaviour capture and synthesis.
Organization
Course module number is ML4508. This course is worth 9 ECTS points. There will be one lecture and one tutorial session per week.
The TAs and their office hours are:
- Riccardo Marin (mail) [Wednesday, 16.30-17.30 (Maria-von-Linden Strasse 6, 2nd floor, 20-8/A14)]
- Xianghui Xie (mail) [Friday, 16.00-17.00 (Maria-von-Linden Strasse 6, 2nd floor, 20-31/A12)]
- Ilya Petrov (mail) [Thursday, 16.00-17.00 (Maria-von-Linden Strasse 6, 2nd floor, 20-7/A18)]
Prerequisites
Basic knowledge of linear algebra, probability theory, optimization, and programming skills in Python are required. Having experience with PyTorch, and knowledge of computer graphics concepts (geometry representation, rendering) is a plus.Location
- Lectures: Lecture Hall A301, Sand 1 (the lectures might be moved to Lecture Hall (ground floor), Maria-von-Linden Strasse 6 based on its availability, the place for the lecture will be announced 1 week before on the website and ILIAS).
- Tutorials: Lecture Hall (ground floor), Maria-von-Linden Strasse 6
Assignments
There will be 3 bi-weekly assignments and a research project.Exam
The final exams will take place on-site in Tuebingen, and you need to be physically present. There is going to be one exam at the beginning of the semester break and one at the end of the semester break. The exam format will be determined by the number of students attending the course (written in case of many people attending or oral in case of smaller attendance). The format will be announced at the start of the course.Tentative date for the exam: 26.02.2024, 10.00-12.00 at Lecture Hall (ground floor) Maria-von-Linden Strasse 6.
Schedule for Lectures
| # | Date & Time | Location | Title | Links |
|---|---|---|---|---|
| 01: | Oct. 25, 800-1000 | A301, Sand 1 | Organization and Introduction to Body Models | YouTube | pdf 1 2 3 | pptx 1 2 3 |
| 02: | Nov. 08, 800-1000 | Lect. Hall, MvL 6 | Image formations and Rotations | YouTube | pdf 1 2 | pptx 1 2 |
| 03: | Nov. 15, 800-1000 | A301, Sand 1 | Surface Representations and Procrustes alignment | YouTube | pdf 1 2 | pptx 1 2 |
| 04: | Nov. 22, 800-1000 | Lect. Hall, MvL 6 | ICP, Non rigid alignment & vertex based models | YouTube | pdf 1 2 | pptx 1 2 |
| 05: | Nov. 29, 800-1000 | Lect. Hall, MvL 6 | Fitting SMPL to scans | YouTube | pdf 1 2 | pptx 1 2 |
| 06: | Dec. 06, 800-1000 | Lect. Hall, MvL 6 | Fitting SMPL to images | YouTube | pdf 1 2 | pptx 1 2 |
| 07: | Dec. 13, 800-1000 | Lect. Hall, MvL 6 | Fitting SMPL to IMUs | YouTube | pdf 1 2 | pptx 1 2 |
| 08: | Dec. 18, 1600-1800 | Lect. Hall, MvL 6 | Vertex based Clothing Models | YouTube | pdf 1 | pptx 1 |
| 09: | Dec. 20, 800-1000 | Lect. Hall, MvL 6 | Neural Implicits and Point based Clothing Models | YouTube | pdf 1 | pptx 1 |
| 10: | Jan. 10, 800-1000 | Lect. Hall, MvL 6 | Humans and NeRFs | YouTube | pdf 1 2 | pptx 1 1 |
| 11: | Jan. 24, 800-1000 | Lect. Hall, MvL 6 | Human behaviour capture | YouTube | pdf 1 2 | pptx 1 2 |
| 12: | Jan. 31, 800-1000 | Lect. Hall, MvL 6 | Human behaviour synthesis | YouTube | pdf 1 | pptx 1 |
| 13: | Feb. 06, 900-1000 | Lect. Hall, MvL 6 | Human motion synthesis and mock exam | | pdf 1 | pptx 1 |
Schedule for Tutorials
| # | Date & Time | Location | Title |
|---|---|---|---|
| 01: | Oct. 26, 1415-1600 | Lect. Hall, MvL 6 | Organization and Introduction to Body Models |
| 02: | Nov. 09, 1415-1600 | Lect. Hall, MvL 6 | Image formations and Rotations |
| 03: | Nov. 16, 1415-1600 | Lect. Hall, MvL 6 | Surface Representations and Procrustes alignment |
| 04: | Nov. 23, 1415-1600 | Lect. Hall, MvL 6 | ICP, Non rigid alignment and vertex based body models |
| 05: | Nov. 30, 1415-1600 | Lect. Hall, MvL 6 | Fitting SMPL to scans |
| 06: | Dec. 07, 1415-1600 | Lect. Hall, MvL 6 | Fitting SMPL to images |
| 07: | Dec. 14, 1415-1600 | Lect. Hall, MvL 6 | Fitting SMPL to IMUs |
| 08: | Dec. 21, 1415-1600 | Lect. Hall, MvL 6 | Vertex based Clothing Models |
| 09: | Jan. 11, 1415-1600 | Lect. Hall, MvL 6 | Neural Implicits and Point based Clothing Models |
| 10: | Jan. 18, 1415-1600 | - | Individual project discussion |
| 11: | Jan. 25, 1415-1600 | - | Individual project discussion |
| 12: | Feb. 01, 1415-1600 | Lect. Hall, MvL 6 | Project presentations |
Registration
- Please register at ILIAS.