Hands-on AI based 3D Vision
Course: Summer Semester 2026
Continuous Learning of Multimodal Data Streams
University of Tuebingen
Description
The goal of this lecture is to learn the most recent AI 3D vision techniques for understanding and reconstructing the 3D world from videos and images. We will cover the fundamentals of multiple view geometry and quickly dive into the most recent methods covering Nerf, Gaussian Splats, 3D diffusion models and token based reconstruction methods.
There will be practical assignments. In contrast to other lectures on computer vision or computer graphics, which are of broader scope, the focus here will be on recent AI advances in the field of 3D computer vision.
Students will learn the mathematical tools and 3D vision techniques, from the fundamentals to the most recent AI based techniques. They will be able to apply the concepts in practice, develop and train models, reproduce research and conduct original research in this area.
Students will learn the mathematical tools and 3D vision techniques, from the fundamentals to the most recent AI based techniques. They will be able to apply the concepts in practice, develop and train models, reproduce research and conduct original research in this area.
Organization
This course is worth 6 ECTS points. There will be one lecture and one tutorial session per week.
The TAs are:
Prerequisites
Basic knowledge of machine learning, linear algebra, probability theory, optimization, and programming skills in Python are required. Having experience with PyTorch, and knowledge of computer graphics concepts (geometry representation, rendering) is a plus.Location
- Lectures: Wed, 10-12am, A1 (A-206) Cyber Valley Campus, MvL 1
- Tutorials: Tue, 2-4pm, TTR2 Cyber Valley Campus, MvL 6
Assignments and Project
There will be 3 bi-weekly assignments and a research project. Both are mandatory and will be taken into account for the final grade (see below).The assignments will contain both theoretical and programming exercises. They will be evaluated based on correctness and clarity.
The projects will be done in groups of 2 over a duration of ~6 weeks. We will evaluate the projects based on
- Completeness of report, including motivation, prior work, methodology, evaluation, limitations, discussion
- Whether the developed methods are substantial
- Robustness of the final result in the scale and scope it was developed
| Event | Date |
|---|---|
| Assignment 1 Release | 22.04 |
| Assignment 1 Deadline | 06.05 |
| Assignment 2 Release | 06.05. |
| Assignment 2 Deadline | 20.04 |
| Assignment 3 Release | 20.05. |
| Assignment 3 Deadline | 17.06. |
| Project Proposals | 16.06. |
| Project Final Presentation | tbd |
| Deliverables Deadline | 31.07. |
Exam and Grading
There will be written exams at the end of the semester. See the exam dates and locations below.The final grade will be a weighted combination of exercises (20%), project (30%) and exam (50%).
To pass the course, you need to pass exercises, project and exam.
| Exam | Date | Time | Location |
|---|---|---|---|
| 01 | 29.07. | 10:00-12:00 | A2 (MvL1) |
| 02 | 23.09. | 10:00-12:00 | TTR2 (MvL6) |
Schedule for Lectures
The lecture happens every Wednesday 10-12am. Location A1 (A-206) Cyber Valley Campus, MvL 1.| Lecture | Date & Time | Title | Material |
|---|---|---|---|
| 01 | 15.04. | Introduction to 3D Computer Vision | pdf Orga Intro | pptx Orga Intro |
| 02 | 22.04. | Image Formation and Rotations | pdf 1 2 | pptx 1 2 |
| 03 | 29.04. | Classical 3D Reconstruction Techniques | pdf Classical | pptx Classical |
| 04 | 06.05. | Stereo Vision and Depth Estimation | pdf Stereo | pptx Stereo |
| 05 | 13.05. | Surface reconstruction and Procrustes alignment | pdf Surface | pptx Surface |
| 06 | 20.05. | Neural Fields and Point Based Representations | pdf 01 02 | pptx 01 02 |
| 07 | 10.06. | Neural Radiance Fields (NeRF) | pdf NeRF | pptx NeRF |
| 08 | 17.06. | Gaussian Splatting | pdf Gaussian Splatting | pptx Gaussian Splatting |
| 09 | 24.06. | Advanced Methods in Learning-Based 3D Reconstruction | pdf Learning Based | pptx Learning Based |
| 10 | 01.07. | Diffusion Models | pdf Diffusion Models | pptx Diffusion Models |
| 11 | 08.07. | Generative Models cont. | pdf Generative Models | pptx Generative Models |
| 12 | 15.07. | Recap |
Schedule for Tutorials
The tutorial happens every Tuesday 2-4pm. Location TTR2 Cyber Valley Campus, MvL 6.| Date | Material |
|---|---|
| Apr 28th | Tutorial 1 |
| May 12th | Tutorial 2 |
| May 19th | TCML Intro |
| June 16th | Project Info Proposals |