Hands-on AI based 3D Vision
Course: Summer Semester 2026
Continuous Learning of Multimodal Data Streams
University of Tuebingen
Description
The goal of this lecture is to learn the most recent AI 3D vision techniques for understanding and reconstructing the 3D world from videos and images. We will cover the fundamentals of multiple view geometry and quickly dive into the most recent methods covering Nerf, Gaussian Splats, 3D diffusion models and token based reconstruction methods.
There will be practical assignments. In contrast to other lectures on computer vision or computer graphics, which are of broader scope, the focus here will be on recent AI advances in the field of 3D computer vision.
Students will learn the mathematical tools and 3D vision techniques, from the fundamentals to the most recent AI based techniques. They will be able to apply the concepts in practice, develop and train models, reproduce research and conduct original research in this area.
Students will learn the mathematical tools and 3D vision techniques, from the fundamentals to the most recent AI based techniques. They will be able to apply the concepts in practice, develop and train models, reproduce research and conduct original research in this area.
Organization
This course is worth 6 ECTS points. There will be one lecture and one tutorial session per week.
The TAs are:
Prerequisites
Basic knowledge of machine learning, linear algebra, probability theory, optimization, and programming skills in Python are required. Having experience with PyTorch, and knowledge of computer graphics concepts (geometry representation, rendering) is a plus.Location
- Lectures: Wed, 10-12am, A1 (A-206) Cyber Valley Campus, MvL 1
- Tutorials: Tue, 2-4pm, TTR2 Cyber Valley Campus, MvL 6
Assignments and Project
There will be 3 bi-weekly assignments and a research project. Both are mandatory and will be taken into account for the final grade (see below).The assignments will contain both theoretical and programming exercises. They will be evaluated based on correctness and clarity.
The projects will be done in groups of 2 over a duration of ~6 weeks. We will evaluate the projects based on
- Completeness of report, including motivation, prior work, methodology, evaluation, limitations, discussion
- Whether the developed methods are substantial
- Robustness of the final result in the scale and scope it was developed
| Event | Date |
|---|---|
| Assignment 1 Release | 22.04 |
| Assignment 1 Deadline | 06.05 |
| Assignment 2 Release | 06.05. |
| Assignment 2 Deadline | 20.04 |
| Assignment 3 Release | 20.05. |
| Assignment 3 Deadline | 17.06. |
| Project Proposals | 17.06. |
| Project Final Presentation | 22.07. |
| Deliverables Deadline | 22.07. |
Exam and Grading
The final exams will take place on-site in Tuebingen, and you need to be physically present. There is going to be one exam at the beginning of the semester break and one at the end of the semester break. The exam format will be determined by the number of students attending the course (written in case of many people attending or oral in case of smaller attendance). The format will be announced at the start of the course.The final grade will be a weighted combination of exercises (20%), project (30%) and exam (50%).
To pass the course, you need to pass exercises, project and exam.
Schedule for Lectures
The lecture happens every Wednesday 10-12am. Location A1 (A-206) Cyber Valley Campus, MvL 1.| Lecture | Date & Time | Title | Material |
|---|---|---|---|
| 01 | 15.04. | Introduction to 3D Computer Vision | pdf Orga Intro | pptx Orga Intro |
| 02 | 22.04. | Image Formation and Rotations | pdf 1 2 | pptx 1 2 |
| 03 | 29.04. | Classical 3D Reconstruction Techniques | pdf Classical | pptx Classical |
| 04 | 06.05. | Stereo Vision and Depth Estimation | pdf Stereo | pptx Stereo |
| 05 | 13.05. | Surface reconstruction and Procrustes alignment | pdf Surface | pptx Surface |
| 06 | 20.05. | Neural Fields and Point Based Representations | |
| 07 | 10.06. | Neural Radiance Fields (NeRF) | |
| 08 | 17.06. | Gaussian Splatting | |
| 09 | 24.06. | Advanced Methods in Learning-Based 3D Reconstruction | |
| 10 | 01.07. | Generative Models | |
| 11 | 08.07. | Generative Models cont. | |
| 12 | 15.07. | Recap |
Schedule for Tutorials
The tutorial happens every Tuesday 2-4pm. Location TTR2 Cyber Valley Campus, MvL 6.| Date | Material |
|---|---|
| Apr 28th | Tutorial 1 |