Hands-on AI based 3D Vision
Course: Summer Semester 2025
Continuous Learning of Multimodal Data Streams
University of Tuebingen
Description
The goal of this lecture is to learn the most recent AI 3D vision techniques for understanding and reconstructing the 3D world from videos and images. We will cover the fundamentals of multiple view geometry and quickly dive into the most recent methods covering Nerf, Gaussian Splats, 3D diffusion models and token based reconstruction methods.
There will be practical assignments. In contrast to other lectures on computer vision or computer graphics, which are of broader scope, the focus here will be on recent AI advances in the field of 3D computer vision.
Students will learn the mathematical tools and 3D vision techniques, from the fundamentals to the most recent AI based techniques. They will be able to apply the concepts in practice, develop and train models, reproduce research and conduct original research in this area.
Students will learn the mathematical tools and 3D vision techniques, from the fundamentals to the most recent AI based techniques. They will be able to apply the concepts in practice, develop and train models, reproduce research and conduct original research in this area.
Organization
This course is worth 6 ECTS points. There will be one lecture and one tutorial session per week.
The TAs are:
- Niklas Berndt (mail)
- Andrea Sanchietti (mail)
- Eyvaz Najafli (mail)
Prerequisites
Basic knowledge of machine learning, linear algebra, probability theory, optimization, and programming skills in Python are required. Having experience with PyTorch, and knowledge of computer graphics concepts (geometry representation, rendering) is a plus.Location
- Lectures: A104 Sand
- Tutorials: A104 Sand
Assignments and Project
There will be 3 bi-weekly assignments and a research project. Both are mandatory and will be taken into account for the final grade (see below).The assignments will contain both theoretical and programming exercises. They will be evaluated based on correctness and clarity.
The projects will be done in groups of 2 over a duration of ~6 weeks. We will evaluate the projects based on
- Completeness of report, including motivation, prior work, methodology, evaluation, limitations, discussion
- Whether the developed methods are substantial
- Robustness of the final result in the scale and scope it was developed
| Event | Date |
|---|---|
| Assignment 1 Release | April 29th |
| Assignment 1 Deadline | May 13th, 12:00 |
| Assignment 2 Release | May 13th |
| Assignment 2 Deadline | May 27th, 12:00 |
| Assignment 3 Release | May 27th |
| Assignment 3 Deadline | June 10th, 12:00 |
| Project Proposals | June 10th |
| Project Deadline | July 22, 00:00 |
| Project Final Presentation | tbd |
Exam and Grading
The final exams will take place on-site in Tuebingen, and you need to be physically present. There is going to be one exam at the beginning of the semester break and one at the end of the semester break. The exam format will be determined by the number of students attending the course (written in case of many people attending or oral in case of smaller attendance). The format will be announced at the start of the course.The final grade will be a weighted combination of exercises (20%), project (30%) and exam (50%).
To pass the course, you need to pass exercises, project and exam.
Schedule for Lectures
The lecture happens every Tuesday 12-2pm. Location A104 Sand.| Lecture | Lecture Date & Time | Lecture Title | Material |
|---|---|---|---|
| Lecture 01 | April 15, 12-14PM | Introduction to 3D Computer Vision | pdf Orga Intro | pptx Orga Intro |
| Lecture 02 | April 29, 12-14PM | Camera Models and Coordinate Systems | tbc |
| Lecture 03 | May 06, 12-14PM | Classical 3D Reconstruction Techniques | tbc |
| Lecture 04 | May 13, 12-14PM | Depth Estimation | tbc |
| Lecture 05 | May 20, 12-14PM | Surface reconstruction and Procrustes alignment | tbc |
| Lecture 06 | May 27, 12-14PM | Neural Fields and Point Based Representations | tbc |
| Lecture 07 | June 03, 12-14PM | Neural Radiance Fields (NeRF) | tbc |
| Lecture 08 | June 10, 12-14PM | Gaussian Splatting and Point Clouds | tbc |
| Lecture 09 | June 17, 12-14PM | Advanced Methods in Learning-Based 3D Reconstruction | tbc |
| Lecture 10 | June 24, 12-14PM | Generative models | tbc |
| Lecture 11 | July 01, 12-14PM | SDS based methods | tbc |
| Lecture 12 | July 15, 12-14PM | Recap of concepts | tbc |
Schedule for Tutorials
The tutorial happens every Tuesday 2-4pm. Location A104 Sand.| Date & Time | Material |
|---|---|
| April 29, 14-16PM | tbc |
| May 06, 14-16PM | tbc |
| May 13, 14-16PM | tbc |
| May 20, 14-16PM | tbc |
| May 27, 14-16PM | tbc |
| June 03, 14-16PM | tbc |
| June 10, 14-16PM | tbc |
| June 17, 14-16PM | tbc |
| June 24, 14-16PM | tbc |
| July 01, 14-16PM | tbc |
| July 15, 14-16PM | tbc |