BEHAVE Dataset

Data License

For non-commercial and scientific research purposes

Please read carefully the following terms and conditions and any accompanying documentation before you download and/or use the BEHAVE data (the "Data"). By downloading and/or using the Data, you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the Data.


The Data has been developed at the Max Planck Institute and is owned by and proprietary material of the Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (hereinafter “MPG”; MPI and MPG hereinafter collectively “Max-Planck”).

License Grant

Max-Planck grants you a non-exclusive, non-transferable, free of charge right: To install the Data on computers owned, leased or otherwise controlled by you and/or your organisation; To use the Data for the sole purpose of performing non-commercial scientific research. Any other use, in particular any use for commercial purposes, is prohibited. This includes, without limitation, incorporation in a commercial product, use in a commercial service, or production of other artefacts for commercial purposes including, for example, 3D models, movies, or video games. The Data may not be reproduced, modified and/or made available in any form to any third party without Max-Planck’s prior written permission.

Usage for publication

In case the images are used for publication or public presentations, you are required to blur all human faces.

Disclaimer of Representations and Warranties

You expressly acknowledge and agree that the Data results from basic research, is provided “AS IS”, may contain errors, and that any use of the Data is at your sole risk. MAX-PLANCK MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE CODE, NEITHER EXPRESS NOR IMPLIED, AND THE ABSENCE OF ANY LEGAL OR ACTUAL DEFECTS, WHETHER DISCOVERABLE OR NOT. Specifically, and not to limit the foregoing, Max-Planck makes no representations or warranties (i) regarding the merchantability or fitness for a particular purpose of the Data, (ii) that the use of the Data will not infringe any patents, copyrights or other intellectual property rights of a third party, and (iii) that the use of the Data will not cause any damage of any kind to you or a third party.

Limitation of Liability

Under no circumstances shall Max-Planck be liable for any incidental, special, indirect or consequential damages arising out of or relating to this license, including but not limited to, any lost profits, business interruption, loss of programs or other data, or all other commercial damages or losses, even if advised of the possibility thereof.

No Maintenance Services

You understand and agree that Max-Planck is under no obligation to provide either maintenance services, update services, notices of latent defects, or corrections of defects with regard to the Data. Max-Planck nevertheless reserves the right to update, modify, or discontinue the Data at any time.

Publication with BEHAVE

You agree to cite the corresponding CVPR'22 paper when reporting results with this dataset. This website lists the most up to date bibliographic information on the front page.


If you use this dataset, you agree to cite the corresponding CVPR'22 paper:

    title = {BEHAVE: Dataset and Method for Tracking Human Object Interactions},
    author={Bhatnagar, Bharat Lal and Xie, Xianghui and Petrov, Ilya and Sminchisescu, Cristian and Theobalt, Christian and Pons-Moll, Gerard},
    booktitle = {{IEEE} Conference on Computer Vision and Pattern Recognition (CVPR)},
    month = {jun},
    organization = {{IEEE}},
    year = {2022},


By downloading the Data, you agree to the license terms. The full dataset contains multi-view RGB-D images paired with SMPL and object registrations. Details about the dataset structure and usage are described in the demo repository.
  • Demo repository

  • Download datasets:
  • Scanned objects
  • Calibration files
  • Train and test split
  • Sequences separated by dates (in total ~140GB):
  • Date01 sequences
  • Date02 sequences
  • Date03 sequences
  • Date04 sequences
  • Date05 sequences
  • Date06 sequences
  • Date07 sequences
  • After downloading all the sequences, run unzip "Date*.zip" -d sequences to extract all sequence files. Please refer to the github repo for more details about the dataset structure.

    Raw videos, we provide the RGB videos and registered depth (i.e. same resolution with color image) videos:
  • Date01: color videos (5.5GB), depth videos (73.6GB), frame timestamps
  • Date02: color videos (5.4GB), depth videos (46.4GB), frame timestamps
  • Date03: color videos (16.5GB), depth part1 (60.8GB), depth part2 (59.7GB), depth part3 (49.5GB), depth part4 (0.5GB), frame timestamps
  • Date04: color videos (6.5GB), depth videos (69.5GB), frame timestamps
  • Date05: color videos (6.8GB), depth videos (86.1GB), frame timestamps
  • Date06: color videos (6.0GB), depth videos (69.2GB), frame timestamps
  • Date07: color videos (10.7GB), depth part1 (92.6GB), depth part2 (60.8GB), depth part3 (0.5GB), frame timestamps

  • Annotations at 30fps:
  • SMPL and object parameters . Interpretation of these parameters can be found in our repo.

  • Most sequences have accurate alignment between registration and point clouds as well as RGB images. For a few seuqencses where object rigistration is very difficult, there might be frames that do not align perfectly. These frames are found empirically by computing the vertex to vertex distance between two consecutive frames. We provide a reference for the possible error frames here.
    Human and object segmentation masks:
  • part1 (~7GB)
  • part2 (~10GB)
  • part3 (~10GB)
  • part4 (~7GB)

  • BEHAVE challenges data:
  • Training sequences: part 1 (3.4G) , part 2 (6.9G) , part 3 (5.1G)
  • Test inputs: all input images and masks
  • Link to the challenges: Human reconstruction track, Object 6D pose estimation track, Joint human object reconstruction track