• 25.08.2023: [New] I am organising a seminar on Quantum Computer Vision and Machine Learning (QCVML) (3CP) in WiSe 23/24.
• 18.08.2023: [New] We are organising a lecture on Advanced Topics in Neural Rendering and 3D Reconstruction (3CP) in WiSe 23/24.
• 03.03.2023: Five papers at CVPR 2023 in Vancouver. Many thanks to all students, interns and collaborators!
  • EventNeRF: Neural Radiance Fields from a Single Colour Event Camera.
  • MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis (CVPR Highlight)
  • Self-supervised Pre-training with Masked Shape Prediction for 3D Scene Understanding.
  • Quantum Multi-Model Fitting (CVPR Highlight).
  • CCuantuMM: Cycle-Consistent Quantum-Hybrid Matching of Multiple Shapes.
• 31.01.2023: We will organise the CVMLCG Seminar 2023.
• 21.01.2023: A paper on quantum annealing with machine learning is accepted at ICLR 2023 as Spotlight.
• 22.12.2022: A state of the art report (STAR) conditionally accepted at EUROGRAPHICS 2023.
• 16.12.2022: We are going to organise a Workshop on Quantum Computer Vision and Machine Learning at CVPR 2023; in collaboration with Tongyang Li, Jan-Nico Zaech, Martin Danelljan, Jacob Biamonte and Tolga Birdal. More details coming soon.
• 13.12.2022: Two articles conditionally accepted at EUROGRAPHICS 2023.
• 08.12.2022: V. Golyanik gave a talk titled Advances in Quantum Computer Vision at the Workshop on Quantum Information at Saarland University (event poster). The slides can be found here. Many thanks to the organisers for making such an event happen!
• 21.10.2022: An article on random number generation on quantum annealers accepted at IEEE Access.
• 05.10.2022: The source code of MoCapDeform is now available here: GitHub link.
• 30.09.2022: A paper on high-fidelity 3D human performance capture accepted at BMVC 2022.
• 12.09.2022: MoCapDeform received Best Student Paper Award at 3DV 2022.
• 05.08.2022: A paper on global 3D human motion capture (MoCapDeform) accepted at 3DV 2022.
• 25.07.2022: 4DQV will present several papers at ECCV 2022:
  • UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture.
  • Quantum Motion Segmentation.
  • HULC: 3D HUman Motion Capture with Pose Manifold Sampling and Dense Contact Guidance.
  • Neural Radiance Fields for Outdoor Scene Relighting.
• 04.07.2022: The source code of Physical Inertial Poser (PIP) is now available here: GitHub link.
• 27.06.2022: The source code of φ-SfT ist now available here: GitHub link.
• 21.04.2022: The source code of Neural PhysCap is now available via GitHub.
• 19.02.2022: We are co-organising a tutorial associated with our STAR on Advances in Neural Rendering on the 27th of April at EUROGRAPHICS'22.
• 28.03.2022: 4DQV will present three papers at CVPR 2022:
  • φ-SfT: Shape-from-Template with a Physics-Based Deformation Model.
  • Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensors.
  • Playable Environments: Video Manipulation in Space and Time.
• 11.02.2022: State of the art report on neural rendering (the updated version) accepted at EUROGRAPHICS'22.
• 25.10.2021: The source code of Convex Joint Graph Matching and Clustering via Semidefinite Relaxations is released.
• 18.10.2021: An article is accepted at TPAMI (HandVoxNet++).
• 17.10.2021: Two papers accepted at 3DV 2021 (about graph matching and clustering and human image generation).
• 13.10.2021: EventHands is mentioned in ICCV 2021 Daily.
• 12.10.2021: The source code of GraviCap is released on GitHub
• 11.10.2021: The source code of Q-Match is released on GitHub.
• 27.09.2021: The source code of EventHands is released on GitHub.
• 08.09.2021: Dennis Willsch gave a 4DQV Lecture Series talk titled Applications on Quantum Annealers at Jülich Supercomputing Centre. The video can be found here.
• 19.08.2021: We are going to present the following works at ICCV 2021: GraviCap, Non-Rigid Neural Radiance Fields, Iterative Shape Matching via Quantum Annealing and EventHands.
• 05.07.2021: The source code of PhysCap is now available here.

State of the Art in Dense Monocular Non-Rigid 3D Reconstruction. E. Tretschk*, N. Kairanda*, M. B R, R. Dabral, A. Kortylewski, B. Egger, M. Habermann, P. Fua, C. Theobalt and and V. Golyanik. * equal contribution. Eurographics 2023 (Full STARs). [draft] [project page] [bibtex]

This survey focuses on state-of-the-art methods for dense non-rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views. It reviews the fundamentals of 3D reconstruction and deformation modeling from 2D image observations. We then start from general methods—that handle arbitrary scenes and make only a few prior assumptions—and proceed towards techniques making stronger assumptions about the observed objects and types of deformations (e.g. human faces, bodies, hands, and animals). A significant part of this STAR is also devoted to classification and a high-level comparison of the methods, as well as an overview of the datasets for training and evaluation of the discussed techniques. We conclude by discussing open challenges in the field and the social aspects associated with the usage of the reviewed methods.

Advances in Neural Rendering. A. Tewari*, J. Thies*, B. Mildenhall*, P. Srinivasan*, E. Tretschk, Y. Wang, C. Lassner, V. Sitzmann, R. Martin-Brualla, S. Lombardi, C. Theobalt, M. Niessner, J. T. Barron, G. Wetzstein, M. Zollhöfer and V. Golyanik. * equal contribution. State of the Art Report at Eurographics 2022. [paper] [project page] [bibtex]

This state-of-the-art report on advances in neural rendering focuses on methods that combine classical rendering principles with learned 3D scene representations, often now referred to as neural scene representations. A key advantage of these methods is that they are 3D-consistent by design, enabling applications such as novel viewpoint synthesis of a captured scene. In addition to methods that handle static scenes, we cover neural scene representations for modeling nonrigidly deforming objects and scene editing and composition. While most of these approaches are scene-specific, we also discuss techniques that generalize across object classes and can be used for generative tasks. In addition to reviewing these state-ofthe-art methods, we provide an overview of fundamental concepts and definitions used in the current literature. We conclude with a discussion on open challenges and social implications.

• Quantum Multi-Model Fitting, Farina et al., CVPR 2023.
• EventNeRF, Rudnev et al., CVPR 2023.
• CCuantuMM, Bhatia et al., CVPR 2023.
• Unbiased 4D, Johnson et al., CVPRW 2023.
• QuAnt, Seelbach Benkner et al., ICLR 2023.
• MHMC, Luvizon et al., EUROGRAPHICS 2023.
• IMoS, Ghosh et al., EUROGRAPHICS 2023.
• QRNG, Bhatia et al., IEEE Access.
• MoCapDeform, Li et al., 3DV 2022.
• UnrealEgo, Akada et al., ECCV 2022.
• QuMoSeg, Arrigoni et al., ECCV 2022.
• NeRF-OSR, Rudnev et al., ECCV 2022.
• Physical Inertial Poser (PIP), Yi et al., CVPR 2022.
• φ-SfT: Shape-from-Template with a Physics-Based Deformation Model., Kairanda et al., CVPR 2022.
• Playable Environments: Video Manipulation in Space and Time, Menapace et al., CVPR 2022.
• Joint Graph Matching and Clustering, Krahn et al., 3DV 2021.
• Neural PhysCap, Shimada et al., SIGGRAPH 2021.
• Non-Rigid NeRF, Tretschk et al., ICCV, 2021.
• EventHands, Rudnev et al., ICCV 2021.
• Q-Match, Seelbach Benkner et al., ICCV 2021.
• GraviCap, Dabral et al., ICCV 2021.
• Human Motion Transfer, Kappel et al., CVPR 2021.
• Differentiable Event Stream Simulator, Nehvi et al., CVPRW, 2021 (*upon request, please get in touch)
• PhysCap, Shimada et al., SIGGRAPH ASIA, 2020.
• Quantum Graph Matching, Seelbach Benkner et al., 3DV, 2020.
• Neural Dense NRSfM, Sidhu et al., ECCV, 2020.
• Hand Texture Model, Qian et al., ECCV, 2020.
• PatchNets, Tretschk et al., ECCV, 2020.
• DispVoxNets, Shimada et al., 3DV, 2019.
• IsMO-GAN, Shimada et al., CVPRW, 2019.