25.09.2024: [New] One paper accepted at NeurIPS 2024:
10.09.2024: Invited Tutorial at GCPR 2024 on Virtual Humans and Quantum-enhanced Computer Vision (QeCV) by M. Habermann and myself. The slides on Quantum-enhanced Computer Vision can be accessed here.
04.09.2024: Invited lecture on Quantum-Enhanced Computer Vision at the 3rd European Summer School on Quantum AI (EQAI 2024).
17.06.2024: Keynote speaker at the Physics Based Vision meets Deep Learning (PBDL) Workshop at CVPR 2024.
31.05.2024: Invited talk at 3D Vision Summer School (3DVSS) 2024.
15.04.2024: We are co-organising two workshops at ECCV 2024. More details coming soon!
- TradiCV: 2nd Workshop on Traditional CV in the Age of Deep Learning
- Quantum Computer Vision and Machine Learning (QCVML) Workshop (2nd Edition)
25.03.2024: We contributed two state-of-the-art reports at EUROGRAPHICS'24:
- Po, Yifan, Golyanik et al. State of the Art on Diffusion Models for Visual Computing.
- Yunus et al. Recent Trends in 3D Reconstruction of General Non-Rigid Scenes.
08.03.2024: We released UnrealEgo Benchmark for stereo egocentric 3D human pose estimation!
01.03.2024: Papers accepted at CVPR 2024:
16.10.2023: Papers accepted at 3DV 2024:
- 3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera (Spotlight).
- Quantum-Hybrid Stereo Matching With Nonlinear Regularization and Spatial Pyramids.
- SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes.
- ROAM: Robust and Object-aware Motion Generation using Neural Pose Descriptors.
- MACS: Mass-Conditioned 3D Hand and Object Motion Synthesis.
15.09.2023: 4DQV/VCAI contributed two journal and one conference proceedings papers to SIGGRAPH Asia 2023:
25.08.2023: We are organising a seminar on Quantum Computer Vision and Machine Learning (QCVML) (3CP) in WiSe 23/24.
18.08.2023: We are organising a lecture on Advanced Topics in Neural Rendering and 3D Reconstruction (3CP) in WiSe 23/24.
03.03.2023: Five papers at CVPR 2023 in Vancouver. Many thanks to all students, interns and collaborators!
- EventNeRF: Neural Radiance Fields from a Single Colour Event Camera.
- MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis (CVPR Highlight)
- Self-supervised Pre-training with Masked Shape Prediction for 3D Scene Understanding.
- Quantum Multi-Model Fitting (CVPR Highlight).
- CCuantuMM: Cycle-Consistent Quantum-Hybrid Matching of Multiple Shapes.
31.01.2023: We will organise the CVMLCG Seminar 2023.
21.01.2023: A paper on quantum annealing with machine learning is accepted at ICLR 2023 as Spotlight.
22.12.2022: A state of the art report (STAR) conditionally accepted at EUROGRAPHICS 2023.
16.12.2022: We are going to organise a Workshop on Quantum Computer Vision and Machine Learning at CVPR 2023; in collaboration with Tongyang Li, Jan-Nico Zaech, Martin Danelljan, Jacob Biamonte and Tolga Birdal. More details coming soon.
13.12.2022: Two articles conditionally accepted at EUROGRAPHICS 2023.
08.12.2022: V. Golyanik gave a talk titled Advances in Quantum Computer Vision at the Workshop on Quantum Information at Saarland University (event poster). The slides can be found here. Many thanks to the organisers for making such an event happen!
21.10.2022: An article on random number generation on quantum annealers accepted at IEEE Access.
05.10.2022: The source code of MoCapDeform is now available here: GitHub link.
30.09.2022: A paper on high-fidelity 3D human performance capture accepted at BMVC 2022.
12.09.2022: MoCapDeform received Best Student Paper Award at 3DV 2022.
05.08.2022: A paper on global 3D human motion capture (MoCapDeform) accepted at 3DV 2022.
25.07.2022: 4DQV will present several papers at ECCV 2022:
04.07.2022: The source code of Physical Inertial Poser (PIP) is now available here: GitHub link.
27.06.2022: The source code of φ-SfT ist now available here: GitHub link.
21.04.2022: The source code of Neural PhysCap is now available via GitHub.
19.02.2022: We are co-organising a tutorial associated with our STAR on Advances in Neural Rendering on the 27th of April at EUROGRAPHICS'22.
28.03.2022: 4DQV will present three papers at CVPR 2022:
11.02.2022: State of the art report on neural rendering (the updated version) accepted at EUROGRAPHICS'22.
25.10.2021: The source code of Convex Joint Graph Matching and Clustering via Semidefinite Relaxations is released.
18.10.2021: An article is accepted at TPAMI (HandVoxNet++).
17.10.2021: Two papers accepted at 3DV 2021 (about graph matching and clustering and human image generation).
13.10.2021: EventHands is mentioned in ICCV 2021 Daily.
12.10.2021: The source code of GraviCap is released on GitHub
11.10.2021: The source code of Q-Match is released on GitHub.
27.09.2021: The source code of EventHands is released on GitHub.
08.09.2021: Dennis Willsch gave a 4DQV Lecture Series talk titled Applications on Quantum Annealers at Jülich Supercomputing Centre. The video can be found here.
19.08.2021: We are going to present the following works at ICCV 2021: GraviCap, Non-Rigid Neural Radiance Fields, Iterative Shape Matching via Quantum Annealing and EventHands.
05.07.2021: The source code of PhysCap is now available here.
Welcome to the web page of the 4D and Quantum Vision (4DQV) group led by Dr. Vladislav Golyanik.
The focus of our team lies on 3D reconstruction and analysis of general deformable scenes, 3D reconstruction of the human body and matching problems on point sets and graphs. We are interested in neural approaches (both supervised and unsupervised), physics-based methods as well as new hardware and sensors (e.g., quantum computers and event cameras). We are integrated in the Visual Computing and Artificial Intelligence Department (D6) and working closely with Prof. Christian Theobalt.
Many research questions at the intersection of computer graphics, computer vision and machine learning involve challenging search problems (e.g., graph matching) or the optimisation of non-convex objectives. For such problems, we develop new algorithmic formulations that can be solved on modern adiabatic quantum annealers or universal quantum computers and investigate which advantages these approaches offer compared to existing classical methods.
Our reserach interests include (but are not limited to):
- 3D Reconstruction and Neural Rendering of Rigid and Non-Rigid Scenes
- 3D Generative Models
- Quantum Algorithms for Computer Vision and Graphics
Publications
2024
2023
2022
2021
4DQV Team
Dr. Vladislav Golyanik 
Navami Kairanda 
Viktor Rudnev 
Shuteng Wang 
Takuya Nakabayashi Co-Supervision
Natacha Kuete Meli Master's Thesis Students
Saurabh Pandey
Sohaib Zahid
Mayur Deshmukh
Alumni
- Suhas Gopal (Master's thesis, 2024)
- Maximilian Krahn (Master's thesis, 2024 (co-supervision with ICL and Aalto University); now a PhD student at Aalto University)
- Christen Millerdurai (Master's thesis, 2024; now PhD student at DFKI)
- Pankhuri Vanjani (Master's thesis, 2023; now PhD student at KIT)
- Cameron Braunstein (Master's thesis, 2023; now PhD student at RTG Neuroexplicit Models)
- Shanthika Naik (intern student, 2023; a PhD student at IIIT Hyderabad)
- Harshil Bhatia (remote BSc student intern from IIT Jodhpur, 2022-2023; now research engineer at Avataar)
- Lakshika Rathi (visiting BSc student from IIT Delhi, 2022; continued Master studies at the University of Wisconsin-Madison)
- Willi Menapace (remotely interning PhD student, 2021-2022; University of Trento)
- Marcel Seelbach Benkner (visiting PhD student, 2022; University of Siegen)
- Hiroyasu Akada (Master's thesis, 2022, co-supervision with KEIO University; now PhD student at VCAI/MPI for Informatics)
- Artur Jesslen (Master's thesis, 2022; now PhD student at Freiburg University)
- Erik Johnson (Master's thesis, 2022; now at Blackberry/QNX, Canada)
- Maximilian Krahn (Bachelor's thesis, 2021; continued Master studies at Aalto University)
Workshops
- QCVML: Workshop on Quantum Computer Vision and Machine Learning (ECCV 2024)
- Traditional Computer Vision in the Age of Deep Learning (TradiCV) (ECCV 2024)
- QCVML: Workshop on Quantum Computer Vision and Machine Learning (CVPR 2023)
Teaching
- Advanced Topics in Neural Rendering and Reconstruction (Lecture at Saarland University in WiSe 24/25)
- Advanced Topics in Neural Rendering and Reconstruction (Lecture at Saarland University in WiSe 23/24)
- Seminar CVMLCG 2023, Saarland University
- Seminar CVMLCG 2021, Saarland University
- Seminar CVMLCG 2020, Saarland University
- Seminar CVMLCG 2019, Saarland University
Recent Talks
- 10.09.2024: Slides on Quantum-enhanced Computer Vision (Part 2 of the Joint Tutorial at GCPR 2024).
- 17.06.2024: Keynote at CVPR 2024 (PDBL Workshop): Neuro-Physics-aware Models for Inverse Problems (slides coming soon)
- 31.05.2024: Promptable Game Models (Invited Expert Lecture at 3D Vision Summer School 2024)
- 18.04.2024: How to Read Academic Papers (CVMLCG'24 talk)
- The Quantum Computing Survival Guide, Part 1 (YouTube Channel)
- The Quantum Computing Survival Guide, Part 2 (YouTube Channel)
- How to Read Academic Papers (CVMLCG'23 seminar)
- Advances in Quantum Computer Vision at the Workshop on Quantum Information (event poster).
- Tutorial on Advances in Neural Rendering (YouTube Channel)
- 3D Computer Vision: From a Classical to a Quantum Perspective, SIC Lecture Series
- CG-Lunch Talk about Q-Match (Seelbach Benkner et al., ICCV 2021)
- How to Give a Good Online Scientific Talk (Presentation, CVMLCG'21 seminar)
Source Codes
- UnrealEgo2, Akada et al., CVPR 2024
- MoNeRF, Kappel et al., TVCG 2024
- Ev2Hands, Millerdurai et al., 3DV 2024
- QSM, Brainstein et al., 3DV 2024
- ROAM, Zhang et al., 3DV 2024
- 3D-QAE, Rathi et al., BMVC, 2023
- EgoLocate, Yi et al., SIGGRAPH 2023
- Quantum Multi-Model Fitting, Farina et al., CVPR 2023.
- EventNeRF, Rudnev et al., CVPR 2023.
- CCuantuMM, Bhatia et al., CVPR 2023.
- Unbiased 4D, Johnson et al., CVPRW 2023.
- QuAnt, Seelbach Benkner et al., ICLR 2023.
- MHMC, Luvizon et al., EUROGRAPHICS 2023.
- IMoS, Ghosh et al., EUROGRAPHICS 2023.
- QRNG, Bhatia et al., IEEE Access.
- MoCapDeform, Li et al., 3DV 2022.
- UnrealEgo, Akada et al., ECCV 2022.
- QuMoSeg, Arrigoni et al., ECCV 2022.
- NeRF-OSR, Rudnev et al., ECCV 2022.
- Physical Inertial Poser (PIP), Yi et al., CVPR 2022.
- φ-SfT: Shape-from-Template with a Physics-Based Deformation Model., Kairanda et al., CVPR 2022.
- Playable Environments: Video Manipulation in Space and Time, Menapace et al., CVPR 2022.
- Joint Graph Matching and Clustering, Krahn et al., 3DV 2021.
- Neural PhysCap, Shimada et al., SIGGRAPH 2021.
- Non-Rigid NeRF, Tretschk et al., ICCV, 2021.
- EventHands, Rudnev et al., ICCV 2021.
- Q-Match, Seelbach Benkner et al., ICCV 2021.
- GraviCap, Dabral et al., ICCV 2021.
- Human Motion Transfer, Kappel et al., CVPR 2021.
- Differentiable Event Stream Simulator, Nehvi et al., CVPRW, 2021 (*upon request, please get in touch)
- PhysCap, Shimada et al., SIGGRAPH ASIA, 2020.
- Quantum Graph Matching, Seelbach Benkner et al., 3DV, 2020.
- Neural Dense NRSfM, Sidhu et al., ECCV, 2020.
- Hand Texture Model, Qian et al., ECCV, 2020.
- PatchNets, Tretschk et al., ECCV, 2020.
- DispVoxNets, Shimada et al., 3DV, 2019.
- IsMO-GAN, Shimada et al., CVPRW, 2019.
Datasets
- UnrealEgo Benchmark Challenge, Akada et al., CVPR 2024
- Ev2Hands, Millerdurai et al, 3DV 2024
- ROAM dataset, Zhang et al., 3DV 2024
- Decaf dataset, Shimada et al., SIGGRAPH Asia, 2023
- EventNeRF, Rudnev et al., CVPR 2023
- Ub4D, Johnson et al., CVPRW 2023
- UnrealEgo dataset, Akada et al., ECCV 2022
- HULC dataset, Shimada et al., ECCV 2022
- NeRF-OSR dataset, Rudnev et al., ECCV 2022
- phi-SfT dataset, Kairanda et al., CVPR 2022
- Postdoc (all fields in the scope of 4DQV): please get in touch if interested
- PhD student at 4DQV (Non-Rigid 3D Scene Reconstruction): how to apply
- PhD student at 4DQV (Neural Rendering): how to apply
- PhD student at 4DQV (Quantum Computer Vision): how to apply
- Visiting PhD student/Intern: how to apply
Dr. Vladislav Golyanik [personal page] [e-mail]