(with audio)

Abstract

This paper proposes GraviCap, i.e., a new approach for joint markerless 3D human motion capture and object trajectory estimation from monocular RGB videos. We focus on scenes with objects partially observed during a free flight. In contrast to existing monocular methods, we can recover scale, object trajectories as well as human bone lengths in meters and the ground plane’s orientation, thanks to the awareness of the gravity constraining object motions. Our objective function is parametrized by the object’s initial velocity and position, gravity direction and focal length, and jointly optimised for one or several free flight episodes. The proposed human-object interaction constraints ensure geometric consistency of the 3D reconstructions and improved physical plausibility of human poses compared to the unconstrained case. We evaluate GraviCap on a new dataset with ground-truth annotations for persons and different objects undergoing free flights. In the experiments, our approach achieves state-of-the-art accuracy in 3D human motion capture on various metrics. We urge the reader to watch our supplementary video. Both the source code and the dataset are released.

Downloads


  • Paper, 10.1 MB

  • Video, 251 MB


  • Source Code (github)

  • GraviCap Dataset (1.0 GB)

Citation

BibTeX, 1 KB

 
@inproceedings{GraviCap2021, 
    author = {Dabral, Rishabh and Shimada, Soshi and Jain, Arjun and Theobalt, Christian and Golyanik, Vladislav}, 
    title = {Gravity-Aware Monocular 3D Human-Object Reconstruction}, 
    booktitle = {International Conference on Computer Vision (ICCV)}, 
    year = {2021} 
}    	
				

Acknowledgments

All data captures and evaluations were performed at MPII by MPII. The authors from MPII were supported by the ERC Consolidator Grant 4DRepLy (770784).

Contact

For questions, clarifications, please get in touch with:
Rishabh Dabral blah@cse.iitb.ac.in (Replace blah by rdabral)
Vladislav Golyanik golyanik@mpi-inf.mpg.de

This page is Zotero translator friendly. Page last updated Imprint. Data Protection.