Description
- Abstract:
- We introduce a method to convert stereo 360° (omnidirectional stereo) imagery into a layered, multi-sphere image representation for six degree-of-freedom (6DoF) rendering. Stereo 360° imagery can be captured from multi-camera systems for virtual reality (VR), but lacks motion parallax and correct-in-all-directions disparity cues. Together, these can quickly lead to VR sickness when viewing content. One solution is to try and generate a format suitable for 6DoF rendering, such as by estimating depth. However, this raises questions as to how to handle disoccluded regions in dynamic scenes. Our approach is to simultaneously learn depth and disocclusions via a multi-sphere image representation, which can be rendered with correct 6DoF disparity and motion parallax in VR. This significantly improves comfort for the viewer, and can be inferred and rendered in real time on modern GPU hardware. Together, these move towards making VR video a more comfortable immersive medium.
- Notes:
- This work was supported by a Brown OVPR Seed Award, RCUK grant CAMERA (EP/M023281/1), and an EPSRC-UKRI Innovation Fellowship (EP/S001050/1)
Download The Dataset
Globus is the platform the Brown Digital Repository uses to make large data sets available
Download using GlobusAccess Conditions
- Rights
- In Copyright
- Restrictions on Use
- Please see license included with dataset. Collection is open for research.
Citation
Attal, Benjamin, Ling, Selena, Gokaslan, Aaron, et al.,
"Data and Trained Models from “MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images”"
(2020).
Data and Trained Models for MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images, Brown University Open Data Collection.
Brown Digital Repository. Brown University Library.
https://doi.org/10.26300/ztph-0j39
Relations
Collections:
-
Data and Trained Models for MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images
MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images Benjamin Attal, Selena Ling, Aaron Gokaslan,Christian Richardt, and James Tompkin European Conference on Computer Vision (ECCV) 2020—Oral Presentation Project website: http://visual.cs.brown.edu/projects/matryodshkawebpage/ and Data set: DOI: https://doi.org/10.26300/spba-rp45... -
Brown University Open Data Collection
This collection contains open and public access data sets created by Brown University faculty and student researchers. Increasingly, publishers and funders are requiring that protocols, data sets, documentation and code underlying publications be retained and preserved in repositories, their locations …...