MR360 : Mixed Reality Rendering for 360° Panoramic Videos
This paper presents a novel immersive system called MR360 that provides interactive mixed reality (MR) experiences using a conventional low dynamic range (LDR) 360° panoramic video (360-video) shown in head mounted displays (HMDs). MR360 seamlessly composites 3D virtual objects into a live 360-video...
| Veröffentlicht in: | IEEE transactions on visualization and computer graphics. - 1996. - 23(2017), 4 vom: 04. Apr., Seite 1379-1388 |
|---|---|
| 1. Verfasser: | |
| Weitere Verfasser: | , , |
| Format: | Online-Aufsatz |
| Sprache: | English |
| Veröffentlicht: |
2017
|
| Zugriff auf das übergeordnete Werk: | IEEE transactions on visualization and computer graphics |
| Schlagworte: | Journal Article Research Support, Non-U.S. Gov't |
| Zusammenfassung: | This paper presents a novel immersive system called MR360 that provides interactive mixed reality (MR) experiences using a conventional low dynamic range (LDR) 360° panoramic video (360-video) shown in head mounted displays (HMDs). MR360 seamlessly composites 3D virtual objects into a live 360-video using the input panoramic video as the lighting source to illuminate the virtual objects. Image based lighting (IBL) is perceptually optimized to provide fast and believable results using the LDR 360-video as the lighting source. Regions of most salient lights in the input panoramic video are detected to optimize the number of lights used to cast perceptible shadows. Then, the areas of the detected lights adjust the penumbra of the shadow to provide realistic soft shadows. Finally, our real-time differential rendering synthesizes illumination of the virtual 3D objects into the 360-video. MR360 provides the illusion of interacting with objects in a video, which are actually 3D virtual objects seamlessly composited into the background of the 360-video. MR360 was implemented in a commercial game engine and tested using various 360-videos. Since our MR360 pipeline does not require any pre-computation, it can synthesize an interactive MR scene using a live 360-video stream while providing realistic high performance rendering suitable for HMDs |
|---|---|
| Beschreibung: | Date Completed 04.10.2018 Date Revised 04.10.2018 published: Print-Electronic Citation Status PubMed-not-MEDLINE |
| ISSN: | 1941-0506 |
| DOI: | 10.1109/TVCG.2017.2657178 |