Shoot360: Normal View Video Creation from
City Panorama Footage
SIGGRAPH 2022

Abstract

We present Shoot360, a system that efficiently generates multi-shot normal view videos with desired content presentation and various cinematic styles, given a collection of 360 video recordings on different environments. The core of our system is a three-step decision process: 1) It firstly semantically analyzes the contents of interest from each panorama environment based on shot units, and produces a guidance that specifies the semantic focus and movement type of its output shot according to the user specification on content presentation and cinematic styles. 2) Based on the obtained guidance, it generates video candidates for each shot with shot-level control parameters for view projections following the filming rules. 3) The system further aggregates the projected normal view shots with the imposed local and global constraints, which incorporates the external knowledge learned from exemplar videos and professional filming rules. Extensive experiments verify the effectiveness of our system design, and we conclude with promising extensions for applying it to more generalized scenarios.

Framework

Overview of Shoot360. 1) The system firstly analyzes the semantic elements (people , building , others) contained in the panorama shots and combines the user specifications on the overall content coverage, cinematic styles and video length to sample the instruction guidance (role of Director). 2) Each instruction determines the source panorama shot, semantic focus, and movement type of the corresponding normal view shot. The system then brings out candidate videos for each shot following the instructions (role of Videographer). 3) Finally, by jointly associating the learned criteria from exemplar videos and hand-crafted criteria from filming rules, it samples shots from the candidates and aggregates the final videos in a local-to-global way (role of Editor).

Demo Session @ SIGGRAPH 2022

User Interface Demo

BibTeX

Acknowledgements

The website template was borrowed from Michaël Gharbi.