The research leading to these results has received funding from the European Union’s Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 248138
Format-agnostic Approach for Production, Delivery and Rendering
of Immersive Media
Felix Lee
HPC I28.10.2013
© FascinatE Consortium - 2 - www.fascinate-project.eu
Real-time Live Panoramic VideoOMNICAM
real-time acquisition
warping & stitching
blending and rendering of 6 HD video streams
one panoramic video of 7k x 2k resolution
on a single PC exploiting GPU processing
© FascinatE Consortium - 3 - www.fascinate-project.eu
Scene Acquisition (1)
OMNICAM ARRI Alexa
HD Broadcast camera
© FascinatE Consortium - 4 - www.fascinate-project.eu
Scene Acquisition (2)
Soundfield® Stereo pair
Gun micsEigenmike®
© FascinatE Consortium - 5 - www.fascinate-project.eu
Football Premier League Game
Chelsea FC – Wolverhampton Wanderers (London, October 2010)
Dance Performance Carmen Suite
Berlin Philharmonic Orchestra conducted by Sir Simon Rattle & Dance Compagnie Sasha Waltz (Berlin, May 2012)
Concert BBC Proms (London, August 2012)
Three Demo Productions (so far)
© FascinatE Consortium - 6 - www.fascinate-project.eu
Saliency Map Center-surround model
inspired from biological models of human visual sensory system Center-surround features
Pixel intensity
Color chromatic opponents RG and BY
Saliency map for still images
© FascinatE Consortium - 7 - www.fascinate-project.eu
Saliency Detection Image pyramid based saliency map
Create image pyramid: blur and shrink the image
Build the image differences Add the image differences
© FascinatE Consortium - 8 - www.fascinate-project.eu
Spatio-temporal Saliency Histograms Spatiotemporal saliency model
Split image into grid cells (cell size: e.g. 20x20 pixels for HD) Compute histogram of pyramid differences for each cell Compare histograms
Current histogram vs. average of histograms from past second Distance function: earth mover distance
Region of interest (ROI) Convert high salient regions into rectangles
Application: automatic view selection
Demo video