iDiff 2008 conference #07 IP-Racine Project Overview

1IP-RACINE presentation: Paris IDIFF

IP-Racine is sponsored by the E.C. FP6


IP-RACINEDigital Cinema

from Scene to Screen

An overview after 40 monthsJosep Blat – Eugenia Fuenmayor


Project Objectives

Increase Interoperability along the Digital Cinema chain

Reduce Costs Advance

the technology


Working areas Digital Cinema improvement needed

‘from Scene to Screen’

Six working groups


Partners• 11+ M€ budget, 8+ M€ EU contribution• Capture Thomson (GVN and DTO)

• Virtual Studios Brainstorm

• Post-production Pandora, FilmLight, DVS

• Exhibition XDC, Barco

• User – Studios MediaPro• Research institutes UoH, JRS, UG,

FSSG, BM-UPF (Coordinators)


Metadata tools (JRS)

Workflow

Clipster(DVS)

SAN (Spycer) (DVS)

e-studio(Brainstorm)

3D interpolation (U.of Hasselt) Baselight Four

(Filmlight)

Solo server(XDC)

Color corrector(Pandora)

Compression(U. of Glasgow)

GraphicsFramework

(UPF)

Projector(Barco)

Auditoriumsound system

Viper/Venom(GVN/DTO)

Quadraphonicsound

(FSSG-CNR)

Virtual SurroundSound (FBM)

Visual Effects,Group Dynamics

(UPF)

De-interlacing(UPF)


Workflow integration examplesDVS - Thomson/GVN:

• Metadata Transfer via SDI• Recording via IT interfaces from Viper camera to DVS-SAN

JRS - DVS:• JRS metadata extraction service integrated into DVS Spycer

DVS - XDC:• DCP master as interface from postproduction to playout

FilmLight - DVS:• Spycer running on FilmLight parallel filesystem

UPF - FilmLight:• Parallelisable graphics framework

Brainstorm – GVN:• Sending Camera tracking position through Standard RS232 protocol

Brainstorm – DVS:• Reading back the Tracking information from the DPX headers

Brainstorm – FBM UPF:• Virtual Image and Audio Integration

Other real-world integration


Achievements

• O1. To create an integrated DC Workflow (“From scene to screen”), – Not a unique workflow, Not a linear workflow, Not a

uniform, homogeneous environment, and Built on top of existing infrastructure

– Left: testbed integration – Right: real world integration


Achievements

• Metadata & Standards Requirements• Workflow Enhancement & Tools • Essence and Metadata Workflow

– Deal with metadata diversity; mapping (standards or extensions); tools(workflow, extraction, preservation, search, conversion, exchange) defined

– Non unique workflow, service oriented architecture defined– Pictorial summaries, WMO, content analysis tools– New Spycer (file server, backup)– New Clipster (NAS solution & bridge; DCDM generation, RT JPEG2000

decoder)

10

Achievements• Supporting Scalable Graphics

– Multiplatform graphics framework supporting modern graphics programming, parallelisation, high-level XML control

• Colour Calibration and Conversion – On-set colour adjustments and monitoring - tools– Colour visualisation, measurement, communication (Truelight)– ICC/Academy standards proposal

• Implementation of highly scalable architecture– Demonstrated at IBC 2007

• NFS-integration of parallel file system; Better SAN integration (with bandwidth throttling); Editorial integration through AAF, MXF, XML; Fast access of another cluster's parallel file system; All 10GigE inter- and intra-cluster fabric


Interoperability (testbed 3)

HIL accelerationboard

(Pandora)

Clipster (DVS)

ViperCapture flamenco

band(Grass Valley)

SAN (DVS) Baselight

(Filmlight)

Solo server(XDC)

Projector(Barco)

Laptop(JRS)

Laptop(DVS)

(UPF)

FBMFSSG-CNR

MXF +XML

HDSDI

(back-up)

Ethernet

Metadata

Software installationSAN(DPX + Metadata)


SAN

Footage (for audio synchronization)5.1 wave tracks

Estudio(Brainstorm)

Color corrector(Pandora) (U. of Glasgow)

Footage Phase I(U.of Hasselt)

HDSDI +DPX

HDSDI + DPX

HDSDI + DPX

HDSDI +

JPEG 2000

External hard disk

HDSDI

5.1 audio synchronized with video

DPX

FBM

Background composition

Image Warpingwith HIL

12

Achievements

• O2 To provide improved digital cameras– To fully integrate electronic acquisition into the digital

workflow• Embedding of metadata in VIPER output to improve workflow• On camera VENOM solid state recorder• More robust mechanics for VIPER• Increased pixel rates• Flexible 10Gig Ethernet transmission


Achievements

• … O2 To provide improved digital cameras– To develop a flexible and robust recording solution for

metadata and uncompressed essence that fits within the workflow

• VENOM solid state recorder• Hard disk field recorder for back-up

Generations of Viper and Venom …Generation of metadata, images, 10GE new transport format


Achievements

• O3 To develop film resolution 3-D virtual studio technologies– High Quality RT ‘preview’ and non RT ‘final’ quality

demonstrator (shown NAB - IBC).• Shaders in HD• New Hardware

– RT Visual Effects and Group Dynamics

15IP-RACINE presentation: Paris IDIFF AAAAA

Achievements• Camera adaptation

– Software from HD to 2K preview; latest SDI, HD & 2K output boards evaluation; new I/O interface through plug-ins

• Shader and code development– eStudio with new DoF shader, diffuse shadows and real objects

casting, higher res motion blur and DoF, improved distortion …

• Advanced camera development– encoder systems for new Camera /

Robot arm or Crane rig; Vinten new Robotic Pedestal; Trackmen, MixTV, Multi KuperBox, TabulaTrack, Total Inmersiondrivers; Motion analysis (Camera and Position tracking)


Achievements

• Visual Effects and Group Dynamics, standalone and integrated– New high-quality RT visual effects: Screen space ambient

occlusion, Variance shadow mapping, Skin rendering– New Group dynamics features: Graph, Auto-navigation, Path-

finding, Semaphores, Attractors– Integration as eStudio and 3D Studio MAX plug-ins, within the

graphics framework

17

Achievements

• O4 Representing, manipulating, annotating and retrieving moving images and sound– High level automated annotation and indexing techniques:

• object-redetection, setting detection;• tracking of regions; • indexing for search and content browsing

– Image representation and manipulation

• Compression for Colour Correction

• Resolution Enhancement; Fractional Resizing


Achievements

• O4 Representing, manipulating, annotating and retrieving moving images and sound– Time slicing/ View Interpolation (next slide)

– Sound rendering based on quadraphony (coll. with WP7):

• In situ prehearingof surround sound in virtual scene

• Postproduction of surround sound adapted to camera pan / zoom / tilt


Time Slicing / View interpolation


Achievements

• O5 To develop and demonstrate real-time hardware solutions:– New colour correction hardware; HIL and drivers for

flexibility; Pandora software; UG and JRS integration

HIL software development integration diagram


Achievements

– New 2k projector 45 % cost reduction; servers supporting DCDM

– Enhancements in projectors / servers usability and technologies

– Quad-audio workflow

– De-interlacing, color enhancement and de-noising

• O6-O7 To develop digital projector and playouttechnology and new formats:


Achievements

• Future digital cinema experience – Projector light efficiency improvements

• Graceful data reduction– More efficient and cost effective movies duplication

• Improved usability – Enhanced projector and (remote) server control (touch

screen plus functionalities); integration– Motorised lamp alignment

• Improved affordability – Servers (G3 improvements) /

projectors (compact platform)


Achievements

• Improved quality of audio – Quad-audio workflow; Rendering; New measurement tools;

DTS conversion; Experimental validation of 1-D sound fields

• Image processing– De-interlacing GPU accelerated, FPGA real-time

implementation under exploration

– First results in Denoising; Color enhancement


Achievements

• O9 To reinforce the European skills base for digital cinema technologies:– Researchers exchange, theses, …– Experimental training: partners events, Virtual Studio on

line (two editions), Media Salles; “making of”; iDiff Seminars


Achievements

• O8 - O10 To promote the understanding and acceptance of EDC and its technologies - market launch– Papers (42); Events (22); Products - Trade shows –Leaflets

– Industrial Days; Standards; Patents

26IP-RACINE presentation: Paris IDIFF AA

Achievements• User Requirements

– Monitoring over the project; Book; Outlook

• IPR and Standards– Patents, EDCF, SMPTE,

ISCDF, MPEG-7, ICC/Academy

• Market awareness• Exploitation

– IBC – NAB, Cannes, … Over 400 feature films

• Dissemination– Book, website, leaflets,

papers, events –industrial days; …


Achievements

• Industrial Days– Digitrain (Media Salles,

Barco)– At NAB (Brainstorm,

Filmlight, Pandora, Grass Valley, DVS)

• Testbeds(Interoperability -Integration)– Testbed 2, Testbed 3

(EP recovered) …


From Scene to Screen …

•THEN (when IP-RACINE started)– DI for postproduction HD -2K– Digital Video; No exhibition

•NOW (when IP-RACINE is about to end)– Digital Cinema Theatres Network, Digital Cameras,

Postproduction 4K and beyond ..– IP-RACINE partners at the industrial forefront (and

research not yet exploited); workflow in the real world


… a summary•High quality research and development, and reports•Patent filings, conference and journal publications•Developed prototypes, many of which incorporated into products•A workflow where products are demonstrated to be interoperable •Future work still ongoing and will bear industrial fruit•Visibility, international conferences and major trade fairs in Europe and the USA.•Representation in relevant standards bodies•Relevant research and development to existing requirements and the emerging interest 3D digital cinema•A coherent workflow support chain from scene to screen, starting on 10GbE transferred from one stage to the next and across equipment platforms.


Metadata tools (JRS)

Workflow

Clipster(DVS)

SAN (Spycer) (DVS)

e-studio(Brainstorm)

3D interpolation (U.of Hasselt) Baselight Four

(Filmlight)

Solo server(XDC)

Color corrector(Pandora)

Compression(U. of Glasgow)

GraphicsFramework

(UPF)

Projector(Barco)


Viper/Venom(GVN/DTO)

Quadraphonicsound

(FSSG-CNR)

Virtual SurroundSound (FBM)

Visual Effects,Group Dynamics

(UPF)

De-interlacing(UPF)


IP-RACINE Office

IP-RACINE Management Officec/o Josep Blat, Eugenia FuenmayorPasseig de Circumval·lació, 808003 Barcelona,Spainjosep.blat at upf.edueugenia.fuenmayor at barcelonamedia.org

www.ipracine.org

mailto:[email protected]

mailto:[email protected]

Technology

iDiff 2008 conference #07 IP-Racine Project Overview