DBB Project Steering Committee Meeting (Draft) - WikiLeaks · PDF fileDBB Project Steering Committee Meeting (Draft) ... – Colorworks PBB Storage update ... Agenda for DBB Project

1

DBB Project Steering Committee Meeting (Draft)

Date: July 27th 　 5:30 p.m. - 7:00 p.m./ PDT 7:30 p.m. - 9:00 p.m./ CDT 8:30 p.m. - 10:00 p.m./ EDT　　　　 July 28th 9:30 a.m. - 11:00 a.m./ JST

Locations:Atsugi, Culver City, Park Ridge, San Jose (Connected by VC ), Austin (Connected by phone)

Attendees:PSG T. Saito, S. Nemoto, M. Kawano, S. Ioka, M. Imamura, M. Soga, T. Fukuda, K. Kamiyama, H.

Yoshinari, H. Kajita, K. Yamanouchi, K. Abe

PSAT. Ohnishi, P. Lude, S. Kanemura, Y. Iwasaki, G. Mirsky, D. Carroll

SCA S. Chennakeshu SPE

C. Cookson, G. Joblove, S. Stephens, P. Squyres, T. Beswick, D. Loughery, B. Masek, R. Kido, J. Houston,T. Yuhaku, M. Nakayama, K. Jinushi

2

Agenda for DBB Project Steering Committee Meeting

• Opening remarks - C. Cookson, T.Saito, S.Nemoto

• B2B Middleware Status Report 　　 - P. Ludé– V1.0 Feature Confirmation

• SPE PBB and Conductor update – Colorworks PBB Storage update - J. Houston– PBB-DAM Related Activities - B. Masek– MB-Conductor Deployment and Testing

• Action items - SPE, PSG

• Ellcami Update - K. Yamanouchi

• Wrap-up - PSG

3







• Wrap-up - PSG

4

Chris Cookson

Opening Remarks

5

Tadashi Saito

Opening Remarks

6

Shoji Nemoto

Opening Remarks

7







• Wrap-up - PSG

8CONFIDENTIAL Ver 6

Software Development Status Report

Steering Committee Meeting

July 27, 2010

Pete LudéSolutions EngineeringPSA San Jose

9CONFIDENTIAL Ver 6

Topics

• Current Activities

• Development Status

• QA Plan

• SPE Support

• Summary and Questions

• Appendix A: Version 1.0 Scope Details

10CONFIDENTIAL Ver 6

SPE Deployment (in progress)

Release -September 30, 2010

Feb 1,2010 July 1,2010May. 1,2010April 1,2010March 1,2010

June 1,2010 Aug. 1,2010Sept.

1, 2010

Jan 1, 2010

NAB 2010

Core Software Development

User Interface Design and Development

Documentation Development

Test Procedures Development Quality Assurance + Software Bug

Fixes

Feedback Collection

Core Software Development

Current Activities - Project Plan


Current Activities - August > September 2010

• Complete Conductor Software Development– July 31, 2010– Some UI elements of Production Asset Database – August 2010

• Quality Assurance– August 1 – Sept 30

• Release Preparation– Documentation, etc

• SPE Support– Colorworks– Post Media Center (PMC)


Topics



• QA Plan

• SPE Support




CATEGORY MoU ColorworksRequiremen

t

As of 2nd March MTG

CurrentStatus

“MUST” Be Supported 46 58 33 37

To Be Determined (more information needed) 1 0 17 13

Not Applicable (e.g. 3rd party no longer in business)

2 2 2 2

Deferred (Cannot implement until third party makes interface available)

0 11 17 17

Delayed (Candidate for future implementation) 0 0 9 9

TOTAL 49 71 78 78

From last Steering Committee

Version 1.0 Project Scope See Details in Appendix A


REQUIREMENT PAGE MUST MAY TOTAL

Top Level Requirements P 5 10 10

Operation Environment P 9 6 6

Platform P 12 24 24

System P 14 93 93

Workflow p 19 25 25

Third Party Products/Services P 21 46 18 64

Web Services P 24 70 70

TOTALS 274 292

Summary of ALL MoU Requirements Version 1.11 Nov 8 2009


What Is Implemented in Ver 1.0? (Device Support)

REQUIREMENT

DESCRIPTION MOU Req. MBC 1.0 DETAILS

FR-04-0002 Transcoding must included Rhozet Carbon Coder/Carbon ServerFR-04-0003 Transcoding must included Digital Rapids Transcode ManagerFR-04-0006 Transcoding must included AmberfinFR-04-0011 Ingest must included AJA Xena 2KFR-04-0012 Ingest must included Blackmagic Decklink HDFR-04-0020 File Transfer must included AsperaFR-04-0022 File Transfer must included SFTP (webmethods ActiveTransfer)FR-04-0023 Sound Editing/Sound Mixing must included Digidesign ProTools (AVID)FR-04-0025-01A Picture Edit (offline editing) included AVID file transfer onlyFR-04-0036 Color Grading/Digital Intermediate must included FilmLight Baselight 8/4/1FR-04-0040 DVD, Blu-ray Encode and Author must included Digital Rapids StreamZHDFR-04-0041 File based Ingest/Record must included Sony XDCAM/XDCAM EXFR-04-0042 VTR Ingest/Record must included Sony HDCAM SR/HDCAM, Sony

DVCAMFR-04-0043 Sony Telefile Import must included Sony HDCAM SRFR-04-0044 Watermarking must included Civolution FR-04-0045 Watermarking, Encryption, DRM must included Thomson NexGuardFR-04-0049 Media Asset Management must included SEL Asset DBFR-04-0054 Conform must included Baselight 1FR-04-0061 QC must included Tektronix Cerify

FR-04-0063File Management must included Imageworks 2012 system (as base

code)FR-04-0064 Metadata must included SPE Digital Slate (parse only)


Bottom Line:

• Significant Majority of 274 requirements of MoU are met

• Not all third-party devices are supported– API’s not available / no web services I/F– Unavailability of units for testing– Time constraints required us to prioritize– We believe we are meeting key SPE requirements

• Not every service is included in Ver 1.0– However, we believe key requirements for Colorworks can be met

• This is complicated…– We need to review specific requirements against MoU goals– We need to reach speedy agreement on Phase 1 close-out plan


Topics



• QA Plan

• SPE Support




QA Status: Schedule


Test Cases as of 26 July


QA3 Testing Status as of 23 July

Percentage C

omplete


Topics



• QA Plan

• SPE Support




SPE Status – Instance #2 for Post Media Center (PMC)

• Timing is later than we would have liked – Concurrent with Ver 1.0 QA cycle

• But… this install will be important for practical testing– Assess usability– Feedback for ver 1.1 (and future) roadmap

• Workflow scope appears to be generally good fit with current plan– Some features beyond current ver 1.0 scope– However, Sony may be able to accommodate via MoU reconciliation


SPE Status – Post Media Center (PMC) Workflow



“Hot folders” not planned for ver 1.0, but

is supported in WebMethods.



Avid UNITY not supported in Ver 1.0 – difficulties in

obtaining documentation & test unit



FlipFactory not yet supported in ver 1.0


SPE Status – Fulfilling MoU Requirements

• Two beta site installation (rather then 1, per MoU) – Colorworks– Post Media Center

• Will require support of SPE installation through Q3-Q4 2010– This is included in MoU– However, future work in 2011 part of optional future SI contract

• Goal: Collect feedback during Q3 2010– Establish consensus on priority list for any additional features

• Future Roadmap– New features determined to be needed for general market:

• CWS Planning to assign to future version release (ver 1.1, ver 1.2 etc)– New features considered unique to SPE requirements

• Offered as extra-cost option


Next step: MoU Reconciliation

• Reach full agreement on Services and Wrappers provided (ver 1.0)– May be additional request by SPE to accommodate some “deferred” items

• Define “punch list” of any remaining items• Clarify final installation support requirements and schedule• Clarify scope of any requested System Integration Work• Service support implementation details under SLA

GOAL: Meeting by week of August 16


Topics



• QA Plan

• SPE Support




Summary

• Release of ver 1.0 on track for Sept 30– Risk: Prod Asset Management UI’s not fully complete at start of QA4 cycle– May be gap in supported services vs. Colorworks requests

• Let’s clarify priority requirements to find solution

• Support for two SPE beta sites ongoing– Colorworks & PMC– We expect these to provide valuable feedback!

• Need to reconcile MoU Requirements – top priority– Services and Devices supported– Maintenance Service/Support per SLA– Future release roadmap requests– Possible optional System Integration work


Questions?

Discussion?


Topics



• QA Plan

• SPE Support




What Is Implemented? (Services) (1)

Requirement ID Function Description

FR-05-0001Transcode Services perform conversion of one media format to another, or one

resolution to anotherFR-05-0002

Watermark Services embed a unique visible watermark in a digital media file

FR-05-0003Transport Services perform data movement of large media file from one server to

anotherFR-05-0004

Repository (MAM) Services add new asset, search, retrieve or manage existing digital asset of a catalog or repository

FR-05-0005

Media Verification Services (QC) perform analysis or validate specific characteristics of the digital media file, normally as part of the quality assurance process

FR-05-0006Fingerprint Services perform forensic fingerprinting analysis of a digital media file

FR-05-0007

Physical Asset Management Services add new asset or manage existing physical asset(e.g., video or audio tapes) in the catalog or repository

FR-05-0008Resource Scheduler Services schedule new or manage current available resources

FR-05-0009Rename/renumber Services perform renaming of files in image sequences for specific

operations like DI

LEGEND: RED:NEW/MODIFED REQ., ITALIC - DEFERRED




FR-05-0010 Digital Rights Management (DRM) Services

manage media asset intellectual rights to helpprevent illegal copying or conversation to other format

FR-05-0011Editorial Services support editorial process, like auto ingest, ALE file generation,

etc.FR-05-0012

Hot Folder Services Monitor folders for new files and create event

FR-05-0013

Conform Services Retrieve files and sequences based on EDL or conform list

FR-05-0014

HSM Migration Services Manage hierarchal storage migration based on media aware rules

FR-05-0015Encryption Services Provide encryption or decryption of files

FR-05-0016

Versioning Services Provide version control of files and sequences, and track approved versions

FR-05-0017Metadata Services Manages metadata for files and sequences




Requirement ID Function DescriptionFR-06-0001

Sync Service Synchronize Video and Audio

A service that Synchronizes Video and Audio files within the backbone.

FR-06-0002Conform Service Pull files and handles

based on EDL A service that conforms files within the backbone

FR-06-0003Retrieve Service Retrieve Files A service that extracts the file from the repository.

FR-06-0004Monitoring Service Monitor Directories This program looks for files in source directories and calls the

ingest service.- INTEGRATION

FR-06-0005Ingestion Service Ingestion Process The Ingestion Service is used to facilitate adding content into

the system.

FR-06-0006Metadata Service Processes Metadata A service that extracts, translates, and processes source

metadata and stores it in the database.

FR-06-0007Notification Service Email Notifications A Service that sends out a notification message.

FR-06-0008

Proxy Service Manage Proxy Relations The Proxy Service manages the relationship of files to proxies (create relationship and retrieve relationship between files)

FR-06-0009Search Service Find assets in the

Backbone A service that submits a search query and retrieves results



What Is Implemented?(Services) (4) Requirement ID

Function Description

FR-06-0010Security Service Authorize Users The Security Service verifies / enforces user roles / privileges / access

within the Backbone.FR-06-0010-1

Transaction Logging Service Logs TransactionsThe Transaction Logging Service creates a record in the database for every transaction. This record shall be used for reporting and tracking purposes.

FR-06-0011Transcode Service Modify Asset Resolutions to

Optimize PerformanceA service that converts assets into higher/lower resolutions or other file formats depending on specifications.

FR-06-0012 Transfer Service (Client to Server) File Transfers HTTP or Aspera Client Transfers from local computers to Backbone

(uploads).FR-06-0013 Transfer Service (Server to

Client) File Transfers HTTP or Aspera Client Transfers from Backbone to local computers (downloads).

FR-06-0014 Transfer Service (Server to Server) File Transfers Backbone Transfers to or from FTP servers, Aspera Console servers,

network directories.FR-06-0015

Transcode Service Modify Asset Resolutions to Optimize Performance

A service that converts assets into higher/lower resolutions or other file formats depending on specifications.

FR-06-0016 Deliver Service (Ext Client to Server) File Transfers SFTP, HTTP or Aspera Client Transfers from external computers to

Backbone (uploads).FR-06-0017 Deliver Service (Server to Ext

Client) File Transfers SFTP, HTTP or Aspera Client Transfers from Backbone to external computers (downloads).

FR-06-0018Watermark Burn In Service File Transfers Add visible watermark or burn ins (copyright, time code, frames)

FR-06-0019Resource Utilization Service resource monitoring track resources used in processes

FR-06-0020Message Authentication Service Message Authentication Authenticate and create digital signatures and encryption (CA based)



What Is Implemented?(Services) (5)


FR-06-0021Data Migration Service Automated file transfer

between storage tiersAutomated movement of data from higher performance storage to lower performance storage and vice versa

FR-06-0022Backup Service Automated backup and

restore Automated backup and restore system for disaster recovery.

FR-06-0023File Spoof Service (Virtual File System)

Create illusion of assets available in all supported formats

Takes request for existing asset in format it does not presently exist in, locate asset and transcoder and initiate transcoding, advise requestor it shall be available in x timeframe

FR-06-0024Encryption Service Encrypt content Encrypts content as directed.

FR-06-0025Deep Archive Service Prepare and store deep

archive formatDirects transcode engine, manages database, and supervises placement in deep archive

FR-06-0026Render Service Render content Render color grading, effects, etc from existing content or render frames

from 3D modelingFR-06-0027 Packaging &

Containerization Service

Packages or wraps content Packages or wraps content

FR-06-0028LUT Service Applies LUTs Applies 1D &3D LUTs plus gamma, logarithmic, and exponential curves for

direct viewing or to bake in proper corrections in dubsFR-06-0029

Service Automatic QC of operations Automatically QC ingests, transfers, etc.

FR-06-0030 Read from Physical Media Service Read Physical Media Retrieve from physical media including videotape, audiotape, optical media.



What Is Implemented? (Services) (6) Requirement ID Function Description

FR-06-0031Write to Physical Media Service Write Physical Media Record on physical media including videotape,

audiotape, optical media.FR-06-0032

Dynamic Service Provider Selection Service Dynamic resource allocation

Determine which provider of a service should be utilized per request, and forward request to that provider.

FR-06-0033Dirt & Scratch Removal Service Remove dirt and scratches

from pictures. Provide cleanup of ingested film.

FR-06-0034Service Gateway

Provide access to other departments or external vendors.

Gateway to connect with and provide programmatic interface to other departments or external vendors.

FR-06-0035Picture Editing Service Picture editing functions. Service that abstracts picture editing process.

FR-06-0036Color Grading Service Color grading functions. Service that abstracts color grading process.

FR-06-0037Visual Effects Service Visual effects functions. Service that abstracts VFX process.

FR-06-0038Sound Editing Service Sound editing functions. Service that abstracts sound editing process.

FR-06-0039

Sound Mixing Service Sound mixing functions. Service that abstracts sound mixing process.

FR-06-0040Metadata Version Control Service

Maintain metadata version control.

Ensures that metadata versions are retained and naming conventions automatically supported.





FR-06-0041

Metadata Latest Version Service Ensure operator has latest version of metadata.

Compares local version to latest network version and provides automatic or semi-automatic transfer and replacement of any outdated metadata.

FR-06-0042

Essence Version Control Service Maintain essence version control.

Ensures that essence versions are retained and naming conventions automatically supported.

FR-06-0043

Essence Latest Version Service Ensure operator has latest version of essence.

Compares local version to latest network version and provides automatic or semi-automatic transfer and replacement of any outdated essence.

FR-06-0044Rename & Renumber Filename Service

Maintain naming and numbering conventions.

Automatic (or semi-automatic) renaming or renumbering of filenames to adhere to predetermined naming conventions.

FR-06-0045

Essence Latest Version Service Ensure operator has latest version of essence.

Compares local version to latest network version and provides automatic or semi-automatic transfer and replacement of any outdated essence.


40







• Wrap-up - PSG

41

SPE PBB and Conductor update

Jul.27th .2010

Sony Pictures Entertainment

42

Colorworks PBB Storage Update

• Resource Management tools– Ability to monitor storage utilization– TSM/Tape Library Monitoring tools– Render Queue

• Smurfs client interface prototype– Asset browser– Asset retrieval request submission and monitoring

43


SMURFS Asset Browser/Extraction UISMURFS Asset Browser/Extraction UI MBC UIMBC UI

TapeTape

ServicesServices

Calypso Media Backbone Conductor

TranscodersTranscoders

Colorworks Development and proposed MBC Integration

Resource ManagementResource Management

MonitoringMonitoring SchedulingScheduling

DiskDisk Render Farm

Render Farm

44


Storage Summary Graph

At-a-glance situation report

• Backbone disk• Backbone tape library• Mouse over any section for further

details

45


Per-project Storage Report

• Ability to sort by any field• Searchable

46


Per-volume Storage Report

• Ability to sort by any field• Searchable

47


Storage Time Series

Shows a breakdown of storage for one or more projects, over time

48


Storage Project Details

• Provides a condensed but complete storage situation report for a project.

• Shows a breakdown of storage for a given project, over multiple storage pools, and over time

• Also shows total occupancy for the project

49


TSM Tape Drives

• Instant status of TSM-controlled tape drives• Pass-through interface between Calypso and TSM

50


TSM Tape Library

• Complete listing of all tapes in the library

51


Render Queue

Front-end to the “live” Rush render management tool

52


SMURFS Component

Asset Browser

• Searchable listing• Asset detail with all collected

metadata and thumbnails

53


SMURFS Component - Asset Retrieval System

• Request submission, validation, and submission to render queue• Designed to handle multi-stage requests (with approval at each stage)• Request browser provides summarized update on the status of a request

54

PBB-DAM Related Activities (Part 1)

• Objectives– Clarify PBB-DAM Requirements across all SPE Production Groups– Provide base documentation to allow architects to select proper PBB-DAM

solution

• Requirements Gathering and Analysis– Effort spanned over 10 weeks– Interviewed 15 Technical Stakeholders (Post Prod Facilities, Colorworks,

VFX, TV, Distribution Backbone)– Identified over 40 asset groups along with their metadata requirements– Developed detailed Functional Specifications– Provides basis to validate potential PBB-DAM solution

55

PBB-DAM Related Activities (Part 2)

• PBB DAM Planning– Architecture/ design team to review findings– Evaluate best PBB-DAM candidates (3rd party, internal, Conductor DAM?)– Currently evaluating DAMs for “MIB-3” and “Spiderman-4”– Collaborate w/ PSA Conductor team to establish best practices for 3 rd party

DAM integration into Conductor

56

Potential PBB-DAM, PBB Storage and Conductor Integrations

1. Interface DAM w/ PBB Storage.

2. Integrate PBB Storage w/ Conductor Media Bus.

3. Integrate PBB DAM w/ Conductor.

SONY WORKFLOW SOLUTIONS

webMethods SOA PLATFORM

SONY WEB SERVICE INTERFACE

webMethods BPM ORCHESTRATION

SONY GUI

SONY WEB SERVICE WRAPPERS

THIRD PARTY SERVICES

SONY UNIQUE SERVICES

web

Method

s ESB

Oracle 11g D

AT

AB

AS

E

SONY MEDIA BUS

Customer STORAGE & NETWORK(e.g. SPE PBB Storage)

PB

B - D

AM

Sony Media Backbone Conductor Platform

3

21

1

2

3

57

MB-Conductor Deployment and Testing (part 1)

• Objectives– Test how to implement Conductor workflows– Evaluate how productions will benefit from Conductor workflow automation– Provide feedback to PSA on initial Conductor learning– Evaluate flexibility, adaptability and usability of toolset by production operators

• Deployments of NAB version near complete– Colorworks

• Oracle and WebMethods systems have been set up• Awaiting work-around to optimize Conductor to Colorworks interface

daemon– Post Media Center (PMC)

• Support and training– Access to WebMethods Sys Admin and WebMethods developer to accelerate

effort– Depending upon outcome and needs, will possibly train more operators

• Upcoming Conductor related activities– Colorworks: Tape archiving/Tape library handling, File movement, Ellcami

integration– PMC: Editorial to Sound orchestrated workflow

58

MB-Conductor Deployment and Testing (part 2)

• PMC Background– Part of SPE Post Production Facilities.– Combines Picture Editorial and Video Transfer Services.

• PMC Conductor Pilot Workflow– Transferring, transcoding and marking Avid files from Editorial to Sound Stages.

• Workflow Benefits– PMC will be able to automate and track key transcode, mark, transfer and notify workflows.– Learning will assess capabilities of Conductor; will share findings with PSA.– Stepping stone towards more complex PBB workflows (Colorworks and others).

59







• Wrap-up - PSG

60

Action Items from SPE

• Review ‘services and wrappers’ list and re-assess prioritization of roadmap of supported devices and functions to ensure Colorworks can leverage greater value out of Conductor.

• Re-visit findings from PBB – Conductor pilot workflows.– Evaluate what worked, investigate areas for improvement.– Assess expanded use of Conductor throughout other productions

workflows.

• Re-visit MB-Conductor DAM capabilities for potential use w/in SPE.

• Plan to re-adjust MoU and SLA based on latest licensing, support and feature prioritizations.

61

Action items from PSG

• Software Site license scheme• Future Roadmap / New Features Management

62







• Wrap-up - PSG

Latest update

ELLCAMI

63

1. Ingest Station for Phil Squyres (TV Production) Target release date of commercial version : Sep. 27

Focused on ingest function only, no file base transcode function We will upgrade Phil’s system to the latest version during the week of Jul. 26

Requests from Phil on Jun. 25 Ingest from SRW to DPX & XDCAM 50 + ASC CDL Tested in Atsugi already

Live ingest function is still under development but with no firm release date yet

2. Transcode Station for Jim Houston (Colorworks) Target release date of Transcode Station is still to be determined

Because of the growing complexity found during beta tests

Nonetheless, DPX – SLIC functionality that Jim is testing will be stable

3. 2K- 4K DEEP upconverter requested by Chris Cookson As promised, we will install the prototype of DEEP during the week of Jul. 26 64

Other questions Status of SLIC

Already implemented in Transcode Station that Jim has been testing

For SPE only

Support of SStP (HDCAM-SR CODEC) Considering both software codec and hardware codec (we will as well consider 4K, DBL Debayer when it comes time to) Release date: TBD

65

66







• Wrap-up - PSG

Documents

DBB Project Steering Committee Meeting (Draft) - WikiLeaks · PDF fileDBB Project Steering Committee Meeting (Draft) ... – Colorworks PBB Storage update ... Agenda for DBB Project