PrestoSpace Fast, Affordable, high Quality Preservation Technologies & Services for Audiovisual Archives An Integrated Project in FP6 Daniel Teruggi, INA

PrestoSpace

Fast, Affordable, high Quality Preservation Technologies & Services for Audiovisual Archives

An Integrated Project in FP6

Daniel Teruggi, INA

Fast, Affordable, high Quality Preservation Technologies & Services for Audiovisual Archives2PrestoSpace

PrestoSpace – Objective To offer to televisions, radios, private

companies, institutions, small archives, general public, the possibility of developing a patrimonial initiative centered on preservation and exploitation of their audiovisual archives.

This will be achieved through technological and industrial innovations, and will be implemented by ensuring the set-up of facilities and services that will provide to these actors an integrated, efficient, scalable and affordable preservation service.


PrestoSpace - the Criteria

Quality

Speed

Cost Efficiency


PrestoSpace - an Integrated Project

FP6-2002-IST 1st Call

Open 17/12/2002, closes 24/4/2003

Strategic Objective 2.3.1.12

Technology-enhanced learning and access to

cultural heritage


PrestoSpace for All

Broadcasters have already invested

considerable efforts in preserving and making

their collections accessible

Effort still required

Small archives crucially need help


Size of the problem

Richard Wright, BBC Information&Archives


The survey of ten major archives found: 1 million hours of film 1.6 million hours of video recordings 2 million hours of audio recordings

Total European holdings of broadcast material are probably 50 times larger

Total European AV holdings are at least double

Most of it is: 1) original; 2) analogue; 3) at risk.

PRESTO Survey of Broadcast ArchivesHoldings:


Preservation Status:

Obsolescence: At least 2/3 of the material in

archives cannot easily be used in its existing form

Deterioration: Approximately 1/3 of the material

has one form or another of deterioration

Fragile media: Roughly ¼ of the material cannot

be released for access because the media are

too easily damaged


Results of the Presto Project Effective Preservation needs a process, not just

good individual items of technology Savings of 2/3 can be made with fullest overall

automation and with elimination of technical bottlenecks

Much more research needed on video, film & metadata preservation processes and technology

Access, new business and transformed business are the most important preservation issues

For many of these issues, real progress needs action on a European scale: technology, rights …


The Solution:Presto Space Preservation Service

Involve all audiovisual collections Profile of user needs Improve audio, video, film and metadata

technology Integrate into an efficient workflow Offer a Service to meet those needs

Tailored to user profile Better, Faster, Cheaper than at present


Why an Integrated Project ? Research areas

Playback optimisation, restoration, process optimisation...

Integration Storage, Delivery & access, Metadata... Developing tools that work

Exploitation Selling the tools Ensuring the set-up of actual services before the

end of the project


Relevance PrestoSpace is fully within the scope of

Strategic Objective 2.3.1.12 : Technology-enhanced learning and access to

cultural heritage Audiovisual Heritage is supported by the

most fragile of all media : film, videotape, audio tape, audio discs


Societal Impact (1) Urgency

Degradation due to age (often autocatalytic) Obsolete playback equipment

Dispersion of efforts Tools required for :

Ensuring harmonised & good practices Interoperability of results Credible exploitation models


Societal Impact (2) The impact should be HUGE :

The services offer resulting from the project will provide to film, broadcast, professional, scientific archives… the opportunity to exploit their assets, according to their requirements, strategic objectives, and capacity.

The project will exploit the advances already made within the Presto, Brava, First… projects.


S&T excellence Objectives clearly defined Major cost reductions will be achieved with

respect to current preservation processes S&T approach(es) will include the

contributions of the best potential partners in the domain, always keeping the integration objective in mind


Quality/Legitimacy of the Consortium (1)

Size of the User base : Project lead by BBC/RAI/INA : three of the largest

European Broadcast Archives Many other Film, TV, & audio archives involved or

represented Experience :

Internal preservation initiatives Joint effort in earlier Projects (Presto…)


Quality/Legitimacy of the Consortium (2)

Stability Main Users are national TV and Radio archives of

European countries Critical mass

More than 50 experienced potential partners have participated in the 1st meeting

Other partners (industrial, business...) being evaluated to cover the full range of the addressed problems


Project Management

Executive Board (BBC/INA/RAI) Steering Board

Leaders of groups (users, research, technical integration, service integration, studies...)

Project Office Administration, cost claims, contracts...

Technical BoardWorkpackage leaders, consultants...


Project Management (2)

The project is lead by major archives that : Have demonstrated their ability to lead such projects Will ensure that the project progresses towards the

objectives Have already a strategy for exploitation :

Services, Tools, Standardisation, Guidelines...


Mobilisation of resources The partners will commit significant

resources towards the realisation of the objectives.

The Management will closely monitor the partners’ efforts and progress.

Our skills and experience allow us to manage a project that is tightly focused and includes every relevant actor in the domain.


PrestoSpace - the WorkSpace

Jean-Hugues Chenot, INA



Preservation

Storage

Delivery & Access

RestorationMetadata

Archive Management


PrestoSpace - the WorkSpacePreservationStock evaluation and monitoringSelection (priorities, cost/benefit)Physical restorationDecisions on formatsDigitisation, quality metadataPlayback process optimisationHuman and automatic checking

StorageLogical and physical formats(costs) : Archive/Delivery/Browsing/Low QShelves/robots/servers, video/dataEncoding, compression…Delivery mediaLife cycle managementVersion management

Delivery & AccessTo the archive video formats, exchange protocols turnkey systems, import systemsTo the final user browsing, selection… links with charging systems

Restoration

Quality evaluationCost/benefit evaluationQuality metadata importRestoration ProcessingRestoration Control

MetadataLegacy/new databases+ documentation/rights/qualityLinks with physical/logical media Assistance to preservation planningProduction metadata Human/Assisted indexationAutomatic transcriptionQuery systems for usersAccess to browsing qualityAccess to actual archive documents Inter-library exchange

Archive ManagementGeneral organisation (planning,

contracts, security, monitoring)Process managementQuality management (all levels)Cost/quality/efficiency process optimisation


PrestoSpace – the Problems


The addressed media

Films.

Video Tapes.

Audio Tapes.

Audio Disks.


Preservation : Films Physical restoration :

Up to 75% of the preservation cost.

Analog to Digital Transfer : Requires medium in adequate condition. Slow operation / Amount of media.


Films Digitisation : the goal A datacine :

Adapted to archive film medium (R,I). Less manual handling (R,I). Faster than Real-Time Digitisation (R,I). Low cost. “Rich” digitisation (R,I) : detection of :

Dust, Line-scratches, Kinescope Moire, …

ResearchIntegration and DevelopmentStudy.


Video Tape Diagnostic goals Chemical or Optical measurement of tape

condition Mechanical devices for Tape Condition

assessment …


Video Tape Digitisation : the goal Low cost. Process Automation. Quality Control. “Rich” digitisation (R,I) : detection of ->

Drop-out, Sync. loss, Color decoding problems, Noise level.


Preservation : Audio Disk

The audio disks (acetate) can be : Fractured, Dusty. Warped.

78 rpm disks are 3’ each… There are millions of these !


Audio Disk Digitisation : the goal

Contactless playback (R,I). Robustness to deteriorations. Whole process Automation.


Preservation : Audio Tape Vinegar Syndrome Flaking Azimuth problems Open reel


Audio Tape Digitisation : the goal

Contactless/optical playback (R,I). Robustness to deteriorations (R,I). Process automation (I).


Separate Magnetic (SepMag) Problems

Vinegar Syndrome : Urgency Shrinkage Synchronisation problems Have to be preserved independently from

telecine


Digitisation : R&D efforts Optical engineering. Sensors (magnetic and optical). Mechanics and robotics. System integration.


Image Sequence Restoration : Problems (1)

Difficulties of the impairment detection from standard digitised media.

Correction difficulties : Artefacts. Unsolved impairments


Image Sequence Restoration : New processes

Integration with the “Rich” digitisation (R) : Line-scratches, Dust, Drop-out…

Improvement of current techniques (R) : Full Sequence Modelling (2D ½). Fast & Efficient Image Reconstruction.

Unsolved impairment problems (R) : Severe film scratches. hair, white spots, … ….


Image Sequence Restoration : Problems (2)

Hardware Restoration : Powerful, but hard to change

Software Restoration : Difficult user adjustment, Long trial and error cycle.

Prohibitive Cost.


Image Sequence Restoration : Fast Low Cost Processing

Fast computing (R, I). Ability to evolve and Easy integration :

Modularity (I), Beyond Plug-ins (I).

Interactive : Easy and fast user adjustment (R, I), Short trial and error cycle (R, I).


Restoration : Efforts Image Sequence Analysis and Processing.

Audio Signal Processing.

Computer architecture.

User Interfaces.


Storage : Problems No guaranty for long term sustainability. Auditability. Ability to evolve. Security. Limited Remote Accessibility.


Storage

Logical and physical formats : Video/data (S). Encoding, compression… (R, I, S) . Delivery media (S).

Pilot Data Grid for Long term

Storage (R, I).

Life cycle management (S).


Storage : R&D efforts

Computer Science : Operating system, Networks. Compression Cryptography. Error correction codes.

PrestoSpace Metadata and Archive Management

Mario Stroppiana, RAI – Radiotelevisione Italiana


Preservation

Storage

Delivery & Access

RestorationMetadata

Archive Management



The role of metadata Metadata types

Identification, Description, Classification, Technical Archive management

Information about the content Access to the content

Internal usage Public services New business opportunities

The archive catalogue represents an asset by itself


Services Retrieval

Textual and content based queries Navigation and Browsing Content exchange

B2B or S2S OPAC (On-line Public Access Catalogue)

Common access to heterogeneous data bases User profiling Multilanguage support Thesauri


Knowledge modelling Conceptual classification

Ontologies Indexing schema Roles Annotation …

Mapping of data models based on conceptual classification


Metadata extraction Tools for automatic content analysis

Speech to text Scene change detection Technical metadata Automatic indexing

Natural Language Engineering Conceptual indexing Semantic nets

Legacy data bases Migration Transition management


Rights Rights data model

Analysis of current situation Definition of a common model for rights

mapping Impact on the documentation model

Guidelines for archive content exploitation Proposal to EU for legacy material rights

Business exploitation Cultural heritage


Metadata – Implementation Essence without Metadata is worthless Preservation is the occasion of enriching

metadata PrestoSpace will concentrate on the

indispensable set of metadata According to legacy information Depending on exploitation requirements


Delivery & Access

To the archive : video formats, exchange protocols (I,S). turnkey systems, import systems (I).

To the final user browsing, selection…(I). links with charging/rights systems (I). data tapes (S, I). Databases Interconnections (I)


Archive Management General organisation (planning, subcontracting,

process monitoring) (S). Tools for process management :

performance/quality/security (at all levels) (I,S). Cost/quality/efficiency process optimisation (I,S). Rights management -> protection systems (R, I). Ensuring that affordable/efficient service

offer is established (I).


PrestoSpace - Efforts

ResearchIntegration and DevelopmentStudy.

Restoration

Quality evaluationCost/benefit evaluationQuality metadata importRestoration ProcessingRestoration Control

PreservationStock evaluation and monitoringSelection (priorities, cost/benefit)Physical restorationDecisions on formatsDigitisation, quality metadataPlayback process optimisationHuman and automatic checking

StorageLogical and physical formats(costs) : Archive/Delivery/Browsing/Low QShelves/robots/servers, video/dataEncoding, compression…Delivery mediaLife cycle managementVersion management

Delivery & AccessTo the archive video formats, exchange protocols turnkey systems, import systemsTo the final user browsing, selection… links with charging systems

MetadataLegacy/new databases+ documentation/rights/qualityLinks with physical/logical media Assistance to preservation planningProduction metadata Human/Assisted indexationAutomatic transcriptionQuery systems for usersAccess to actual archive documents Inter-library exchange

Archive ManagementRight managementGeneral organisation (planning,

subcontracting, security, monitoring)

Process managementQuality management (all levels)Cost/quality/efficiency process optimisation


PrestoSpace - expected Results

Solutions

Products

Procedures/Standards

Services

PrestoSpace should be the project in the field



Documents

PrestoSpace Fast, Affordable, high Quality Preservation Technologies & Services for Audiovisual Archives An Integrated Project in FP6 Daniel Teruggi, INA