Planning Beyond Digitization: Digital Preservation for Audiovisual Collections

PLANNING BEYOND

DIGITIZATION Kara Van Malssen

AMERICAN ARCHIVE /Broadway Video Digital Media

kvanmalssen@broadwayvideo.com

DIGITAL PRESERVATION OF AUDIOVISUAL COLLECTIONS

The Real Thing17 November 2010

Photo by chrisdlugosz via Flickr http://www.flickr.com/photos/chrisdlugosz/3403751594/

Digital AV Media:1. Born Digital2. Digitized from an analog / physical source

a lot more born-

digital!

Digital Universe 2010 = 1.2 zetabytes (1 billion terabytes)

Why is preserving digital audiovisual content

CHALLENGING

very large filesvery large filesvery large files

?

Photo from the Library of Congress via Flickr http://www.flickr.com/photos/library_of_congress/3738806589/

Static Media Viewed with the naked eyeUtilized without additional componentsVery long useful life if stored properly

Magnetic MediaDependent on machines to view and utilizeMedia are fragileIndustry changes results in format obsolescence

Photo by Martin Deutsch via Flickr http://www.flickr.com/photos/teflon/2352818254/

Digital MediaNot tangibleEven more dependenciesMore frequent obsolescence of various componentsPlayback requirements not obviousMore content than ever before

VIDEO: no standard format for

production or archiving

many LARGE files

Analog formats cannot wait!

“There is simply not enough equipment to play them all back.”

“the small population of decks make it mathematically improbable that a great deal of this work can ever be transferred - there is simply not enough equipment to do it - at any price.”

“We have lost the chance to save it all - now we must move quickly to identify and save what is critical.”  

-Jim Lindner, “End of Quad and One Inch” Discussion on AMIA-L, 21 May 2009

Preservation of physical media...Primarily required good storage and disaster protection

Photo by vodstrup via Flickr http://www.flickr.com/photos/vodstrup/1486839907/

Photo by Scoobay via Flickr http://www.flickr.com/photos/scoobay/3163954667/

Preservation of digital mediaMuch more than just good storage and disaster protection

Value creation for today’s users

RISK FACTORS

Photo by kino-eye via Flickr http://www.flickr.com/photos/kino-eye/39529915/

Bit Rot

Photo by damclean via Flickr http://www.flickr.com/photos/damclean/2143598772/

Software Obsolescence+ Upgrades

Hardware Obsolescence

Processor Obsolescence

Photo by huangjiahui via Flickr http://www.flickr.com/photos/huangjiahui/3179858408/

Storage Media Obsolescence

Photo by Ian-S via Flickr http://www.flickr.com/photos/ian-s/2785762687/

Storage Media Failure

Photo by massdistraction via Flickr http://www.flickr.com/photos/sharynmorrow/3718174646

Despite storing CD-Rs recorded in ideal conditions, tech site TechARP unboxed

300 CDs recorded between 7 to 9 years ago, and found that

they have a failure rate approaching 10 percent for

the first 173 discs--the restoration is still on-going.

Paul Mah, “The Problem of Bit Rot Revisited.” FierceCIO: Tech Watch, 21 July 2009

http://www.fiercecio.com/techwatch/story/problem-cd-bit-rot-revisited/2009-07-21?utm_medium=nl&utm_source=internal

“

”

Lack of METADATA“If a piece of program material is not

correctly placed and identified on a digital system, it might as well not be there - no one will be able to find it or even know it exists.”

Cox, Tadic, Mulder. Descriptive Metadata for Television. Focal Press, 2006. p63.

Photo by Roby (C) via Flickr http://www.flickr.com/photos/roby72/2401722298/

Lack / loss of organizational support

and resources

Photo by wallyg via Flickr http://www.flickr.com/photos/wallyg/2488178506/

REQUIREMENTS

1. BIT PRESERVATIONSustaining the 0s and 1s, or ensuring that the video, audio, and ancillary files remain intact over time with no loss or corruption of bits

Photo by adrenalin via Flickr http://www.flickr.com/photos/adrenalin/4250667/

2. ACCESIBILITY AND USABILITY OF CONTENTEnsuring that video, audio, and ancillary files can be found, retrieved, interpreted, played back, and delivered to the appropriate users.

Photo by Sunshine Junior via Flickr http://www.flickr.com/photos/sunshinetoday168/1323387457

3. ORGANIZATIONAL INFRASTRUCTUREAn entity (repository) explicitly responsible for keeping the content alive and accessible.

Photo by stefan1024 via Flickr http://www.flickr.com/photos/stefan1024/3682770758

Authenticity Requirements

“The archive accomplishes [preservation and access] by taking ownership of the records, ensuring that they are understandable to the accessing community, and managing them so as to preserve their information content and authenticity.” - OAIS

B6.10 Repository enables the dissemination of authentic copies of the original or objects traceable to originals. - Trustworthy Repositories Audit and Certification

“Authentication, or the demonstration of authenticity...includes both technical and procedural aspects. Technical approaches may include the maintenance of detailed documentation of digital provenance (the history of the object), the preservation of a version of the object that is, bit-wise, identical to the content as submitted” - PREMIS

STRATEGIES

efficiency

costs

authenticity

Refer to the ISO standard OAIS (Open Archival Information System) Reference Model

PRODUCER

CONSUMER

MANAGEMENT

ADMINISTRATION

ACCESSINGEST

PRESERVATION PLANNING

SIP

AIP

DIP

DATAMANAGEMENT

ARCHIVAL STORAGE

AIP

DESCRIPTIVE

INFO

DESCRIPTIVE

INFO

queries

result setsorders

INGESTSIP

Brings content into the repository

INGEST STRATEGY Establish Submission Information

Package (SIP) requirements

essence + metadata

Limit formats accepteduse LOC sustainability criteria

http://www.digitalpreservation.gov/formats/

Descriptive

Technical

Digital Provenance

Rights

ARCHIVAL STORAGEAIP

Manages storage & backups, integrity, security

ARCHIVAL STORAGE STRATEGY

Establish AIP requirements using metadata standards

METS

amdSec

fileSec

structMap

dmdSec techMD rightsMD

HD Broadcast

Master (mov/data)

SD Broadcast

Master (mov/aiff/

m2v)

Production Master (mxf)

ARCHIVAL STORAGE STRATEGY

Regularly audit files, repair corrupt files with backups

by oosp via Flickr http://www.flickr.com/photos/mhl20/4197661773/

Results of data corruption, from “Digital Preservation Strategies for AV Content”

M. Addis, R. Wright, R. Weerakkody (IBC 2010)

DATAMANAGEMENT

DESCRIPTIVE

INFO

Administers database about repository holdings

Where is the content?What is the content?

Who made it?Can I use it?

How can I display it properly?How can I preserve it?

Where did it come from?

Identification & OrganizationDescriptive MetadataDescriptive MetadataRights MetadataTechnical & Structural MetadataPreservation MetadataSource Metadata

Create, update, manage, and maintain good metadata throughout the life cycle of the

digital object

DATA MANAGEMENT STRATEGY

by cirox via flickr http://www.flickr.com/photos/cirox/2217372790/

ACCESSDIP

Facilitates requests to archival storage and data management, generates Dissemination Information Packages, delivers information in the appropriate format to users.

ACCESS STRATEGY

Meeting users needs

http://www.flickr.com/photos/myradphotos/4337804/

by Tom Rafferty via flickr http://www.flickr.com/photos/traftery/4311797662/

PRESERVATION PLANNING

responsible for planning, reviewing, and updating the repository’s preservation strategy

PRESERVATION PLANNING STRATEGY

Technology watch

Photo by jacobian via Flickr http://www.flickr.com/photos/jacobian/2385490476/

ADMINISTRATION

Oversees the operation of the system

ADMINISTRATION STRATEGY

Self Assessment or Certification on Digital Object Management, Technological Infrastructure, and Governance & Organizational Viability

Implementation of the repository depends on the services to be provided to the producer/customer

by future15pic via Flickr http://www.flickr.com/photos/future15/2036935569/

CASE STUDIES

PBS

Library of Congress

NYU

WNET WGBH

SIP site

Repository

NDIIPP - PRESERVING DIGITAL PUBLIC TELEVISION

AMERICANARCHIVE

MUSEUM OF MODERN ART

Digital Repository for

Museum Collections

THANK YOU!kara.vanmalssen@broadwayvideo.com