Catherine Lacken Preservation in Television Archives Catherine Lacken, SWR

Catherine Lacken

Preservation in Television Archives

Catherine Lacken, SWR

< >Tape Workshop, Glasgow 12 –16 May 2008, Catherine Lacken© SWR 2

Preservation Television Archives

Overview

• Broadcast archives: Mandates• Preservation management• Formats in television archives• Magnetic tapes• DVDs• Setting Preservation Priorities• Preservation Projects• Obsolescence• Moving from the analogue to digital domain



Broadcast Archives

1. Department within broadcaster: Service department to support Production (and preserve cultural heritage) – Public service broadcasters– Commercial broadcasters– Small and or local community broadcasters

2. National audiovisual archives / National Broadcast Archives: Preserve cultural Heritage (and support production for external broadcasters)) – May be part of national archive or national library



Television Archives within Broadcasting Corporations

• Audiovisual archives• One of many departments in a company

– often lack of awareness of archival needs within company, competition for resources

• House programmes produced by the company usually after transmission

• High percentage of holdings are unique and are not available elsewhere

• Rights held on high percentage of programmes: Assets• Professional tapes and recording equipment in house• Technical infrastructure and know-how available



Mandates of Television Archives

• Preservation– Safeguard company assets: programmes produced by

company– Preserve audiovisual cultural heritage (important where no

legal deposit laws exist e.g. Germany)

• Access– Serve the needs of programme makers: provision of resources

(stock footage) and information– Provide access to audiovisual cultural heritage



Aims of AV Preservation

• To ensure the long-term survival of audio-visual content and preserve its integrity

• To provide access to content in the present and in the future– unknown factor: access needs of the future

• Preservation and Access go hand in hand



Users of Television Archives

In-house• Programme planers

– Repeat broadcasts– Re-versioning

• Programme Makers– Stock Footage

• PR• Other departments

– Financial, legal etc.

External• Programme exchange

– Partners, TV networks

• Programme Sales• Cultural Organisations• Academic research• General Public (public

service broadcasters)



Access

• Descriptive metadata as means to find content that is sought

• Technical metadata with information on storage format: carrier

• Access to audiovisual content is only possible via replay equipment

• Logistics– Traffic - loan management– Copying facilities for external loans or where no access copy

held– Satellite links for fast delivery



Preservation Tasks

• Slow down natural processes of decomposition by keeping physical carriers in optimal storage conditions

• Eliminate environmental hazards• Protect originals by providing access copies• Monitor physical condition of holdings• Restore damaged carriers • Maintain replay equipment for all formats held (especially

for obsolete formats in collection)• Migrate obsolete formats to newer formats to ensure

continued accessibility



Preservation

• Passive– Preserving carriers for restoration/digitisation at a later stage– Storing in ideal environment (controlled temperature, relative

humidity) to prevent or slow down decay

• Active– Transferring to newer formats– Restoration of carriers

• When financial resources are limited, emphasis on passive preservation measures



Formats in Television Archives

• Broadcast Masters– transmission tapes produced in advance of transmission

(technical quality control), master for copies

• Production formats– post-production, news, current affairs; IT-sound track, clean

feeds (important for stock footage)

• Viewing formats– to select footage or for general viewing; consumer formats



Common Formats in TV Archives

• 35mm Film (Negative and Positive)• 16mm Film (Positive and Reversal)• 16mm Negative Film• 17,5 and 16mm separate magnetic sound tracks• 16/35 mm Film with magnetic or optical sound track• 2”, 1” tapes (B or C Format) • U-Matic Low or High Band • Betacam, Beta SP, Betamax • D1, D2, D3, D5 ……• DigiBeta• Hi8• DVCpro25, DVCpro 50• IMX



History of TV Formats at SWR

• Broadcast Masters– 1954- 1962: 35mm Positive– 1957 - 1962: 16mm Positive– 1961 - 1984: 2” Quad tapes– 1984 - 1996: 1”B-Format– 1991 - 1998: Betacam SP– 1996 - 2003: D5– 1996 - DigiBeta– 2003 - IMX– 2005- Video Files




• Production Formats– 1954 - 35mm Negative/Positive– 1959 - 16mm Negative/Positive– 1963 - 1987: 16mm Reversal Film– 1983 - 1991: U-Matic (3/4”)– 1991 - 2003 Betacam SP– 1996 - DigiBeta– 2001 - DVCpro25, DVCpro50– 2002- IMX– 2003- Video Files




• Viewing Formats– 1954- 35mm Positive– 1959- 16mm Positive– 1963- 16mm reversal– 1973-1979: VCR – 1977-1990: VCC / V2000– 1978 - 2006 VHS– 1999 - mpeg1/real video files– 2006 - DVD



Television Formats: Carrier Categories

• Original recordings– Raw material / rough cuts / camera tapes– Off-air recordings of live broadcasts

• Transmission tapes• Broadcast masters• Different versions of programmes

– With / without captions– With / without mixed commentary track (broadcast version)– With / without credits, title

• Copies– Loan for selection of footage or general viewing– Loan for broadcast (external users)

• All categories except copies are covered by preservation management



Formats in TV archives today

• Vast majority – Magnetic Tapes– Studio productions – Magazines– Recordings of live events, off-air recordings– Transmission tapes

• Film – high end productions – features, documentaries – Newsreels - reversal

• Files: news, current affairs; files for transmission



Magnetic Tapes

• Audio, video, analogue, digital• Various band widths (<1/4” – 2”)• Reel to reel, cassette• Different recording techniques: helical scan, longitudinal

scan • Professional and consumer formats



Magnetic Tapes

• Physical composition– Base layer– Magnetic coating in which recorded signal is embedded– Adhesive holds two layers together



Magnetic tapes: storage

• Storage environment: 15 – 20° C, low humidity (35 – 55% RH)

• Production environment: often 20°+, acclimatisation• Cassette: always rewind• Store in upright position on shelf• Keep away from magnets: danger of signal loss (= loss of

content)• Keep clean: tiny particles on surface interfere with

interaction with heads on replay equipment



Threats to magnetic tapes

• Wear and tear – use in replay equiment, ejecting from player if not rewound

• Surface dirt / debris • Binder degradition – hydrolisis • Sticky shed syndrom• Magnetism• Obsolesence

– Experience has shown that most magnetic tapes are obsolescent before the end of their natural life cycles (+/- 30 years)



DVDs

• Consumer format designed for viewing • Not archival format – data reduction• Instability of medium: recording process

– Dyes – Pressing

• Fragile – layer structure• Compatibility issues



Preservation Management

Metadata • Tracking

– Records of carrier loans – to whom, for what purpose• Records of transferral from one format to another

– Generation History: New carrier is copy of which old carrier• Technical data

– Machine / unit on which content recorded, used for transfers • Condition monitoring

– Records of checks and findings• Brand names

– Record with carrier information• Digital tapes

– Monitoring of BER (block error rates)



Preservation Strategy

• First Step: Assessment of Preservation Needs– Gain knowledge of general condition of collection – Define areas for active and passive preservation– Establish priority areas for active preservation



A B

C

A: Carrier conditionChemical decayPhysical decayObsolete

B: Carrier statusOriginal/Master

C: ContentUniqueHigh demandHigh value

Setting Preservation Priorities



Television Archives SWR Stuttgart

Format (approx. figures) Period of use

• 30,000 IMX tapes 2003 -• 15,000 D5 tapes 1996 - 2004• 10,000 Digi-Beta tapes 1998 - • 5,000 DVDs 2005 - • 200,000 Beta SP tapes 1990 - 2005• 80,000 VHS tapes 1985 -

2006• 4,000 1“ MAZ open reel 1984 - 1996• 40,000 rolls 16mm Film + sep mag 1954 - 1998• 2,500 rolls 35mm Film + sep mag 1954 –1995• No 2“ MAZ open reel (migrated) 1962 – 1986• No U-Matic cassettes (migrated) 1981 - 1992



Assessment of Collection: Problem areasSWR Television Archives

• Reversal film (news/current affairs 1962 – 1983)– original: held only on film (no copies, no rushes)– Degradation: bad splices, sticky shed, colour fade, wear and

tear– Of long-term value/unique: depicts all aspects of society/no

commercial broadcasters in Germany until 1980– Obsolescent format within broadcaster: no duplication

transfer facilities for short-term access needs– Restoration highly labour intensive: expensive



Assessment of Collection: Problem areasSWR Television Archives

• 35mm/16mm Film– Older b/w film: physical degradation: wear and tear,

scratches etc. (in past used as viewing format)– Colour fading of earlier colour TV productions– Some evidence of chemical decomposition: Vinegar

syndrome most noticeable for productions of 1960s, separate sound track especially vulnerable; not in danger zone yet

– Transmission tapes and access formats available for most 16mm/35mm positive film: no more wear and tear of film likely



Assessment of Collection Condition: Problem areas

• 1“ tapes – Degradation of certain tape brands (mainly Agfa), oldest

tapes in poor condition (1980s)– All tapes threatened by obsolescence: replay equipment

within company being phased out– Part of collection original status: live recordings, original

recording no longer exists– Inferior technical quality where 1“ represents 3rd archival

generation (reflects technology limits of 60s/70s and not content integrity)



Implementation Steps

• Funding– Create awareness of value of assets at top management level:

support + money• What are the expected benefits, return on investment? • How much has to be invested?• Proposed plan of action

– Lobby users– Publicity

• Collection gems• Service provided by archive



Business Plan

Putting a figure on the costs• Assessment of collection

– Volume– Condition– Content

• Assessment of Requisites for Project Realisation– Technical Personnel – Archival personnel (logistics, cataloguing backlog and deficits)– Equipment – Time

• Prioritisation– Selection criteria



Preservation Strategy: Concept

• Passive preservation where no immediate danger to content• Transfer high priority film and 1“ tapes to digital format • Active transfer when there is access demand• Provision of optimal storage conditions for film and tapes before

and after transfer to digital access format



Implementation: PrioritiesSWR Preservation Project

• High priority– Transfer 1” tapes with sticky shed to IMX tape

• Medium priority– Transfer remaining 1” originals to IMX

• 1” will not be supported by broadcaster after 2007

– 16mm newsreels that have not been transferred to tape

• Later– Transfer remaining film to digital carrier– Create browsing files for legacy holdings– Transfer to high-res video mass storage system



Preservation and Digitisation Projects

• 2002 – 2003– 1“ tapes with sticky shed

(approximately 1500 tapes)

• 2004 – 2012– 28,000 1” tapes – 5,200 hours of Film (newsreel + features, documentaries on

negative/positive)– From 2008: files gradually to replace physical carriers as target format

• 2012 – 2018 ? – 5,000 hours film (productions on negative/positive, remaining newsreel)

• 2012 - ? – Content on other obsolete formats (D5, Beta SP) to be transferred to

digital archival storage system



Preservation Project and Archival Personnel

• Logistics – moving material, tracking records etc.• Parallel to the transfer to new carrier cataloguing input

– technical metadata on carriers that was not recorded in database in the past

– Missing content description– Updating, additional information, correction of entries etc.



Preservation strategy today

• Cost conscious– Avoid large-scale format transfers– Balance between active and passive preservation

measures– Application of selection criteria for active preservation

• Reflect digitisation policy of broadcaster– Help archive meet new production requirements:

integration in digital workflows• Lay foundation for video mass storage system



Criteria for digital archival preservation formats

– Uncompressed, no data reduction– Standards based– Independent of storage media– Robust format– Non-proprietary systems– Affordable



Disdvantages of data reduction for archives

• Present day bandwidth constraints and prohibitive cost of uncompressed video storage likely to be resolved in not too distant future

• Today’s limitations should not become tomorrow’s handicap

• Transfers involving different systems of encoding and decoding within the production workflow lead to artifacts

• Data reduction does not preserve content integrity



Preservation Formats: Guidelines

• Broadcast archives must be involved in the decision-making process when new formats are being introduced

• Formats that do not meet the criteria of archival preservation formats should be avoided

• Loss of quality is the result of every transfer involving analogue recordings and when migrating from one digital encoding/decoding system and/or compression format to another

• Every format migration incurs enormous expense: if unsure about format, wait!



Obsolesence

When new technology for production, transmission is introduced:

• Survival of content not necessarily endangered• Access to content a problem when replay equipment

becomes unavailable• Access for production purposes problematical, especially

for news and current affaires where there is demand for fast / instant access (copying takes time)

• Particular format information on carrier becomes important: – Old



Metadata TV formats - Film

• 35mm or 16mm• Negative, Positive, Reversal• Original, Copy• Version: Complete production with or without titles and

credits; short item for magazine programme, rushes, edited or unedited

• Separate sound track, optical or magnetic track on film, separate IT sound track



Metadata TV formats: Video - magnetic video

• Tape width: 2 inch, 1 inch, ¾ inch, ½ inch, ¼ inch• Type: Quad, Betamax, U-Matic, Betacam Digibeta, VHS,

V2000, IMX, DVCpro, D1, D2……..• Version: Off-air recording, broadcast master, take• Original, copy, telecine• Sound track information



Milestones in Television Development

• Film -> Magnetic tape• B/w television -> colour television• Linear editing (film) -> non-linear

editing• 4:3 aspect ratio ->16:9 aspect ration • Analogue -> Digital broadcasting• Content storage on physical carrier ->

file• Standard Definition -> High definition

Year SWR (approx.)

• 1962• 1968• 1983• 2006• 2006 – 2010• 2006• 2010



TV Archives and New Formats: 1967 –1969 Introduction of colour TV

• In beginning colour productions denoted as such; no information = b/w production

• When b/w became the exception to the rule, no information = colour

• Transfer of information in paper catalogues to electronic database: information on all productions b/w or colour or colour with b/w



Attributes not recorded in earlier information systems

• Black-and-white• Mono• Analogue• Aspect ratio 4:3• Standard definition

As long as there is no alternative to such systems, this information of minor relevance



TV Archives and New Formats: 2006 Introduction of 16:9 aspect ration

• Standardisation with introduction of new format: – Regulation of Technical Department that all new tapes must be

labelled as one of the following:• 4:3 • 4:3 letterbox• 16:9 Full format• 16:9 full format Letterbox

• This information entered in Archive Info System– Tapes prior to 2006 are not labelled 4:3 – Some older carriers are labelled “Letterbox”, usually meaning 4:3

letterbox; occasionally “16:9” as additional carrier information in free text data field

• Next step: High Definition TV – Necessity to distinguish between 16:9 SD – and 16:9 HD



Parallels between introduction of colour TV and 16:9 aspect ratio

• Improved image quality immediately obvious to viewer with “new technology” TV set – i.e. colour TV in late 1960’s, widescreen TV set today

• In changeover period problems with re-use of recent archive footage in “old” format (b/w – 4:3)

• Necessity to distinguish between old and new formats in archive information systems

• Problem: deficit in catalogue information for carriers deposited in archives before introduction of new technology– Not of major relevance to present, possibly to future



Differences Colour TV / aspect ration

• B/w cannot be converted to colour• Colour viewed on b/w TV set looks like b/w production; no

irritation on part of viewer• 4:3 can be converted to 16:9• 16:9 can be converted to 4:3

– However, the conversion methods entail loss of information (top, bottom or side of image), distortion of image (stretching, squeezing) or distracting black bars, non screen-filling image

• Conversion / reconversion increases the loss of picture quality



Metadata on TV formats in TV archive information systems: SWR

• Paper based catalogues– free text, more or less standardised carrier descriptions

• Databases introduced in late 1970’s, 1980’s– Standardised text fields, e.g. FESAD 1985: lists of carrier

attributes to chose from, only listed terms allowed; one text field for additional non-standardised information

• Database updates, new data bases: 1990’s, 2000’s • Increased standardisation; e.g. New FESAD 2004: enlarged

list of carrier attributes to reflect new technology – pop-up lists, tick-box options

• Databases with video files (content) and metadata on both content and file format – Video files and metadata generated during production are linked

with descriptive in archival cataloguing database



Film, Tapes,One item user at a time

Heavy tape traffic Lossy format migration

Tapes, FilesMultiple, simultaneous and remote

Access = reduced tape traffic,Lossless and automatic migration

Bridge

On the way from analogue to digitalTV-archives today



Expectations for digital archives

• No loss of information in copying/migration process • Automatic migration to new format generations• Automatic quality control• No external loans of carriers:

– wear and tear minimised– no loss of carriers

• Savings in storage costs



Why digitise?

• To combat format obsolescence?• For Access?

– To provide a better service?– To meet new demands?– To reach new user groups?

• To save costs in long run?– Labour, storage, migration

• To make money?– Exploit new digital dissemination possibilities

• For preservation?• Combination of some or all of the above?



Going Digital

Definition of Digitisation• To transfer content from analogue formats to digital

carriers?• Access for viewing/selection purposes using browsing files?• Integration of TV archives in tapeless production

environment? (high resolution files)• Long-term storage of content in video mass storage

systems instead of on shelf?



Digitisation Policy

• Combine aims of provision of access with those of preservation

• Cost-conscious– Avoid large-scale manual format transfers – Balance between active and passive preservation measures– Application of selection criteria for active preservation

• Reflect new production requirements– Facilitate integration of archive within new digital production

workflows• Lay foundation for future digital archive system

– Long-term perspective – now just the here and now



Documents

Catherine Lacken Preservation in Television Archives Catherine Lacken, SWR