WORLD CONFERENCE 2013OCTOBER 25 - 28 2013 DUBAI, UAE

Charles Fairall, Helen Edmunds (2013). Turning a national archive digital, by degrees….

TURNING A NATIONAL ARCHIVE DIGITAL, BY DEGREES…Charles FAIRALL IEng MIET *a, Helen EDMUNDS†b

a Head of Conservation, BFI National Archive; b Collections Manager, BFI National Archive

This paper addresses the BFI National Archive’s transition from analogue archive workflows to digital through initiatives driven largely by pragmatic response to evolving needs. At a time when the BFI now consolidates its digital ambitions through the Film Heritage Unlocked project, the paper looks at the various approaches aimed to deliver mass digital access alongside preservation initiatives and lessons learnt along the way to what we all acknowledge will be a more digitally-oriented future than could ever have been predicted.

Keywords: Digital capability | Digitisation | Analogue to digital transformation

AN EVOLUTION FROM ANALOGUE TO DIGITAL

Two or three decades ago, when the authors of this paper embarked upon their working lifetime in a national film and television archive, there was little mention of digital. Yes, engineers and technicians working in the broadcast industry at that time would have been using and maintaining elements of digital technology within processes such as timebase correction and some network transmission paths, but on the whole the word digital was for geeks and simply didn’t feature in daily vocabulary; not really even in the emerging world of personal computing. Today however, the story is very different. Just about everyone on the planet has something to say about digital. Whether its television or tablets, mp3s or mobiles; consumers and businesses alike are now hard wired for digital.

For archivists working with moving images in the 80s and 90s, the prospect of digitisation was almost unthinkable. There was certainly a fascination for the subject, but every time the calculators came out to start working out the raw data sizes for digitised film frames, video signals and the like, we smiled and quickly switched them off again. The technology required most likely could never exist or even if it could, would not be affordable for archives to store or transport the vast volumes involved.

Of course the rapid evolution of the World Wide Web through the 1990s, combined with the reality of Moore’s law and an insatiable consumer hunger for technology, virtually took and tore up the rule books of the previous 50 years - in what felt at the time like just a few days. Like everyone else determined to progress and succeed in the digital revolution, BFI archivists had to react quickly to maintain viability, in what had inexorably metamorphosed into a new world: a digital world. So, with all the right technologies necessary to overcome the perceived impossibilities of a decade before becoming more and more available and affordable, what would we as archivists do with them? During the late 90s and early 00s some highly significant factors were at play which helped form a cultural-technological strategy.

* Charles FAIRALL: Collections and Information | British Film InstituteJ Paul Getty Jnr Conservation Centre | Berkhamsted HP4 3TP | United Kingdone-mail: Charles.fairall@bfi.org.uk

† Helen EDMUNDS: Collections and Information | British Film InstituteJ Paul Getty Jnr Conservation Centre | Berkhamsted HP4 3TP | United Kingdone-mail: helen.edmunds@bfi.org.uk

Copyright © of this paper is the property of the author(s). FIAT/IFTA is granted permission to reproduce copies of this work for purposes relevant to the above conference and future communication by FIAT/IFTA without limitation, provided that the author(s), source and copyright notice are included in each copy. For other uses, including extended quotation, please contact the author(s).

Charles Fairall, Helen Edmunds

Responsible for one of the world’s greatest film and television collections, held in entirely analogue form and spread among the most diverse range of legacy formats imaginable, the BFI had its work cut out - not only to make as much of it available to its audiences as possible but also to secure the physicality of its artefacts, all of which were unstable one way or another through decay and/or format obsolescence.

One of the very earliest digital initiatives at the BFI National Archive came by virtue of a project sponsored by the HLF (Heritage Lottery Fund). Titled, Building an Archive for the 21st Century, this 5 year project set out to address collections management issues surrounding legacy acquisitions spanning many decades which required further preparation to meet expectations for the coming years. Television was at the forefront of this campaign, with some 36,000 titles from the history of the UK’s major independent broadcaster ITV, held only in their native format of 2” Quadruplex videotape. When the green light came on for the project, key personnel were recruited and a bespoke technical facility was quickly created during 1999 within the BFI Conservation Centre at Berkhamsted to complement the existing Video Unit which was built in the mid-80s for the capture and preservation of off-air television.

Figure 1 2” Quadruplex tapes awaiting transfer to digital format by the ‘Quad Squad’

Working 12 hour shifts, 7 days each week, with leadership from the archive’s Senior Curator of Television and a Senior Engineer with specific expertise recruited for the project, a team of 8 operational engineers (fondly known as the “Quad Squad”) transferred in excess of 30,000 programmes to Digital Betacam - literally saving them from a tape format that due to diminishing skills and spare parts could simply not be sustained for the long term in such high volume. The HLF 2” project, which enjoyed strong support from the ITV companies, was deemed a great success and provided a timely opportunity to adopt and demonstrate many new methods and working practices. Through this entirely positive experience many barriers were broken down, with personnel from right across the archive operation being involved and far more sophisticated collections and project management in place than had ever been known before.

The HLF project, which had brought in additional staffing, created new skills and overall broke new ground, inevitably revealed further challenges. In addition to the tens of thousands of television programmes transferred during the 5 year period, the BFI’s film collections were examined in thorough detail to determine cultural value and also physical condition. In excess of 65 million feet of film were inspected and condition documented. At that time a number of disparate legacy databases existed with specific, categorised information in each. While these were largely fit for purpose in previous

2 FIAT/IFTA World Conference 2013 in Dubai

Turning a national archive digital, by degrees…

decades, when ambitions for preservation, collections management and access were perceived very differently, they were not linked or necessarily in a form befitting of an era driven by the needs of digitisation for preservation and high volume, popular access.

From inspecting such a large quantity of film during the HLF project, the BFI now had evidence of magnetic film materials at significant risk of loss and began a programme of digitising all preservation status magnetic tracks. This early digitisation for preservation programme at the conservation centre necessitated developing technologies, equipment installation and changes to the bespoke database Tecrec, in use at that time to accommodate documentation of the first digital files to be preserved within the national collection.

Around the same time the BFI, working with several partners including UK Parliament and The National Archives, were funded by JISC (Joint Information Systems Committee) to deliver 1,000 hours of newly digitised content, which would be curated and contextualised for further and higher education users. This project came to be known as InView: British history through the lens. To deliver this project the conservation technology team at the archive defined the necessary equipment and managed installation. The BFI National Archive could now create preservation quality SD digital files and transcode them, within the existing video laboratory at the conservation centre. The project also provided an opportunity to develop multi-skilling, as technical archivists worked with a selected title, often from the point of acquisition, through film inspection and preparation, to quality checking new tape assets and the creation of digital files.

The mid to late 00s at the BFI National Archive were to provide further exciting challenges and significant progress, with a successful bid to UK government for funding to build upon the strengths and necessary findings of the HLF project. A further major project, known as SHUK (Screen Heritage UK) was begun, with archival emphasis being focused on securing national collections. For the BFI archive, this would impact on three significant areas by:

Creating a new database CID (collections information database) with an integrated workflow system fit for the future needs of managing UK national archives and capable of interfacing with the public domain

Improved collections stabilisation and management, with the creation of a new Master Film Store that would radically control inherent deterioration due to decomposition and remove the significant and immediate threats of fire and mould

Developing existing conservation centre technical capabilities through the introduction of new equipment and tools for film cleaning, photochemical printing, digital scanning, grading and restoration

CULTURAL PROGRAMME

Coincident with these major technical initiatives, ever more ambitious cultural programmes were evolving at the BFI, with projects being created to collect, preserve and deliver moving image content and information to audiences and education in a variety of new and innovative ways.

2003 - BFI Screenonline‡ a web based resource for licensed public libraries, schools and colleges was among the first of its major digital initiatives. Intended as an educational tool Screenonline presents short -form clips from its film and television collections, along with supporting text written by subject and genre experts from the archive’s curatorial team.

‡ BFI Screenoline can be accessed at http://www.screenonline.org.uk/

FIAT/IFTA World Conference 2013 in Dubai 3

Charles Fairall, Helen Edmunds

2004 - BFI Mediatheque§ a free digital public access portal, presenting browse-able, curatorially contextualised, full-length content to attending customers. Initially based within the BFI Southbank complex, BFI mediatheques and their growing body of digitised programmes, currently over 2,500, are being opened throughout the UK

2007 – BFI YouTube** channel is launched providing the public free access to film culture and heritage. 11 million hits and up, viewers can catch up with BFI restoration programmes such as The Genius of Hitchcock or see Santa Claus as imagined by G. A. Smith in 1898.

2009 – BFI /JISC funded InView: British history through the lens†† online access across Super Janet networks to over 1,000 hours of curatorially interpreted film and television works for academic and scholarly use by the UK’s higher education community

2009 – BFI Digital National Television Archive preservation and access to television collections through processes for acquisition of current digital off-air broadcasts and selected production quality programmes along with a programme of mass digitisation for collections locked within obsolete analogue legacy videotape formats

Figure 2 Digital operations at the BFI National Archive’s Conservation Centre

Bridging the analogue-digital divide presented the archive’s conservation centre preservation and collections management teams with some considerable challenges. There was little time to reflect on the outcomes of both HLF and SHUK projects if we were to keep up with demand for digital access to the BFI’s collections, while maintaining appropriate levels of preservation and collections care. Joining together more intensely than ever before to maintain impetus, preservation, collections management and

§ Information on the curated collections of moving image available at BFI Mediathques can be found at http://www.bfi.org.uk/archive-collections/introduction-bfi-collections/bfi-mediatheques

** The BFI YouTube channel http://www.youtube.com/user/BFIfilms

†† Access InView: British history through the lens at http://www.bfi.org.uk/inview/

4 FIAT/IFTA World Conference 2013 in Dubai

4K film scanning for restoration Off air TV preservation

Real time HD film scanning for digital access

Turning a national archive digital, by degrees…

information teams worked relentlessly to create new workflows and processes to provide a file based digital approach to access and preservation that would complement the existing and no less important needs of a predominantly analogue archive.

EVOLUTION OF NEW SKILLS PROFILES AND STAFFING STRUCTURES

In 2010 the BFI National Archive underwent a reorganisation, the Collections and Information department restructured to form conservation, curatorial, collections management and documentation teams.

The new Collections Management team focuses on archive procedures and standards, storage and access. Within it, the Collections Gateway team, dedicated to facilitating access to collections materials was formed. This team works with all users, both internal and external, to enable access to the film and TV heritage collection. This can range, for example, from access requirements to deliver BFI’s own cultural programme; to working with donors and rights holders or providing direct access by UK broadcasters or UK Parliament. Collections Gateway receive requests to access archive content via CID’s integrated workflow and with an understanding of the available resources balanced with the client’s deadline, determine the most effective workflow at the conservation centre to deliver the clients requirements. The team have had to develop knowledge and understanding of digital workflows, file types and digital delivery mechanisms in response to the changing nature of access requests.

The Collections Gateway team of access officers are also responsible for delivering the BFI’s viewing service for researchers. Historically, this function has been about providing physical access to the moving image viewing collection, by transporting film prints or videotapes from the Conservation Centre to central London for a viewing session. More recently this service has evolved, now offering digital access at the BFI Reuben Library, BFI Southbank. Two dedicated work streams have been established at the conservation centre to digitise a range of video tape and film viewing copies on demand.

To address the ever increasing digital access and preservation demands a digital preservation engineer has been appointed and a small Digital Operations team created, drawing upon the talents of existing staff and their considerable experience gained across the archive’s film and video labs.

Traditional analogue conservation skills and processes have been firmly maintained, covering all areas of film and video. Through a combination of training programmes and experience gained on the job within the archive, all film conservation specialists are now trained in the use of HD film access scanners to meet demands for on demand digital research. These changes to working practice provide benefits all round and allow archive staff to engage more fully in projects and interface effectively with the BFI’s framework of commercial suppliers.

TOOLS AND STANDARDS

Tremendous benefits were gained from the Securing the National Collection strand of the SHUK programme, which impacted the national collection in three main ways: radically improved film storage through the BFI master film store (MFS); a powerful new collections information database (CID) and the addition of more technologically advanced laboratory equipment to enhance film conservation processes.

In combination with a new organisational structure that included a dedicated documentation team, with a Head of Information, responsible for BFI collections data, a

FIAT/IFTA World Conference 2013 in Dubai 5

Charles Fairall, Helen Edmunds

significant period of change was underway and the foundations laid for the archive to function appropriately in the digital age.

FILM STORAGE

The construction of the BFI’s MFS, an advanced low temperature and low humidity storage facility for the long-term preservation of master status film materials ensures that nitrate and safety film remains viable for future use which will inevitably include high levels of digitisation.‡‡ Over the last two and half years a collection management review for all holdings was completed, focusing primarily on the preservation status master film collection. A key component of this programme was to move 330,000 film cans to BFI MFS, with some 225,000 cans requiring a barcode label to be applied prior to the move. Additional preparatory work in advance of the move programme included identifying and segregating film with magnetic components which are considered unsuited to low temperature storage and replacing in excess of 125,000 film cans which were below the necessary standard.

Figure 3 BFI Master Film Store, Gaydon, Warwickshire

Photo credit: Edmund Sumner (c) BFI

COLLECTIONS INFORMATION DATABASE WITH INTEGRATED WORKFLOW SYSTEM

The BFI Collection Policy, reviewed in November 2011, stated that: documentation enables accountability for moving image works and physical objects in the collections by recording provenance, ownership and rights. It supports collection management by tracking location, recording technical data and conservation activity. Documentation underpins access by describing the collections and providing associated information, interpretation and meaning. The BFI aims to adhere to existing standards and best practice in the film archive sector or those communities of practice relevant to a particular part of the collection.”(BFI Collection Policy, 2011: section 5, p.17)

In parallel with the design and construction of the Master Film Store, the BFI worked closely with Adlib Information Systems§§, the company behind the BFI’s Collection Information Database (CID), to develop a new collections management system, this included the important initial work of integrating 35 diverse datasets and varied discrete legacy databases. The development of CID was informed by the BFI’s new policy for documentation*** ensuring that data now had a common standard and taxonomy controls.

‡‡ BFI Collection Policy on conservation, section 6.4, http://www.bfi.org.uk/about-bfi/policy-strategy/bfi-national-archive-collections-policy

§§ Adlib Information Systems, http://www.adlibsoft.com/

*** The BFI’s approach to the principles and standards of documentation are referred to in section 5.1 and 5.2 of the BFI Collection Policy

6 FIAT/IFTA World Conference 2013 in Dubai

Turning a national archive digital, by degrees…

The plan when developing CID was to ensure duplication in data entry no longer occurred; that outdated management processes were revised and improved; accurate reporting to inform collection management policies was achievable; interoperability for data sharing was possible and that the ambition to unify all collections data was fulfilled.

CID now enables users to search across all film and television information and the BFI’s collections affording the opportunity for the national collection to be interpreted flexibly and with significantly greater functionality.

The CID development also included an integrated workflow solution. Our aim when developing this system with Adlib, was to achieve efficiency across a wide range of activities, tasks and workflows; to standardise collection management procedures with wider museum and archive community; integrate barcode and tracking for improved collection security, enhanced location, inventory and movement control; achieve accurate reporting and remove reliance of paper trails; know what resources are available, effectively manage deadlines and employ an efficient ordering process for accessing items from the collection.

A comprehensive review of conservation centre activities was undertaken, resulting in a defined list of activity tasks which were then built into the workflow system. These activity tasks are used to construct workflow jobs to deliver clients requests.

Figure 4 Activity tasks defined within CID’s integrated workflow system

At the time of developing CID’s integrated workflow, activity tasks were future proofed for digital workflows, such as 2K/4K scanning and data migration to LTO.

Figure 5 is an example from CID’s integrated workflow system, where a workflow job has been constructed to undertake digital quality control of a newly acquired DCP, at the request of a fiction curator.

FIAT/IFTA World Conference 2013 in Dubai 7

Vaults operations

Pick items

Return items

Transport out

Transport in

Loan in

Loan out

Dry lab operations

Inspection

Technical selection

Preparation for printing

Preparaion for scanning

Preparation for

projection

Other preparation

Service on return

Digital operations

SD scanning

HD scanning

2K/4K scanning

Audio encoding

Video encoding

Transcoding

Ingest data

Data migration

Data migration to

LTO

Dubbing

Video copy

Audio copy

Wet lab operations

Film cleaning

Film printing

Film processing

Film image quality

Analogue image

grading

Digital image grading

Digital image restoration

Silent intertitle

restoration

New title creation

Quality control

Technical acceptance -

Theatre

Video quality control

Audio quality control

Digital quality control

Charles Fairall, Helen Edmunds

Figure 5 Example of a job in CID’s integrated workflow system, illustrating the use of activity tasks

Figure 6 is an example where a request from a researcher has resulted in a workflow job being constructed and implemented to enable digital access. In this example three VHS tapes were encoded, with proxy files to be viewed by the client at the BFI Reuben Library via CID.

Figure 6 Example of a job in CID’s integrated workflow system, illustrating part of the structure for BFI conservation centre workflows for the digitisation on demand of videotape source materials

CID’s integrated workflow system provides the BFI Archive with the tools to move collections items through various laboratory processes reducing a previous reliance on a plethora of paperwork for coordinating workflow in combination with MS Excel or MS Access databases; it provides real time information on the location of materials as they move through conservation centre film, video and digital laboratories; documents metrics and information on the reason for access by virtue of a series of activity codes.

CONSERVATION TECHNOLOGY

Conservation technology is a critical aspect of operations at the BFI National Archive’s conservation centre. Ensuring film, video and digital technologies remain operational requires skilled engineers, robust maintenance schedules and a commitment to documenting knowledge of obsolete machinery, formats and processes. A determined ‘hunter, gatherer’ approach to acquiring obsolete technology over the years allows us the continued ability to access and thereby preserve a range of obsolete video formats.

SHUK funding supported the development of existing conservation centre technical capabilities through the introduction of new equipment and tools for film cleaning, photochemical printing, digital scanning, grading and restoration, thereby ensuring the archive is equipped to digitise film for preservation, access and restoration purposes.

The purchase of four HD film access scanners in 2011 heralded the start of file based digital access to the moving image collection for research purposes.

8 FIAT/IFTA World Conference 2013 in Dubai

Turning a national archive digital, by degrees…

BFI FILM FOREVER 2012-2017

Alongside major plans for film production funding and education, Film Heritage Unlocked is the archive strand of the three key strategic priorities of the BFI’s 2012-2017 Film Forever plan†††, which advances the UK’s film and television archive’s ambitions for digital access and preservation by several orders of magnitude.

To that end, over the 5 year project, funded primarily by Lottery funds, 10,000 film titles from across the UK’s archives and rights holders will be digitised with the primary intention of providing public access. To achieve this, the BFI National archive has to further build upon its capabilities, providing digital access and preservation to some 5,000 of the titles from its own vaults.

Digital Capability: to complement film cleaning, scanning, restoration and grading functions developed through the SHUK project, UK archives’ capacity for digitising film must grow to meet the ambitions of Film Heritage Unlocked. In particular, faster scanning will be sought, which combined with new skills and efficient workflows will create digital packages suitable for a range of digital channels, ranging from digital cinema through to the online BFI Player.

Digital Infrastructure: requirements for long-term data preservation and digital asset management were gathered through a BFI led working group to provide the basis for a competitive dialogue procurement procedure. At the time of writing, 6 bidders have been selected to present scalable archive data storage and management solutions with cost and management plans detailed for a projected 20 year period.

Film Digitisation: as digital capability increases and working within a framework of 6 chosen commercial facilities, regular batches of selected film elements are being are being prepared by conservation specialists and processed through workflows that are specific to both physical type and appropriate digital deliverables. Raw, preservation quality scans are to be maintained for future use within the long-term data preservation function of Digital Infrastructure.

Stills digitisation: the collections management team having relocated the BFI’s photographic collection from central London office storage to conservation centre conditioned vault storage in 2012, followed immediately on with a procurement process to identify a digitisation partner. Save Photo are working with the BFI to digitise the collection of 1.2 million still images and 156,000 transparencies. The collection of images represent more than 150,000 film and television programmes, comprised of portraits, capturing events on and off screen, images of studios, cinema and events from 1896 to the present day. This exciting project commenced in 2013, to complete in 2016. Throughout the 3 year programme newly digitised images will be regularly added to CID. Public access to these images is available on BFI premises in the BFI Reuben Library, Southbank, London.

While we are by no means at the end of the digital revolution, it is clear that the major advances made in the area of digital archiving now make the analogue-only past virtually unrecognisable: maybe even unthinkable.

The Film Forever plan states that: …making our rich screen heritage available is integral to the BFI’s broader cultural ambitions to support British film and British talent, and to provide a varied programme of British and international film which can attract a wide range of new audience – public and professional – to a deeper and richer range of film. The plan clearly indicates that the “ultimate goal is to digitise and make accessible to the public all of the UK’s screen heritage” and proposes a “new programme driven by

††† Details of the BFI plan Film Forever can be found at http://www.bfi.org.uk/about-bfi/policy-strategy/film-forever

FIAT/IFTA World Conference 2013 in Dubai 9

Charles Fairall, Helen Edmunds

both public demand and curatorial expertise which will use digital delivery across a range of platforms to bring collections to the public with an ease of access that was previously impossible.” (Film Forever Supporting UK Film, BFI Plan 2012-2017, October 2012, p.27)

10 FIAT/IFTA World Conference 2013 in Dubai