A centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Funded by: This work is licensed under the Creative

a centre of expertise in data curation and preservation

Preserving Digital Archives LUCAS March 2006

Funded by:This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 2.5 UK: Scotland License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/2.5/scotland/ ; or, (b) send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.

Digital Archiving or Digital Archaeology?

Dealing with Digital Records

Maureen Pennock Digital Curation Centre



Today’s talk• The DCC

• Background & Context

• What We Do

• Digital Preservation & Archiving• Preservation & Curation: Issues & challenges

• Specific challenges for repositories receiving digital deposits

• Examples

• Proactive solutions



UK Digital Curation Centre• JISC Circular 6/03 called for bids in digital curation

• JISC and the e-Science Core Programme funding• for development, services and outreach in digital

curation• for a research programme

• Impetus to action• Growth in e-Science activity and data creation• Recognition that continuing access to digital

information is needed



Partners• University of Edinburgh (lead site)

• Chris Rusbridge, Prof Peter Buneman

• University of Glasgow - HATII• Prof Seamus Ross, Director of HATII and Erpanet

• University of Bath - UKOLN• Dr Liz Lyon, Director of UKOLN

• Councils for the Central Laboratory of the Research Councils (CCLRC)• Dr David Giaretta



Objectives• Lead a vibrant international research programme to improve

quality in data curation and digital preservation

• Deliver effective, efficient and high demand services

• undertake evaluation of tools, methods, standards and policies

• work with the community to establish registries of tools and technical information

• Create an active, innovative and collaborative Associates Network

• Connect communities

• Universities and Research institutions

• Scientific data and documents

• International & cross-sector



•Industry

•research •collaborators

•standards bodies

•testbeds•& tools

•communities of •practice: users

•community •support & •outreach

•research

•development •co-ordination

•service •definition •& delivery

•management •& admin •support

•Collaborative •Associates •Network of •Data•Organisations

•curation organisations •eg DPC



Research• Annotation in Databases• Data archiving• Socio-economic and legal issues• Metadata extraction and curation• Provenance and databases• Data transformation, integration and publishing• Security• Supporting technologies• Organisational and cultural challenges to digital

curation



Development• DCC Approach to Digital Curation (white paper) –

sets out the path for development activities:• Monitoring international standards• Development of a Representation Information

Registry/Repository (DCC RIR)• Development of recommendations for tools and methods for

generating Representation Information• Creating testbeds for digital curation tools• Creating auditing and certification processes for trusted

repositories



Services• Information Services

• Community-developed Digital Curation Manual• Briefing Papers & FAQ’s• Technology Watch• Case Studies• Best Practice Checklists

• Advisory Services• Events: information days, workshops, training,

conferences• Helpdesk

• Audit and Certification Services



Out of scope

• We don’t• Accept digital archival materials for curation,

storage, or preservation• Maintain a computer hardware museum• Offer file migration or conversion• Repair damaged data resources• Carry out digital archaeology



Summary• Support and promote continuing improvement

in the quality of data curation and preservation activity

• Nurture strong community relationships between practitioners, researchers, and curators

• Address digital curation from all aspects of the records life-cycle

• Develop and promote curation knowledge, tools and techniques

• Identify and research new organisational, technical, and supporting curation challenges



Digital Curation• Digital curation is all about maintaining and

adding value to a trusted body of digital information for current and future use; specifically, we mean the active management and appraisal of data over the life-cycle of scholarly and scientific materials.

• Digital Curation brings a whole host of

challenges• The range of stakeholders that affect the

survival of digital material cuts across the whole life-cycle

• Everyone plays an important role



Preserving Digital Archives I• The main problems:

• Technology obsolescence• Hardware and software dependencies• Leading to authenticity and integrity issues

• Fragility of digital media• Bit deterioration • Media breakage

• Lack of good practice examples for• Data creation• Data documentation



Preserving Digital Archives II• Some other problems:

• Cultural • Creators don’t (want to) understand the technical

issues

• Contextual• Metadata identifying resources is often insufficient

• Financial• Difficult to anticipate costs of different activities

• Organisational• Need for collaboration between parties is often

overlooked• Infrastructure and existing procedures often inadequate

• Legal• Largely an unknown quantity



Receiving Digital Archives I• Specific challenges for organisations receiving

externally originating digital deposits• Possible lack of control/influence over

• resource creation• resource & collection documentation• transfer metadata• technical metadata• file storage format• media storage format



Receiving Digital Archives II• Further challenges:

• Lack of required hardware to read discs• Lack of required software to read file formats• Lack of in-house knowledge about digital deposits• May be a hybrid collection – paper & digital

• This causes further challenges

• Final archive may only accept limited formats• You *might* have to convert the material for archiving

• Solving all of these issues may cause you money



Digital archaeology for digital archiving• Digital archaeology

• A form of data recovery• Exploring what is buried below the surface• Rescuing neglected and damaged data resources• May reveal a lot of unwanted material, but…• … may also reveals some valuable treasures

• May be necessary if repositories receiving digital deposits are to ascertain

• What material has been deposited• Whether the material is valuable• How to access and store the material for the longer-

term



Digital Archaeology - Examples• NCUACS

• Received Old BBC Discs – obsolete storage format

• National Archives/CAMiLEON project• Domesday Project – rescuing data from obsolete media and

data formats

• NARA• Hurricane Marilyn resulted in weather damaged diskettes and

12 inch WORM discs

• German Unification• West German Archivists recovering obsolete East German

data



What can you do?• Find a reliable way to explore deposits and

ascertain the value of the records• Some records may be damaged or fragmented

• Rescue records where possible• Store those that are not

• You never know what we will be capable of in the future

• Most data can be rescued – given enough time and money



How can you do it?• Learn from experience

• What you have done in the past• What others have done in the past• Collaborate and share knowledge

• Be prepared • To change processes• To embark on a learning curve• To communicate issues to potential depositors

• Develop in-house knowledge• Cuts costs• But be prepared to combine it with specialised

external knowledge when necessary



What can help you?• Identify and use tools developed by others

• DCC tools such as the RIR• Erpanet tools on Ingest, Costing, Selecting

Technologies, Developing a Policy• Tools from the NAA• Tools from the NLA• Tools from the Digital Preservation Testbed

• Identify and use valuable case studies• Ross & Gow: Digital Archaeology: Rescuing

Neglected and Damaged Data Resources• Jeremy John, British Library: Digital Manuscripts

Project• Studies presented at this workshop



Thank you.

Questions?

Maureen [email protected]

Join the DCC Associates Network at http://www.dcc.ac.uk

Documents

A centre of expertise in data curation and preservation Preserving Digital ArchivesLUCAS March 2006 Funded by: This work is licensed under the Creative