Principles and Practice of Managing Digital Stuff

CARDIFF, JASNA, MICHIGAN | APRIL 2015@digitalfay

ED FAYEXECUTIVE DIRECTOR, OPEN PRESERVATION FOUNDATION

PRINCIPLES AND PRACTICE OF

MANAGING DIGITAL ‘STUFF’



Everything is impermanent.

Source: The Buddha

“



Attachment causes suffering.

Source: The Buddha

“



Attachment causes suffering.

Source: The Buddha

Change is also the cause of

suffering in the digital world!

“




Digital preservation efforts don't

get the attention they should

because of perceived high total

cost of ownership (TCO) and a

lack of monetary return on

investment (ROI)

“

Source: University of Minnesota, Hype Cycle for Education (https://hypecycle.umn.edu/)



Ok, digital preservation…

Preserve all the things!

The trough of complexity

Let’s do this!

Humans can preserve after all



National newspaper collection

1762 Thomason collectionCivil war and other 17C newsbooks and papers

1818 Burney collectionBound volumes covering 1603-1817

1822 Stamp OfficeCollected for tax purposes and transferred

1869 Legal deposit legislationDirect deposit to the British Museum


A person who publishes in the

United Kingdom a work to which

this Act applies must at his own

expense deliver a copy of it to …

any deposit library entitled to

delivery.

“

Source: UK Legal Deposit Legislation 2003 (updated in 2013 to include born-digital)


Why preserve

Legal obligation

Regulatory requirement

Funder mandate

Risk management

Opportunity



Digital collection types (from a survey of 139 institutions worldwide)

Source: http://openpreservation.org/knowledge/surveys

What to preserve


Threat Digital Risk Physical

Disasters Operator error

Natural disaster

Similar

Attack External attack

Internal attack

Similar

Management Economic failure

Organisational failure

Similar

Digital vs physical threats

Based on Rosenthal et al. (http://www.dlib.org/dlib/november05/rosenthal/11rosenthal.html)


Threat Digital Risk Physical

Process Software failure

Software obsolescence

Self-rendering

Data Media failure

Media obsolescence

Acid paper

Cave painting

Infrastructure Hardware failure

Hardware obsolescence

Furniture

Buildings

Communication error

Network failure

Not network

reliant

Digital vs physical threats

Based on Rosenthal et al. (http://www.dlib.org/dlib/november05/rosenthal/11rosenthal.html)



Carrier Lifetime (years)

Stone tablet 10,000

Paper 1,000

Film 100

Disk 10

Source: Richard Wright (http://dx.doi.org/10.7207/twr12-01)


Carrier Lifetime (years)

Stone tablet 10,000

Paper 1,000

Film 100

Disk 10

Source: Richard Wright (http://dx.doi.org/10.7207/twr12-01)





File systemStructures data for retrieval

Provides data as files

Hardware

interaction


NetworkDistributed environment or data

SoftwareProvides interactions with files

Off-the-shelf or bespoke

File formatsEncode data in meaningful ways

Proprietary or open standards


Suffering can be overcome.

Source: The Buddha

“


Suffering can be overcome.

There is a path to nirvana.

Source: The Buddha

“


Digital preservation is best

understood as the active

management of digital

information to ensure its

continued accessibility over

time.



Physical preservation

Is keeping the bits safe


Physical preservation

Is keeping the bits safe

ReplicationMultiple copies

Multiple storage media

Multiple locations

FixityEnsure identity over time: authenticity (using ‘checksums’)

Prevent natural, accidental, or malicious alteration


Logical preservation

Is understanding dependencies




Dependencies are social because

environments are social.

Creators and depositors.

Archive and repository.

User community.




Characterise collectionsIdentify formats

Identify features

Characterise environmentsTechnical environments

Social environments




Metadata is a love note

to the future

Source: Jason Scott (@textfiles)

“


Digital preservation

Is mitigating risk



Is mitigating risk

PolicyCollection/deposit guidelines

Preservation intent

PlanningPreservation approaches

Process and skills development

Technology watchMonitoring the environment and your collections



Is mitigating risk

Open standards and softwareConformance to standards and control of features

Transparent processing and quality assurance

InterventionMigration of file formats

Emulation of legacy environments


Digital repository

Digital Object

Repository


Digital repository++ preservation

Component Management

Technology Watch

Execution Platform

Digital Object

Repository

Preservation Planning


Community

Digital repository++ preservation

Component Management

Technology Watch

Execution Platform

Digital Object

Repository

Preservation Planning

Institution



DPN (US)

JISC (UK)

…

National EU International

Technical

Business

Political DCC/DPC (UK)

NCDD (NL)

NESTOR (DE)

NDIPP (US)

NSLA (AU/NZ)

…

EC

OPF/IMPACT/PRESTO/…

APA/DLM/RDA/…

UNESCO

Based on H. Balk (Centres of Competence and Virtual Centres of Excellence)

CARDIFF, JASNA, MICHIGAN | APRIL 2015@digitalfaySource: http://cordis.europa.eu/fp7/ict/telearn-digicult/digicult-projects_en.html

2006-10 (FP6)€19m (€80m)

2011-14 (FP7)€68m (€120m)


Memory

Institutions

Research and

Development

Service Providers(repository systems)

Visibility/Adoption

Sustainability

Requirements

Testing

Software

Theory

Sustainable software

Market opportunity

New communities

(tool adoption/testing)

Sustainable software

Knowledge exchange

Development efficiency

Agile/risk tolerance

Experience

Requirements

Testing

Data

Global benefits

- sustainability

- data broker

- multiple communities

- TRUST/CONFIDENCE

CARDIFF, JASNA, MICHIGAN | APRIL 2015@digitalfaySource: Mariana Mazzucato



Final thoughts

The system is not the solution.

(People are!)

Continuous improvement is

better than over-engineering.

Diversify by collaboration*

(* Shaun Hendy at iPres 2014)

!