© Philip W. Lord & DCC 20041

Philip Lord Digital Archiving ConsultancyAlison Macdonald Digital Archiving ConsultancyLiz Lyon Digital Curation CentreDavid Giaretta Digital Curation Centre

From Data Deluge to Data Curation

e-Science All Hands Meeting 2004 Nottingham, September 2004

© DAC Ltd & DCC 2004 2

e-Science Curation

1. Curation2. Curation and e-Science3. The role of JISC/JCSR and

e-Science Core Programme4. Moving towards curatorial

services for UK (e-)Science5. The Digital Curation Centre

© DAC Ltd & DCC 2004 3

e-Research and change

• Changes in technology enable e-ScienceBUT

• These same advances put data at riskAND

• Raise issues of policy and practiceIMPLYING A need for continuing care and management of data

This continuing care requires data CURATION

© DAC Ltd & DCC 2004 4

Curation for e-Research

• CURATION:The activity of managing and promoting the use of data from its creation to ensure it is fit-for-purpose and remains available for discovery and re-use.

• ARCHIVING:A curation activity ensuring data is properly selected, stored making it available for access over the long term.

• PRESERVATION:An archiving activity in which data are maintained over time so they can still be accessed and undertood through changes in technology

© DAC Ltd & DCC 2004 5

Curation, JISC and the e-Science Core Programme

• JISC has recognised this as an area of need and instituted a Preservation and Records Management Programme

• The e-Science Core Programme is also addressing the data curation question

• JCSR commissioned a report into the status of e-Science curation in the UK, which reported in 2003

• JISC and e-Science Core Programme funding the newly formed Digital Curation Centre

© DAC Ltd & DCC 2004 6

The current status of curation

The e-Science Curation Report:• Surveyed the UK HE and Research Councils’

current practice and provision for data curation

• Surveyed the bench-scientists perceptions and awareness of the issues of data curation

• Made 10 strategic and 19 tactical recommendations for future action

• Developed a process model for academic science curation

© DAC Ltd & DCC 2004 7

Perceived need at the bench

• 79% of data generator responses said their primary data would be of value after project end

• 90% said their summary/derived data would be of value after project end

• 94% said their published data would be of value after project end

BUT• A generally low understanding of the curation

issues attached to maintaining that value

© DAC Ltd & DCC 2004 8

But short-comings in provision

Table percentages (over all responses)

Has financial provision been made for keeping data?

Yes No Don’t know

Do the terms of your funding require data to be archived/preserved?

Yes 17% 21% 8%

Some cases

- 8% -

No 4% 23% 4%

Don’t know

- 4% 10%

© DAC Ltd & DCC 2004 9

Issues needing addressing

1. Curation needs recognition at all levels – including the highest

2. Overall responsibility needs to be assigned and a task force established

3. The Research Councils should make curation an article of policy

4. The Digital Curation Centre is urgently needed to provide generic services

5. Discipline-specific centres are needed to support the user communities

© DAC Ltd & DCC 2004 10

Issues needing addressing

6. Exemplar research projects are needed to identify measurable benefits and rescue valuable data at risk

7. Work should be co-ordinated:a. With e-Library initiativesb. Between and within the Research Councilsc. With supra-national bodies, specifically in Europe

8. Institutions should be encouraged to adopt OAIS-compliant repositories

9. Training and awareness-raising are key

© DAC Ltd & DCC 2004 11

A model for curation - level 1

Scientist

Research Process

Secondary(derived)

data

Tertiarydata for

publication

Primary publication

Secondarypublication

Tertiarypublication

PeerReview

Pre-prints& e-Prints

Publicationarchives

Library - Peers - Public - Industry

PublicationProcess

Primary data

Web Content

Patent data

Research ProcessLevel 1curation

© Philip Lord, 2003

Scientist

Research Process

Secondary(derived)

data

Tertiarydata for

publication

Primary publication

Secondarypublication

Tertiarypublication

PeerReview

Pre-prints& e-Prints

Publicationarchives

Library - Peers - Public - Industry

PublicationProcess

Primary data

Web Content

Patent data

Research ProcessLevel 1curation

© Philip Lord, 2003

© DAC Ltd & DCC 2004 12

A model for curation – level 2

Scientist

Research Process

Secondary(derived)

data

Tertiarydata for

publication

Primary publication

Secondarypublication

Tertiarypublication

PeerReview

e-Prints

Publicationarchives

Library - Peers - Public - Industry

PublicationProcess

Primary data

Web Content

Patent data

Research Process

Researchbased on

data

Metadata

Archivist

© Philip Lord, 2003

Level 2curation

Archiveddata

Scientist

Research Process

Secondary(derived)

data

Tertiarydata for

publication

Primary publication

Secondarypublication

Tertiarypublication

PeerReview

e-Prints

Publicationarchives

Library - Peers - Public - Industry

PublicationProcess

Primary data

Web Content

Patent data

Research Process

Researchbased on

data

Metadata

Archivist

© Philip Lord, 2003

Level 2curation

Archiveddata

© DAC Ltd & DCC 2004 13

A model for curation - level 3

Scientist

Research Process

Secondary(derived)

data

Tertiarydata for

publication

Primary publication

Secondarypublication

Tertiarypublication

PeerReview

e-Prints

Publicationarchives

Library - Peers - Public - Industry

PublicationProcess

Primary data

Web Content

Patent data

Research Process

Researchbased on

data

Metadata

CurationCurator

Curation Process

Data repositories

© Philip Lord, 2003

Level 3curation

Archiveddata

Scientist

Research Process

Secondary(derived)

data

Tertiarydata for

publication

Primary publication

Secondarypublication

Tertiarypublication

PeerReview

e-Prints

Publicationarchives

Library - Peers - Public - Industry

PublicationProcess

Primary data

Web Content

Patent data

Research Process

Researchbased on

data

Metadata

CurationCurator

Curation Process

Data repositories

© Philip Lord, 2003

Level 3curation

Archiveddata

© DAC Ltd & DCC 2004 14

The Digital Curation Centre

• Funding awarded in early 2004 to– University of Edinburgh (Lead)– University of Glasgow– UKOLN (Bath)– CCLRC

• Formal launch in autumn 2004• Funding from JISC and e-Science Core

Programme assured for three years– Continuity beyond year three to be established

© DAC Ltd & DCC 2004 15

DCC Programme

• Establish a vibrant research programme– addressing wider issues of data curation

• Nurture strong community relationships– Centres for community advice, help and training

• Develop services– for testing and evaluating tools, methods, standards and

policies in realistic settings– offering a repository of tools and technical information, a

focal point for digital curators

• Achieve the 'virtuous circle'– feeding expertise, experience and need into its research

programme on data curation– transforming research-led innovation into services that

enhance productivity of practice

© DAC Ltd & DCC 2004 16

Information and contacts

SEE:

Digital Curation Centre:www.dcc.ac.uk digitalcuration@ed.ac.uk Digital Archiving Consultancy:www.d-archiving.com support@d-archiving.com

JISC Preservation and RM Programme:www.jisc.ac.uk/index.cfm?name=programme_preservation

e-Science Report:www.jisc.ac.uk/uploaded_documents/e-ScienceReportFinal.pdf