Report on Preservation of ETDs:The LOCKSS Prototype

The work of Kamini Santhanagopalan Virginia Tech Graduate Student in Computer Science

Reported at the 9th International Symposium on ETDs, Quebec City

Presented By:Gail McMillan, Director

Digital Library and ArchivesVirginia Tech

Agenda

Goals What is LOCKSS? Participating Universities International ETD Preservation Analysis and Results Conclusion

Digital Preservation

Goal: Information should be Readable Usable in the future

Preservation – NOT just backup Existing preservation techniques

Floppy, CD and hard disk drives Central and distributed database

servers

Technical Infrastructure Goals

Build on successful LOCKSS open-source model

Create dark archive for locally produced digital content

Use off-the-shelf hardware Use open-source software Easy replication Demonstrate LOCKSS scalability

LOCKSS Lots of Copies Keep Stuff Safe

Peer-to-peer digital preservation system Open source software Turns an inexpensive desktop computer

into a digital preservation appliance Easy, inexpensive way to

Collect Store Preserve Provide access to the contents--or, not.

Functions of LOCKSS (1)

CollectVia a web crawler

Appropriate crawl rules are specified

Preserve and AuditEvery institution preserves

Its own contents Contents of partner universities Contents are polled to determine

authenticity and reinstate bad files

Functions of LOCKSS (2)

Provide access By running web proxies Open or restricted access

Dark Archives for partners’ ETDs Levels of access controlled at originating

institutions Administration

Via a web user interface Controlling access to cached contents

and other functions

LOCKSS Preservation

Contents of each university (nodes M1 through M5) preserved at every other university Multiple, dispersed

copies Not a backup--

nothing is overwritten

All versions retained

M1

M3

M2

M5

M4

ASERL-LOCKSS-ETD Initiative

Florida State University Georgia Institute of Technology University of Kentucky University of Tennessee Vanderbilt University Virginia Polytechnic Institute and State University

http://www.aserl.org/

Preservation using LOCKSS

Prerequisites Minimum hardware configurationLOCKSS software installed on all

participating partners’ systemsPermissions for the LOCKSS system

to collect, preserve, periodically validate, repair ETDs

Example Hardware Configuration Enterprise (3TB)

Dell PowerEdge Server 1850 LOCKSS - $3500

Dell PowerEdge Server 1850 Firewall - $2500

Dell/EMC AX100 SAN (3TB) - $10,000

RedHat Enterprise AS – 2@$50 = $100

UPS - $700 Server Rack - $1200

Grand Total - $16,800.00 w/ Rack - $18,000.00

Desktop (200Gb)

Intel Based Desktop LOCKSS (200Gb) - $500

Intel Based Desktop Firewall - $350

CentOS Linux - $0 UPS - $50

Grand Total - $900.00

Participating Universities

International universities Pontifícia Universidade Católica do Rio

de Janeiro, Brazil Humboldt-Universität, Germany University of Cape Town, South Africa

US universities Florida State University Georgia Tech Virginia Tech

International ETDs Preservation (1)

For international universities KS wrote plug-ins to collect contents

(ETDs) from the 3 universities For US universities

Verified and reused OAI plug-ins for the 3 universities

International ETD Preservation (2)

Example ETD collectionUniversity of Cape Town ETD collectionManifest (i.e., permissions) page:

http://pubs.cs.uct.ac.za/lockss/manifest.html

Screen shots of UCT plug-in and the crawl results of contents follow

University of Cape Town Plug-in (1)

UCT plug-in:

Crawl Results with

• Level (depth) =4

• Fetch delay = 6 seconds

Harvested International ETD Collections

Harvested American ETD Collection [source: http://lockss-etd.lib.vt.edu:8081/DaemonStatus ]

Tutorial on how to write plug-ins

KS developed mini-tutorial http://scholar.lib.vt.edu/lockss/introduction.htm

10 screens This tutorial can be

Generalized for ETD plug-ins Extended to write OAI plug-ins

Conclusion and Future Work

International ETDs can be harvested and preserved using LOCKSS and OAI-PMH

It requires cooperation and collaboration from participating universities

Future Work An online portal open for the public to view

certain details Brazil expressed interest in formalizing ETD

preservation for the NDLTD using LOCKSS

Acknowledgements

Special thanks to LOCKSS (Stanford University) Thomas Robertson Seth Morabito

Thanks to all participating universities Florida State Georgia Tech Humboldt-Universität, Germany Pontifícia Universidade Católica do Rio de

Janeiro, Brazil University of Cape Town, South Africa Virginia Tech

Send Questions/Comments to

ksanthan@vt.edu