Transcript
Page 1: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Digital Libraries from Theory to Framework to Application:

5S Model, Open Archives Initiative, and Networked Digital Library of

Theses and Dissertations

December 6, 2000 --- Seoul

Edward A. Fox

[email protected] http://fox.cs.vt.edu

CS DLRL Internet TIC

Virginia Tech, Blacksburg, VA, USA

Page 2: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Acknowledgements (Selected)

Mentors: JCR Licklider, Michael Kessler, Gerard Salton Sponsors: Adobe, IBM, Microsoft, NLM, NSF, OCLC, SOLINET,

SURA, UNESCO, US Dept. of Ed. (FIPSE), … VT Faculty/Staff: Tony Atkins, Thomas Dunbar, Debra Dudley,

John Eaton, Gwen Ewing, Peter Haggerty, Gary Hooper, Gail McMillan, Len Peters, James Powell, …

VT Students: Emilio Arce, Fernando Das Neves, Brian DeVane, Robert France, Marcos Goncalves, Scott Guyer, Robert Hall, Neill Kipp, Paul Mather, Tim McGonigle, Todd Miller, Constantinos Phanouriou, William Schweiker, Ohm Sornil, Hussein Suleman, Patrick Van Metre, Laura Weiss, …

Page 3: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

JCDL 2001 First Joint ACM/IEEE Conference on

Digital Libraries (+ NSF DLI-2 PI mtg)

http://www.jcdl.org June 24-28, 2001 in Roanoke, VA Conference Committee: General Chair: Edward A. Fox, Virginia Tech Program Chair: Christine Borgman, UCLA Treasurer: Neil Rowe, Naval Postgraduate School Posters Chair: Craig Nevill-Manning, Rutgers U.

Page 4: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Virginia Tech Background

Largest university in Virginia, land-grant, football, town population 35K plus 25K students

Blacksburg Electronic Village, since 1992, with > 80% of community on Internet

Net.Work.Virginia, largest ATM network, with over 900 sites, for education, research, government

LMDS, Local Multipoint Distribution Service, gigabit wireless networking - 1/3 of Virginia

Math Emporium, 500 workstations Faculty Development Initiative, round 2 Torgersen Hall, $30M Advanced Communications and Information

Technology Center, opening, w. DLRL

Page 5: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Internet TechnologyInnovation Center

Supported by Virginia’s Center for Innovative Technology

Statewide University Partners - Governing Board:

Christopher Newport University– William Winter, William Muir, Virginia Electronic Commerce Technology Center /

Southeastern Virginia Network (VECTEC/SEVAnet)

George Mason University– Scott Martin, Internet Multimedia Center (ICM)– Steven Ruth, International Center for Applied Studies in IT (ICASIT)

University of Virginia– Alf Weaver, Internet Commerce Group (InterCom)– Jim French, Internet Digital Library

Virginia Tech– Edward Fox, Digital Library Research Laboratory (DLRL), CC, CS– Scott Midkiff, Center for Wireless Telecomm. (CWT), VTISC, ECpE

Page 6: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

ITIC @ VTResearch Areas

Collaboration (e.g., group decision support) Community networking (e.g., BEV) Internet access (e.g., statewide network) Information services (e.g., digital libraries) Modeling and simulation (e.g., Web traffic) Usability (e.g., human factors engineering) Virtual environments (e.g., CAVE, visualization)

Page 7: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

OUTLINE

Introduction, Grand ChallengeDefinitions, 5SOAINDLTDMARIAN, ENVISIONPetaPlexFor more information

Page 8: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Digital Library Courseware

http://ei.cs.vt.edu/~dlib/ WWW pages or large PDF copy files CourseInfo quizzes based on books by Michael

Lesk (MKP.com) and William Arms (MIT Press) Contents based on books, with other popular

topics added (e.g., agents) Separate pages to supplement: Definitions,

Resources (People, Projects), and References

Page 9: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Digital Libraries --- Virginia Tech

MARIAN (NLM) CS DL Prototype - ENVISION (NSF, ACM) TULIP (Elsevier, OCLC) BEV History Base (NSF, Blacksburg) DL for CS Education - EI (NSF, ACM) WATERS, NCSTRL (NSF) NDLTD (SURA, US Dept. of Education) CSTC (NSF, ACM), CRIM (NSF, SIGMM) WCA (Log) Repository (W3C) VT-PetaPlex-1 (Knowledge Systems)

Page 10: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Libraries of the FutureJCR Licklider, 1965, MIT Press

World

Nation

Province

City

Community

Page 11: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

DL-Related Timeline

1985

CoRR

DLI

ETDs

1990xxx

1995

XML

OAI

2000

NCSTRL

PDF

ProposedUgrad DL

NDLTD

DLI2

HyperCard

TEI

SGML

PCs

Hypertext Conf. DC RDF

CSTR

NSDL

ScholarlyEPub in U’s

JPEG, MPEG MPEG-7

WWW

Java

Page 12: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Computing (flops)Digital content

Com

mun

icat

ions

(ban

dwid

th, c

onne

ctiv

ity)

Locating Digital Libraries in Computing andCommunications Technology Space

Digital Libraries technologytrajectory: intellectualaccess to globally distributed information

less more

(Slide from S. Griffin, NSF)

Page 13: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

D ig ita l L ib r a ry C on te n t

A r ticle s,R e po r ts,

B o o ks

T e xtD o cum e n ts

S p ee ch ,M u s ic

V ideoA ud io

(A er ia l)P h o tos

G eo gra ph icIn fo rm ation

M od e lsS im u la tio ns

S o ftw are ,P rog ram s

G e no m eH u m a n,a n im a l,

p lan t

B ioIn fo rm ation

2 D , 3 D ,V R ,C A T

Im ag es a ndG ra ph ics

C o nte n tT yp es

Page 14: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Structured Video Browser(making video into hypermedia)

www.learn.umd.edu

IBrowse

Expository multimedia Narrative Structures

Page 15: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

ICUInformation and CommunicationUniversity

Users Web Search Engines

WWW

Servlet Engine

Web Server

OSDB

Search Server

Servlet Servlet ServletMPEG-7

DescriptionModule

1

2

3

4

5

3’

4’

5’

MPEG-7 Image Library Systems Tech. M

PE

G-7 Im

age Library System

s

Page 16: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

MP

EG

-7 Video Library S

ystems T

ech.

ICUInformation and CommunicationUniversity

MPEG-7 Video Library Systems Tech.

Video Data

Description GeneratorDescription Schemes

Design Tool

DescriptionScheme

MetaDatabase

VideoDatabase

Retrieval ServerModule

PlayerP

resentation

Module

Architecture

Page 17: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Integrated Integrated CCLINC CCLINC Translingual Information SystemTranslingual Information System

Integrated Integrated CCLINC CCLINC Translingual Information SystemTranslingual Information System

DARPA

Extraction

What is th

e north korean

movement in th

e front li

ne?

CCLINC SERVER

Info Detection

Summarization

It seems that North Korea launch a missile againAfter North Korea launched a Daipodong missilelast month, NK is perceived to proceed to an additionaltest launch. Korea, US and Japan enter into an alertstate, and prepare for a joint response policy. Korea estimates that the additional launch will be on 09/05. Japan estimates that NK’s missile range is short. USinformation says that there is no sign of launch yet.

Translation

What is th

e status of nk

missile la

unch against japan?

BugHanI IlBonE Ddo MiSaIlEul

BalSaHan Deus HaDa

2-w

a yS

pe e

c h T

ran

s ati

on

Page 18: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Digital LibrariesShorten the Chain from

Editor

Publisher

A&I

Consolidator

Library

Reviewer

Page 19: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

DL = Users Direct(Organized Artifact Mediated Communication)

Author

Reader

Digital

LibraryEditor

Reviewer

Teacher

Learner

LibrarianDr. Patient

Page 20: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Digital Libraries --- Objectives

World Lit.: 24hr / 7day / from desktop Integrated “super” information systems: 5S:

streams, structures, spaces, scenarios, societies Ubiquitous, Higher Quality, Lower Cost Education, Knowledge Sharing, Discovery Disintermediation -> Collaboration Universities Reclaim Property Interactive Courseware, Student Works Scalable, Sustainable, Usable, Useful

Page 21: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Benefits

Ease of use Effectiveness

“The benefits of digital libraries will not be appreciated unless they are easy to use effectively.” - IITA Workshop report

Page 22: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

DLs: Why of Global Interest? National projects can preserve antiquities and

heritage: cultural, historical, linguistic, scholarly Knowledge and information are essential to

economic and technological growth, education DL - a domain for international collaboration

– wherein all can contribute and benefit– which leverages investment in networking– which provides useful content on Internet & WWW– which will tie nations and peoples together more

strongly and through deeper understanding

Page 23: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Borgman et al.:Workshop Report onSocial Aspects ofDigital Libraries: http://www-lis.gseis.ucla.edu/DL/

InformationLifeCycle

Page 24: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Grand Challenges Can

Mobilize the community Spur creativity Lead to important benefits in society Push researchers to develop relevant theories Force people to work in teams/groups Convince funding agencies to invest Help bring about integration of systems,

interoperability, and seamless interfaces

Page 25: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

DL Challenges

World Digital Library (Libraries) Preservation - so people with trust DLs Scalability, sustainability, interoperability (Supporting infrastructure - networks, …) DL industry - critical mass by covering libraries,

archives, museums, corporate info, govt info, personal info - “quality WWW” integrating IR, HT, MM, ...– Need tools & methods to make them easier to build

Page 26: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Licklider – Unified Theory?

Not ready in 1960s Analog – unified field theory in physics “Mess” today – segmented field, specialities

– Database <-> Knowledge <-> Content Mgmnt– Multimedia, Hypermedia, Hypertext– Logic, Algebra, Artificial Intelligence, …

Expensive, annoying for users– Don’t know where to look– Don’t know how to use services

Page 27: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

OUTLINE

Introduction, Grand ChallengeDefinitions, 5SOAINDLTDMARIAN, ENVISIONPetaPlexFor more information

Page 28: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Definitions

Library ++ (library+archive+museum+…) Distributed information system + organization

+ effective interface User community + collection + services Digital objects, repositories, IPR management,

handles, indexes, federated search, hyperbase, annotation

Page 29: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Definition: Digital Libraries are complex systems that

help satisfy info needs of users (societies) provide info services (scenarios) organize info in usable ways (structures) present info in usable ways (spaces) communicate info with users (streams)

Page 30: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

5S Layers

Societies

Scenarios

Spaces

Structures

Streams

Page 31: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Definition: 5S Framework Societies: interacting people (, computers) Scenarios: services, functions, operations, methods Spaces: domains + constraints (e.g., distance,

adjacency): 2D, vector, probability Structures: relations, trees, nodes and arcs Streams: sequences of items (text, audio, video,

network traffic) (5 Element System: Fire, Wood, Earth, Metal, Water)

Page 32: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

5S: Combinations

Societies + Scenarios = user model Societies + Scenarios + Spaces = user interface Streams + Structures = markup Streams + Structures + Scenarios = object Structures + Scenarios = DBMS

Page 33: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Core of DL Collecting

– Authoring, Repositories, Archives, Museums, … Organizing

– Packaging of Data and Metadata, Storing– Naming/Identifying and Cataloging– Classification, Clustering, …

Serving– Indexing, Linking, Summarizing, Visualizing– Browsing, Accessing, Searching, Filtering,

Retrieving, Distributing, Using, …

Page 34: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

DL Components

User Interfaces

Workflow Mgr

DBMS

Search Engines, Classifiers, …

Data, MM Info

Gateways

Repository

Rights Mgr

MM/ HT Renderer

Page 35: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Document Models, Representations, and Accesses

Doc = stream + structure + use-scenario; hybrid (paper/electronic), digital only

Multilingual: content, summary, metadata Multimedia: structure, quality (oS), search Structured: MARC, SGML, by user: MVD Distributed collection: Kleisli, CIMI, Z39.50 Federated search: collecting, picking site(s),

parallel search / fall-back, fusing results Access: IPR, payment, security, scenarios

Page 36: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Architectural Issues

Internet middleware Independent system / part of federation Decompositions vary

– search engine, browser, DBMS, MM support– repository, handle server, client– information resources + mediators, bus or agent

collection + client with workspace/environment Metrics: e.g., for federated search

Page 37: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Standards

Protocols/federation– Z39.50, CIMI– Dienst, NCSTRL– OAi protocol

Metadata– TEI: inline, detailed (structure in stream)– MARC: two-level, fine-grained– Dublin Core: high-level, 15 elements– RDF: describing resources/collections, annotation– OAMS and others used in OAi

Page 38: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

(2 slides from Lee Zia, NSF)Presidential Directive - 12/17/1999

Subject: Use of Information Technology to Improve Our Society

“13. The Secretary of the Smithsonian Institution, the Director of the National Science Foundation, the Director of the National Park Service, and the Director of the Institute of Museum and Library Services shall work with the private sector and cultural and educational institutions across the country to create a Digital Library of Education to house this country's cultural and educational resources.”

Page 39: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Programmatic History

Digital Libraries Initiative (DLI 1) - NSF/NASA/ARPA, FY 94-97

DLI 2 - NSF, et al., initiated in FY 98, continuing

Education FY 98-99

DLI 2 Special Emphasis in Undergrad

NSDL ProgramNSF: FY 00-02

DL

Operational

Fall, 2002DLs & UG Earth Systems Educationinitiated FY 99, continuing

Page 40: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

CS -> CSTC -> CRIM -> JERIC NSF and ACM Education Committee are funding

a 2 year project “A Computer Science Teaching Center” - CSTC - http://www.cstc.org/

College of NJ, U. Ill. Springfield, Virginia Tech Focus initially on labs, visualization, multimedia Multimedia part is also supported by a 2nd grant

to Virginia Tech and The George Washington University: http://www.cstc.org/~crim/

ACM J. of Educational Resources in Computing

Page 41: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 42: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Author‘s toolswww.physik.uni-oldenburg.de/EPS/mmm

Page 43: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 44: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

DL Components

User Interfaces

Workflow Mgr

DBMS

Search Engines, Classifiers, …

Data, MM Info

Gateways

Repository

Rights Mgr

MM/ HT Renderer

Page 45: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

OUTLINE

Introduction, Grand ChallengeDefinitions, 5SOAINDLTDMARIAN, ENVISIONPetaPlexFor more information

Page 46: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Open Archives Initiative

OAIwww.openarchives.org

[email protected]

Page 47: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Open Archives Initiative (OAI) xxx@LANL, high-energy physics (Ginsparg, 1991) CSTR + WATERS = NCSTRL (Lagoze,1994) xxx + NCSTRL = CoRR collaboration (1998) Universal Preprint Service protoproto, Oct. 21-22, 1999, Santa Fe

– led by LANL, CNI, DLF, Mellon --> OAi Santa Fe Convention (see Feb. D-Lib Magazine article) Follow-on mtgs: 6/3@San Antonio, 9/21@Lisbon (ECDL) Archives -> Open Archives

– Support unique archive identifiers– Implement Open Archives metadata set (DC, using XML)– Implement OA harvesting protocol (derived from Dienst protocol)– Register the archive

Build tools, layer other services: linking, searching, …

Page 48: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Metadata: Complex -> Simple

MARC ($50) Dublin Core (DC)

Page 49: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

OAI – Repository Perspective

Required: Protocol

DODO DO DO

MDO

MDO MDOMDOMDO

MDOMDOMDO

Page 50: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

OAI – Black Box Perspective

OA 1

OA 2

OA 4

OA 3

OA 5OA 6

OA 7

Page 51: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Black Box OAI-ETD Perspective

ISTEC(Ibero

America)

PhysDis

NSYSU(Taiwan)

ADT(Australia)

BN.PT(Portugal)

www.theses.org

CyberTheses(Francophone)

VT

Dissert.Online(Germany)

MITOhioLINK

CBUC(Catalunya)

NDC(Greece)

SEALS(S.Africa)

CIC U. Bergen(Norway)

Page 52: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Tiered Model of Interoperability

Mediator services

Metadata harvesting

Document models

Page 53: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

OAi Philosophy

Self-archiving = submission mechanismLong-term storage system = archiveOpen interface = harvesting mechanismData provider + service providerStart with “gray literature”

– e-prints/pre-prints, reports, dissertations, …

Page 54: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Archive of Digital Objects

ArchiveAccessProtocol

Handle(ID)

Digital object

terms and conditions

Page 55: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Open Archives (protoproto)

ArXiv & Los Alamos National LabCogPrints & U. SouthamptonNACA & NASA (reports)NCSTRL & Cornell U.NDLTD & Virginia TechRePEc & U. SurreyTotal of around 200K records

Page 56: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Original Open Archives Members

American Physical Society California Digital Library Caltech Coalition for Networked Info. Cornell University Harvard University Library of Congress Los Alamos Nat’l Lab Mellon Foundation

NASA Langley Research Cntr Old Dominion University Stanford University U. of Ghent U. of Surrey U. of Southampton Vanderbilt University Virginia Tech Washington University

Page 57: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Open Archives Future

EconWPA (U. Washington) e-biomed -> PubMed Central (NIH) PubScience (DOE) Clinical Medicine Netprints (+ other HighWire Press holdings ) University ePub (California Digital Library) All public e-prints (MIT) Scholar’s Forum (Caltech) Int’l: CERN, Germany, India, Mexico, … Goal: millions of books/articles/reports / yr

Page 58: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Approaches to Open Archives

Build ByDiscipline

Build By Institution

Page 59: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Approaches to Open Archives

Build ByDiscipline

Build By Institution

AuthorCategoryInterdisciplinaryYearLanguageQuery …

Page 60: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Figure 1. Layers Related to Open Archives Initiative

Services

Search/Browse

Authoring Citation Checking Submission

Metadata Creation

Editorial: Reviewing, Certification

Registry

Archives: Name, ID, Description, Terms and Conditions, …

Metadata Formats: Name, XML DTD, …

Archive Formats: Name, Standard, Preservation Process, …

Protocols Tools

Services

Copy-Edit / Add Value Citation DB Updating

Authority Control

Preservation Conversion

Text/MM Editing

Gazetteer Cataloging

Collaboration

Annotation

Summarization

Citation / Linking

SFX

CiteSeer

Repository NCSTRL Repository

EconWPA Repository

RePEc Repository

Repository for NDLTD Open Archives Harvesting Protocol

Metadata Formats: OA Metadata Set, NDLTD Standard (DC-based) Set

Transaction Log

Training Resources

VT Partition

Record (Metadata)

Record (Full Content)

… …

UVA Partition

Metadata Content

Caltech Partition

Metadata Content

Page 61: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Mechanisms

Sharing– Join federation, run software– Make metadata and archive available

Aggregating– By discipline– By institution– By genre

Automating– Workflow– Harvesting and providing services– Federated searching– Dynamic linking (e.g., with SFX)

Page 62: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

VT View of the Open Archives Initiative (OAI)

Enable sharing of publication metadata and full-text by digital libraries

Standardize low-level mechanisms to share contents of libraries

Build higher-level user-centric and administrative services in meta-libraries

Install organizational mechanisms to support the technical processes

Page 63: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Virginia Tech Projects

MARC XML-DTD Computer Science Teaching Centre (CSTC) W3C Web Characterization Repository OAI Repository Explorer Networked Digital Library of Theses and

Dissertations (NDLTD)

Page 64: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

MARC XML-DTD

XML Transport format for US-MARC records

Standardized metadata exchange format for traditional library services joining OAI

Page 65: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

CS Teaching Center (CSTC)

Collection of reviewed online resources used to aid in teaching of Computer Science

Supports author submission and peer-review process for new ACM Journal of Educational Resources In Computing (JERIC)

Connected with NSDL (NSF 00-44)

http://www.cstc.org

Page 66: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

W3C Web Characterization Repository

Online database of metadata related to publications, tools and data sets dealing with Web characterization

Project of the Web Characterization Activity working group of the World-Wide-Web Consortium (www.w3c.org/WCA)

http://purl.org/net/repository

Page 67: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 68: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 69: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 70: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

OAI Repository Explorer

Serves as a compliancy test Allows browsing of open archives using only OAI

protocol Sends requests on behalf of user, parses and

checks responses and displays browsable interface Will detect most discrepancies in protocol http://purl.org/net/explorer

Page 71: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 72: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 73: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 74: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

OUTLINE

Introduction, Grand ChallengeDefinitions, 5SOAINDLTDMARIAN, ENVISIONPetaPlexFor more information

Page 75: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

A Digital Library Case Study

Domain: graduate education, research

Genre: ETDs = electronic theses & dissertations

Submission: http://etd.vt.edu

Collection: http://www.theses.org

Project: Networked Digital Library

of Theses & Dissertations http://www.ndltd.org

(NDLTD – remember: ND LTD / NDL TD) (also, newer NUDL:

Networked University Digital Library, with e-courseware, etc.)

Page 76: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

The Networked Digital Library of Theses and Dissertations

www.NDLTD.org

Leader of the Worldwide ETD(Electronic Thesis and Dissertation) Initiative

Training AuthorsExpanding Access

Preserving KnowledgeImproving Graduate Education

Enhancing Scholarly CommunicationEmpowering Students & Universities

Page 77: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

GradProgram Library IT Ed.

(Tech)

Page 78: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Media

ETD Web Sitehttp://www.ndltd.org/

ETDs Got Your Interest?

Graduate Students

Singapore AMChronicle of Higher Ed.National Public RadioNY Times ...

U. Laval

Page 79: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Key Ideas: Networked infrastructure

Scalability

Education is the rationale

University collaboration

Workflow, automation

Authors must submitMaximalAccess

PDF, SGML, MM,MARC, DC, URNs,Federated search

Standards

8th graders vs. grads

Page 80: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

What led to today’s meeting? 1987 mtg in Ann Arbor: UMI, VT, … 1992 mtg in Washington: CNI, CGS, UMI, VT and 10 universities with 3 reps

each 1993 mtg in Atlanta to start Monticello Electronic Library (MEL): SURA,

SOLINET 1994 mtg in Blacksburg re ETD project: std of PDF + SGML + multimedia

objects 1996 funding by SURA, US Dept. of Education (FIPSE) for regional, national

projects 1997 meetings in UK, Germany, ... 1998 – 1st symposium – Memphis (20) 1999 – 2nd symposium – Blacksburg (70) 2000 – 3rd symposium – St. Petersburg (225) -> Caltech

Page 81: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

What are the long term goals?

Attract all TDs/yr: 50K D-US, 25K D-Germany, 10K TD-Canada, …

>200K/yr rich hypermedia ETDs that may turn into electronic portfolios (images, video, audio, …)

Dramatic increase in knowledge sharing: literature reviews, bibliographies, …

Services providing lifelong access for students: browse, search, prior searches, citation links

Hundreds/thousands of downloads / year / work

Page 82: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

ETDs: Library Goals Improve library services

–Better turn-around time –Always available

Reduce work –catalog from e-text –eliminate handling: mailing to UMI, bindery

prep, check-out, check-in, reshelving, etc.Save space

Page 83: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Aiding universities to enhance graduate education, publishing and IPR efforts

Helping improve the availability and content of theses and dissertations

Educating ALL future scholars so they can publish electronically and effectively use digital libraries (i.e., are Information Literate and can be more expressive)

What are we doing?

Page 84: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

NDLTD

Computer Resources

Research

Literature

Student Prepares Thesis/Dissertation

Page 85: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Student Defends & Finalizes ETD

My Thesis

ETD

Page 86: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 87: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Student Gets CommitteeSignatures and Submits ETD

Signed

Grad School

Page 88: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Graduate School Approves ETD, Student is Graduated

Ph.D.

Page 89: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Library Catalogs ETD, Access isOpened to the New Research

WWW

NDLTD

Page 90: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Status of the Local Project Approved by university governance Spring

1996; required starting 1/1/97 Submission & access software in place Submission workshops for students (and

faculty) occur often: beginner/adv. Faculty training as part of Faculty

Development Initiative Over 3000 ETDs in collection – some have

audio, video, large images, software, …

Page 91: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Archiving ETDs

Every 15 minutes back-ups made of not-yet-approved submissions

Hourly back-ups of newly approved ETDs

Weekly back-ups of entire ETD collection

Copies stored on-site and off-site

Page 92: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

VT ETD Cataloging

same as current cataloging policies, except:– author-assigned keywords (not LCSH)– generic (not LC) call no.– fields/subfields as required for computer files– full abstracts

time savings– cataloger familiar with computer files– equipment, software for word processing– 5 minutes avg. (10-15 minutes for paper TDs)

Page 93: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Library Costs

$12/vol. for paper thesis processing– catalog, bind, security strip, label, shelve– @950 vols./yr. = $11,466

$3.20/vol. ETD processing– cataloging @950 vols./yr. = $3040

$.07/vol. shelving $.04/vol. circulation

Page 94: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Costs/Savings at VT

Graduate School stopped shipping to the library 3000 copies of paper TDs/year

Library stopped binding, shelving, and circulating 3000 copies of TDs/year

166 ft of shelf space saved/year by the library VT used existing equipment in Library (vs. start-up

costs for staff, hardware and software from from a zero-base estimate: $65,000 – see http://scholar.lib.vt.edu/theses/)

Page 95: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Institutional Members Coalition for Networked Information (CNI) Committee on Institutional Cooperation (CIC) Diplomica.com Dissertation.com Dissertationen Online (Germany) ETDweb, a Division of Answer4.com Ibero-American Science & Technology Education Consortium (ISTEC) National Documentation Centre (NDC), Greece National Library of Portugal (for all universities) OCLC Online Computer Library Center Organization of American States (SEDI/OAS) Southeastern Library Network (SOLINET) UNESCO (www.unesco.org/webworld/etd)

Page 96: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

National / Regional Projects Australia

– U. New South Wales (lead)– U. of Melbourne– U. of Queensland– U. of Sydney– Australian National U.– Curtin U. of Technology– Griffith U.

Germany– Humboldt University (lead)

– 3 other universities

– 5 learned societies: Math, Physics, Chemistry, Sociology, Education

– 1 computing center

– 2 major libraries

Consorci de Biblioteques Universitàries de Catalunya, as group, www.cbuc.es:– Universitat de Barcelona– Universitat Autonòma de Barcelona– Universitat Politècnica de Catalunya– Universitat Pompeu Fabra– Universitat de Girona– Universitat de Lleida– Universitat Rovira i Virgili– Universitat Oberta de Catalunya– Biblioteca de Catalunya

South Africa: ECHEA/SEALS India, Portugal, …

Page 97: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

OhioLINK

Statewide Consortium Represents 79 colleges, universities, libraries Public Universities Private Universities and Colleges 2-Year Colleges Only a few (e.g., Miami U. of Ohio) are also

NDLTD members on their own

Page 98: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

US University Members (44) Penn. State University Rochester Institute of Tech. U. of Colorado Health Science Center U. of Florida U. of Georgia University of Hawaii, Manoa U. of Iowa U. of Kentucky U. of Maine U. of North Texas – required since 8/99 U. of Oklahoma U. of South Florida U. of Tennessee, Knoxville U. of Tennessee, Memphis U. of Texas at Austin – required in 2001 U. of Virginia U. Wisconsin - Madison Vanderbilt U. Virginia Commonwealth U. Virginia Tech - required since 1/97 West Virginia U. - required fall 1998 Western Michigan U. Worcester Polytechnic Inst.

Air University (Alabama)Baylor UniversityBrigham Young University (part, whole)CaltechClemson UniversityCollege of William & MaryConcordia University (Illinois)East Carolina UniversityEast Tenn. State U. – require fall 2000Florida Institute of TechnologyFlorida International UniversityGeorge Washington UniversityLouisiana State UniversityMarshall University (W. Va.)Miami University of OhioMichigan TechMississippi State UniversityMITNaval Postgraduate School (CA)New Mexico TechNorth Carolina State University

Page 99: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Other Countries with Members

Belgium Brazil Canada Germany Hong Kong India Italy Korea Mexico

Netherland Norway Russia Singapore S. Africa S. Korea Spain Taiwan UK

Page 100: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

For professional societies

Like “writing across the curriculum”, e.g., Chemical Markup Language, MathML, …

Besides writing: computing/communications, information literacy, personal digital library management, tool use, research methods, collaboration, archiving/preservation

Data sets, communities of users of them Classification systems / browsing / searching NRC’s “On becoming a researcher”

Page 101: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

NDLTD Members

0

10

20

30

40

50

60

70

803/

11/9

7

5/11

/97

7/11

/97

9/11

/97

11/1

1/97

1/11

/98

3/11

/98

5/11

/98

7/11

/98

9/11

/98

11/1

1/98

1/11

/99

3/11

/99

5/11

/99

7/11

/99

9/11

/99

11/1

1/99

Date Joined

Nu

mb

er o

f M

emb

ers

Page 102: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Usage of ETDs in VT Collections

1996 1997 1998 1999

Jan-Aug

Total

requests

37,171 247,537 465,974 907,104

Daily

Requests

102 685 1,722 3,121

Abstract

requests

25,829 112,633 177,647 143,056

Hosts

served

9,015 22,725 28,022 52,663

Page 103: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 104: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Popular Works 1996458 Seevers, Gary L. Identification of Criteria for Delivery of Theological Education Through Distance Education: An International Delphi Study (Ph.D., Educational Research and Evaluation, April 1993; 1353Kb)

432 Hohauser, Robyn Lisa. The Social Construction of Technology: The Case of LSD (MS in Science and Technology Studies, Feb. 1995; 244Kb)

390 Childress, Vincent William. The Effects of Technology Education, Science, and Mathematics Integration Upon Eighth Grader's Technological Problem-Solving Ability (Ph.D. in Vocational and Technical Education, July 1994; 285Kb)

310 Kuhn, William B. Design of Integrated, Low Power, Radio Receivers in BiCMOS Technologies (Ph.D. in Electrical Engineering, Dec. 1995; 2Mb)

287 Sprague, Milo D. A High Performance DSP Based System Architecture for Motor Drive Control ( MS in Electrical Engineering, May 1993; 878Kb)

165 Wallace, Richard A. Regional Differences in the Treatment of Karl Marx by the Founders of American Academic Sociology (MS in Sociology, Nov. 1993; 479Kb)

150 McKeel, Scott Andrew. Numerical Simulation of the Transition Region in Hypersonic Flow (Ph.D. in Aerospace Engineering, Feb. 1996; 3Mb)

Page 105: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Popular Works 19979920 Liu, Xiangdong. Analysis and Reduction of Moire Patterns in Scanned Halftone Pictures (Ph.D. in Computer Science, May 1996; 6.6Mb)

7656 Petrus, Paul. Novel Adaptive Array Algorithms and Their Impact on Cellular System Capacity (Ph.D. in Electrical Engineering, March 1997; 5Mb)

2781 Agnes, Gregory Stephen. Performance of Nonlinear Mechanical, Resonant-Shunted Piezoelectric, and Electronic Vibration Absorbers for Multi-Degree-of-Freedom Structures (Ph.D. in Engineering Mechanics, Sept. 1997; ? + 7926Kb)

2492 Gonzalez, Reinaldo J. Raman, Infrared, X-ray, and EELS Studies of Nanophase Titania (Ph.D. in Physics, July 1996; 4607Kb)

1877 Shih, Po-Jen. On-Line Consolidation of Thermoplastic Composites (Ph.D. in Engineering Mechanics, Feb. 1997; 3.3Mb)

1791 Saldanha, Kevin J. Performance Evaluation of DECT in Different Radio Environments (MS in Electrical Engineering, Aug. 1996; 3.2Mb)

1431 DeVaux, David. A Tutorial on Authorware (MS in CS, April 1996; 2.3Mb)

1394 Kuhn, William B. Design of Integrated, Low Power, Radio Receivers in BiCMOS Technologies (Ph.D. in Electrical Engineering, Dec. 1995; 2518Kb)

Page 106: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

International Use 1996 1997 1998 850 2992 8170 United Kingdom 608 2,501 4223 Australia 346 2378 7373 Germany 713 2367 3970 Canada 387 1264 2201 South Korea 463 1161 4431 France 250 725 2553 Italy 191 867 2781 Netherlands 183 1130 1449 Brazil 22 967 1089 Thailand 83 958 1414 Greece

Page 107: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

NUDL 1/15/99 Networked University Digital Library

(NUDL) proposal to NSF under DLI2 international program– VT: Library, Grad School, Industrial&Systems Eng.– Partners: UK (2) , Singapore, Russia, Korea, Greece, Germany, plus

Iberoamerican group (Spain, Portugal, Argentina, Brazil, Chile, Mexico)

– Problems: Multilingual search, multimedia submissions, requirements/usability, international performance, …

Start with ETDs, then expand to other student works, portfolios, data sets, (CS) courseware, ...

Page 108: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

NUDL Research Partners

Ricardo A. Baeza-Yates, Universidad de Chile, Chile José Luis Brinquete Borbinha, Biblioteca Nacional, Portugal José Hilario Canós Cerdá, Universidad Politécnica de Valencia, Spain Stavros Christodoulakis, Technical University of Crete, Greece Eberhard Hilf, University of Oldenburg, Germany Peter Diepold, Susanne Dobratz, …, Humboldt University, Germany David Garza-Salazar, Monterrey Inst. of Technology (ITESM), Mexico Sung Hyon Myaeng, Chungnam National University, Korea (+ colleagues) Ana Maria Beltran Pavani, Prédio Cardeal Leme, Brazil Lim Ee Peng, Nanyang Technological University, Singapore Alexander I. Plemnek, St.-Petersburg State Technical University, Russia Berthier Ribeiro-Neto, Federal U. of Minas Gerais (UFMG), Brazil J. Alfredo Sánchez, Universidad de las Américas-Puebla, Mexico

Page 109: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Who are sponsors / cooperators? Funding, Donations of hardware/software

– SURA– US Dept. of Education (FIPSE)– Adobe Systems– IBM– Microsoft– OCLC

Others Serving on Steering Committee– National/Regional Projects: Australia, French

speaking group (Montreal, Lyon), Germany, IberoAmerica (ISTEC, OAS), UK (UTOG)

– CGS, National Lib. Canada, NSF, SOLINET, UMI, UNESCO, ...

Page 110: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Relationship with publishers

Concern of faculty and students that still wish to publish books or journal articles, voiced: campus, Chronicle, NPR, Times

Solution: Approval Form gives students, faculty choices on access, when to change access condition; use IPR controls in DL

Solution: by case, work with publishers and publisher associations to increase access– AAP, AAUP– AAAS, ACM, ACS, Elsevier, ...

Page 111: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Some responses from publishers

ACM: need to acknowledge copyright Elsevier: need to acknowledge copyright IEEE-CS: endorse initiative ACS: After first publication, can release Textbook publishers: different market,

manuscript significantly reworked General: restricting access to local campus

will not cause any problems

Page 112: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

How does this relate to UMI?

Generally, they are independent decisions. 1987 UMI workshop was first to explore ETDs. UMI wrote support letter for US Dept. of Ed. proposal. UMI is on Steering Committee. ProQuest Direct pilot of scanning works started 1/1/97,

with free 2 yr access to front part. We are collaborating on:

– accepting electronic author submissions– standards (e.g., representation)

Page 113: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

ETD Initiative (and UMI)

StudentsLearn aboutDL, EPub

TDsbecome more

expressive

N. Amer. (T)Ds areaccessible, archived

Global TDsbecome more

accessible,archived

UMI

Universities

Page 114: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

User Search Support(multilingual, XML)

NDLTD W orld FederatedSearch

Virg in ia Tech ...(univ)

D issertationsO nline

(G erm any)

O hioLink(lib / un iv group)

Portugese N L ...(national lib)

Austra lia(regional)

O AS,ISTEC(Latin

Am erica)

UserInterface

Note: All groups shown are connected with NDLTD.

Page 115: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

www.theses.org

James Powell student project, D-Lib Magazine description in Sept. 1998

XML description of each site– type of search engine / service

– language

– coverage (for resource discovery) Adding Z39.50 gateway capability and

integrating with MARIAN, along with Harvest and Open Archives protocols

Page 116: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Interoperability Tests Planned IBM DL: donated equipment, technical support,

powerful IPR (see TOIS, D-Lib) Z39.50: OCLC SiteSearch / VT tailored s/w

– university libraries w. catalogs of freely shared MARC records pointing to archival copies

– via URNs: handles & PURLs Dienst / NCSTRL - www.ncstrl.org: CS depts.,

DARPA, NSF, CNRI, Cornell – UVA, MIT have worked on extensions for ETDs - Portugal is studying use for Europe - VT is working on Dienst to Z39.50 gateway

Page 117: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Access Approaches

Goal: Maximize access and services, e.g., by encouraging:

UMI centralized services VTLS: planned free union collection of metadata Distributed service: Dienst, Z39.50 Regional services (e.g., OhioLink, AZ/NM) Local servers with browse, search

– From local catalogs to local archives WWW robot indexing and search services

Page 118: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Access Possibilities

Websearchengines

librarycatalogclients

www.theses.org

www.openarchives.org

3rd

PartyServices(e.g.,UMI)

VirginiaTech

NationalLibrary ofPortugal

CBUC(Spain)

OhioLink

MIT NationalProjects:AU, GE, …

Page 119: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Why might a university want to be involved?

To improve graduate education / better prepare your students / increase their knowledge and visibility

To unlock university information To save money for students and for the

university / improve workflow To build an important digital library

Page 120: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Multiple objectives

Sharing research results

– Decrease costs, increase services

– Increase knowledge of users Adding to author knowledge/skills

– Epub, DL, IPR Enhancing organization’s infrastructure

– CS department, library

– University, Laboratory

Page 121: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

DL Submission Software

Similar software developed for W3C’s WCA, CSTC, and NDLTD

CSTC version field-tested to manage papers for ACM Digital Libraries ‘99

May generalize for

– conferences

– electronic journal

– resource description (e.g., courses, Web content)

Page 122: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

How can a university get involved?

Select planning/implementation team– Graduate School– Library– Computing / Information Technology– Institutional Research / Educ. Tech.

Send us letter, give us contact names– www.ndltd.org/join

Adapt Virginia Tech solution– Build interest and consensus– Start trial / allow optional submission

Page 123: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Contact Our Project Team

[email protected]

Phone CallVisit

Video Tape

Page 124: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Convene Local Planning Group

ETD

Page 125: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Build Local ETD Site

Digital Library

Policies

Inspection/Approval

Workshop/Training

ETD

ETD

Page 126: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Responsibilities

Handle local education and collection– Contact information for helpers– Archive

Utilize standards– Metadata: MARC / DC-based concensus specification

Share metadata– Union services, mirrored services

Allow access– www.theses.org / www.dissertations.org– Open Archives Initiative (www.openarchives.org)

Page 127: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Type 1 Members University Requires ETDs

Adobe Acrobat and/or XML/SGML tools Automated submission & processing Archive/access through UMI, (OCLC,)

Virginia Tech, ... (Local) WWW site, publicity (Local) Assistance provided as requested:

email, phone, listserv(s)

Page 128: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Type 2 Members University Agrees to Require ETDs

Like Type 1 but set date not reached Usually has an option or pilot May: wait for new AY; start with all who enter

after; … Build grass roots support

– Advisory committee: representative? expert?– Champions to spread by word of mouth– Approval: Senates, Commissions, Deans, Students– Publicity to reach community

Page 129: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

NDLTD Members, Types 3-7

3. Part of university requires ETDs 4. University allows ETDs 5. University investigating, has pilot 6. University consortium joins:

– CIC (Big 10 coordinating body) 7. Non-university organization joins

– CNI (Coalition for Networked Info.)

Page 130: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Support Services Developed

WWW site with > 300 Mb, CD, videotape Automated submission system (MySQL, UNIX,

WWW scripts - grad school/library) Student guidelines, style sheets, multimedia

training materials, FAQs, press info SGML and XML DTDs for ETDs SGML to HTML (web generator) LaTeX, Word templates, converters FTP site for PS to PDF conversion with UNIX

distiller

Page 131: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Special Support

OCLC– Hosting Jan. 9-10 DC-XML metadata std meeting– Might help with preservation or Worldcat extract

VTLS– Machine, software, support staff to run free– Union catalog server: multilingual, MARC/DC-XML

UNESCO– ETD Guide in English, French, Spanish: book, Web– Scholarships to attend Int’l ETD Conference– Assistance with regional workshops: Columbia, …

Page 132: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 133: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 134: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Accessibility Activities / Plans

Interface design (simple, 3D, VR) Usability studies Generic multi-lingual support Support for those with disabilities Hybrid collection (paper, MARC,

abstracts, full-text, multimedia) Disciplinary classifications, tools Visualization of results, collection

Page 135: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

CAVE Experiments

Use a familiar metaphor– building / floor / room / shelf / book

Rearrange orderings / shelving– use categories, clustering, ranking – use visualization: colors and gaps– study space mappings: physical, logical

Simplify movement for key tasks

Page 136: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 137: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

CAVE-ETD CAVE-ETD is a simulation of a library that

runs in a CAVE (VR environment). Populated with a subset of ETD records.

Main Foyerroom

room

room

room

Page 138: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Book Browsing

Page 139: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Reading Book Abstract

Page 140: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

SPIRE Visualization

Page 141: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Support Offered

Software, documentation, tech support Email, listservs ([email protected],

-eval, -grad, -library, -technical) Donations: Adobe, Microsoft Evaluation: instruments, analysis

http://scholar.lib.vt.edu - solutions/statistics (Temporary storage / archiving; aid - in setting

up an int’l service & archive)

Page 142: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

OUTLINE

Introduction, Grand ChallengeDefinitions, 5SOAINDLTDMARIAN, ENVISIONPetaPlexFor more information

Page 143: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

MARIAN

Multiple Access Retrieval of Information with Annotations

(Marian the Librarian …) Evolved from CODER system to a distributed

Online Public Access Catalog (OPAC), then DL backend, now becoming a full DL system

From C/C++ to Java Future: NDLTD, NUDL, PetaPlex Use for campus collection management Use for www.theses.org as centralized system with

gateway services: OAi, Harvest, Z39.50, …

Page 144: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

MARIAN Layers

Database Layer

Search Engine Layer

User Information Layer

User Interface Layer

User User User User

Page 145: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

MARIAN Parallelism

Java part response time vs. query rate comparation

(type 1 requests)

01000200030004000

0 100 200 300 400 500

query rate (#/min)

resp

onse

tim

e (m

s)

all modules in one machine one "webgate"

two "webgate"s four "webgate"s

Page 146: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 147: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 148: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 149: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 150: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 151: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 152: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 153: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

GermanPhysDis

Collection

5SL Source

Description

wrapper wrapper

Harvestprotocol

VT OAI

Collection

MARIAN/DEByE Mediation Middleware

MIT ETDCollection...

Open Archives

protocol

wrapper...Dienst

protocol

SOIF

DublinCore RFC1807

NDLTD/NUDL/Digital Library User

Queries + Results

Belief Network LayerFusion Layer

Additional Evidential

Information

GreekHellenic Dissertations

Collection

wrapper

MARCZ39.50

protocol

WrapperGenerator

Local Data Store

Search ServicesRecommendation Services, etc

AnalysisIndexingLinking

Page 154: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

ENVISION

NSF “A User-Centered Database from the Computer Science Literature” (1991-93)

Collected bib/typesetter data, converted to SGML Scanned thousands of page images MARIAN search engine - can be made available (also

applied to the Virginia Tech library catalog) used as part of a prototype object-based DL, with tailored visualization interface (L. Nowell dissertation)

Page 155: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 156: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 157: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 158: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 159: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

OUTLINE

Introduction, Grand ChallengeDefinitions, 5SOAINDLTDMARIAN, ENVISIONPetaPlexFor more information

Page 160: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 161: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

PetaPlex Top View

4 ft.

side

Page 162: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

PetaPlex Side View

4 ft. wide

8 ft.

high

Roles:* Support* Cooling* Power

15

shelves

Page 163: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

PetaPlex

Digital Library Machine (“super” object store): Parallel computer / storage utility

Research: inverted files, video server, … Knowledge Systems Incorporated is supplying

VT-PetaPlex-1 with 2.5 terabytes through 100 nodes:

Net connection + 25GB disk + 233 MHz Pentium + Linux

Page 164: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

PetaPlex Complex

FRONT END MACHINERS/6000, 1G RAM, 4 Proc.

Nanoserver

Nanoserver

Nanoserver

Nanoserver

Nanoserver

Nanoserver

Nanoserver

Nanoserver

Nanoserver

Nanoserver Nanoserver

Nanoserver

Nanoserver Nanoserver

Service

Machine 1

Service

Machine 2

Service

Machine 3

Service

Machine 4

Page 165: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

PetaPlex Service Machine Possibilities

Front-end provides handle/repository abstraction through hashing

Small object server Large object server

– video on demand– streaming audio

Information retrieval server Proxy / cache server (e.g., 1 terabyte server

of 1000 worldwide for Comsat/Intelsat)

Page 166: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Sornil & Mather Dissertations

Mather: efficiently handling very large numbers of objects of varying sizes

Sornil: efficiently handling IR for very large dynamic collections, large numbers of users, high transaction rates, large inverted files– modeling and simulation– data organization – parallelization of algorithms, alone and in

combination for retrieval (related) tasks

Page 167: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 168: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 169: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 170: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 171: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations
Page 172: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Comparison  Network of

Workstations (NOW)

Beowulf PetaPlex

Archi- tecture

Cluster of general purpose workstation class machines using off-the-shelf network interconnect

General purpose PCs, interconnected with a custo- mized network

Special purpose architecture tuned for superstorage. Uses a mix of off-the-shelf PC compo- nents and specialized network interconnects.

Cost per node

Workstation prices. Between $2000-$2500/node

Mid to low-end PC prices. Between $1200-$1800 per node

Mass produced components will reduce price to around $100/node

Target area

Computation Computation Storage, computation is a secondary function

Filesystem support

UNIX flavors UNIX flavors Replaces location dependant files with location independent fine-grained URN named objects

Page 173: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

OUTLINE

Introduction, Grand ChallengeDefinitions, 5SOAINDLTDMARIAN, ENVISIONPetaPlexFor more information

Page 174: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Outline of Online Course - 1

Section 1. Foundations– Early visions, definitions, resources/references,

projects Section 2. Search, Retrieval, Resource Discovery

– Information storage and retrieval, Boolean vs. natural language

– Indexing: Phrases, Thesauri, Concepts – Federated search and harvesting, OAI,

Crawlers/spiders/metasearch – Integrating links and ratings

Page 175: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

NESTOR Map: Intro to DLs

Page 176: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Outline of Online Course - 2 Section 3. Multimedia, Representations

– Text/audio/image/video/graphics/animation – Capture, Digitization, Compression – Standards, Interchange: JPEG, MPEG – Content-based retrieval, Playback, QoS, SMIL

Section 4. Architectures– Modular/componentized, Protocols – InfoBus (Stanford, Java), Mediators, Wrappers

(TSIMMIS)

Page 177: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Outline of Online Course - 3

Section 5. Interfaces – Workflow, Environments, Taxonomy of interface

components, Visualization – Design, Usability testing

Section 6. Metadata – Ontologies, RDF – MARC, Dublin Core, IMS – Mappings, Crosswalks

Page 178: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Outline of Online Course - 4

Section 7. Electronic Publishing, SGML, XML– Authoring, Presenting, Rendering, Document Object

Model (DOM) – Dual-publishing, Styles (XSL) – Structure, Semi-structured information, Tagging/markup,

Structure queries Section 8. Database Issues

– Extending database technology – Structured and unstructured information – Multimedia databases, Link databases – Performance/replication/storage

Page 179: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Outline of Online Course - 5

Section 9. Agents– Distributed issues – Protocols, Negotiation

Section 10. Commerce, Economics, Publishers– Preservation and archives – Terms and conditions, Open collections, Self-archiving – Economic models, Micropayments

Page 180: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Outline of Online Course - 6

Section 11. Intellectual Property Rights, Security– Legal issues – Copyright, Rights management

Section 12. Social Issues– Cooperation and collaboration, Ratings, Annotation

(PICS) – Educational applications (NSDL), Digital divide – Museums (AMICO), Cultural heritage, International

concerns – Organizational acceptance/issues, Personalization

Page 181: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Remember

Introduction, Grand ChallengeDefinitions, 5SOAINDLTDMARIAN, ENVISIONPetaPlexFor more information

Page 182: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

Conclusions

Consider DLs: to use, to teach, to add to, to build Education is one important application of DLs Cultural heritage, linguistic diversity, new

knowledge – all are important to preserve Technology opens up exciting opportunities in DLs

to yield seamless “super” information systems Having a framework and theory may lead to better

(more effective) systems and broader applicability Interoperability is part of the DL grand challenge

Page 183: Digital Libraries from Theory to Framework to Application: 5S Model, Open Archives Initiative, and Networked Digital Library of Theses and Dissertations

URLs

http://fox.cs.vt.eduhttp://ei.cs.vt.edu/~dlib (Courseware)http://www.dlib.org (D-Lib Magazine)www.smete.org and later

www.nsf.gov/nsdlwww.ndltd.org and www.theses.orgwww.cstc.org (CSTC and JERIC)www.openarchives.orgwww.jcdl.org (JCDL’2001 – June 24-28)


Recommended