IST- 2001-320015 OAForum: A European View on the Open Archives Initiative Museum Information on the Basis of an Open Standard Uwe Müller Humboldt University

OAForum: A European View on the Open Archives Initiative

Museum Information on the Basis of an Open Standard

Uwe MüllerHumboldt University Berlin, Germany

Electronic Publishing GroupUniversity Library / Computing Centre

[email protected]

Uwe Müller, 21.11.2002, Les rencontres d'Aliénor: OAForum - A European view on the Open Archives Initiative

Agenda

1. Open Archives

2. The Open Archives Initiative

3. OAForum: The European View

4. Open Access to Museum Information


Open Archives

Different types of “archives”scholarly publication server (pre-prints, e-prints)libraries (OPAC, e-journals)museum databases (object metadata)archives (historical documents) and cultural heritageeducation

Origin self archivingunclosing existing databasesestablishing new databases

“open archives” approach gains popularitycross archive accesslow cost dissemination of previously “hidden” resourcesbuilding of new service provision


Open Archives: Problems

Problem: lacking interoperabilitydifferent metadata standards

Different terminology

different languages

different access strategies

different interfaces / transfer protocols

different copyright regulations

Difficulty to establish new services cross archive access approach: Z39.50 protocol

search distributed databases homogenously

mainly used for bibliographic data


Open Archives: Problems (2)

Metadatadifferent formats (DC, MAB, MARC, …)

different interpretation possibilities (Creator: Author vs. Artist vs. Photographer)

Transfer protocolslibraries: Z39.50

NCSTRL: DIENST protocol

harvest systems: HTTP based

Search methodsdatabase based vs. file system based

DB: real time, ODBC, SQL, PHP, Perl etc.

FS: no real time, index file, SOIF


Agenda

1. Open Archives





The Open Archives Initiative (OAI)

Main ideasworld-wide consolidation of scholarly archives

free access on the archives (at least: metadata)

consistent interfaces for archives and service provider

effortless implementation

based on existing standards (e.g. HTTP, XML, DC)

Basic functioning

Harvester Repository

Requests (based on HTTP)

Metadata (encoded in XML)

Metadata(Documents)

Metadata

Service Provider Data Provider

„Service”


OAI: Organisational Structure

Steering Committeegeneral policy of OAI

12 members (USA, United Kingdom, Germany)

Executive Committeeoperational decision

Carl Lagoze (Cornell University)

Herbert van de Sompel (Los Alamos National Laboratory)

Technical Committeeformulation of the OAI protocol specification

contact with OAI community

Executive Committee + 14 members (USA, United Kingdom, Denmark, Switzerland)


OAI: General Assumptions

2 groups of participants Data Providers (Open Archives, Repositories)

free access of metadatanot necessarily: free access to full texts / resourceseasy to implement, low barriers

useable for small institutions Service Providers

use OAI interfaces of the Data Providers harvest and store metadatamay select certain subsets from Data Providers

(set hierarchy, date stamp)may enrich metadataoffer (value-added) service on the basis of the metadata


OAI: Technical Model

Se

rvic

e P

rovi

der

e-print

Da

ta

Pro

vid

er e-prints

e-print

Da

ta

Pro

vid

er Images

e-print

Da

ta

Pro

vid

er

OPAC

e-print

Da

ta

Pro

vid

er Museum

e-print

Da

ta

Pro

vid

er Archive

Requests:

Identify

ListMetadataformats

ListSets

ListIdentifiers

ListRecords

GetRecord

Responses:

General information

Metadata formats

Set structure

Record identifier

Metadata

Da

ta

Pro

vid

er Harvester

Repository

Repository

Repository

Repository

Repository


OAI: Protocol Basics and Details

Basics of OAI-PMHprotocol based on HTTP

request arguments as GET or POST parameters

six request types

e.g. http://archive.org?verb=ListRecords&from=2002-11-01

responses are encoded in XML syntax

supports any metadata format (at least: Dublin Core)

Details of OAI-PMHlogical set hierarchy (definition: data providers)

date stamps (last change of metadata set)

error messages

flow control


OAI: Protocol Basics and Details (2)

Metadata sets (Records)1. header

unique identifier (key for further archive requests),e.g. oai:HUBerlin.de:30000231

datestamp, e.g. 2002-11-01

logical sets in which the record is contained

2. metadata

metadata prefix

metadata set (at least: Dublin Core)


Example: http://edoc.hu-berlin.de/OAI-2.0?verb=ListIdentifiers&from=2002-01-03&until=2002-01-08&metadataPrefix=oai_dc&set=doctypes:dissertations

<?xml version="1.0" encoding="UTF-8"?> <OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"> <responseDate>2002-10-22T17:49:49+01:00</responseDate> <request verb="ListIdentifiers" from="2002-01-03" until="2002-01-08" metadataPrefix="oai_dc" set="doctypes:dissertations">http://edoc.hu-berlin.de/OAI-2.0</request> <ListIdentifiers> <header> <identifier>oai:HUBerlin.de:3000819</identifier> <datestamp>2002-01-08</datestamp> <setSpec>doctypes</setSpec> <setSpec>doctypes:dissertations</setSpec> <setSpec>dnb</setSpec> <setSpec>dnb:dnb33</setSpec> </header> <header> <identifier>oai:HUBerlin.de:3000831</identifier> <datestamp>2002-01-07</datestamp> <setSpec>doctypes</setSpec> <setSpec>doctypes:dissertations</setSpec> <setSpec>dnb</setSpec> <setSpec>dnb:dnb27</setSpec> </header> </ListIdentifiers> </OAI-PMH>


OAI: Data Provider Architecture

SQL-Database

OAI Data Provider

Web server (e.g. Apache, IIS)

OAI request(HTTP request) Programming

extension (e.g. PHP, Perl)

SQL request

DB response

OAI response(XML instance)


OAI: Service Provider Architektur

Data Provider Data Provider Data Provider

Scheduler

Flow controlXML Parser

Normaliser

Database

Service module

User Harvester User

OAI Service Provider

Dublication checker

Update mechanism


OAI: Examples / Participants

Data Provider / Repositoriessee http://www.openarchives.org/Register/BrowseSites.pl

Service ProviderRepository Explorer: http://oai.dlib.vt.edu/cgi-bin/Explorer/oai2.0/testoai/

Cross Archive Searching Service: http://arc.cs.odu.edu/

MyOAI: http://www.myoai.org/

DINI: http://edoc.hu-berlin.de/oaisearch/

Physnet: http://physnet.uni-oldenburg.de/oai/query.php

…


Agenda

1. Open Archives





OAForum: Background

Project Open Archives ForumEuropean Union Information Society Technologies (IST) Programmeaccompanying measureproject start: October 1st, 2001 (duration: 2 years)project partners:

UKOLN, University of Bath, United Kingdom I.E.I.-CNR, Pisa, Italy Humboldt University Berlin, Germany

http://www.oaforum.org/ Motivation

increasing discussion about open archives approachsetting up a framework for the approach in generalpromotion of the approachEuropean view ...


OAForum: Project Partners

UKOLN, Bathvarious projects contexts of metadata and interoperability, cross searching

Renardus project, Schemas, DESIRE

I.E.I-CNR, PisaCYCLADES project, DELOS

development of services on top of OAI specification

Humboldt University, BerlinDissertation Online, NDLTD

DINI workshops on OAI


OAForum: Objectives

raise and sharpen awareness on the open archives approach

promote “low barrier” interoperability

opening cultural resources

detecting potentials for new services

encourage collaborative development of solutionsdiscussion of problems

exchange of experiences and information

support European liaison with OAI

… build community of interest


OAForum: Objectives (2)

Establish an information repository activities related to the open archives approachexperiences, developed services (e.g. document delivery, searching, browsing, summarisation, linking)share the database with other organisations (e.g. OAI)

Validation of European experience concerningimplementing and using the OAI-PMH and other similar approachesrequirements from implementers and users

Organisational review and analysis possible business modelsIntellectual Property Rights

Dissemination of the open archives approachworkshops


OAForum: Ask and Answer Questions

Not: yet another OAI implementation project … supporting activities

European initiatives with open-archives based approach clustering activities

existing and new communitiesIST projectsnational initiatives

dissemination activitiesshare experiences on Open Archivesinvestigation of usage: different paradigmsglobal availabilityshare developments


OAForum: Benefits

tool to reach communities bring together what is happening in Europe raise awareness on and discuss main issues

common terminology on digital repositories

metadata / full text harvesting models

needs of users and communities

advanced services

make European projects ready for actiondevelop possible solutions

establish business models


Communitiesinstitutions of cultural heritage

museums

European digitising projects

scholarly institutions

public libraries

special user groups

publishers

commercial sector

education

Service Providere-print archives

subject gateways with aggregating functions

value-added services

Data Providerexisting metadata repositories

new metadata collections

OAForum: Participants

Carriers and Users of Open Archives


OAForum: Topics

1. Workshops, Distribution distribute information and experiences on technologies, etc.produce interest for topics connected with open archivesarticles, talks, etc.

2. Organisational Evaluation analyse business modelstackle issues of copyrights (IPR …)

3. Technical Evaluationapply the technical framework of OAIdevelop an information portal with

information on projects, repositories, service providermetadata schemas software and implementations

discuss problems of interoperability


OAForum: Workshops, Distribution

Contact: I.E.I.-CNR, Pisa – Donatella Castelli May 2002, Pisa, Italy

experiences from the European e-prints community

establish the forum

December 2002, Lisbon, Portugalarchives and libraries

open access to hidden archives

March 2003, Berlin, Germanymetadata schemas

museums and institutions of cultural heritage

September 2003, Bath, United Kingdomcooperation with LIBER and SPARC initiative

final workshop – bigger conference


OAForum: Organisational Evaluation

Contact: UKOLN, Bath – Leona Carpenter Business models

co-operations between data providers to establish service networks

metadata exchange between archives and services

provision of value-added services (e.g. enrichment of metadata, automatic classification, OpenURL)

Copyright issuesIPR and copyright (influence on producers and distributors)

property rights on metadata (collective use of metadata, metadata exchange, agreements with publishers etc.)

long-term availability of digital resources

Discussion group Dennis Nicholson (University Glasgow)


OAForum: Technical Evaluation

Contact: Humboldt University, Berlin – Susanne Dobratz Interoperability

integration of new (open archive) approaches with existing technologiesIs unqualified Dublin Core sufficient?

Issues on database management concurrency and update problemsscalabilityde-dublication

Software and toolscollect and share experiences and existing solutionsestimation of necessary expenditure

establish data provider / service …skills, manpower, time


Agenda

1. Open Archives





Museum and Open Archives

rich information resources often not accessible for search engines

information is hold in databases

other non-HTML forms

other problem for overlapping search: no precise, useful, reliable information

possibility: collecting & indexing metadata records

solution: combination ofbest traditional library and museum techniques

new internet techniques

OAI …


Museums: Existing Approaches

CIMI (Consortium for the Interchange of Museum Information)

pre-release tester for OAI protocolgeneric OAI repository implementationdownloadable source code (v1.0, v2.0 to come)(http://cimi.webfarm.com/wg/oai/OAI_signup.html)planned: large-scale test (“real world” applications)XML schema for SPECTRUM (alpha test)(http://www.cimi.org/wg/xml_spectrum/index.html)

“Canada Place Portal”Canadian federal Department of Canadian Heritage (PCH) Canadian Heritage Information Network (CHIN) National Library of Canada OAI for harvesting and aggregating object recordshttp://www.pch.gc.ca/progs/em-cr/2002/2002_5/02_e.cfm


Museums: Existing Approaches (2)

AMOL (Australian Museums Online)OAI-enabling regional servers supply records to National Database of Museum Objects target: regional server network entirely based on OAI protocol http://amol.org.au/oai/service/ http://www.amol.org.au/collection/collection_index.asp

Adlib (company for museum software)implemented OAI-PMH in Internet Server module(works with any CGI web server)target: joined web presence of museumsNetherlands: > 10 museum OAI compatible

Mellon Foundation set of experiments in seven USA institutions, e.g. RLG http://www.arl.org/newsltr/217/waters.html


Museums: Open Questions

Should we unclose museum resources? benefits, costs, risks, …analysis of user necessities

Access strategies / protocolsIs OAI useable for exchange of museum information?

hierarchical descriptions, scalability, required extensions, presentation and partitioning community extensions for OAI / DCalternatives to DC, e.g. SPECTRUM

technical and organisational pre-conditionsaggregation, integration, access and presentation(bibliographic, textual, multiple media, and object metadata) data qualityneed of a central registry (scope and services)utility of Dublin Core for meeting requirements


Museums: Open Questions (2)

Co-operationnational / international

best practice guidelines

Business modelssustainability

Copyright problemsaccess control and rights

IPR (Intellectual Property Rights)


Museums: Ask and Answer Questions

OAForum Questionnaire Technical Validation Glossary Resource / Project Information OAForum workshop on “Museums and Open

Archives” just get in contact with OAForum …


http://www.oaforum.org/resources/tecvalq2.php


http://www.oaforum.org/resources/glossary.php


Information Gateway

Please help us!with information on your projectswith questions you have concerning open archiveswith experiences and existing solutionswith ideas and suggestions …

Endorse the workshop in March 2003 in Berlin27th-29th March 2003especially for museum community suggestions for topicstextual contributionsparticipationhttp://www.oaforum.org/workshops/berl_invitation.phpContact: Susanne Dobratz, HU Berlin <[email protected]>


http://www.oaforum.org/oaf_db/register/


www.oaforum.org/oaf_db/list_db/list_services.php


Thank you …

Questions?

http://www.oaforum.org/

[email protected]

Leona Carpenter (Bath)Donatella Castelli (Pisa)Susanne Dobratz (Berlin)

Uwe Müller

Humboldt University Berlin, Germany

[email protected]


Additional Information Open Archives Initiative

http://www.openarchives.org/ http://www.openarchives.org/OAI/openarchivesprotocol.html (OAI-PMH)http://www.openarchives.org/service/listproviders.html (Service Providers)http://www.openarchives.org/Register/BrowseSites.pl (Data Providers)http://www.openarchives.org/tools/index.html (Tools, ...)

Open Archives Forumhttp://www.oaforum.org/ http://www.oaforum.org/workshops/berl_invitation.php (workshop, Berlin, March 2003)

Signal-Hill OAI Sitehttp://www.signal-hill.org/nav/archives2.html

Eprints.orghttp://software.eprints.org/#sites

CIMIhttp://www.archimuse.com/mw2001/papers/perkins/perkins.html (article by John Perkins)http://www.mcn.edu/mcn2001/workshop6.html (CIMI OAI workshop)http://www.cimi.org/wg/xml_spectrum/index.html (SPECTRUM XML schema)