Digital Libraries: a Reference Model LIDA 2009 29 May 2009, Zadar, Croatia Vittore Casarosa ISTI-CNR, Pisa and University of Parma, Italy

Digital Libraries:a Reference Model

LIDA 200929 May 2009, Zadar, Croatia

Vittore CasarosaISTI-CNR, Pisa and

University of Parma, Italy

LIDA 2009 Vittore Casarosa, CNR and University of Parma, Italy 2

The traditional role of libraries

• Mediators between information and users• Selection

– Definition of collections

• Acquisition– Physical objects

• Description– Catalogs

• Access– Shelves

• Preservation– Controlled enviroment


Libraries: some figures

• Volumes (in millions)

• Journals– From 10.000 in 1950 to 150.000 in 2002

• Alexandria principle beginning to fade

Institution 1910 1995 2002 Library of Congress 1,8 23 26 Harvard Univ. 0,8 12,9 14,9 Yale Univ. 0,55 9,5 10,9 U Illinois (Urbana) 0,1 8,5 9,9 U California (Berkeley) 0,24 8,1 9,4

British Library 2 15 18 Cambridge Univ. 0,5 3,5 7 Oxford Univ. 0,8 4,8 6 Bibl. Nat. De France 3 11 12


Evolution of technology

• Computer technology– CPU and integrated chips– Random Access Memories

• RAM – from KB to GB

– External memories• Tapes, hard disks, floppy disks• Memory sticks• CDs• DVDs• from MB to GB to TB to PB to EB

• Communication technology (networks)– (Telephone) line speed– Point to point (leased lines)– Local Area Networks– Inter-networking (TCP/IP)


The World Wide Web

• Combination of computer technology and communication technology

• It all started with the “hyperlink”• Then came the “browser” (Mosaic)• Then came the first wave• Then came the “dot come, dot gone”• Then came the second wave• Finally came the “information explosion”

– An estimate of 110 to 560 million hosts– An estimate of 15 to 30 billion pages on line

• And now we have Web 2.0 (with Web 3.0 just around the corner)


DELOS main objective

To define and conduct a joint program of activities in order to integrate and coordinate the on-going research activities of the major European research teams in the field of digital libraries for the purpose of developing the next generation digital library technologies


Digital Libraries in the Information Space

Structure of Data

Structure of User Behavior Digital

Libraries

Databases/IR

Web

Low

High

High


Definition of “Digital Library”

• Click to add text


What is a Digital Library ?

• A DL is the combination of “content and services” • A DL is an “entity” providing the functionality to mediate

between information objects and information users in the context of distributed collections of information objects. This (external) functionality includes access, publish, delivery, preservation, personalization, etc.

• A Digital Library is a “tool” at the centre of intellectual activity having no logical, conceptual, physical, temporal, or personal borders or barriers on information

• A Digital Library is an “institution” in charge of providing at least the functionality of a traditional library in the context of distributed and networked collections of information objects


DELOS - Grand 10610-Year Vision #1

Digital libraries should enable any citizento access all human knowledge anytime and anywhere,in a friendly, multi-modal, efficient, and effective way, by overcoming barriers of distance, language, and culture and by using multiple Internet-connected devices


DELOS - Grand 10-Year Vision #2

The potential exists for digital libraries to become the universal knowledge repositories and communication conduits for the future, a common vehicle by which everyone will access, discuss, evaluate, and enhance information of all forms


Conceptual Framework

Contents

Usage

Digital Library System

Management


Building a Digital Library


Research Directions in DLs

Reference Model for DLS

Foundations

ArchitecturesInformation

AccessAudio/Visual

SemanticInterop

Systems

UserInterfaces

Visualization Personalization

User

e-Health, e-Learning, e-Culture, etc.…

ApplicationsH

orizo

nta

l

Cura

tion

Pre

serv

atio

nR

eposito

ries

Evalu

atio

nM

eth

odolo

gie

sEvalu

atio

nIn

frastru

cture

s


Foundations Research Issues

Reference Model for DLS

Formalize a conceptual framework for Digital Library systems

to serve as a yardstick of quality and richness

to specify features and properties of generic DLMS

to clarify relationships among

digital libraries, digital repositories, digital archives,

search engines, information infrastructures,

knowledge commons


System-related Research Issues

Architectures

Peer-to-peer architectures

Grid middleware

Service-oriented architectures

Information Access

Indexing for complex and novel data

Query routing in complex distributed Digital Libraries


System-related Research Issues

Audio/Visual

Automatic metadata extraction

Context-aware content-based retrieval

Audio/visual interfaces

Semantic Interop.

Methods for the integration of heterogeneous ontologies

and domain-specific knowledge organization systems

Interoperability with e-Learning applications


User-related Research Issues

User Interfaces Framework for new digital library interfaces Task-oriented user interfaces Cooperation/collaboration tools, e.g., annotations

Visualization

Personalization

Self-adaptability to small screens Visual analysis and exploration of query results

Modeling foundations for user preferences and context Personalization of user interactions Peer-similarity-based query routing decisions User log analysis for profiling


Horizontal Research Issues

Curation/Preservation

Integration of preservation functionality

Establishment of a testbed and evaluation framework

for preservation techniques

Automating selection and ingest processes

Evaluation Methodologies

Standard frameworks for comparative evaluation of DL Systems

Definition of standard events in a DL environment

Identification of appropriate metrics

Establishment of information repositories


Applications Research Issues

e-Health

Virtual electronic health records

Integration of multiple medical information streams

e-Learning

Interoperability of e-Learning applications

e-Culture

…

Integration of upper-level ontologies

Mapping of core ontologies to schemas and KOS


New name for Digital Libraries ?

<adjective><content abstraction><created entity>

• adjective := Digital• content abstraction := ‘’• created entity := Library

Digital REALMDigital REsources for

Archives,Libraries and Museums


Alternatives for new name

Libraries

Environments

Spaces

Agoras

Commons

Forums

Architectures

Worlds

Realms

Data

Information

Knowledge

Digital

Pervasive

Dynamic

Universal

Ubiquitous

Collaborative


What’s in a name ?

DIGITAL LIBRARIESDIGITAL LIBRARIES

... what's in a name? that which we call a rose, by any other name would smell as sweet…...

DDYNAMICYNAMICUUNIVERSALNIVERSALKKNOWLEDGENOWLEDGEEENVIRONMENTSNVIRONMENTS

KKNOWLEDGENOWLEDGECCOMMONSOMMONS


Need for a Reference Model

• A reference model is an abstract framework for understanding significant relationships among the entities of some environment, and for the development of consistent standards or specifications supporting that environment

• A reference model is based on a small number of unifying concepts and may be used as a basis for education and explaining standards to a non-specialist

• A reference model is not directly tied to any standards, technologies or other concrete implementation details, but it does seek to provide a common semantics that can be used unambiguously across and between different implementations


A Three-Entity Framework


The three Entities

• Digital Library– An organization, which might be virtual, that comprehensively

collects, manages, and preserves for the long term rich digital content, and offers to its user communities specialized functionality on that content, of measurable quality and according to codified policies

• Digital Library System– A software system that is based on a defined (possibly

distributed) architecture and provides all functionality required by a particular Digital Library. Users interact with a Digital Library through the corresponding Digital Library System

• Digital Library Management System– A generic software system that provides the appropriate

software infrastructure both (i) to produce and administer a Digital Library System incorporating the suite of functionality considered foundational for Digital Libraries and (ii) to integrate additional software offering more refined, specialized, or advanced functionality


Different types of DLMS

• Extensible Digital Library System – A complete Digital Library System that is fully operational with respect to

basic/ foundational functionality required. It is based on an open software architecture, so that further software components can be incorporated on top of the ones already there with ease (DelosDLMS, GreenStone)

• Digital Library System Warehouse – A collection of software components that encapsulate the core suite of

DL functionality and a set of tools that can be used to combine these components in a variety of ways (in Lego-like fashion) to create Digital Library Systems offering a tailored integration of functionalities. New software components can easily be incorporated into the Warehouse for subsequent combination with those already there (BRICKS, DILIGENT)

• Digital Library System Generator – A highly parameterized software system that encapsulates templates

covering a broad range of functionalities, including a defined core suite of DL functionality as well as any advanced functionality that has been deemed appropriate to meet the needs of the specific application domain Through an initialization session, the appropriate parameters are set and configured; at the end of that session, an application is automatically generated, and this constitutes the Digital Library System ready for installation and deployment (MARIAN)


DELOS DLMS


Containment of models


Actors in the Digital Library


Main roles of Actors (1)

• DL End-Users – They exploit the DL functionality for providing, consuming, and

managing the DL Content as well as some of its other constituents They perceive the DL as a stateful entity that serves their functional needs The behaviour and output of the DL depend on its state at the time a particular part of its functionality is activated. DL end-users may be further partitioned into

• Content Creators• Content Consumers• Librarians (end user)

• DL Designers (Digital Librarian) – They exploit their knowledge of the semantic of the application domain

to define, customize, and maintain the Digital Library so that it is aligned with the information and functional needs of its end-users To perform this task, they interact with the DLMS providing functional and content configuration parameters The values of these parameters, which can be modified during the DL lifetime, configure the specific DL perceived by the end-users because they determine the particular Digital Library System instance serving the Digital Library


Main roles of Actors (2)

• DL System Administrators (System Librarian)– They select the software components necessary to create the Digital

Library System needed to serve the required DL (as specified by the DL Designer) and decide where and how to deploy them They interact with the DLMS by providing architectural configuration parameters, such as the selected software components, the hosting nodes, and the components allocation. The value of the architectural configuration parameters can be changed over the DL lifetime Any change of these parameters may result in the provision of different DL functionality and/or different quality

• DL Application Developers – They develop the software components of DLMSs and DLSs,

implementing the necessary functionality


Hierarchy of Actors’ Views


Main concepts (1)


Main concepts (2)

• Content – The Content concept encompasses the data and information that the Digital Library

handles and makes available to its users Content is an umbrella concept used to aggregate all forms of information objects that a Digital Library collects, manages, and delivers It encompasses the diverse range of information objects, including such resources as objects, annotations, and metadata

• User – The User concept covers the various actors (whether human or machine) entitled to

interact with Digital Libraries. Digital Libraries connect actors with information and support them in their ability to consume and make creative use of it to generate new information User is an umbrella concept including all notions related to the representation and management of actor entities within a Digital Library. It encompasses such elements as the rights that actors have within the system and the profiles of the actors with characteristics that personalize the system’s behaviour or represent these actors in collaborations

• Functionality – The Functionality concept encapsulates the services that a Digital Library offers to its

different users, whether classes of users or individual users While the general expectation is that DLs will be rich in capabilities and services, the bare minimum of functions would include such aspects as new information object registration, search, and browse Beyond that, the system seeks to manage the functions of the Digital Library to ensure that the functions reflect the particular needs of the digital library’s community of users and/or the specific requirements relating to the Content it contains


Main concepts (3)

• Policy – The Policy concept represents the set (or sets) of conditions, rules, terms and

regulations governing interaction between the Digital Library and users, whether virtual or real. Examples of policies include acceptable user behaviour, digital rights management, privacy and confidentiality, charges to users, and collection delivery

• Quality – The Quality concept represents the parameters that can be used to

characterize and evaluate the content and behaviour of a Digital Library. Quality can be associated not only with each class of content or functionality but also with specific information objects or services Some of these parameters are objective in nature and can be automatically measured, whereas others are subjective in nature and can only be measured through user evaluations

• Architecture – The Architecture concept refers to the Digital Library System entity and

represents a mapping of the functionality and content offered by a Digital Library onto hardware and software components There are two primary reasons for having Architecture as a core concept: (i) Digital Libraries are often assumed to be among the most complex and advanced forms of information systems; and (ii) interoperability across Digital Libraries is recognized as a substantial research challenge A clear architectural framework for the Digital Library System offers ammunition in addressing both these issues effectively


The main concepts in perspective

User


The Digital Library Development Framework


The Reference Model


Concept Maps


Digital Library Domains


The Resource Domain


The Content Domain


The User Domain


The Functionality Domain


Main functions

C32 Access ResourceC33 Discover

C34 BrowseC35 Search

C36 AcquireC37 Visualize

C38 Manage ResourceC39 CreateC40 SubmitC41 WithdrawC42 UpdateC43 ValidateC44 AnnotateC45 Manage Inform ObjectC64 Manage ActorC71 Manage FunctionC72 Manage PolicyC73 Manage Quality Parameter

C74 CollaborateC75 Exchange InformationC76 Converse C77 Find CollaboratorC78 Author Collaboratively

C79 Manage DLC80 Manage Content C85 Manage UserC90 Manage FunctionalityC92 Manage QualityC93 Manage Policy Domain

C94 Manage & Configure DLSC95 Manage DLS C104 Configure DLS


Manage Information Object

• C46 Disseminate– C47 Publish

• C48 Author – C49 Compose

• C50 Process– C51 Analyze

• C52 Linguistic Analysis• C53 Qualitative Analysis

– C54 Examine Preservation State• C55 Statistical Analysis • C56 Scientific Analysis• C57 Create Structured Representation• C58 Compare

– C59 Transform • C60 Physically Convert

– C61 Translate• C62 Convert to a Different Format• C63 Extract


Manage Actor

• C65 Establish Actor – C66 Register

• C67 Sign Up

– C68 Login

• C69 Personalise– C70 Apply Profile


Manage DL

• C80 Manage Content – C81 Manage Collection

• C82 Import Collection• C83 Export Collection

– C84 Preserve • C85 Manage User

– C86 Manage Membership – C87 Manage Group – C88 Manage Role – C89 Manage Actor Profile

• C90 Manage Functionality – C91 Monitor Usage

• C92 Manage Quality • C93 Manage Policy Domain


Manage & Configure DLS

• C95 Manage DLS – C96 Create DLS – C97 Withdraw DLS – C98 Update DLS – C99 Manage Architecture

• C100 Manage Architectural Component• C101 Configure Architectural Componebnt• C102 Deploy Architectural Component• C103 Monitor Architectural Component

• C104 Configure DLS – C105 Configure Resource Format – C106 Configure Content – C107 Configure User – C108 Configure Functionality – C109 Configure Policy – C110 Configure Quality


Access Resource


Manage Resource (1)


Manage Resource (2)


Manage Actor


Collaborate


Manage DL


Manage & Configure DLS


The Policy domain


Categorization of Policies


The Quality Domain


The Architecture Domain


DLS Reference architecture


DLS Concrete architecture


Conclusions

Before you think that now you know everything about (the technical aspects of) Digital Libraries, there is one (recurring) question:

Will the Web become the ultimate Digital Library ?

The Web vs. Digital Libraries

Opinion n. 1


Claim #1

• All the information needs of an “IT Society” (research, education, entertainment, business, etc.) will be provided by this huge heap of information called the Web

NO

NO


NO

Claim #2

• For all those activities that require organized and controlled information, the actual institutions (notably libraries, archives and museums) will continue to have a significant role

NO

(NOT ONLY THEM)


Digital Libraries in the Information Space

Structure of Data

Structure of User Behavior

Digital Libraries

Databases

Web

Low

High

High

Info Retrieval

……

Wikis/blogsCMS/DAMS


Basic concepts of an Info Mgmt System

ContentUserFunctionality

QualityPolicy

Architecture


Refuting Claim #1

• All the information needs of an “IT Society” (research, education, entertainment, business, etc.) will NOT be provided by this huge heap of information called the Web


The Web as an Info Mgmt System

Architecture

FunctionalityContentUser

Limited

QualityNo guarantees

PolicyMinimal


Three roads to “Web Knowledge”

1. Handcrafted high-quality curated knowledge bases (ontologies, encyclopedias, etc.)

2. Large-scale information extraction & harvesting (pattern matching, NLP, statist. learning, etc.)

3. Social wisdom from communities (social tagging, folksonomies, etc.)


Refuting Claim #2

• For all those activities that require organized and controlled information, the actual institutions (notably libraries, archives and museums) will continue to have a (much less) significant role


Libraries as Info Mgmt Systems

ContentUserFunctionality

QualityPolicy

Architecture

Small %Strict roleassignment

Overprescribed


No barriers to knowledge exchange

• More and more of the worlds info/knowledge lives in specialized digital libraries that

– Have content added/created by members of a community

– Are curated by specialists of that library’s topic– Are maintained by (designated members of) the

community

• No strict separation between producer, curator, and consumer roles wrt which actors plays which

• Advanced services: annotation, personalization, contextualization, preservation, collaboration, etc.


Hopeful Conclusion: DLs vs. the Web

• They are not going to fight or replace each other, but in the end they are going to complement each other.

With lots ofnew technology help

for both

Hm!

Yes!

Probably!


Opinion n. 2


Libraries – Digital Libraries - Web

Will the Web be theDigital Library ?Library ?

• Content • Structure • Use(fullness)


Use(fullness):Visits to libraries:

- Approx 10 / year- Of which “private/touristic” visits: approx 10 / year

• Visits to Digital Libraries:- Several times per week

• Visits to the Web:- Several times per day– Usually finding what is needed, specifically for professional

purposes

• People increasingly use simple Google-style interfaces for search rather than more complex DL-like interfaces, not to mention library catalogues



Structure (Information and Metadata)Libraries:

- very high, professional- but inflexible: one sorting, rest in (digital) catalogues

searching these is actually like a DL without content

• Digital Libraries:- Very high, manually cared for, sometimes community-

driven- Flexible ways of interaction

• Web: - None, but:- Increasingly created automatically: CiteSeer, DBWorld,

GoogleBooks, Genre Classification, Topic analysis, Named Entity Detection,…



Content:• Libraries:

- High-quality, selected content- but increasingly: not the content I need: proceedings,

papers are first/only in DL before they make it to the library catalogue

• Digital Libraries:- Sometimes the only place where I can find certain

material (sufficiently easily)- Increasingly also older content digitized

• Web: - Sometimes even more comprehensive than theDLs- What’s not on-line does not exist (cf. CiteSeer)- Increasing amounts of traditional, high-qualiyt content

on the Web (or in a DL?) (e.g. Internet Archive)



So:Looks like (parts of) the Web will turn into a

Digital Library, which will eventually replace the Conventional Library

• Requiring only- More content (will come)- More structure (will be provided by better computer

programs), which will lead to- More users using this, assisted by better interfaces



Opinion n. 3


Google’s Mission

Google's mission is to organize the world's information and make it universally accessible and useful.

Organize• By vertical/property: Scholar, Book Search, Product Search,

News, Maps, etc• By search

World’s information• What we can reach through the web• What we license

Universally accessible• Via internet• Internationalized and localized

Useful: focussing on and meeting our users’ needs85

Is Google(*) the world’s Digital Library?

* put here your favourite search engine


A Digital Library’s Mission

A Digital Library 's mission is, for a selected user community, to organize that community’s information and make it universally accessible and useful to that community.

Organize• According to the needs of the user community (art,

photographs, scientific data, ... )

Community’s information• Information (including data) generated by the community• That can be reached through the web• That can be licensed (or purchased)

(Universally) accessible• Via internet (including via web search)• Internationalized and localized

Useful: focussing on and meeting users’ needs within the selected user community

86


Major Differences in Missions

• Scale• Information: broad versus deep coverage• General versus specific communities (and

therefore needs)• Organizing principles (can be very different)• Services provided: how we “add value” to

information/dataOther considerations• Profit• Quality, conservation and preservation• Authority

87


Conclusions (opinion 3)

• Web Search (Google) and Digital Libraries share similar but complementary missions

• Celebrate the diversity of missions, and concentrate on strengths whether as web search engine or digital library• Search engines: scale, universal delivery, universal services• Digital libraries: specialized collections, specialized services,

“library” services

• Focus on delivering value to users through useful and relevant (web) services (“Focus on the user and all else will follow”)

• Web search is a service that Digital Libraries should exploit to ensure universal access to information and services

88


The evolution of “libraries”

“digital librarians”, “digital curators”, etc

???????

Digital Library Digital library

paper, pictures, audio, video

digital “surrogates”born digital objects

librarians, curators, etc

people

libraries, museums, archives, ....

institutions

clay tablets, papyrus, paper

contents


Final conclusion

Getting … “There”

The important thing is ....

Documents

Digital Libraries: a Reference Model LIDA 2009 29 May 2009, Zadar, Croatia Vittore Casarosa ISTI-CNR, Pisa and University of Parma, Italy