An International Journalbookstore.teri.res.in/docs/journals/Sample copy-WDL-June2008.pdfDocumentation Research & Training Centre Indian Statistical Institute, India Prof. Dr Michael

Volume 1 Issue 1June 2008

An International Journal

World

Dig

ital L

ibra

ries: A

n In

tern

atio

nal J

ourn

al V

olu

me 1

(1) J

une 2

008

World Digital Libraries, Volume 1, Issue 1, June 2008

Editorial ... v

Viewpoints on digital library issues ... vii

A work-centred approach to evaluation of a digital library: annotation system for knowledge sharing in collaborative workAnnelise Mark Pejtersen, Preben Hansen, and Hanne Albrechtsen ... 1

E-research: a new genre of digital library servicesAhmed Taha ... 19

Digitizing cultural heritage and history archivesCase studies from Russia (Comintern Archives) and Albania (National Archives)Jean-Marc Comment ... 27

MEDLIS: Model for Evaluation of Digital Libraries and Information ServicesGobinda Chowdhury, David McMenemy, and Alan Poulter ... 35

Recognition-free search in graphics stream of PDFA Balasubramanian and C V Jawahar ... 47

Open Access: the new frontier connecting the learning commons through hassle-free and seamless scholarly communicationM G Sreekumar ... 61

Book review ... 77Open Access to Knowledge and Information: scholarly literature and digital initiatives - the South Asian scenarioAnup Kumar Das, UNESCO Office, New DelhiReviewed by S M Dhawan

Forthcoming events ... 79

Published by Dr R K Pachauri for The Energy and Resources Institute,Darbari Seth Block, IHC Complex, Lodhi Road, New Delhi – 110 003, andprinted by him at Innovative Designers & Printers, New Delhi – 110 020.

For more information log on to http//bookstore.teriin.org/journal_inside.php?material_id=477&qty=1

Dr Jagdish AroraDirector, INFLIBNETIndia

Prof. José Luis BorbinhaEngineering School, Lisbon Technical UniversityPortugal

Prof. Daniel Chandran, PhDFaculty of Information TechnologyUniversity of TechnologySydney, Australia

Prof. Ching-chih Chen, PhDGraduate School of Library and Information ScienceSimmons College, USA

Dr Gobinda ChowdhuryDepartment of Computer and Information SciencesUniversity of StrathclydeUK

Dr P R GoswamiDirector, National Social Science Documentation CentreICSSRIndia

Prof. Alan HopkinsonLearning Resources, The Sheppard LibraryMiddlesex UniversityUK

Prof. Ee-Peng Lim, PhDDivision of Information Systems, School of Computer

EngineeringNanyang Technological UniversitySingapore

Prof. Gary MarchioniniSchool of Information and Library ScienceUniversity of North Carolina at Chapel HillUSA

Dr A Lakshmana MoorthyDirector, Defence Scientific Information and

Documentation CentreIndia

Dr Usha Mujoo-MunshiChief LibrarianIndian Statistical Institute, India

Prof. Dr Erich J NeuholdComputer Science, University of ViennaAustria

Prof. Paul NieuwenhuysenVrije Universiteit Brussel, Belgium

Mr John PaschoudInformation Systems EngineerLondon School of Economics, UK

Dr A R D PrasadDocumentation Research & Training CentreIndian Statistical Institute, India

Prof. Dr Michael SeadleDirector, Institute for Library and Information ScienceHumboldt University in Berlin, Germany

Dr V N ShuklaDirector (Application), CDAC, Noida, India

Prof. Ingeborg Torvik Sølvberg, PhDDept. of Computer and Information ScienceNorwegian University of Science and TechnologyNorway

Prof. M G Sreekumar, PhDDepartment of Information ScienceFaculty of Computer Science & Information TechnologyUniversity of Malaya, Malaysia

Prof. Shigeo SugimotoUniversity of Library and Information ScienceJapan

Prof. P Tapio Varis PhDActing President, Global University System,Media Education, University of Tampere, Finland

Prof. Shalini R. Urs, PhDExecutive DirectorInternational School of Information ManagementUniversity of Mysore, Mysore, India

Prof. Om VikasFormer Vice-Chancellor, ABV Indian Institute of

Information Technology and ManagementIndia

Prof Ian H. WittenDepartment of Computer ScienceUniversity of Waikato, New Zealand

Editorial Advisory Board

Assistant EditorS GopalakrishnanT E R INew Delhi, India

Editor-in-ChiefProf. N BalakrishnanIndian Institute of ScienceBangalore, India

Associate EditorP K BhattacharyaT E R INew Delhi, India

EditorDebal Chandra KarT E R INew Delhi, India

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

World Digital Librariesan international journal

Volume 1, 2008

The Energy and Resources Institute

© The Energy and Resources Institute 2008

No part of this publication may be reproduced, stored in a retrieval system or transmitted inany form or by any means, electronic, mechanical, photocopying, recording or otherwise,without the prior permission of the publishers, TERI Press, The Energy and Resources Institute,New Delhi 110 003, India.

The publisher does not assume any responsibility for any injury and/or damage to persons orproperty as a matter of products liability, negligence or otherwise, or from any use or operationof any methods, instructions or ideas contained in the material herein.

Contents

v

vii

1

19

27

35

47

61

77

79

Editorial

Viewpoints on digital library issues

A work-centred approach to evaluation of a digital library: annotationsystem for knowledge sharing in collaborative workAnnelise Mark Pejtersen, Preben Hansen, and Hanne Albrechtsen

E-research: a new genre of digital library servicesAhmed Taha

Digitizing cultural heritage and history archivesCase studies from Russia (Comintern Archives) and Albania (National

Archives)Jean-Marc Comment

MEDLIS: Model for Evaluation of Digital Libraries and Information ServicesGobinda Chowdhury, David McMenemy, and Alan Poulter

Recognition-free search in graphics stream of PDFA Balasubramanian and C V Jawahar

Open Access: the new frontier connecting the learning commons throughhassle-free and seamless scholarly communicationM G Sreekumar

Book reviewOpen Access to Knowledge and Information: scholarly literature and

digital initiatives – the South Asian scenarioAnup Kumar Das, UNESCO Office, New DelhiReviewed by S M Dhawan

Forthcoming events

WDL (World Digital Libraries)Volume 1, Number 1 • June 2008

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

World Digital Libraries 1(1)

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○


Editorial

N BALAKRISHNAN

Indian Institute of Science, Bangalore

World Digital Libraries 1(1) v–vi

The world of computers and communicationhas witnessed a stupendous and relentlessgrowth in the past few decades. Libraryscience exploited this trend like no other field.The more recent ‘storage and content’revolution has come as a boon to libraryscientists. The Internet and its large plethoraof search engines have changed library scienceto information science. The ever-changingface of the traditional library, and the easewith which information can be stored,processed, and accessed independent of thelocation, time, and language has, in largemeasure, democratized access to information.Information science has now become anintellectually alert field, tapping innovationsand newer ideas. These innovations now drivethe field of computers, and storage andcommunication, and, indeed, they havebecome more like the hunters rather than thehunted. Like many new inter-disciplinaryareas, information science is poised at theinterface between science and engineering,and it is from here that future innovationswould emerge.

Digital libraries epitomize the advances ininformation science and form a strong pillaron which the impact of all advances would befelt. One of the fundamental paradigms ininformation science is that information growswhen shared. The WDL (World Digital

Libraries) journal has been launched toprovide a platform to those who use newapplications and technologies, professionalsand young students for sharing knowledge –knowledge that is authenticated by peer reviewand validated by an international-baseclientele.

The major topics that have become morerelevant in the digital world are digitalpreservation and management of electronicinformation, its access, multilingualism, andsecurity. In the digital world, the existingcopyright laws and business models need acomplete rethinking. The digital librarydevelopments have also initiated new thoughtsin open access and the need for newerbusiness models. All these will form thesubject matter of the journal. The WDLjournal would provide an avenue for reportingoriginal research contributions and sharing ofbest practices and experiences across theworld.

The members of the editorial board are thebest in their respective field. But keeping inmind that knowledge knows no boundaries,they would continue to rely on inputs fromthe readers.

There are six peer-reviewed articles in theinaugural issue of the journal. The topics spanfrom evaluating digital libraries, providingopen access for scholarly communication,

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○


using e-research as a new genre in digitallibrary services, preserving cultural heritage –a Russian experience, and searching acrossPDF documents. Hopefully, the readers would

vi Editorial

benefit immensely from these articles andcontribute more, so that there is a non-linearexpansion of knowledge in the field of digitallibraries.

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○


*Director, Institute for Library and Information Science, Humboldt University in Berlin, Germany

Viewpoints on digital library issues

Digital libraries: economics andarchiving

MICHAEL SEADLE*

Digital libraries face three key issues—interaction with humans, economic viability,and the long-term archiving of contents.

Interaction

The user interfaces for most digital librariesfail in key aspects. Many assume that someform of search strategy is the best way todiscover contents and do little to take context,organization or cognitive linkages intoaccount. The help systems for these interfacestend towards the verbose and somehowgenerally fail to answer the user’s question.Colour, image, structure, and shape aretreated as if they are purely decorative issuesand not integral parts of a human informationsystem. Cultural assumptions also permeatethe interfaces with a very strong emphasis onthe verbal over the visual or oral.

Economics

Gobinda Chowdhury raises the question ofeconomics in his article on MEDLIS and formost institutions, this issue is both critical andinvisible. With the exception of JSTOR,

arguably, no contemporary digital library canpay its own way. This is true of conventionallibraries too, of course, but conventionallibraries have funding in place, while digitallibraries tend to subsist from project toproject and slide slowly into technologicalobsolescence, once the project funding ends.OPACs, in so far as they could be considereddigital libraries, escape this fate because theyhave become so integral to conventionallibrary operations, and this may be one reasonwhy OPACs continue to expend to supportmore digital library functions.

Archiving

The problem of data loss is well known andbroadly discussed. The solution does not liein finding a long-lasting medium but in thetimely copying of digital objects. Withenough copies and sufficient distribution,digital objects can in theory exist indefinitely– a distinct advantage over any purelyphysical medium. This form of archivingraises issues about integrity, authenticity, andreadability, which need to be solved andperhaps redefined over a period of time.Above all, digital archiving needs to emergefrom the phase where multiple systems claimto handle all problems. Rigorous andreproducible testing needs to become a keycomponent of any archiving program.

World Digital Libraries 1(1) vii–ix

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○


Digital libraries: Europeanapproaches

TAPIO VARIS*

2008 is the European Year of InterculturalDialogue. In 2006, the European Commissionset up a Media Literacy Expert Group with theaim of analysing and defining media literacyobjectives and trends, highlighting andpromoting good practices at European level,and proposing actions in the field. Theanalysis and assessment of the EuropeanCommission’s previous activity (for instance,media literacy projects funded within theeLearning programme) were also discussedand examined. The work of this group wasused for the communication ‘A Europeanapproach to media literacy in the digitalenvironment’ by the Commission of theEuropean Communities in December 2007(COM(2007) 833 final).

A study on ‘Current trends and approachesto media literacy in Europe’ wascommissioned in May 2006 (I was in thesteering committee). It maps current practicesin implementing media literacy in Europe,confirms the tendencies that emerged in thepublic consultation, and recommends somemeasures to be implemented at communitylevel to help foster and increase the level ofmedia literacy. Finally, it briefly outlined thepossible economic and social impacts of an EU(European Union) intervention in this field.1

The concept of digital literacy in a broadsense is a way of thinking but it can also beunderstood as complementary to the conceptof media education and even synonymouswith media literacy. Digital literacy as medialiteracy aims to develop both criticalunderstanding of, and active participation in,

the media. Digital and media literacy is aboutdeveloping people’s critical and creativeabilities. Using a computer requires diverseand complex knowledge. It also introduces theindividual and humanity to new contexts,which demands mental, intellectual,profound, and complex changes. In essence,digital literacy is a complicated process thatconsists of acquiring a new tekne, ability of artor craft. Creativity and culture becomeessential raw materials for the knowledgeeconomy.

The introduction of eLearning requires newcompetencies. A competency is an area ofknowledge or skill that is critical forproducing key output. The competencies canbe grouped into generic categories such asgeneral, management, distribution method,and presentation method, which helpillustrate the relationship among certaincompetencies.

The definition on digital competence asfrom the key competencies of life longlearning recommendation of the EU is asfollows.

Digital competence involves the confidentand critical use of IST (information societytechnology) for work, leisure, andcommunication. It is underpinned by basicskills in ICT (information and communicationtechnology): the use of computers to retrieve,assess, store, produce, present, and exchangeinformation, and to communicate andparticipate in collaborative networks via theInternet.

The use of ICT is quite a basic skill. Wethink it is about a combination of basic ICTskills, innovation ability and domain-specificknowledge (such as education) which makesinnovations happen. The question is what arethe real skills one needs in eSociety, where all

*Acting President, Global University System, Media Education, University of Tampere, Finland1 (http://ec.europa.eu/avpolicy/media_literacy/index_en.htm)

viii Viewpoints on digital library issues

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○


the services and commerce are web based?Those skills differ quite a lot from the ‘old’world’s ICT basics.

We are facing a major educationalinvention in technology after phoneticalphabet, the and printing. It is telematics,which means computers connected tonetworks. There is a need for a new globalstrategy for promoting the role of ICT indifferent fields of the working life of theemerging knowledge societies and developingeducational and training approaches on howto learn the use of ICT and become digitallyliterate in the spirit of sustainabledevelopment.

Traditional alphabetical competenciesinclude basic operational competencies relatedto texts, psycho-cognitive competencies relatedto alphabetic signs, basic reading and writing,basic mathematical and textual comprehension,and socio-communicative competencies.Digital competence includes basic operationalcompetencies related to screens and computers,psycho-cognitive competencies related tocomputer signs, basic computer competence,

interactive media and on-line competence, andglobal socio-communicative competenceassociated to cyberspace.

Digital literacy is a fundamental element ofthe knowledge society. In Europe, it is nowseen as a right for all. Ensuring that everyonehas the necessary skills, competences,experiences, and attitudes to make effectiveuse of ICT is probably the biggest challenge ofall. The illiterates of the 21st century are notthose who cannot read and write but thosewho cannot unlearn, learn, and relearn.

It is widely understood that the mostimportant skills of the future would becommunication skills. Today everyone is ableto access vast amount of data without amediator. Critical thinking skills are needed asa productive and positive activity. Criticalthinkers see the future as open and malleable,not as closed and fixed. They are aware of thediversity of values, behaviour, socialstructures, and artistic forms in the world.Critical thinking is a process, not an outcome,and it is emotive as well as rational.

Viewpoints on digital library issues ix

Abstract

This paper presents an expert evaluation of a collaborative annotation system named DiLAS (DigitalLibrary Annotation Service). It can be globally accessed by individuals as well as different usercommunities, and knowledge is created and shared within English-speaking communities. It containsa collection of textual documents on information science and software science, and it gives access toall kinds of related material such as authors’ home pages, photos, articles, and so on. An analyticalevaluation was conducted as a participatory group evaluation, which involved presentation beyondthe objectives and rationale papers and development of the prototype. The empirical evaluation ofDiLAS consisted of two experiments. The first evaluation was a bottom up evaluation that began atthe elementary level with an evaluation of the usability of the interface. A Cognitive Walkthroughapproach was chosen using a qualitative approach. The next evaluation moved towards the broaderwork context with a user- and work-centred evaluation involving an entire collaborative tasksituation, which required knowledge sharing on a common real life work task. This evaluationresulted in a large number of user requirements for the next prototype.

World Digital Libraries 1(1): 1–17

A work-centred approach to evaluation of a digitallibrary: annotation system for knowledge sharing incollaborative work

ANNELISE MARK PEJTERSEN, Center of Cognitive Systems Engineering, DenmarkPREBEN HANSEN, Swedish Institute of Computer Science, SwedenHANNE ALBRECHTSEN, Institute of Knowledge Sharing, Denmark

Annelise Mark Pejtersen, Preben Hansen, and Hanne Albrechtsen2


○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

Introduction

Most contemporary digital libraries arerepositories of information wherein contentscan be searched or browsed by the user.However, when the users’ activity is part of alarger problem solving or work process,searching and browsing of repositories areoften not sufficient, since complex problemsolving tasks also involve the generation ofnew idea and knowledge, which often requirescollaboration and knowledge sharing of withothers. A digital library can support problemsolving and work tasks in many different ways.One way is to establish collaborativeannotation system that allows the users to workwith annotations and share knowledge withoutlosing active support from the digital library.This paper presents a collaborative annotationsystem named DiLAS (Digital LibraryAnnotation Service). It can be accessedglobally by individual user as well as differentuser communities, and knowledge can begenerated and shared globally within English-speaking communities, and the content ofannotations can also be written in the nativelanguage of user groups. It contains acollection of textual documents oninformation science and software science, andgives access to all kinds of related materialsuch as authors’ home pages, photos, articles,and so on.

The implementation of an annotationsystem in a DLMS changes the characteristicsof the application and the behaviour of itsusers. Writing an annotation may lead the userto a dialogue with the author of a document orthe author of an annotation, which eventuallycan turn into a more elaborated collaborationand creation of additional annotations andnew knowledge. For further information onannotation systems, see Agosti M, Benfante L,and Orio N, 2003; Agosti M and Ferro N,2005a; Agosti M and Ferro N, 2005b; AgostiM, Ferro N, Frommholz I, Thiel U, 2004;

Agosti M, Ferro N, and Orio, N, 2005;Andersen H H K, Albrechtsen H, and Cleal BR, 2004; Bottoni P, Civica R, Levialdi S, OrsoL, Panizzi E, and Trinchese R, 2004; Bottoni P,Civica R, Levialdi S, Orso L, Panizzi E, andTrinchese R, 2005; Klas C P, Fuhr N, andSchaefer A, 2004; Ovsiannikov I A, McNeill TH, and Arbib M A, 1999.

Annotations are supposed to be associatedwith the documents in digital libraries, andthey are regarded as an additional informationor as new content of the digital library.Annotation work can also be more or lessstructured user information on a researchtopic for which documents are searched andannotations are prepared. Making annotationsand talking about them is not an isolated task;on the contrary, it is deeply embedded in thework task. Therefore, the writing ofannotations cannot be separated from theactual task to which it is linked. Annotationwork and sharing of knowledge may involveknowledge associated with the common worktask as well as with the documents in theDLMS. How annotation work is intertwinedwith work tasks as a means, among others, toaccomplish work is very dependent on thework domain and the tasks. In addition to theestablishment of a network collaborativeannotation system, it is, therefore, necessary toconsider how the interplay of sharingknowledge on documents and knowledge onwork tasks can be facilitated.

Furthermore, this interplay is performed bydistributed users with different cognitivecapabilities and domain knowledge, differentsocial infrastructures and nationalities – andpossibly, different tasks, preferences, andexperiences, too. It is important to investigatewhat are the common ground and thecommon needs, as well as clarify theimportance of the differences and how theycan eventually be overcome.

A thorough understanding of these issues isonly possible through empirical studies, which

3


○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

A work-centred approach to evaluation of a digital library

ideally speaking should be performed ahead ofthe design and before the prototypedevelopment is completed. However, forpragmatic reasons, often this approach is nottaken, and DiLAS is no exception, since it isprimarily developed on the basis of alreadyexisting DL (digital libraries) prototypestogether with the input of good ideas anddomain knowledge from expert DL researchersand developers. Given this condition,empirical evaluation of DiLAS prototypes byuser and work studies will therefore informthe design as an ongoing, iterative process,concurrently with the development of the nextprototypes in response to the users’evaluations of each new design.

Extensive user and work studies can besttake place independent of a specificapplication, which the user has adapted to,since users’ behaviour will reflect the optionsof the application. However, a collaborativeannotation system is not yet a common tool; itis a novel design idea with prototypes, whichcan be developed in an iterative participatorydesign and evaluation process with theparticipation of users, developers, designers,and evaluators.

Aims of the paper

The primary aim of this paper is to presentwork in progress on a user- and work-centredevaluation approach which took place prior toplanning upcoming, empirical end-userevaluations with a sample of real life,heterogeneous end-users.

The second aim is to illustrate the kind ofresults that these evaluation experimentsproduced. Only a few examples can be given ina short paper. Many of the suggestions havealready been implemented in the next versionof the DiLAS prototype, which will beevaluated in the near future.

The third aim of this paper is to introduce anovel design of a DLMS with a collaborativeannotation system that has an extensive

functionality, which may inspire other DLMSresearchers.

Related work

Knowledge about users’ requirements fromreal life situations is very important for thefuture development of DLMS with acollaborative annotation system in general andfor the DiLAS prototype in particular. Twoprevious approaches to design and evaluationof different collaborative annotation systemsfor knowledge sharing are briefly introducedbecause comprehensive field studies andevaluations by end-users were a strongcomponent in both.

Annotation, indexing, and retrieval ofdigitized historical archive

Collate, a collaborator for annotation,indexing, and retrieval of digitized historicalarchive material (<www.collate.de>) has acollaborative annotation function somewhatsimilar to DiLAS. It has a different knowledgesharing model for annotation content that hasbeen adopted from linguistic discoursetheories. The collate annotation function wasevaluated by professional archivists, whocollaborated synchronously in an evaluationworkshop that lasted two days on makingannotations on documents in a common worktask on which they all had some knowledge.The archivists had different expertise in films,and they belonged to different scientificparadigms, which reflected their differentnational cultures. They were from the TjeckRepublic, Germany, and Austria. Thearchivists realized the usefulness of having ageneric knowledge model to support sharingand creation of knowledge among diversedomain experts, who had not collaborated.They found this particular model somewhatartificial compared to their intuitive expertbehaviour, when arguing with their colleagues.The knowledge model had to accommodatethe discourse of experts in an unfamiliar



○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

knowledge sharing situation, and not the otherway around. There was no doubt that the usersexperienced the idea of using a collaborativeannotation system very positively with respectto sharing knowledge and learning. (Brocks H,Stein A, Thiel U, Frommholz I, Dirsch-Weigand, 2002; Cleal B R, Andersen H H K,and Albrechtsen H, 2004; Frommholz I,Brocks H, Thiel U, Neuhold E, Iannone L,Semeraro G, Berardi M, Ceci M, 2003;Pejtersen A M, Albrechtsen H, Cleal B, HansenC B, Hertzum M, 2001).

Annotations in a collaborativebusiness system

The goal was to enhance a business systemwith a combined recommendation andannotation system for more effectivecollaboration within a specific work groupwithin a large worldwide software company.To this end, a user and usability study wasconducted that included the collaborativeaspects of the system. Different stages of themethodology were (1) preliminaryrequirements specification: an informal initialdefinition of users’ needs was gathered via adiscussion with expert representatives;(2) general scenario outline: on the basis ofwhat emerged in the informal discussionabove, a proposed scenario was suggested,representing the work-tasks and theirinteraction with the proposed system; and(3) scenario and preliminary visualization.One result from the study was thatannotations were used to keep track of theexperts and colleagues within a project. In thisway an annotation system can be used as anawareness tool that keeps track of variousaspects of the project like what is being done,who is doing what, and how things are beingdone. However, if the (physical) distance istoo small between the colleagues in a group(say that they have rooms close to each other),the annotation system will not be used because

human–human discussion is actuallypreferred. This will, of course, depend on thedomain and the tasks. The collaborativeactivities in the project were affected byparameters such as time, cultures, andlanguages due to the fact that this is a world-wide operating company (Hansen P andJärvelin K, 2004; Hansen P and Järvelin K,2005).

The DiLAS prototype

DiLAS is intended to be used by a wide varietyof users like, among others, scientists of anyarea dealing with scientific, individual andcollaborative work in computer science andinformation science. They include studentspreparing for their exams or thesis, PhDstudents organizing their relevant literatureand documenting new ideas, as well asscientists involved in projects or in commonproblem solving exercise or those sharingdocuments and discussing questions andmatter of common interests.

Task of writing a new text

Amongst the various work tasks, that can besupported by annotation work, is theprototypical task of writing a new text, whichis a very common task among scientists, asevidenced by following points.� Writing a thesis (PhD, master) and

organizing relevant literature (writingsummaries and comments) in annotations.

� Preparing a submission and to accomplishthis, accessing summaries and comments ofdocuments in annotations.

� Collaboratively preparing a submission of aproject proposal and in relation to thisusing annotations to respond to questions.

� Discussing new ideas, and interpreting thedocuments and material at hand.

� Reviewing articles.� Writing diaries.

5


○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○


A user’s task of writing a new text throughthe support of annotation work can, forinstance, cover the following knowledge-sharing activities.� Discussion of the digital library content

with other users including the annotations.� Discussion of the content of annotations

with other users (annotation thread).� Collaborative or individual integration of

annotation content in a new text (editingfacilities).

� Access for the user to his/her previousannotations and to other users’ annotations

� Decision to revise a current text accordingto new information from another user’sannotation on the topic of the new text.

All these activities can be involved in thetask of writing of a new text. The annotationsystem of the digital library becomes an openand dynamic work environment. The users arefree to choose their communicative discourseduring knowledge sharing and become visiblecontributors to the content of the digital library.

Annotation work with DiLAS

Provided subsequently is a list of some of thefunctions of DiLAS which support theannotation work before or after an annotationhas been made.

Create annotations

Users are able to annotate different kinds ofdigital objects and type links between digitalobjects or markings and highlighting. Thisprototype only contains text. In order to findout an explicit meaning of an annotation, theuser may select one or more annotation types.It is possible to attach some metadata to anewly created annotation, like title and author.

Save annotations in folders

Retrieved references, links, and full-textdocuments can be saved in a personal folder.It is possible to create a community folder for

collaborating users in order to support theusers’ knowledge sharing.

Make relationships among annotations

No links are yet available for the user to makelinks among related annotations. It is possibleto mark a passage in an annotation in order toindicate that a new annotation refers to thistext passage.

Modify and delete annotations

In a collaborative environment, the ‘modifyand delete’ annotation function may becritical, since annotations made by othersmight depend on the annotation to bemodified and deleted. Hence, a certain policyfor deletion and modification of annotations isnecessary.

List annotations

When users view a selected digital object, theyare able to get a list of all associatedannotations, be it their own annotations orthose made by others to which they haveaccess.

Browse annotations

In case of nested annotations, users are able tobrowse annotation threads. If annotations arereferences to other digital objects, it should bepossible to browse the network of digitalobjects and referential annotations. This is notyet implemented.

Search annotations

Users can search all annotations referring to adigital object, and the annotations being of acertain type or belonging to a certain user orgroup. Searching for annotations is also astrategy to search for documents.

Set the scope of annotations

Users can decide about the scope of theirannotations. It should be possible for the users



○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

to determine if an annotation is private,shared within a group or public. This is notimplemented as yet.

Group and user management

In order to be able to map the users with theDiLAS policies for groups and individuals,modelling of user groups may be required toaddress the question of authorities toadminister the policy of groups and appointmoderators of an annotation writing group.This not implemented as yet. The functions of‘set the scope of annotations’ and ‘group anduser management of annotations’ are the mostgeneral functions, ‘meta functions’, comparedto the other functions. The ‘set scope’ functionallows the users to decide in which context(for example, work task) the annotation workis performed, and the ‘management function’allows the user to decide on the participants ofthe user groups and what they can access. Apolicy for the use of DiLAS will be developedin the next prototypes.

Examples of user group policies can beestablished by creating a number of taskscenarios and organizing focus groupinterviews, which will provide the necessaryinformation about possible future usesituations and the degree to which users needlocal policies to use the tool in their work.Hence, such policies can be given as examplesfor the users to look at. The control of changeof policy also needs to be made clear. Hence, itshould be possible for the user to select abutton on the interface named ‘policy’. Themenu could then give the user the option ofprivate-shared-public. In a more complex anddistributed environment, it may be necessaryto have different access levels. Depending onthe role the user has in a group, he/she may beable to access a specific set of information.

If annotations are not created, all thefunctions in Table 1 will become superfluous.Therefore, it becomes particularly importantto make sure that the creation of annotations

is easily and successfully carried out by theend-users. There are several means that canhelp to accomplish this goal, and althoughmany have been implemented in DiLAS, westill lack the empirical knowledge about how itworks for inexperienced end-users and how it

Figure 1 Menu with the view annotation andrework annotation option

Table 1 The model with different types of

knowledge

Create AnnotationAnnouncementIntegrationSummaryExplanationCommentMemorandumSolutionQuestionExample

7


○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○


can help collaborating users to share theirknowledge.

A user- and work-centredframework for design andevaluation

The design and evaluation of a Digital LibraryAnnotation System that is able to supportglobal knowledge sharing and collaborativeannotation work require a framework that isable to inform and guide the complex,semantic problems involved (Patton M, 1990;Rasmussen J, Pejtersen A M, and L PGoodstein, 1994). The semantic complexity inuser and work-centred design and evaluationis found in the need to bridge the conceptualgap between the analyses of users and theanalysis of their work environment. Inaddition, it is necessary to bridge theconceptual gap between these two kinds ofanalyses and the physical, functional and nonfunctional, components of a system like DiLAS.

The framework is represented as an onionmodel that spans many generic layers thatprogressively bridge the gap between the usersand their work environment. However, thedesigners and developers are responsible forthe challenge of bridging the gap between theanalysis of users and their work and thephysical components of an application, sincethere is no generic relationship between thephysical components of an application and theanalysis of users and their work. It is,therefore, their task to meet the design andevaluation requirements from qualitativeempirical user and work studies to theindividual physical components of, in thiscase, the DiLAS annotation system.

To illustrate this last bridging, each layer ofthe framework is presented first with anevaluation question reflecting the user andwork perspective, second, with a componentof DiLAS, which will be impacted by theanswer to the evaluation question.

User characteristics

Does DiLAS match users’ resources, such asusers’ knowledge background, level ofexpertise, cognitive and perceptual skills,language, social values, and culture and theirsubjective preferences, and so on? Theevaluation question addresses the content andform of the interface and the usability of thedisplays – such as the meaning of the icons,which appear together with the informationon a retrieved document in order to inform,for example, that the document has alreadybeen annotated by somebody else.

Strategies

Does DiLAS support all relevant strategiesadopted by the users? A strategy is based on aparticular kind of mental model and therelated interpretation of information, and ontactical rules (we do not know these yet withannotation work with DiLAS users).Evaluation question addresses the access androute to the database content.

Decision making

Does DiLAS support individual andcollaborative decision making and thinkingand learning? – such as analysis andinterpretation of knowledge, evaluationcreated knowledge, formulation of goals forknowledge sharing, planning to compare andcomment on annotations? Evaluation questionaddresses the content and structure of thedatabase – is the information adequate forcollaborative analysis, evaluation, and planningduring annotation work and sharing ofknowledge.

Task situation

Does DiLAS support the whole individual andcollaborative task repertoire—for example, thecollaborative work task of writing a new text,which involves search for annotations, createknowledge using the knowledge sharing



○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

model, integrate annotations in a new text,communication with colleagues. Does itsupport processes such as write an annotation,read an annotation, and mark the content ofan annotation? The evaluation questionaddresses all the functions of DiLAS todetermine if there is – or not – a function tosupport the users’ attempt to perform a task,and how well the users can apply, and whetherthey like, a function designed to support thetask, like listing annotations of a document inorder to decide which annotation should beread by a colleague.

Collaboration and management

Does DiLAS support collaboration,coordination, and management of theannotation work? For example, coordinationof individual and common decisions duringcollaborative analysis of documents andannotation writings. Also, set the scope ofannotations for the common work task of agroup or in a public work space, appoint theperson who is responsible for managing theannotations of the work task, and verify thatthe organization of the annotation work iscorrect. The evaluation question addresses thesufficiency of functions of DiLAS designed forthis purpose, like set the scope of annotationsand user and group management.

The work domain in the environment

Qualitative, empirical-user- and work-centreddesign and evaluation are concerned with thecorrespondence of a DLMS with the actualneeds of the end-users and its compliance withthe needs of the work domain. It is anassessment of how well DiLAS actually servesthis coupling. Does DiLAS improve the� quality of their work;� efficient accomplishment of their work; and� enjoyment of their work.

Is DiLAS knowledge domain of informationscience and software science research an

adequate common work space to accomplishthese goals? Do the functional relationshipswork well among these goals, the tasks, theprocesses, the available resources? Is thesystem appreciated by the users and theirenvironment? This evaluation questionaddresses all the relationships amongcomponents of a full-scale DiLAS system.

The focus of the evaluation experimentswith the DiLAS prototype with respect to theselayers was on requirements for modificationand further development of the prototype withrespect to how well the� users can understand the interface;� functions support the task situation of

annotation work; and� functions support collaboration and

coordination in annotation work.

Analytical and empirical evaluationof DiLAS

The DiLAS project defines the overall aim of aformative evaluation as gaining knowledgeabout how the present design of DiLAS as adecentralized annotation service complies withthe needs of the prospective users.

Figure 2 shows that evaluation can beperformed either analytically or empirically, orboth approaches can be applied, since they aremutually supportive. An analytical evaluationis a comparison of requirements derived fromanalysis of work, and from users anddevelopers in order to verify that anapplication actually meets the requirements.Analytical evaluation will normally precedeempirical evaluation in order to detectproblems and address these before theevaluation of compliance with users’ needs.Second, it serves to estimate if the prototype isin an appropriate shape for an end-userevaluation.

The analytical evaluation was conducted asa participatory group evaluation, whichinvolved presentation beyond the written

9


○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○


papers of the rationale behind thedevelopment of the prototype. Discussions ofhow well the prototype met the requirementsinvolved different viewpoints from fourexperts including knowledge about softwaredevelopment, DLMS user and work-centredsystem design and evaluation.

The relationship between analytical andempirical evaluation can be viewed ascomplementary. Empirical evaluation involvesthe application’s performance when applied byactual end-users, and usually a path fromdetails to global features will be the bestapproach. An end-user evaluation of aprototype will have validity relative to thecorrespondence of the evaluation with theconditions in a real environment. It will bemore valid if the evaluation includesperformance criteria that reflect theperformance criteria adopted by end-users in areal-life situation.

The empirical evaluation of DiLASconsisted of two experiments. The firstexperiment was a bottom up evaluation that

began at the elementary level with anevaluation of the usability of the interface. ACognitive Walkthrough approach was chosenbecause it would be pointless to conductexperiments to evaluate the support of thefunctionality of the prototype in acollaborative work task, if the DiLAS interfacewas not easily understandable by the users.Using a qualitative approach, the nextevaluation moved towards the broader workcontext with a user- and work-centredevaluation involving an entire, collaborativetask situation, which required knowledgesharing on a common real-life work task. Itwas important to investigate whether theprototype was sufficiently well developed withreference to the overall goal of DiLAS.

Expert users

At present, only the experienced users can usethis prototype, but in principle, everybody canget access. Expert users were chosen in allevaluation sessions, since the developers hadpointed to the fact that the current prototypecould only be used by experienced users andscientists. Thus, the evaluation was conducted bythree experts with long-standing and extensiveexperience in system design and evaluation,information science, human-computerinteraction, psychology, and sociology.

Participatory evaluation of thedesign objectives

This evaluation was carried out to determine ifthe system meets the goals for which it wasdesigned, and how it does so. It isintellectually very difficult to envision thedesign intentions embedded in a complex tool.Therefore, developers’ design rationale duringthe prototype development is important forplanning the end-user evaluation. Mostsystems are much richer in functionality thandescribed in the research reports. The aim was to� learn and evaluate the design objectives and

assumptions;

Figure 2 The framework of user- and work–centred evaluation and design



○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

� demonstrate as to how they are implementedin the prototype; and

� indicate what was not accomplished.

The evaluation question wasHow well does the DiLAS prototype meet thedevelopers’ design objectives?

Participatory evaluation emphasizes therelationship and mutual understanding amongthe participants, the designers, developers,users, evaluators, and so on, and theirindividual reflections on the design. Theevaluators’ role is to understand their needs,their ideas, and their constraints, and totransform this understanding into suggestionsfor changes in users’ behaviour andmodifications of the developers’ ideas aboutthe users’ needs (Spinuzzi C, 2005).Participants of the group evaluation includeda DiLAS developer with expertise in thearchitecture of DLMS and three other experts.Dynamically, a number of different scenariosand cases popped up and were discussed fromthe participants’ viewpoints during thepresentation of DiLAS. This served well toexplain the ideas that had not been

implemented as yet, and, therefore, should notbe involved in the evaluation. For example,the feature of searching for documents basedon the topic of their associated annotations isstill to be developed.

It is the vision that the DiLAS collaborativeannotation system will enhance the users’sharing of knowledge and thereby alsomotivate a change in users’ behaviour usingthe digital library towards more collaborativework. In order to accomplish this goal, userswill have to change their social interaction withthe DLMS away from the traditional passiveand neutral use for solving single problemindividually. With a good annotation system,users will have the opportunity to becomeactive and visible contributors of knowledge,personalize the existing knowledge, and createand share knowledge with others whilecontinuously learning and solving theirproblems in a better way (Figure 3).

This social behaviour during interactionwith the collaborative annotation systeminvolves a repertoire of cognitive capabilities,which become a space of common resourcesduring collaboration; for example,

Figure 3 The cognitive and social behaviour of users, which DiLAS isdesigned to obtain

11


○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○


remembering the discourse of one’s ownannotations and those of others, thinkingabout an argument, clarification of anambiguous terminology, interpretation of themeaning of an annotation from anotherperson, evaluation and comparison ofopinions of different authors on the sametopic, and learning about a new research topicat the edge of one’s own discipline.

Users’ cognitive behaviour

� Remembering� Thinking� Clarification� Interpretation� Evaluation� Learning

Users’ social behaviour

� Become active and visible contributors ofnew knowledge.

� Enrich the content of a digital library bypersonalizing the information.

� Share their knowledge and collaborate.� Learn from annotations made available by

others.

With reference to this, an obvious questionis:

Does DiLAS support and change the users’cognitive and social behaviour and does itimprove the way in which users share knowledgeand collaborate?

It is the vision that DiLAS fosters change inusers’ cognitive and social behaviour whenthey interact with digital libraries. The next setof empirical evaluation experiments will beconcerned with the correspondence of thenext DiLAS prototype 2 with the actual needsof the end-users who are not experts. It is anassessment of the degree to which DiLASactually serves the needs of the end-usersduring its actual performance in the ultimatecontext of use in a real work place.

Usability evaluation of the DiLASinterface

DiLAS is developed as an add-on facility to theagent-based DAFFODIL and to the MADCOWinterface. The DiLAS interface is a graphical,text-based interface with windows, menus andbuttons. Through agent-based facilities forcross-searching databases and webinformation, DiLAS provides the users withthe option of managing annotations acrossdifferent digital libraries independently of aparticular digital library.

The question was:Is DiLAS’ interface easy to understand by a

first time expert user? Will it be easy tounderstand by inexperienced end-users?

A Cognitive Walk Through was applied inorder to determine the problems a user mayencounter using DiLAS for the first time.Cognitive Walkthrough is a structured andsystematic usability inspection method,wherein a list of the sequence of actions ismade with reference to the steps a userinterface will require a user to perform inorder to accomplish a task (Hertzum M andJacobsen N E, 1999; Wharton C, Rieman J,Lewis C, Polson P, 1994). The user’s task wasto know if weblogs are relevant in acommercial website in order to decide whetherit is worthwhile to join a course on how to setup weblogs. The user searched on the topic‘weblogs’ in DiLAS and wanted to annotatethe most relevant documents as a memo.Searching documents is the intendedbeginning of annotation work, and the expertuser could use the DiLAS Search Functionsuccessfully. Reading about relevantdocuments made the user learn enough to takea decision on the participation in a course onweblogs. There was no need for more supportin this single search task with no collaborationinvolved, but the user had no success in



○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

creating an annotation. In conclusion, theinterface could not be easily understood by theless experienced end-users.

Pop-up menus and windows

Many of the options are now made availableby pop-up menus and right clicking. Some ofthe main options could be moved up to theuser interface and not hidden in menus, whichwill reduce the number of steps the interfacerequires the user to take. Too many windowsmake it a rather cumbersome and cognitively ahard task to perform. For example, access tothe DiLAS annotation system seemed to behidden from the user in a long list of menus.One menu had a text line called ‘createannotation’ but it contained five otherdifferent and irrelevant functions, which wasconfusing.

Consistent display of only relevantoptions

When the view annotation button is selected,the annotations are shown in hierarchy. If theuser selects the title with the mouse and clicks

the right button, a menu appears. This menuis different from the menu that appears with aright click on the view annotation button inthe earlier window, and it contains differentoptions that are different from those that arerelevant for DiLAS annotations. Some of theseproblems are due to the integration of DiLASwith two other digital libraries. The fact thatDiLAS is a front end should not be visible tothe users. A separate and unique interfaceshould be designed for the collaborativeannotation system as one global window, thatis the users’ collaborative workspace. Thisworkspace could explicitly display thefunctions involved: search a document, createannotation, view annotation, and so on. Allthese functions should be at the main level andcould have a menu bar, for example, on thetop with buttons. This would allow the usercomplete flexibility together with a verysimple user interface that shows the user allthe relevant functions.

Figure 4 shows the display for creatingannotations and viewing annotations. The rightside of the display represents the options for

Figure 4 The window for ‘Create Annotations’ and for ‘ViewingAnnotations’ (left)

13


○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○


creating an annotation: write the user’s namein the upper window, select a ‘type’ ofannotation content among the displayedcategories of the knowledge model (Table 1),write the title of the annotation, and write thecontent of the annotation in the windowcalled ‘Text’ in accordance with the selectedtype of annotation. The left side of the screendisplays the option of viewing annotations in ahierarchical list, which shows both the createdannotations and the replies to the createdannotations. These replies can be the user’sown or that of other users.

User- and work-centred evaluationof knowledge sharing incollaborative work

The goal was to empirically evaluate thesupport of DiLAS in sharing of knowledgeduring a collaborative real life work taskthrough a more comprehensive evaluationthan the first evaluation of the interface. Theunderstanding of the developers’ objectives asachieved in the previous analytical evaluationwith the developer of the system wasembedded in the planning of this evaluation.

The question was:How well does DiLAS’ support knowledge

sharing and collaboration?The method applied was a user- and work-

centred evaluation as described above. Theevaluation addressed the support of the worktask and the annotation task and the supportof knowledge sharing and collaboration inthese two task situations. The threeprofessional expert users knew each other verywell as collaborators in research projects.Collaboration amongst the three professionalevaluation experts was performed in acommon, real life task, which requiredknowledge sharing and communication amongthe experts.

Collaborative task of writing a newtext for a research report

The experts had many common tasks, amongwhich is the task of writing a new text on thestate-of-the-art in research on annotations andannotation systems. After a group discussion,this task was chosen for the evaluation, alsobecause the writing of a new text isprototypical for DiLAS. A pre-requisite forwriting a new text is the search for documentson the topic of the new text and subsequentlyto write annotations on the retrieveddocuments after they have been looked at. Theaspiration was that they, through the use ofDiLAS, would reach a common knowledgeground that would be useful for their futurecollaborative task of writing a research report.Before that could be accomplished, they wouldhave to share knowledge due to the diversityin their current knowledge as well as theexpected diversity in their annotations due todifferent contents of the documents theywould retrieve and annotate. There was nocollaborative planning as to how to search orhow to perform annotation work. The expertsperformed their search and began theirannotation work individually, planning tocommunicate ad hoc as they went along, andfinish with a group discussion on the topic oftheir task. After working with DiLAS, theevaluators meet to share the experiences anddiscuss the problems they encountered duringtheir use of DiLAS and their discussion onnew ideas for the redesign of the next DiLASprototype.

Knowledge sharing

It became clear, quite early, that the threeexperts seriously wanted to share knowledgeon their experiences of the prototype withoutlosing support from DiLAS. In the currentsystem the users have to open their own e-mailsystem, and write e-mails separately. DiLASonly supports asynchronous communication,



○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

but it also needs to support synchronouscommunication. It should be possible for thecollaborators to notify and contact each otherin order to discuss the annotations, shareknowledge, ask questions, and confirm thatthe same observations were made.Synchronous collaboration at a distancerequires that a communication tool like email,voice mail, and the like is integrated into thedigital libraries, which and thus makes onlineinteractive knowledge creation and sharingpossible.

Collaboration

It is necessary for a group working together tobe able to share knowledge within a commonworkspace about an annotation that is related toa common task. As a consequence, there shouldbe a function available for the participantswithin a group to create a common workingdocument for all to work on. This can be donein several ways, for example, a commonworkspace could include two kinds ofannotation functions: annotations associatedwith documents as it is now, and annotationsassociated with collaboration on the commonwork task. This will also make it possible tokeep track of the tasks for which theannotations are created. In the current system,it is only possible to create a community folderfor collaborating users, which contains all thematerials retrieved from the digital libraries. Inaddition to the establishment of a networkedcollaborative annotation system, it is necessaryto consider how the interplay of sharingknowledge on documents and knowledge onwork tasks can be facilitated.

Evaluation of the knowledgemodel for creation of annotations

The developers of DiLAS have strived todefine a comprehensive model for structuringthe content of annotations. The list belowenumerates the mandatory categories, which

have been chosen as a support to users, whenthey write annotations for themselves or whenthey communicate with others.

The users have to decide as to what theywant to create within these eight categories.There are several advantages of this approach.One advantage is that knowledge managementis supported by an inherent knowledge model,which forces the user to analyse what kind ofknowledge the user actually wants tocommunicate, either prior to or after havingwritten it down. In addition, when theannotation content is organized consistently,it can also be searched across collections, forexample, all annotations with announcementson a specific document can be gathered.Another advantage of this is that theknowledge that is created by different users iscommunicated in a compatible format, andthus the content can be easily understood,communicated, recognized, and shared amongcollaborating users. Such a semantic modellingof knowledge becomes the common workspaceof annotations for users and user communitiesin which they can both communicate, andshare and manage knowledge at the same time.

A prerequisite is, however, that thisknowledge model is experienced as intuitiveand natural for users. If that is not the case,the knowledge model will be experienced as anartificial constraint that is imposed on theuser’s way of thinking, which is cumbersometo use and which will take time and demandunnecessary mental resources. This negativeattitude was experienced in earlier work asmentioned above. In order to avoid thisproblem, a knowledge model can be developedbottom up, for example, it can be derived foran empirical analysis of how users writeannotations in a free fashion duringcollaborative knowledge sharing. It is verylikely that it will be possible to identify acommon discourse with generic categories ofknowledge in annotations written by a user

15


○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○


community in a specific knowledge domain. Itis also very likely that some of the annotationtypes that are used in the current prototypewill appear. Such a bottom up approach toknowledge modelling has been successfullyadopted earlier in a different context.

However, the complexity of the developmentof an active knowledge sharing model thatreflects the collaborating users’ commondiscourse in a specific knowledge domain isenormous. As mentioned earlier, knowledgesharing is performed by distributed users withdifferent educational background, differentresearch paradigms, different organizationalvalues and cultures, and so on. The solutionmay be to propose a non-scientific commonsense knowledge sharing model, which theusers can learn and adopt fast, if they find itacceptable, when they create knowledge.

The question is:How well does DiLAS knowledge model

support the creation and sharing of knowledge?

This will be part of the future evaluationexperiments with DiLAS prototype 2.

Conclusion

The main topic of this paper was the initialevaluation and assessing (re)design

requirements of the first DiLAS prototype. Itresulted in both short-term and long-termrequirements. Short-term requirements areabsolutely necessary to be implementedbefore the end-user evaluation within aminimum of functions. Long-termrequirements are those that are beyond thecurrent system design by adding functionsthat will conceptually improve the design. Anew design of a DiLAS prototype 2 based onthe short-term requirements is already inplace together with a planning of a largerevaluation experiment with non-expert end-users.

Acknowledgement

The authors acknowledge the collaborators ofDELOS WP.4.10, in particular IngoFromholz, who presented the prototype inthe first evaluation experiment. DiLAS hasbeen developed in the context of theEuropean DELOS Network of Excellence onDigital Libraries (2003–08). DiLAS is anintegrated effort towards ‘Design,Implementation and Evaluation ofMultimedia Annotations for UserCollaboration’ involving extensive contactwith DELOS colleagues.

References

Agosti M, Benfante L, and Orio N. 2003

IPSA: a digital archive of herbals to support scientific research

In Proceedings of 6th International Conference on Asian Digital Libraries

Digital Libraries – Digital Libraries: Technology and Management of Indigenous Knowledge (ICADL 2003), pp. 253–264.

LNCS 2911

Germany: Springer,

Agosti M and Ferro N. 2005a

Annotations as context for searching documents

In Proceedings of 5th International Conference on Conceptions of Library and Information Science – Context: nature, impact

and role (Colis5), pp. 155–170. LNCS 3507

Germany: Springer



○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

Agosti M and Ferro N. 2005b

A system architecture as a support to a flexible annotation service

In Peer-to-Peer, Grid, and Service-Orientation in Digital Library Architectures: 6th Thematic Workshop of the EU Network of

Excellence DELOS. Revised Selected Papers, pp. 147–166. LNCS 3664

Germany: Springer

Agosti M, Ferro N, Frommholz I, Thiel U. 2004

Annotations in digital libraries and collaboratories – facets, models and usage

In Proceedings of 8th European Conference on Research and Advanced Technology for Digital Libraries (ECDL 2004),

pp. 244–255. LNCS 3232

Germany: Springer

Agosti M, Ferro N, and Orio, N. 2005

Annotating illuminated manuscripts: an effective tool for research and education

In Proceedings of 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL 2005), pp. 121–130

New York: ACM Press.

Andersen H H K, Albrechtsen H, and Cleal B R. 2004

Structuring collaborative research: experiences from an evaluation study of a collaboratory

In Proceedings of 3 Danish human–computer interaction research symposium, Roskilde Universitetscenter, Datalogisk

afdeling, Roskilde, Datalogiske Skrifter, 98, p. 13–16

Bottoni P, Civica R, Levialdi S, Orso L, Panizzi E, and Trinchese R. 2004

MADCOW: a multimedia digital annotation system

In Proceeding of Working Conference on Advanced Visual Interfaces (AVI 2004), pp. 55–62

New York: ACM Press

Bottoni P, Civica R, Levialdi S, Orso L, Panizzi E, and Trinchese R. 2005

Storing and retrieving multimedia web notes

In Proceedings of 4th International Workshop on Databases in Networked Information Systems (DNIS 2005), pp. 119–137.

LNCS 3433

Germany: Springer

Brocks H, Stein A, Thiel U, Frommholz I, Dirsch-Weigand. 2002

How to incorporate collaborative discourse in cultural digital libraries

In Proceedings of the ECAI 2002 Workshop on Semantic Authoring, Annotation and Knowledge Markup (SAAKM02)

Lyon, France

Cleal B R, Andersen H H K, and Albrechtsen H. 2004

Collaboration, communication and categorical complexity: a case study in collaboratory evaluation

Journal of Digital Information Management 2(1): 13–20

Cousins S, Paepke A, Winograd T, Bier E, and K Bier. 1997

The digital library integrated environment

In Proceedings of the 2nd International Conference on Digital Libraries, pp. 142–151

Frommholz I, Brocks H, Thiel U, Neuhold E, Iannone L, Semeraro G, Berardi M, Ceci M. 2003

Document-centred collaboration for scholars in the humanities – the COLLATE system

In Proceedings of 7th European Conference on Research and Advanced Technology for Digital Libraries (ECDL 2003), pp.

434–445. LNCS 2769

Germany: Springer

17


○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○

○


Hansen P and Järvelin K. 2004

Collaborative information searching in an information-intensive work domain: preliminary results

Journal of Digital Information Management, 2(1): 26–30

Hansen P and Järvelin K. 2005

Collaborative information retrieval in an information-intensive domain

Information Processing & Management, 41(5): 1101–1119

Hertzum M and Jacobsen N E. 1999

The evaluator effect during first-time use of the cognitive walkthrough technique

In Human-Computer Interaction: ergonomics and user interfaces, vol. 1, pp. 1063–1067, edited by H J Bullinger and J Ziegler

[Proceedings of HCI International 1999 (Munich, 22–23 August 1999)]

London: Lawrence Erlbaum Associates

Klas C P, Fuhr N, and Schaefer A. 2004

Evaluating strategic support for information access in the DAFFODIL system

In Proceedings of the 8th European Conference on Research and Advanced Technology for Digital Libraries (ECDL 2004),

pp. 476–487. LNCS 3232

Germany: Springer

Ovsiannikov I A, McNeill T H, and Arbib M A. 1999

Annotation technology

International Journal of Human–Computer Studies 50(4): pp. 329–362

Patton M. 1990

Qualitative Evaluation Methods

Beverly Hills: Sage Publications

Pejtersen A M, Albrechtsen H, Cleal B, Hansen C B, Hertzum M. 2001

A Web-based Multimedia Collaboratory: empirical work studies in film archives

Roskilde: Risø National Laboratory

Rasmussen J, Pejtersen A M, and L P Goodstein. 1994

Cognitive Systems Engineering

New York: John Wiley & Sons

Spinuzzi C. 2005

The Methodology of Participatory Design

Technical Communication 52(2): 163–174

Wharton C, Rieman J, Lewis C, Polson P. 1994

The Cognitive Walkthrough Method: a practitioner’s guide

In Usability Inspection Methods, J Nielsen and R L Mack (eds.), pp. 105–140

New York: John Wiley & Sons

Documents

An International Journalbookstore.teri.res.in/docs/journals/Sample copy-WDL-June2008.pdfDocumentation Research & Training Centre Indian Statistical Institute, India Prof. Dr Michael