Upload
giannis-tsakonas
View
452
Download
2
Tags:
Embed Size (px)
DESCRIPTION
A presentation by Konstantinos Koumoutsos, Angelos Mitrelis and Giannis Tsakonas in Libraries In the Digital Age 2010 in Zadar, Croatia.
Citation preview
Evaluation Insights to Key Processes of Digital Repositories
Konstantinos Koumoutsos, Angelos Mitrelis Giannis TsakonasLibrary Information Center, University of Patras, Greece.
Libraries In the Digital Age 2010, May 24-28, 2010, Zadar, Croatia
Digital Repositories
• A familiar stranger:– Important channels for disseminating scientific/academic
productivity.– Important means for preserving scientific/academic outputs.– Representing different types of communities (institutional/
thematic).– Wide-spread systems, but with high degree of uniqueness.– Reflecting different modes of policies and interactions.
2
Digital Repositories’ Evaluation
• Institutional Repositories:– Social aspects• impact of scientific documents, motives of self-archiving activity,
relationships with other types of digital repositories, etc.– Technical aspects• effectiveness/efficiency measures of submission, interfaces, etc.
• DSpace* in particular:– Comparative evaluation of system features– Evaluation of end user performance– Administration/configuration easiness
* Used in this study
3
is Research
• Part of a wider evaluation initiative for the ‘Nemertes’ IR.• Emphasizes in key processes held inside the repository:– Submission (users as self-submitters).– Retrieval (users as searchers).– Editing (librarians, moderators, administrators).
4
Context
• ‘Nemertes’, the University of Patras’ IR.– Developed by Library and Information Center.– Since 2006 on a DSpace installation (v 1.4).– Link: http://nemertes.lis.upatras.gr
• Collections:– Self-submission.– ‘eses and Dissertations of University of Patras’
(primary collection, also on OPAC, printed/digital format)– ‘Technical Reports’– ‘Journals/Proceedings Publications of the LIC personnel’– ‘Faculty Members Publications’
(pre/post-prints of the faculty members of the University) 5
Methodology ~ Overview
Submission process: a questionnaire survey addressed to people who had earlier submitted in the IR.
Retrieval process / Interfaces: a HCI-based user study.
Editing processes: interviews with librarians.
6
Methodology ~ Some Details
• uestionnaire survey:– Previously submitted in the IR.– From 1129 registered, 126 participated.– Online, 25 questions, scale from 1 (-) to 5 (+).
• HCI-based user study:– 24 graduate students, Department of Electrical Engineering,
HCI background.– Based on usability heuristic evaluation principles (10 principles,
4 levels of problem significance).• Interviews:– 5 librarians supporting the service.– Semi-structured, approx. 19 minutes duration.
7
Insights to Submission
• Reported: Overall satisfaction with the process.• Fairly satisfied with help facilities and submission wizard.• Satisfied with the support of the librarians.• Very satisfied with the self-submission and the
bilingualism of the system.
8
Further Insights to Submission
• Satisfied with copyright (now: institutional license/plus an option for a Creative Commons one).– Yet 15.1 % do not know or do not answer.
• Satisfied with exclusion period (publication embargo)– Alternatives: most: up to 2 years / a few: up to 5 years.
• Fairly satisfied with access policy (now: free access to all)– Alternatives: most: controlled access to everyone / a few: closed
to everyone.
9
Insights to Retrieval Interfaces
Principle Unique problems
Number of Reports (by Severity)
Number of Reports (by Severity)
Number of Reports (by Severity)
Minor Important SeriousVisibility of system status 4 2 4 3Match between system and the real world 17 10 29 20User control and freedom 6 3 9 3Consistency and standards 12 5 24 8Error prevention 4 1 14 2Recognition rather than recall 3 1 11 4Flexibility and efficiency of use 9 9 13 5Aesthetic and minimalist design 20 0 74 18Help users recognize, diagnose, & recover from errors 4 0 11 7Help and documentation 5 5 6 2
36 (11.88%)
195 (64.36%)
72 (23.76)
10
Further Insights to Interfaces
11
x16
x6
x9
x8x3
Further Insights to Retrieval
12
x16
x1
Insights to Editing
• e editing process is long and time consuming. – Yet minimizes the possibility of errors.
• Retrospective editing of metadata and corrections to files is considered essential.
• Submitters’ keywords are considered invaluable.• Reporting problems in users’ navigation in the physical
space to accomplish a submission.• Receiving complaints for the completion of the license.
13
Further Insights to Editing
• Usual problems:– e absence of abstracts and keywords in parallel language.– e errors in bibliographic descriptions (capitals, mixed
keyboard languages, ‘greeklish’).– e uploading of damaged or non compliant files.
• Auto-suggestion fields are recommended.• Faculties and Departments should assist in promoting
‘Nemertes’.• Library should be proactive.
14
A Wide View of Findings
• Submitters seem very positive. Librarians and ‘HCI users’ were strict.
• Search functionalities need revision.• Interface features need clarity and consistency.• Editing needs revision to enhance efficiency.
15
Response
• Migration to new DSpace version.• Resolve problems in information retrieval.• Apply add-ons that assist submitters and librarians.– Secure correct entry of metadata.
• Careful consideration of labels and terms.• Exploit submitters’ keywords.• Design a new theme and graphical elements.
16
Conclusions
• An obvious conclusion: more research. – More views are needed.– Each collection poses new challenges.
• e three processes are linked. – Each process has an effect on the other.– e physical aspect of the service affects the digital.
17
e End
• ank you for your attention.• uestions?
Addendum:• ‘Nemertes’: http://nemertes.lis.upatras.gr/dspace/handle/123456789/3069• ‘E-LIS’: http://eprints.rclis.org/18502/• Contact: Giannis Tsakonas, [email protected]
18