Embed Size (px)
Citation preview
ezDL: An Interactive Search and
Evaluation SystemWorkshop on Open Source Information Retrieval (OSIR)
@ SIGIR 2012
Thomas Beckers, Sebastian Dungs, Norbert Fuhr, MatthiasJordan, Sascha Kriewel, Vu Tuan Tran (Demo)
Universitat Duisburg-Essen, Germany
Portland, USA, August 16th 2012
Beckers et al. (UDE) ezDL August 16th 2012 1 / 35
Outline
1 Introduction and Concepts
2 Architecture
3 Evaluations
4 Use Cases
5 Conclusion
Beckers et al. (UDE) ezDL August 16th 2012 2 / 35
Introduction and Concepts
What is ezDL?
ezDL is a. . .
interactive search tool,
development platform for searchapplications and
system for different types ofuser-oriented evaluations.
Beckers et al. (UDE) ezDL August 16th 2012 3 / 35
Introduction and Concepts
Desktop frontend
BBC
D
E
A
F
Beckers et al. (UDE) ezDL August 16th 2012 4 / 35
Introduction and Concepts
Major concepts as an interactive search tool
Digital Library Life Cycle (Paepcke)
Beckers et al. (UDE) ezDL August 16th 2012 5 / 35
Introduction and Concepts
Major concepts as an interactive search tool
Support for tactics and stratagems (Bates)
Beckers et al. (UDE) ezDL August 16th 2012 6 / 35
Introduction and Concepts
Major concepts as an interactive search tool
Proactive support
Beckers et al. (UDE) ezDL August 16th 2012 7 / 35
Introduction and Concepts
Major concepts as development platform
Easy integration of new data sources (DLs)
Extendable user interface
Plugin system (OSGi)
Beckers et al. (UDE) ezDL August 16th 2012 8 / 35
Introduction and Concepts
Major concepts as evaluation tool
Support for eye-tracking with dynamic Areas of Interest (AOIs)
Automated logging of user and system actions
Beckers et al. (UDE) ezDL August 16th 2012 9 / 35
Architecture
1 Introduction and Concepts
2 Architecture
3 Evaluations
4 Use Cases
5 Conclusion
Beckers et al. (UDE) ezDL August 16th 2012 10 / 35
Architecture
Implementation
Licensed under GPLv3 (other licenses also possible)
Fully implemented with Java 1.6 (1.7 is planned) no legacycode
Unit tests that help maintain stability
JavaDoc documentation available
Beckers et al. (UDE) ezDL August 16th 2012 11 / 35
Architecture
Architecture
Backend
Service-oriented architecture (Agents)Message-driven communication (via JMS-based infrastructure)
Frontend
Desktop (Swing)Web (Google Web Toolkit)Mobile devices (Android, iOS), under development
Beckers et al. (UDE) ezDL August 16th 2012 12 / 35
Architecture
Architecture
Backend
MTA 1
MTA 2
MTA n
ACMWrapper
IEEEWrapper
DBLPWrapper
WileyWrapper
DirectoryQueryHistoryAgent
RepositoryAgent
SearchAgent
Communication bus (e.g. CORBA or JMS)
Client 1
Client 2
Client 3
Client m
Client 4
Beckers et al. (UDE) ezDL August 16th 2012 13 / 35
Architecture
Wrapper for data sources (e.g. DLs)
Wrapper toolkit for HTML-based sources
currently used for ACM DL, SpringerLink, PubMed . . .
Connectors for common IR systems
Apache Lucene/SolrTerrier, Lemur, MG4J (in development)
Beckers et al. (UDE) ezDL August 16th 2012 14 / 35
Architecture
The desktop frontend architecture
Beckers et al. (UDE) ezDL August 16th 2012 15 / 35
Architecture
Extending ezDL
New data sources
Patent databases: Google Patents, ESPACENET, CLEF-IPMedical databases. . .
New features
tool (desktop frontend)backend agent (if required)
Beckers et al. (UDE) ezDL August 16th 2012 16 / 35
Architecture
Extending ezDL
New visualisationse.g. Hypergrid visualisation of search results
Beckers et al. (UDE) ezDL August 16th 2012 17 / 35
Evaluations
1 Introduction and Concepts
2 Architecture
3 Evaluations
4 Use Cases
5 Conclusion
Beckers et al. (UDE) ezDL August 16th 2012 18 / 35
Evaluations
Built-in support for user evaluations
Logging user actions
Example
User searches for information retrieval in 2 connected DLs(DBLP, ACM)
event:
name: "search"
clientTimestamp: 1/4/2012 15:26:32,1234
timestamp: 1/4/2012 15:26:32,3456
sequenceNumber: 10
parameters:
query: "information retrieval"
sources: dblp, acm
Beckers et al. (UDE) ezDL August 16th 2012 19 / 35
Evaluations
Built-in support for user evaluations
Tracking AOIs
Beckers et al. (UDE) ezDL August 16th 2012 20 / 35
Evaluations
Built-in support for user evaluations
Beckers et al. (UDE) ezDL August 16th 2012 21 / 35
Evaluations
Built-in support for user evaluations
Fixed layout on screen
Loading predefined perspectives
Splash screen for choosing evaluation settings
Beckers et al. (UDE) ezDL August 16th 2012 22 / 35
Evaluations
Evaluation infrastructure
frontend
Questionnaire tool
backend
Evaluationweb app.
Experiment database
IR engine
Eye tracking tool
logging data
eye tracking data
questionnaire answers
Beckers et al. (UDE) ezDL August 16th 2012 23 / 35
Evaluations
Evaluation infrastructure (cont’d)
Start
Training Task
Task
Pre-ExpQuestionnaire
End
Pre-TaskQuestionnaire
Post-TaskQuestionnaire
Post-ExpQuestionnaire
Log Viewer
Database
System
Evaluation web app.
Questionnaire & Session Data
Starts System
Log Data
Session Data
Analysis Tool(Example)
Collected Data
Beckers et al. (UDE) ezDL August 16th 2012 24 / 35
Use Cases
1 Introduction and Concepts
2 Architecture
3 Evaluations
4 Use Cases
5 Conclusion
Beckers et al. (UDE) ezDL August 16th 2012 25 / 35
Use Cases
Khresmoi project
Multilingual + multimodal search in the medical domain
Use cases
General publicClinicians and general practitionersRadiologists
Beckers et al. (UDE) ezDL August 16th 2012 26 / 35
Use Cases
Web-based frontend for general public
Beckers et al. (UDE) ezDL August 16th 2012 27 / 35
Use Cases
Desktop frontend for general practitioners
Beckers et al. (UDE) ezDL August 16th 2012 28 / 35
Use Cases
CAIR project
User-oriented clustering of search results
Clustering was added to ezDL (clustering agent and adaptedsearch tool)
Collections:
Amazon Book databaseWikipediaWeb search
Beckers et al. (UDE) ezDL August 16th 2012 29 / 35
Use Cases
CAIR Web Search (Integrated Browser)
Beckers et al. (UDE) ezDL August 16th 2012 30 / 35
Use Cases
iPerFedPat: Federated Search for Patents(Mike Salampasis, U. Thessaloniki, Greece)
Beckers et al. (UDE) ezDL August 16th 2012 31 / 35
Use Cases
iPerFedPat New Search ToolsResult Clustering, Entity Mining
Beckers et al. (UDE) ezDL August 16th 2012 32 / 35
Conclusion
1 Introduction and Concepts
2 Architecture
3 Evaluations
4 Use Cases
5 Conclusion
Beckers et al. (UDE) ezDL August 16th 2012 33 / 35
Conclusion
Conclusion
ezDL is a
interactive search tool,development platform andsystem for user-oriented evaluations
Efficient and effective IR systems need an appropriate userinterface
Framework to avoid “reinventing the wheel” when developingsearch UIs
Participation by other persons/groups is welcome
Beckers et al. (UDE) ezDL August 16th 2012 34 / 35
Thank you for your attention.
Beckers et al. (UDE) ezDL August 16th 2012 35 / 35
