Community Data Evaluation using a Semantically Enhanced Modelling

Process

e-mail: mhh@comp.leeds.ac.uk • , Mohammed Haji 1, Peter Dew 1, Chris Martin 1,2

• 1 School of Computing, University of Leeds

• 2 School of Chemistry, University of Leeds

2

Content

• Community Data Evaluation using a Semantically Enhanced Modelling Process

• Capturing Provenance and Data

• Current practices and the Electronic Lab Notebook

• Evaluation

• Conclusion

3

Community Data Evaluation

• The Motivation

– Study how to transition from today's ad-hoc process practises

– Sustainable process of

• Gathering, community evaluation and sharing data & models between scientists

• Minimising changes to proven working practises of the scientist

• Operate within world-wide co-laboratories

• Progress in many scientific communities depends on complementary

experimental and theoretical development.

• These communities require high quality data to evaluate findings.

- Our primary community is the Atmospheric Community .

4

Capturing Provenance Data

• Provenance is captured in three forms namely Inline (during the experiment execution), pre-hoc and post-hoc, before and after the experiment.

• Broadly speaking there are two categories for capturing provenance data in e-Science projects:

• System oriented: There are usually tightly coupled with the workflow paradigm and seek to automatically capture provenance.

• User oriented: Adopting key practises from the scientific approach and use domain specific scientific terminologies.

• In this research we seek to develop a user oriented approach and reconcile with the system orientation to automate process provenance capture. Specifically capturing inline annotation.

5

Current Evaluation Processes for the MCM

6

Envisioned Evaluation Processes

LaboratoryArchiveCommunity Semantic Database

Inputs to the modelling process:Benchmark data

Model parameter sets etc.

Scientist’s Personal ELN Archive

Workgroup database

ELN Capture of the Model Development Provenance

Model Development

Model ExecutionAnalysis

Links to experimental dataand provenance generation

processes

Data sources

Community EvaluationSubjective

SeMEEP

Semantic-enabled

ELN

7

Modelling Plan

Ontology

Compare to generate metadata

Mechanism Editing Model Execution Model Output Analysis

Mechanism version n

Mechanism version n-1

Scientific Process

Automatic Metadata Capture

Planning the

Scientific Process

User Annotation

Metadata Storeage

Metadata Storeage

Capture Metadata at run time

ELN Process

8

• Prompts displayed when changing the chemical mechanism;

• Editing a reaction

• Adding a new reaction

ELN Screenshots

9

Evaluation Methodology

• In-depth interviews with members of the atmospheric chemistry model group at Leeds, covering:

– Demonstration of the prototype

– User testing of the prototype

– Discussion of scenarios involving the use of the prototype.

• Analysis

– Interviews recorded and transcribed

– Analysed using techniques from grounded theory

10

Evaluation

• Barriers to adoption:

– Effort required at modelling time for provenance capture

• “[in] your lab book you can write down what ever you want [but with an ELN] it is going to take time to go through the different protocol steps”.

– When asked if they would use an ELN requiring a similar amount of user input to the prototype the response was positive:

• “Yeah, I think it would be a good thing. I don’t think it is too much extra … work.”

– Rather than viewing the prompts for user annotation as interruption to their normal work the user recognised the value of being prompted

• “is a good way to do it because otherwise you won’t [record the provenance].”

11

Conclusion

• Outlined the Community Data Evaluation using a Semantically Enhanced Modelling Process and the ELN.

• The work is focused on a user-oriented approach using domain specific scientific terminologies.

• Showed the community evaluation vision.

• Discussed the ELN evaluation method.

• Future work

– Carry out further investigation into the atmospheric chemistry community.

– Look into other community that would benefit from this work such as Geomagnetism.

Acknowledgement

- Peter Jimack, David Allen and Mike Pilling