An Introduction to Evaluation in Medical

VisualizationNoeska Smit1 and Kai Lawonn2

1TU Delft, The Netherlands 2University of Koblenz · Landau, Germany

2

3

4https://commons.wikimedia.org/w/index.php?curid=10570695

5

And he liked it very much…

6

Evaluation• Munzner’s nested model for visualization design and validation

Munzner: "A nested model for visualization design and validation."Visualization and Computer Graphics, IEEE Transactions on 15.6 (2009): 921-928.

7

Evaluation• Isenberg et al.: 10 years of VIS paper evaluations

• Trend: more evaluations involving participants

Isenberg et al.: "A systematic review on the practice of evaluating visualization.“IEEE Transactions on Visualization and Computer Graphics, 19.12 (2013): 2818-2827.

8

Evaluation Types• Performance evaluation

• Case study

• User study

9

Performance Evaluation• Measurable quantities:

• Timings: rendering, preprocessing, framerates

• Compared to similar methods

• On the same data (OSIRIX, AIM@SHAPE)

10

Case Study• Apply method to clinical data

• What is shown/new insights

• Should be done by domain experts, otherwise usage scenarios

11

User Study• Observation

• Task comparison

• Interview

• Evaluation form

12

Which Users?• Domain experts, students in the field

• People in the lab, students

• Depending on task: random people on the street

13

Practical Examples• Medical education application

• Visualization of multimodal PET/CT data

14

The Online Anatomical Human

Smit et al.: “The Online Anatomical Human: Web-based Anatomy Education” Proceedings of Eurographics Education Papers 2016

15

The Online Anatomical Human

https://vimeo.com/156236130

16

MOOC Deployment• Massive Open Online Course (MOOC) on human anatomy via

Coursera: • Over 14000 active learners worldwide• https://www.coursera.org/learn/anatomy

17

Online Evaluation• Large scale user study

• Survey:• Statements with Likert scale responses• Open questions• Linked to Coursera user profiles

User experience

User performance

18

Illustrative Multi-volume Rendering for PET/CT Scans

Lawonn et al.: "Illustrative multi-volume rendering for PET/CT scans.“ Proceedings of the Eurographics Workshop on Visual Computing for Biology and Medicine 2015

19

20

21

22

23

24

Discussion• Standardized evaluation procedures

• Standardized datasets

• Medvis challenges?

25

Conclusion• Evaluation methods for medical visualization:

• Performance• Case study• User study

• Medvis Challenge?

26

Thank you

• http://anatomy.tudelft.nl• Live OAHction

• http://medvis.org• The blog on all things medical visualization