1

The Medical Exploration Toolkit – an efficientsupport for visual computing in surgical planning

and trainingKonrad Muhler Student Member, IEEE, Christian Tietjen, Felix Ritter, and Bernhard Preim

Abstract—Application development is often guided by the usage of software libraries and toolkits. For medical applications, the toolkitscurrently available focus on image analysis and volume rendering. Advanced interactive visualizations and user interface issues are notadequately supported. Hence, we present a toolkit for application development in the field of medical intervention planning, training andpresentation – the MEDICALEXPLORATIONTOOLKIT (METK). The METK is based on the rapid prototyping platform MeVisLab and offersa large variety of facilities for an easy and efficient application development process. We present dedicated techniques for advancedmedical visualizations, exploration, standardized documentation and interface widgets for common tasks. These include, e.g., advancedanimation facilities, viewpoint selection, several illustrative rendering techniques, and new techniques for object selection in 3D surfacemodels. No extended programming skills are needed for application building, since a graphical programming approach can be used.The toolkit is freely available and well documented to facilitate the use and extension of the toolkit.

Index Terms—Visualization Applications, Life and Medical Sciences, Visualization Techniques and Methodologies, Software Engineer-ing, Medical Visualization, Software Library, Graphical Programming.

✦

1 INTRODUCTION

Software assistants for intervention planning, e.g., forsurgery, interventional radiology or radiation treatmentplanning are a relatively recent development. Surgicalapplications have special demands on visualization andinteraction. It is not sufficient to display and analyze slicedata and to create volume-rendered images. Instead, anin-depth analysis of the image data needs to be sup-ported with appropriate 3D interaction techniques andadvanced visualization techniques. With the MEDICAL-EXPLORATIONTOOLKIT (METK) we present a widely ap-plicable library for application development that closesthe gap between image analysis, processing, and basicvisualizations on the one hand, and the surgical needsconcerning visualization and interaction on the otherhand. The METK is based on the image processing anddevelopment environment MeVisLab [21], [27].

Visual computing in surgical applications has to pro-vide comprehensive patient-related information, includ-ing visualizations of the relevant anatomical and patho-logical structures, and enabling a faithful representationof the area around the pathologies. Moreover, measure-ments, annotations, resection lines and other informationmay be important to directly support pre-interventionaldecisions. On the one hand, flexible visualization andinteraction is needed to cope with the peculiarities of

• K. Muhler, C. Tietjen and B. Preim are with the Department of Simulationand Graphics, University of Magdeburg, 39106 Magdeburg, Germany.E-mail: muehler@isg.cs.uni-magdeburg.de

• F. Ritter is with MeVis Research – Center for Medical Image Computing,Bremen, Germany.

individual cases, but on the other hand, strong guidanceis desirable to avoid that surgeons are overwhelmed bythese facilities. Most of this information, e.g., measure-ments of the structure’s extent, can only be derived afterthe segmentation of relevant structures. Further analysessuch as infiltration of and distances between structuresas well as safety margins and volumes of structures canonly be performed after segmentation.

Visualizations based on pre-segmented structures aremandatory for operation planning in many fields, dueto a high density of soft tissue structures with over-lapping image intensity values. Thus, operation plan-ning in the abdominal region (e.g., liver, pancreas orkidney), the neck region, and orthopedic interventionsis preferably performed using segmented surfaces incombination with the original 2D slices, while in neurosurgery or in emergency cases volume rendering of theoriginal image data is preferred. The METK develop-ment focuses on segmentation-based visualizations butdoes not support the segmentation process itself, sincethis process is supported in the underlying MeVisLab.Furthermore, several applications (e.g., HepaVision forliver surgery [5] or NeckVision for neck surgery [8]) andservice providers [20] are available to perform this task.The METK can import DICOM segmentation masks aswell as polygonal meshes of structures (e.g., in OpenInventor or STL format).

Open source and freely available toolkits arewidespread in the research community. Using toolkits,application prototypes can be built up quickly, revertingto ready-to-use basic functions. In the medical domain,toolkits and libraries for image analysis and volume

2

rendering are widely available [7], e.g., the MITK [39],3DSlicer [24] or VolumeShop [6]. However, they aredifficult to extend for professionals without a substantialbackground in computer science, since substantial C++knowledge is required. The METK supports an easierapplication building process for surgical applicationswhere no extended programming skills are needed.Instead graphical programming in combination withscript-based interface design is employed. The METK isa turn-key environment because all described functionsare fully implemented and basic applications in form ofexample networks and datasets are provided with theMETK. The METK extends the underlying MeVisLabdevelopment environment in the following ways:

1) The METK provides advanced visualization tech-niques that fit especially the requirements of sur-gical planning, like safety margins, vessel visuali-zation or extensions for 2D viewers.

2) Animation facilities are provided that enable theusage of animations as pre-rendered videos or as asupport for interactive exploration.

3) The METK provides a case management that en-ables applications to load and save collections ofmany segmented structures, including the imagedata and further information about the patient.

4) The METK includes automatic synchronizationbetween all modules of an application. Thus,changes in one part of an application may directlyaffect other parts.

5) The METK provides special facilities to save dif-ferent states of a visualization for later reuse.

6) To support fast application development, manyuser interface widgets are available in the METK,e.g., structure lists and interface templates.

7) To use the METK and to build up own ready-to-use applications, no extended programming skillsare needed. Almost all tasks can be solved usingscript languages like python.

8) The open source and well documented data inter-face enables developers to easily extend the METKby new modules.

All these techniques and facilities are not available inMeVisLab. In comparison to other toolkits, only a veryfew of these techniques are available in different systems,e.g., illustrative techniques in VolumeShop [6].

Outline

In Section 2 we present conceptual considerations aswell as different surgical application scenarios and de-rive requirements concerning visualization, interaction,case management, and interface design. In Section 3 wereview related toolkits and discuss the differences to theMETK. In Section 4 we give an overview of the keyconcepts and present the techniques that we integratedin the METK. Furthermore, details with respect to newtechniques for 2D overlay visualizations, to advancedfacilities for selecting objects, to reuse once defined

visualization parameters are presented. In Section 5 wedescribe how application building is achieved using theMETK and present some applications where the METKhas been successfully applied. In Section 6 we presentand compare the evaluation of implementing a referenceapplication in MeVisLab with and without the METK. InSection 7 and 8 we close with a discussion of the toolkitand the lessons learned, a summary and an outlook onfuture developments.

2 REQUIREMENT ANALYSIS FOR SURGICALPLANNING AND TRAINING

The design of the METK is guided by experiences inseveral fields of surgical application development. Thus,we present conceptual considerations, selected appli-cation scenarios and derive requirements for surgicalapplications.

2.1 Conceptual Considerations

A challenging problem, if not the most challengingproblem in computer aided surgery, is a precise un-derstanding of requirements and needs, priorities andrelations between them. It turns out that fast prototypingand discussions of initial suggestions are essential forthis process. Surgical departments often are too small toinclude people with substantial IT background. Further-more, specific features of new applications are hard todescribe verbally. Medical doctors are often not awareof the available options. If application prototypes areavailable in an early stage, the discussions with the usersamount to substantial input for the development process.This input is crucial in application development, sincethe wishes, aims, and experiences are rarely apparentafter a first specification of the application. Therefore,we follow a process where we offer different solutionsto the medical doctors and ask them to comment onthese solutions. To come up very quickly with suchprototypes was one of our central requirements duringthe development of the METK.

A second major goal was to enable that once devel-oped visualization and interaction techniques are easilyreused in a larger department with many projects. Pro-viding a broad common base of techniques, in particularmanagement of image data and related information toa particular case, enables researchers to focus on newaspects and current applications. This involves a largeinvestment in software design but pays off later. Fromthe first project on, the METK significantly enhancedthe reuse of newly developed techniques. Two majordevelopers from different projects guided and drove thisprocess. The application of the METK by a broader usergroup lead to a refinement of the requirements andprovided input for technical challenges.

For a toolkit for application development it is crucialto provide a modular character, where every applicationcan just use the modules it needs. Extendibility is a

3

second major aspect, since the toolkit may be used for alarge variety of applications with specific requirements.To guarantee the extendibility, it is important to designan open and extendable structure for communicationand data management, where new modules are able toinstantly communicate with all other existing modules.

Our concerns lead to the following requirements forthe METK development:

• Ready-to-use applications should be created quickly.Thereby, essential feedback of end users can beobtained in an early development stage.

• The application building process should not requireextended programming, like implementing a largenumber of classes for application logic in C++.

• Existing applications as well as new applicationsshould be able to be extended by new techniques,e.g., for special surgical requirements, obtained fromfeedback or techniques from newer visualizationdevelopments.

• The developers should be able to handle the largeoverhead of loading and managing medical imagedata.

In the following, we describe selected applications sce-narios which lead to refined and more specific require-ments of surgical applications.

2.2 Application Scenarios

Scenario 1: Neck dissections are carried out for patientswith malignant tumors in the neck or head regionto remove lymph node metastases. Depending on thebroadening of enlarged lymph nodes, only a few of themor a large part of the neck including muscles must beresected. A large number of structures have to be takeninto account (e.g., vessels, muscles and up to 60 lymphnodes). The surgeons must explore the distances of alllarger lymph nodes to vital structures of risk in orderto judge if there is enough space to safely resect themindividually [17].

Scenario 2: In abdominal surgery, the resection oftumors in the liver, kidney or pancreas is rather sim-ilar with respect to the demands of software support.Here, a tumor or several metastases need to be resectedwith a specific safety margin. In difficult cases (e.g.,a tumor in a central or dorsal position, where frontalaccess is severely limited), this intervention requires in-depth computer-based operation planning. The tumorand especially the surrounding vessels must be care-fully inspected in 2D as well as in 3D. The remainingliver volume must be calculated with respect to vesselsupply and drainage. The results need to be adequatelyvisualized in 3D as well as in 2D [25].

Scenario 3: In spine surgery, small changes of thespine’s anatomy can evoke symptomatic disorders forthe patient. Hence, the spine surgeon must inspect thespatial relations between nerval and spinal structuresas well as the relation of the spine to surroundingmuscles, vessels and glands. The surgeons need to place

virtual needles and implants in the spine region toplan different strategies for the access route in the laterintervention. Dedicated 3D visualizations can help theplanning surgeons to locate such access routes withoutinjuring important structures [9].

2.3 Requirements for Surgical Applications

For all scenarios, the exploration of the dataset must beas fast as possible in the clinical routine. Presentationsare essential for collaborative intervention planning suchas tumor board discussions, where a complex case ispresented by one medical doctor to initiate an interdisci-plinary discussion to finally come to treatment decisions,e.g. a combination of radiation treatment planning andchemotherapy or a chemotherapy to downsize a tumorpreoperatively.

In general, surgeons are medical experts, usually withonly modest computer experience. They benefit fromfaithful spatial renditions of the patient’s individualanatomy, but they usually have no special abilities toexplore and handle 3D data. From our experience withsurgical applications, we derive basic requirements forsuch applications:

1) Surgical applications must primarily support thesurgeon’s decision-making process.

2) Measurement capabilities must be provided to sup-port, e.g., distance, volume, and angle measure-ments, since these measurements are often closelyrelated to surgical decisions.

3) Due to the importance of 2D slice data, 2D and3D views of data should be coherent and synchro-nized, while the exploration of 3D data must besupported in particular.

4) Important anatomic structures need to be empha-sized, preserving the context.

5) Dedicated techniques for special surgical fieldsshould be provided (e.g., resection techniques forabdominal surgery, DTI visualization for neurosurgery, and multimodal data visualization for car-diac surgery), while many techniques are usable forseveral fields.

6) In many areas, such as Web applications and learn-ing environments, it is essential that systems arenot only easy to use, but also are perceived as mo-tivating and appealing which is often summarizedas providing a convenient user experience. Surgeonsnowadays use highly appealing interfaces designedby educated visual designers, thus expecting anappealing user interface [10], even from researchprototypes.

Case Management. One further requirement is an ef-ficient case management. We define a case as a singledataset (e.g., an MRI scan), or a collection of datasets,relating to one patient, with additional data such assegmentation information, information about the typeof structures, and information about the patient. Thisinformation is necessary to provide sophisticated and

4

adapted visualizations. For every application, differentstandard and default visualization styles may be avail-able. These styles must be identified in many interviewswith potential users and medical experts [17].

Nearly all surgical applications need an efficient casemanagement, advanced 3D visualizations of segmentedstructures and guidance for their exploration. Measure-ment facilities are less important for patient’s consul-tation, while resection techniques are primarily neces-sary for abdominal surgery planning. Thus, a modularapproach is necessary where only required features areintegrated in an application.

3 RELATED WORK

Since medical visualization and medical image analysisare fields of active research, several toolkits are available.We will discuss them with respect to the presentedrequirements. Besides related toolkits there is relatedwork for the individual visualization and explorationtechniques we integrated in the METK, e.g. the LiveSync-System [38] as an extended viewpoint selection tech-nique or the work of Viola et al. [35] that discussesaspects of good camera paths for medical volume visu-alizations. We refer the reader to the cited papers foreach technique for a deeper insight – also with respectto related work in these specific methods.

3.1 Visualization Toolkits for Medical Purposes

For the fast generation of visualizations, Bavoil et al. [2]presented VisTrails. VisTrails is a publicly availablepipeline-based environment, where visualizations formany fields of use, e.g. time-varying or diffusion tensordata, can be created. The system also provides the abilityto reuse several parts of previous visualization pipelinesusing visualization by analogy [29]. VisTrails focuses onthe generation and exploration of datasets and not onapplication building.

VolumeShop [6] is a standalone prototype for theinteractive direct volume illustration of single datasets.Providing impressive facilities to create illustrations, itis very useful for presentation purposes where an artistcarefully prepares aesthetic visualizations. VolumeShopalso enables the testing and development of new visu-alization techniques, because it is an open source sys-tem. However, extensive programming skills are needed.Another system that also aims at the support of visu-alization experts and artists was presented by Roßleret al. [28]. They provide a layer on top of the shaderprogramming level to create impressive visualizationsof multimodal datasets: a render graph that can becreated with different sets of components like lightning,transformations, slicing or cutting. These graphs areautomatically transformed into shader scripts to createvisualizations. However, Bruckner et al. [6] and Roßlleret al. [28] do not support the building of applicationsbased on the presented visualization and interactiontechniques.

The 3DSlicer [24] is an open-source environment usedin a variety of research applications, mainly in neuroimaging. As the name implies, the focus of the 3DSlicerlies on the 2D slicing of volume data. It only providessome basic 3D visualization techniques.

The Medical Imaging Interaction Toolkit (MITK) [39]is a C++ framework that is built up on ITK and VTK.It focuses on image analysis algorithms and interactionsupport for the segmentation and registration of medicalimages. The developers themselves state that MITK isnot intended to be ‘an application framework’.

Julius [16] is a software framework that consists of acore application which can be extended by plug-ins. Itfocuses on the medical image analysis and deformablemodels and provides algorithms for segmentation, regis-tration and intra-operative navigation. For visualizationsVTK is used, while no special visualizations for surgicalapplications were mentioned.

A C++ toolkit specializing in intra-operative support isthe Image-Guided Surgery Toolkit (IGSTK) [11]. It sup-ports the development of applications for interventionalradiology procedures and image-guided surgery, whereexternal tracking devices are applied.

A closed framework for the visualization of mul-timodal medical images is presented by Manssouret al. [19]. They focus on segmentation and registration,but do not demonstrate how individual applications canbe created with their framework.

Another framework for medical image analysis isCAVASS [12]. It provides simple volume rendering andsurface visualizations, but no dedicated support for ap-plication development, since CAVASS is a fixed systemthat is only extendable by its open source interface.

Botha et al. [4] recently introduced a developmentenvironment for fast visualization prototyping, the DE-VIDE system, which provides substantial capabilities inaccessing and changing code and underlying dataflownetworks during runtime. DEVIDE supports the de-velopment of new visualization and segmentation tech-niques well.

SciRun [37] is a problem solving environment that isbased on dataflow networks consisting of single mod-ules, e.g., for scalar and vector visualization, simula-tion, and image processing. Also end user applicationscan be created, called PowerApps. This generation ofPowerApps is similar to the module definition languageof MeVisLab that is used to create consistent interfacesabove the panels of single modules.

Amira [31] is a commercially available object-orientedextensible toolkit for scientific visualization. It providesa wide range of analysis, simulation and visualizationtechniques. Amira offers a visual programming ap-proach where the dataflow metaphor is used to connectmodules to a network. Even though there are scriptingand basic GUI facilities available, there is a lack ofsupport for application development. Using Amira, noend user applications can be created.

5

3.2 The MeVisLab Environment

MeVisLab [27] is a freely as well as commercially avail-able visual programming and rapid prototyping plat-form for image processing research and developmentwith a focus on medical imaging and visualization (foran example, see Figure 1). For the commercial version,comprehensive support is provided by MeVis Medi-cal Solutions. The free version is restricted to privateor academic research purposes. Complete applicationsincluding user interfaces can be built within a com-mon, cross-platform framework. Beside general imageprocessing and visualization tools, MeVisLab includesadvanced medical imaging algorithms for segmenta-tion, registration, and quantitative morphological andfunctional analysis. New image processing algorithmsand visualization tools can be integrated as modulesusing a standardized software interface. Macro modulesthat allow for a hierarchical encapsulation of networksfacilitate the reuse of available developments. Efficientdesigns of graphical user interfaces can be achievedusing an abstract, hierarchical module definition lan-guage (MDL) that hides the complexity of the under-lying module network to the end user. To add dynamicfunctionality, Python or JavaScript code may control bothnetwork and user interface level elements.

An integral part of MeVisLab is the object-orientedMeVis Image Processing Library (ML) that provides ageneral framework for image processing. Each algorithmis represented as a self-descriptive module inside thedevelopment environment. Image processing is accom-plished in a strictly request-driven manner using paging,caching, and multi threading strategies. In addition, theopen source Insight Toolkit (ITK) for performing regis-tration and segmentation has been wrapped into nativeMeVisLab modules. The MeVis Giga Voxel Renderer(GVR) presents a state-of-the-art multi-volume rendererthat combines a texture-based multi-resolution approachwith advanced per object shading techniques [18]. Inter-active responsiveness is guaranteed during interactiverendering by time slot management. For visualizationand interactive graphics programming, the Open Inven-tor 3D visualization library as well as the VisualizationToolKit (VTK) are fully integrated into MeVisLab. Eventhe combination of Open Inventor and VTK modules ispossible. Based on Open Inventor, additional functiona-lity has been added, such as customizable 2D and 3Dviewer frameworks, annotations, advanced MPR tech-niques, and support for the OpenGL Shading Language(GLSL).

3.3 Discussion of Toolkits

MeVisLab has many similarities to Amira and SciRunand a broad overlapping of functionality (see Bitteret al. [3] for a comparison of MeVisLab, Amira, SciRunand MITK), since all offer visual programming andAmira and MeVisLab use Open Inventor. However,MeVisLab focuses on medical image data, quantitative

Fig. 1. Screenshot of MeVisLab with a network of mod-ules and several panels opened for parameter input andvisualization.

image analysis and provides facilities for applicationdevelopment, like a higher definition language to designuser interfaces. However, there is no support for capa-bilities to handle whole cases as described in Section 2.Even if all techniques of the METK can be developedusing MeVisLab (as the complete METK was), MeVisLabitself does not provide those high-level building blocks.MeVisLab focuses on algorithmic functionality and offersthe ability to construct reusable building blocks. Usingthe METK, a developer of medical applications candesign a running prototype more efficiently.

In essence, there are many toolkits and frameworksfor medical image analysis and visualization. But someof them are focused on the creation of singular im-pressive visualizations (e.g., VolumeShop [6]), some ofthem focus on medical image analysis (e.g., MITK [39],3DSlicer [24]), and only a few support application build-ing (e.g., SciRun [37], MeVisLab [27]). To the best of ourknowledge there is no toolkit or framework to create effi-cient medical applications with high-end visualizations,adequate interaction techniques and user interface guid-ance.

4 THE ARCHITECTURE OF THE METKIn the MEDICALEXPLORATIONTOOLKIT, each functionis encapsulated in a module. Using MeVisLab’s visualprogramming environment, modules can be freely com-bined in a dataflow network to build up applicationswith an individual feature profile. This allows the de-veloper to design applications that support the specificworkflow of different surgical intervention planning pro-cesses in an efficient and fast manner.

All functions are conceptually organized in three lay-ers: the data management and communication layer, thevisualization layer, and the exploration layer (see Figure 2).The lowest layer imports the case data and provides datamanagement functions. The visualization layer comprisesviewer classes and special rendering modules. Basicviewer classes and the volume rendering are reusedfrom MeVisLab. All high-end interaction and exploration

6

Data Management and Communica�on Layer

Smooth Polygonal Surfaces and Vessel Visualiza�ons

Mul� CodedSegmenta�on Masks

Mul�modal Tomographic Images

Segmenta�on Masks of Structures

Basic MeVisLab Func�ons

Volume Rendering

Li� ChartColored 2D Overlays using MCSM

Illustra�ve Visualiza�ons

Visualiza�on Layer

Viewer

Explora�on Layer

Widgets andLayout Templates

Anima�ons

Object Selec�on andFast Object Manipula�on

Viewpoint Selec�on

MeasurementTools

Key states and Undo Facili�es

ColoredPolygonal Surfaces

Fig. 2. Layer structure of the METK. A large variety ofdifferent visualization and interaction techniques as wellas case management facilities and user interface widgetscan be freely combined to build up individual surgicalapplications.

techniques that are necessary to create powerful surgicalapplications, are available within the exploration layer.The layers and their provided functions will be describedin the next sections.

4.1 Data Management and Communication Layer

The data management and communication layer containsfunctions for case data management, inter-applicationcommunication and generation of 3D polygonal sur-faces. Since DICOM is widely used and standardized, theMETK is focused on processing DICOM data. Addition-ally, segmentation masks can be loaded to automaticallygenerate 3D polygonal surfaces. This operation mustonly be performed once, since the surfaces are storedfor further loadings of a case. Depending on the typeof structures, different algorithms for generating thepolygonal surfaces are used depending on the metadata, acquired during the segmentation. In most cases,Marching Cubes in combination with a surface smooth-ing is applied. For vascular structures we use a model-based surface reconstruction that respects the thin andbranching structure of vessel trees [30]. Besides segmen-tation masks, also polygonal meshes of structures andsecondary objects (e.g., medical probes) can be imported.

Multimodal data. As the underlying MeVisLab, theMETK has the abilities to load multiple modalities likescans from CT, MRI or PET. The visualization techniquesare able to combine different registered data volumes,e.g. a CT and PET scan. To perform the necessaryregistration between the different datasets, MeVisLabprovides a wide range of capabilities in its free version.Thus, they are no integral part of the METK. However,the METK provides an interface to use the multimodalcapabilities of MeVisLab in the context of the METKtechniques.

Case and cache management. To reduce the memoryconsumption and to speed up the whole process ofloading and exploring a case, we integrated an efficientcase management in the METK. Each structure and eachimage stack is only loaded once and distributed virtuallyin the application network. Even if the structures arevisualized with different techniques in different viewers(see Figure 10), it will only be maintained once in theapplication cache.

Communication. Besides the data management, theMETK provides a communication structure between allmodules. Events can be sent between specific modulesfor a direct inter-module communication and be broad-casted to reach all modules. Therefore, all changes ofunderlying data and parameters are communicated toall modules ’listening’ to those parameters, so they canadjust their own parameters, data, and visualizations.This leads to identical visual properties of all structuresin all viewers and widgets, and thus to a consistent viewof all data.

Synchronization. Moreover, the currently selected ob-ject (CSO) is automatically communicated in the METK.Hence, a synchronized view in different viewers can beprovided. If the user selects a structure in a 3D viewer,all 2D viewers can display the suitable slice for thisstructure and vice versa. If the user picks a structurein a 2D slice, it can be emphasized in all 3D viewers,

7

moving the camera automatically to a good viewpointon this structure (see Section 4.3).

Moreover, different 3D viewers can be automaticallysynchronized in the METK by connecting its camera pa-rameters (position, orientation etc.). This can be used toexplore different datasets from the same viewpoints, tocompare different intervention strategies for one patientor to compare pre- and post-operative data.

Multi-coded segmentations. Usually, e.g. with MeVis-Lab, each segmentation mask, representing one struc-ture, is stored in a single file. This is inefficient withrespect to memory and performance. Storing all seg-mentation masks in one image stack can overcome thisproblem. However, one voxel of an image may belongto more than one anatomic structure, when structuresoverlap each other. For example a voxel in the livertissue may belong to a tumor and the liver tissue itself.Thus, we cannot assign one label to each structure for theresulting segmentation mask of all structures. A straight-forward approach is to assign each structure to one bit ofan 8 byte voxel value. But this approach is limited by thenumber of bits: e.g., only 64 labels could be stored in an8 byte value. Since in real data only a small subset of allpossible combinations of overlapping structures occurs,we developed a more efficient solution and refer to it asmulti-coded segmentation masks (MCSM).

An MCSM contains all segmentation masks of allstructures of a case. Each combination of labels of avoxel that appears in the data is encoded with a distinctvoxel value (see Figure 3). For example all voxels simul-taneously belonging to the liver tissue and the hepaticvein (and to no other structure) are assigned one uniquevoxel value. The mapping of voxel values to structurelists is stored separately in the case data. An MCSM iscreated by sequentially adding one segmentation maskafter another. If a new combination of voxel labels occursat a voxel position, a new number is assigned to thiscombination. After all single segmentation masks wereadded to the MCSM, it can be used e.g., as an efficientbase for colored overlays. The upper bound of 264 labelswill never be reached with medical datasets, since eventhe theoretical case that in a dataset of 5123 voxels eachvoxel represents another combination of structures iscovered by the MCSM.

In Table 1 for 4 cases from the scenarios describedin Section 2 the number of segmented structures andthe number of labels needed in an MCSM are opposed.For the 81 segmented structures of the living liver donortransplantation 11 bits can be used instead of 81 bits.

4.2 Visualization Layer

In the visualization layer, all actions are performed thatare necessary to provide basic and advanced 3D visuali-zation techniques to the user.

Based on the surfaces stored in the cache, a material isassigned to each structure to achieve appealing surfacevisualizations. Important structures are visualized with

Liver

Liver , Tumor

Liver , Hepatic Artery

1

Hepatic Artery

Liver , Hepatic Vein

Liver , Tumor , Hepatic Vein

2

3

4

5

6

Fig. 3. Multi-Coded segmentation masks. Segmenta-tion masks of single structures are sequentially added tothe multi-coded segmentation masks (MCSM). For eachstructure all values of voxels that belong to the structureare stored separately. Thus, many segmentation masksof overlapping structures can be saved in one MCSM. For4 structures, instead of 16 possible combinations, actuallyonly 6 occur.

TABLE 1Number of labels in MCSM for example cases. Larger

numbers of labels for liver cases result from a morefrequent overlapping of structures.

CaseNumber ofsegmentedstructures

Number oflabels in

the MCSM

Neck dissection 1 36 61

Neck dissection 2 21 36

Liver tumorresection 49 361

Living liver donortransplantation

81 656

a high opacity. For structures which serve as anatomiccontext, e.g., organs or large skeletal structures, we pro-vide silhouette rendering. Thus, they are still visible, butdo not hide the view onto other important structures.

For volume rendering the METK employs the MeVis-Lab GigaVoxelRenderer [18]. It enables the tagged vol-ume rendering of segmented structures. Thus, differentstructures can be visualized with local transfer functions.For the sake of consistency, the colors of structures arethe same as for their surface visualization. To visualizeunsegmented tissue, a global transfer function can beapplied and the volume rendering can be combined withthe polygonal surface visualizations.

Advanced 2D visualizations. The basic problem of theslice-based visualization, namely the lack of an overviewin cross-sectional images, has been tackled with a 2.5Dapproach to provide the essential information, the so-called LIFTCHART [32]. Using this technique, the range

8

of slices that a specific structure spans over can bequickly seen. A narrow frame attached next to the cross-sectional image represents the overall extent of slicesin the volume dataset. The top and bottom boundaryof the frame correspond to the top and bottom slice ofthe dataset (see Figure 4). Each segmented structure isdisplayed as a bar at the equivalent vertical positioninside this frame. Upper bars correspond to higher struc-tures in the body. Different arrangements of the bars arepossible, e.g., condensing all structures of the same typeinto one column. The currently displayed slice of thevolume dataset is depicted by a horizontal line in theLIFTCHART widget (see Figure 4).

To support the correlation between structures in 3Dscenes and 2D slices, structures can be visualized in2D slice data as colored and semi-transparent overlays,so the underlying gray values are still visible. If morethan one structure should be displayed at the samevoxel position, the combined color can be calculated indifferent ways:

1) Only the color of the most important structure ischosen.1

2) A weighted mixture of all colors of the overlappingstructures is calculated.

3) Application-dependent overlapping regions can beemphasized separately in dependency of involvedstructures, e.g., the infiltration of lymph nodes in amuscle can be colored red with a silhouette, evenif this visualization style does not appear in one ofthe two structures.

The calculation of the overlays is performed based onour multi-coded segmentation masks.

Safety margins around tumors and metastases areessential for intervention planning and intraoperativenavigation. Therefore, for all structures at risk, a 3DEuclidean distance transform is performed. Depictingimportant distance thresholds (e.g., yellow and red rep-resenting 5 and 10 mm) turned out to be appropriate.The distances may be displayed in 2D as well as in 3D.In the 2D view, silhouette lines visualize the importantdistances (see Figure 4(a)). In 3D, unicolored surfaces aredrawn on structures visualizing the range of distanceto critical structures, e.g., lymph nodes to vessels andmuscles (see Figure 4(b)).

Illustrative visualizations. It became apparent thatapplying transparency is not sufficient to visualize com-plex structures. In particular if the density of anatomicstructures is high, illustrative techniques are employedto better convey object shapes and relations. For thatreason, illustrative visualization techniques were devel-oped. Illustrative visualization was found to be usefulfor selected therapy planning tasks, e.g., hatching linesconveys surface shape better compared to conventional

1. The importance is acquired from the meta data in the casedescription or, if this data does not exist, indirectly from the size ofthe structure. In surgical planning mostly small structures are moreimportant than large ones.

(a)

(b)

Fig. 4. LIFTCHART. 4(a) LIFTCHART in a 2D viewer andthe corresponding dataset in 3D 4(b). The location ofdifferent structures in the slice stack can be identified bytheir color. Selecting a structure in the LIFTCHART selectsthe corresponding slice in the viewer. Furthermore, safetymargins are depicted with red and yellow.

shading for radiation treatment planning [14]. The appli-cation of silhouettes to strongly transparent structures in-creases the recognizability. The use of local transparencywas also promising (e.g., cut aways or ghosting [36]).These illustrative techniques are provided in the METKand can be flexibly combined with surface and volumerendering. Silhouette rendering is the default style inthe NECKSURGERYPLANNER [33] which is used in theclinical routine and appreciated by medical doctors forproviding adequate support. However, so far there is noevidence for an advantage of stippling and hatching insurgical planning.

Medical viewers. The visualization layer also providesseveral viewers that consist of wrapped and extendedMeVisLab viewers. The extended viewers are able tocommunicate their parameters (e.g. camera position andorientation) to other METK modules and can receivecommands, e.g. to control the viewers remotely by theanimation system. 2D viewers can display slices in manyways: singular or in a multi-slice view, where axial,

9

sagittal, and coronal as well as free multi planar refor-mations can be shown. 3D and 2D viewers can be freelycombined in an arbitrary number and arrangements.

4.3 Exploration Layer

To support the exploration process, we provide severaltechniques, interaction facilities and interface widgets.

Animation. To guide the user as well as to providesmooth transitions between different viewpoints andvisualization styles, we integrated the animation frame-work described by Muhler et al. [22]. Using an adaptivescript language, one script with an animation descrip-tion can be reused for many similar cases, providedthat segmentation results are stored and named in astandardized way. For example, all cases matching onescenario described in Section 2 are similar. The goalsof our animation facilities are similar to Iserhardt-Baueret al. [15] who developed standardized video generationfor a specific problem, namely the diagnosis of cerebralvasculature. The script-based animations can also beused in an interactive application to guide the user’s ex-ploration process. Selected structures can be approacheddue to automatic camera flights, and appearance changescan be smoothly animated to preserve the user’s orienta-tion. The flexibility of the animation facilities enables theusage of advanced animation techniques such as thosebased on story telling principles [38]. Thus, applicationsbased on the METK can provide both interactive anima-tions in real time for exploration support, and renderedvideos for presentation and interdisciplinary discussions,e.g., in a tumor board.

Viewpoint selection. Finding good views on singlestructures or groups of structures is essential for anautomatically guided exploration. In contrast to previ-ous approaches, e.g. the viewpoint entropy of Vazquezet al. [34], the viewpoint selection technique of Muhleret al. [23] incorporates anatomic knowledge. Animationsin the METK are enhanced by this dedicated viewpointselection technique for multi-object 3D scenes that hasbeen applied in several intervention planning tasks. Af-ter selecting a structure, the camera position is automati-cally transformed to a good view on this structure. Thequality of a viewpoint is affected by many parameters.The structure should be visible to a maximum extent.A good viewpoint should be stable, i.e., minor rotationsmust not completely hide the selected structure. Medicaldoctors may have different preferred regions to look ata 3D scene of segmented structures. Thus, the preferredregion is also an important parameter for viewpointestimation. These and other parameters are consideredby our viewpoint selection technique. As discussed byMuhler et al. [23], relatively good presets for certainapplication scenarios may be defined.

Good viewpoints are employed in the METK to gener-ate standardized views for documentation (in combina-tion with other standardized visualization parameters).If the user picks a structure from a list or from the

viewer, the camera can be automatically moved to thebest viewpoint of the structure.

Camera paths. To produce appealing movements ofthe camera from one viewpoint to another, we developedand integrated a set of path algorithms in the METK. Topreserve the orientation on long distance, movementsof the camera, we first zoom out to a global view onthe scene and zoom into the target structure at the endof the flight. We also make camera movements moreappealing by slow acceleration at the beginning and theend instead of abrupt speed changes.

Measurement tools [26] are also integrated, e.g., fordistance measurements and its appropriate visualizationby means of arrows. The proposed measurement toolsare extended by automatic measurement facilities forcomputing minimal distances between two structuresand by calculating a structure’s volume.

Key states. For presentation purposes, interdisci-plinary discussions or patient consultation, several viewsand visualizations of the explored data need to besaved. Instead of only saving screenshots, we employkey states, which store all information about a sceneand its visualization. This includes camera parametersas well as visualization properties. Thus, a completestate of a visualization can later be restored for furtherexplorations or demonstrations. Since key states arestored in the case data, they can be transfered fromone application (e.g., a surgical planning software) toanother (e.g., an application for patient consultation).Naturally, key states can also be exported as screenshotsfor usage in documents or presentations. Usually, asurgeon creates a couple of key states during a planningprocess (see Figure 5). In combination with animationfacilities, videos can be created automatically from a setof key states, where smooth transitions between the keystates are computed. These videos, for example, are usedto teach other surgeons. Key states can also be used todefine presets. Applying a once defined key state to anew case with similar structures, these structures arevisualized with the same properties.

Undo. One important feature, especially for surgeonswho are inexperienced in 3D exploration, is an undofunction for 3D scene manipulations. In the METK, afterevery performed action (e.g., a camera movement or avisualization change) the whole scene is stored in a keystate. Changes performed in a very narrow time range(e.g., automatic changes of the visualization) are com-bined in one key state. The user can return to arbitrarysteps.

Automatic object selection. We provide several newtechniques to select objects in 3D scenes with manyobjects of different transparencies. In such scenes, theselection is ambiguous, if there is more than one object inthe picking ray. The simplest approach to disambiguatethe selection is always to select the first object in the pickray. However, if this object has a strong transparency, theuser probably intends to select another object behind.In complex medical scenes, some objects are completely

10

Title:

Overview

Comment:

Liver tissue with portal veins and suspected metastases

Title:

Liver territories

Comment:

Affected territories for two central metastases

Title:

Resection volume

Comment:

Suggested resection volume for one metastasis

Fig. 5. Key states. Several key states that were createdfor planning a liver surgery. The surgeon stored the keystates together with a title and a short comment.

enclosed by others, so they are never the first object inthe pick ray. For example in liver surgery, the liver tissuealways encloses nearly all intrahepatic structures, e.g.,vessels and tumors (see Figure 6(a)).

We developed a procedure to automatically select anobject after the user has clicked on the scene. It isassumed that the user points the mouse consciously.That means, when the mouse cursor is placed over avery large and a very small object, the user placedit deliberately over the small object. Furthermore, itis assumed that the perception of strongly transparentobjects appears less prominent than the perception ofmore opaque objects. To identify the desired object, thealgorithm proceeds as follows (Algorithm 1):

All objects hit by the pick ray are determined andsorted by the depth distance of their intersection point.Only objects that are visible by at least 10% at theintersection point are taken into account. As a nextstep, the size of the projected bounding box on theviewport of all objects and the transparency degree of thesingle objects is determined. The impact of transparencyand the projected bounding box size are adjustable toconsider different types of scenes (e.g., scenes with ob-jects of rather equal size). The object with the highestrating is selected at the end. Thus, for example opaquestructures behind structures with a strong transparencyare selected.

Interaction support of object selection. The algorithmreveals its limitations when an opaque structure lies be-hind a semi-transparent structure, while their projectionsare almost equal (see Figure 6(b)). In such cases, at allpoints where the user picks the transparent structure,the opaque one in the background is selected. Therefore,we provide two interaction techniques in the METK.

Input: All objects {O} hit by the pick rayInput: Impact IT of object’s transparency OT

Input: Impact IS of object’s viewport size OS

Input: Total viewport size VS

Output: Object Oresult with highest ratingSort objects by the depth distance of theirintersection point;Aray ⇐ 1.0 ; // Ray attenuationR ⇐ 0 ; // Overall rating initializationforeach Oi ∈ {O} do

OS ⇐ Calculate object’s viewport size;

Rs ⇐ (1−OS)VS

IS ; // Compute Ratings

Rt ⇐ OT IT ;if Aray > 0.1 and R < (RS + RT ) then

R ⇐ RS + RT ;Oresult ⇐ Oi;

endAray ⇐ ArayOT ;

endAlgorithm 1: Object selection.

(a) (b)

Fig. 6. Object selection. In 6(a) the selection of innerstructures such as vessels is enabled in the first place,while in 6(b) the transparent oesophagus in front of theopaque spine cannot be selected automatically.

The first allows the user to scroll between all structuresin the pick ray, using the mouse wheel or the cursorkeys. Starting with the structure that the automaticalgorithm would choose, the user can scroll back andforth between objects adjacent in depth. The CSO willbe clearly emphasized, using a thick silhouette and anopaque color. The second selection technique offers a listof all structures in the pick ray directly at the cursorposition in a small panel, so the user does not need tode-focus from the scene (see Figure 7(a)). We extend thelist of textual structure names by pictorial representa-tions of the structures. However, this technique aims atexperienced users who know all structures by name.

Object manipulation. Even if users adjust the appear-ance of a visualization globally by selecting a preset,they might want to adapt the appearance of singlestructures individually. We provide some GUI widgetsthat can be integrated in a panel or window, to adjust allvisualization parameters such as color, transparency or

11

(a) (b)

Fig. 7. Fast exploration popup menus. 7(a) All struc-tures in the pick ray are presented as a fast accessiblelist to the user. 7(b) For quick parameter manipulation, acontext popup is presented right to the cursor position.

silhouette width. In addition, we provide an explorationtechnique, where the user can easily adjust the mostimportant parameters directly in the viewer (see Figure7(b)). The provided list of parameters can be adaptedand extended for individual application requirements.

Graphical application interface. For a fast and effi-cient application development, predefined widgets forcommon and recurrent tasks are provided, e.g., lists toselect structures or to change their visibility. We providepanels to change the visualization parameters of struc-tures such as color, transparency or silhouette width,efficiently.

Feedback from surgeons clearly revealed that a ratherlow level of flexibility is needed and guidance is con-sidered essential. Surgeons prefer clear and easy to un-derstand interfaces [10] instead of interfaces with manyparameter sliders and value inputs. They want to geta good visualization for the current task or medicalquestion automatically or at the utmost selecting a well-defined preset from a small list of choices. The METKsupports this, e.g., with key states and the animation fa-cilities. Furthermore, ready designed graphical interfacesfor surgical applications are provided as templates. Theinterfaces were gathered by many interviews with ourmedical partners and approved by evaluation [10].

5 APPLICATION DEVELOPMENT WITH THEMETK

Since the METK is an extension of MeVisLab, all METKapplications can be built up by creating networks ofmodules in a visual programming environment. TheMETK modules can be arbitrarily mixed up with otherMeVisLab modules. Due to the high level functions pro-vided by the METK, a developer can focus on applicationlogic. This quickly yields applications that a surgeon canuse (see Figure 9), which often elicits essential feedback.Thus, this stage should be achieved as fast as possibleand the development cycles of new applications needto be accelerated. The application’s logic is defined in

1. Create a new module using the MeVisLab module wizard

2. Build a network, including METK modules for communica�on, to get the selected object, to get the 3D data and a viewer to visualize the object

3. Write a script to define the layout of the graphic interface

4. Result: A panel where the current selected object is visualized.

Fig. 8. Steps that are necessary to create an own METKmodule that visualizes the currently selected object withits name and anatomical affiliation.

Python scripts. We support application building by inte-grating many ready-to-use Python scripts in the METK.The design of the graphical user interfaces is scripted aswell, using the module definition language of MeVisLab.We extend the basic set of widgets such as buttons andsliders by more complex widgets that can be integratedin an application with minimum scripting effort.

To extend the METK or to supplement existing func-tions, developers can refine existing METK modules orcreate new modules. Depending on the complexity ofextensions there are basically two options: Developerscan implement simple functions, e.g., a patient datamanagement or widget panels in modules, written inPython. Advanced and especially performance criticalissues can be implemented in C++ libraries. Since this isonly necessary for special visualization techniques notincorporated in the METK so far, such as DTI visuali-zation, this does not contradict the supposed low pro-gramming skills that are needed to build up ready-to-useapplications with the METK. In Figure 8, we illustrate thesteps that are necessary to build a new METK modulethat visualizes the currently selected object (CSO) withits name and anatomical affiliation. This module can, for

12

Fig. 10. NeckSurgeryPlanner. The NECKSURGERY-PLANNER supports the operation planning for neck dis-sections. To provide deep insight in the original 2D dataas well as in the segmented 3D structures, 3D and 2Dviews are used synchronized. On the left, a browser forenabling and disabling structures and key state previewsare provided.

Fig. 11. LiverSurgeryTrainer. The LIVERSURGERY-TRAINER is an application to teach abdominal surgeonsthe planning workflow for liver resection and living liverdonor transplantations. The application layout containsonly a few widgets.

example, be integrated as part of a larger application.Several full-fledged applications were designed and

developed with our toolkit. Using the METK, a trainingsystem for liver surgeons was developed, the LIVER-SURGERYTRAINER [1] (see Figure 11). The feedbackfrom the surgeons within the evaluation of the LIVER-SURGERYTRAINER [10] inspired several refinements ofthe METK. One inspiration was the large importance of2D slice-based visualizations in contrast to a pure focuson 3D visualizations. The NECKSURGERYPLANNER sup-ports the decision-making process for neck dissections.Here, the target group are experienced surgeons [33] (seeFigure 10).

6 EVALUATION BY IMPLEMENTING A REFE-RENCE APPLICATION

To assess the effectiveness of the METK in comparisonto current application development, we implemented asmall reference application in both the standard versionof MeVisLab and MeVisLab extended with the METK.The application should load a given CT image andmultiple segmentation masks, and visualize them as 3Dsurfaces and as 2D images. In 3D the structures shouldbe visualized with their standardized style and theirvisibility (on/off) should be changed individually. Afterthe description of the particular development process wewill compare both solutions by means of developmenttime, resulting application and usability.

For the solution with the basic MeVisLab version, allsegmentation masks must be loaded manually. After-wards, modules to create the surface for each singlestructure must be added and parameterized individually.To achieve a correct visualization style, the structures’nodes in the network must be categorized (e.g., vesselor muscle) and a material (color and transparency) mustbe attached to each group. The scene is visualized in asimple 3D viewer. For the 2D visualization the image isadded to the network and visualized in a 2D viewer. Theviewers are put together in one application interface bywriting a script that defines position and extension ofeach viewer. To switch the visibility of each structure,each visibility parameter of every structure must beadded manually to the script. This process must beperformed again for every case, since only the networkcan be saved and no case data management is available.

Using the METK, we first add a manager modulethat enables an application to load and save cases andprovides the communication functions for the whole ap-plication. A second module provides the import facilitiesfor the slices and segmentation masks. For the 3D visu-alization, the module for surfaces visualization and anMETK 3D viewer are added. For 2D slice visualization,an METK module for image loading and an METK 2Dviewer are added. Afterwards, the script to arrange thewidgets in the application window is written, whereas aspecial METK list widget is integrated for fast visibilitychanges of single structures. After executing the appli-cation, a new case can be created by importing an imageand segmentation masks located in the same directory.For each structure, its type and anatomical affiliationcan be entered. This is only necessary if the case wasnot segmented with an METK compatible application asmentioned in Section 1. For later reuse, the case can besaved, so this procedure must be performed only once.

Since with the basic version of MeVisLab for eachcase a network must be created manually, it took 51minutes to add all 148 modules for the sample case andparametrize them individually. Writing the script took8 minutes. This process of about 1 hour needs to berepeated for every new case. The required time primarilydepends on the number of segmented structures.

13

(a) (b)

Fig. 9. Sample METK application networks. 9(a) A network to present segmented datasets in a 3D viewer with aGUI widget to change visibility of structures. A spine surgery dataset is displayed. 9(b) A network of an application tosynchronously explore 3D and 2D data, extended with silhouettes and a colored distance transformation in 2D and3D. A neck surgery dataset is displayed.

With the METK it took less than 1 minute to create theapplication network consisting of only 6 METK modules.Additional 5 minutes were used to write the script forthe application layout, and it took another 8 minutes toimport the data into the application.

From the beginning until the ready visualization ofthe data it took 14 minutes with the METK and 59minutes with the basic version of MeVisLab. For newcases (that only consist of images and segmentationmasks and come without any METK compatible metadata) with the METK it took only the time of importingthe data, whereas with the free MeVisLab version thewhole network would have to be re-created.

7 DISCUSSION

There are many development environments for sci-entific visualization available. Some of them supportthe developing process by sophisticated techniques,e.g., graphical network programming (e.g., SciRun [37],MeVisLab, and DEVIDE [4]). Even though SciRun andMeVisLab provide basic facilities for application de-velopment, it is still very complex to efficiently createapplications that use new visualization techniques andcan be used by ’real users’ independently. In a compari-son study of four visualization frameworks (thereunderSciRun and MeVisLab), MeVisLab was determined asthe ‘best framework for creating applications’ [3]. Hence,we extended MeVisLab with a toolkit that especiallysupports the application building process for surgicalplanning. Even if there is some knowledge of pythonand the layout language MDL of MeVisLab necessaryto build up complete applications, it is much easier andfaster to come up with fully functional prototypes in anearly development stage, since many of the most usedvisualization and exploration techniques are already im-plemented in the METK. The application networks arealso easier to maintain and an reduced cognitive effort isnecessary to integrate changes or extend the application.

8 CONCLUSION AND FUTURE WORK

We presented an extensive toolkit for surgical ap-plication development – the MEDICALEXPLORATION-TOOLKIT (http://www.metk.net). Using the METK,applications that fulfill surgical requirements of explo-ration support and visualization techniques, can be builtup quickly. Introducing the multi-coded segmentationmasks, we provide an efficient way to store multipleoverlapping segmentation masks in one mask, support-ing colored overlays in 2D. Advanced 3D selection tech-niques and key states for storing of visualization stateare also dedicated to surgical planning, but useful forother application areas. With animation facilities, theviewpoint selection as well as new support for objectselection, a substantial guidance for the exploration of3D scenes is provided. Although the METK is a goodbasis for solving many intervention planning problems,special applications will yield new requirements.

To bridge the gap between pre-operative planningsoftware and intra-operative usage of the planning re-sults is a challenge for future work. The adaption oftechniques such as the automatic viewpoint selection foran intra-operative use would be a useful extension, sincemore guidance in 3D exploration is needed there, due tothe particular surroundings.

Our experiences with automatic techniques such asthe object and viewpoint selection showed that moresemantic information about the importance of structuresand their relations into the visualizations needs to beintegrated. Therewith, the presented context to struc-tures of interest or the intended user interactions canbe adjusted in a more appropriate manner.

Recent developments with user interface devices usedin surgical applications [13] necessitate the integrationof a wider variety of input devices in the METK. Thoseextensions require the integration of device drivers inthe system as well as a refinement of interaction andexploration techniques.

14

ACKNOWLEDGMENTS

This work was supported by the BMBF in the frameworkof the SOMIT-FUSION project (FKZ 01—BE 03B) and bythe Deutsche Forschungsgemeinschaft (DFG) (PriorityProgramme 1124, PR 660/3-1 and PR 660/3-2).

REFERENCES

[1] R. Bade, I. Riedel, L. Schmidt, K. J. Oldhafer, and B. Preim. Com-bining Training and Computer-assisted Planning of OncologicLiver Surgery. In Proc. of Bildverarbeitung fur die Medizin (BVM),pages 409–413, 2006.

[2] L. Bavoil, S. P. Callahan, P. J. Crossno, J. Freire, C. E. Scheidegger,C. T. Silva, and H. T. Vo. Vistrails: enabling interactive multiple-view visualizations. In Proc. of IEEE Visualization (VIS), pages135–142, 2005.

[3] I. Bitter, R. V. Uitert, I. Wolf, L. Ibanez, and J.-M. Kuhnigk.Comparison of Four Freely Available Frameworks for ImageProcessing and Visualization That Use ITK. Transactions onVisualization and Computer Graphics, 13(3):483–493, 2007.

[4] C. P. Botha and F. H. Post. Hybrid Scheduling in the DeVIDEDataflow Visualisation Environment. In Proc. of Simulation andVisualization (SimVis), pages 309–322, 2008.

[5] H. Bourquain, A. Schenk, F. Link, B. Preim, G. Prause, and H.-O. Peitgen. HepaVision2: A software assistant for preoperativeplanning in living-related liver transplantation and oncologic liversurgery. In Proc. of Computer Assisted Radiology and Surgery (CARS),pages 341–346, 2002.

[6] S. Bruckner and M. E. Groller. Volumeshop: An interactive systemfor direct volume illustration. In Proc. of IEEE Visualization (VIS),pages 671–678, 2005.

[7] J. J. Caban, A. Joshi, and P. Nagy. Rapid Development of MedicalImaging Tools with Open-Source Libraries. Journal of DigitalImaging, 20:83–93, 2007.

[8] J. Cordes, J. Dornheim, B. Preim, I. Hertel, and G. Strauss.Preoperative Segmentation of Neck CT Datasets for the Planningof Neck Dissections. In Proc. of SPIE Medical Imaging, 2006.

[9] J. Cordes, K. Hintz, J. Franke, C. Bochwitz, and B. Preim. Con-ceptual design and prototyping implementation of a case-basedtraining system for spine surgery. In Proc. of 1st International eLBaScience Conference (eLearning Baltics), pages 169–178, 2008.

[10] J. Cordes, K. Muhler, K. Oldhafer, G. Stavrou, C. Hillert, andB. Preim. Evaluation of a training system of the computer-basedplanning of liver surgery. In Proc. of CURAC, pages 151–154, 2007.

[11] A. Enquobahrie, P. Cheng, K. Gary, L. Ibanez, D. Gobbi, F. Lind-seth, Z. Yaniv, S. Aylward, J. Jomier, and K. Cleary. The Image-Guided Surgery Toolkit IGSTK: An Open Source C++ SoftwareToolkit. Journal of Digital Imaging, 20:21–33, 2007.

[12] G. Grevera, J. Udupa, D. Odhner, Y. Zhuge, A. Souza, T. Iwanaga,and S. Mishra. CAVASS: A Computer-Assisted Visualization andAnalysis Software System. Journal of Digital Imaging, 20:101–118,2007.

[13] C. Hansen, A. Koehn, S. Schlichting, F.Weiler, S. Zidowitz,M. Kleemann, and H.-O. Peitgen. Intraoperative modification ofresection plans for liver surgery. International Journal of ComputerAssisted Radiology and Surgery, 3:291–297, 2008.

[14] V. Interrante, H. Fuchs, and S. M. Pizer. Conveying the 3dshape of smoothly curving transparent surfaces via texture. IEEETransactions on Visualization and Computer Graphics, 3(2):98–117,1997.

[15] S. Iserhardt-Bauer, C. Rezk-Salama, T. Ertl, P. Hastreiter, andB. Tomandl. Automated 3D Video Documentation for the Anal-ysis of Medical Data. In Proc. of Bildverarbeitung fur die Medizin(BVM), pages 409–413, 2001.

[16] T. Jansen, B. von Rymon-Lipinski, Z. Krol, L. Ritter, and E. Keeve.An Extendable Application Framework for Medical Visualizationand Surgical Planning. In Proc. SPIE Medical Imaging, pages 349–357, 2001.

[17] A. Kruger, C. Tietjen, J. Hintze, B. Preim, I. Hertel, and G. Strauss.Interactive Visualization for Neck Dissection Planning. In Proc.of IEEE/Eurographics Symposium on Visualization (EuroVis), pages295–302, 2005.

[18] F. Link, M. Konig, and H.-O. Peitgen. Multi-resolution volumerendering with per object shading. In Proc. of Vision Modelling andVisualization (VMV), pages 185–191, 2006.

[19] I. H. Manssour, S. S. Furuie, L. P. Nedel, and C. M. D. S. Freitas.A framework to visualize and interact with multimodal medicalimages. In Proc. of Joint IEEE TCVG and Eurographics Workshop onVolume Graphics, pages 385–398, 2001.

[20] MeVis Medical Solutions - Distant Services. http://mms.mevis.de/en/Distant Services.html, Mar 2008.

[21] MeVisLab. http://www.mevislab.de, Mar 2008.[22] K. Muhler, R. Bade, and B. Preim. Adaptive script based an-

imations for intervention planning. In Proc. of Medical ImageComputing and Computer-Assisted Intervention (MICCAI), pages478–485, 2006.

[23] K. Muhler, M. Neugebauer, C. Tietjen, and B. Preim. ViewpointSelection for Intervention Planning. In Proc. IEEE/EurographicsSymposium on Visualization (EuroVis), pages 267–274, 2007.

[24] S. Pieper, M. Halle, and R. Kikinis. 3D Slicer. In Proc. of IEEEInternational Symposium on Biomedical Imaging: Nano to Macro, 2004.

[25] B. Preim, D. Selle, W. Spindler, K. J. Oldhafer, and H.-O. Peitgen.Interaction Techniques and Vessel Analysis for Preoperative Plan-ning in Liver Surgery. In Proc. of Medical Imaging and Computer-Assisted Intervention, MICCAI, pages 608–617, 2000.

[26] B. Preim, C. Tietjen, W. Spindler, and H.-O. Peitgen. Integrationof Measurement Tools in Medical Visualizations. In Proc. of IEEEVisualization (VIS), pages 21–28, 2002.

[27] J. Rexilius, J.-M. Kuhnigk, H. K. Hahn, and H.-O. Peitgen. AnApplication Framework for Rapid Prototyping of Clinically Ap-plicable Software Assistants. In Proc. of Informatik fur Menschen -Band 1, pages 522–528, 2006.

[28] F. Roßler, R. P. Botchen, and T. Ertl. Dynamic shader generationfor flexible multi-volume visualization. In Proc. of IEEE PacificVisualization Symposium 2008 (PacificVis ’08), pages 17–24, 2008.

[29] C. Scheidegger, H. Vo, D. Koop, J. Freire, and C. Silva. Queryingand creating visualizations by analogy. IEEE Transactions onVisualization and Computer Graphics, 13(6):1560–1567, 2007.

[30] C. Schumann, S. Oeltze, R. Bade, and B. Preim. Model-free SurfaceVisualization of Vascular Trees. In Proc. of IEEE/EurographicsSymposium on Visualization (EuroVis), pages 283–290, 2007.

[31] D. Stalling, M. Westerhoff, and H.-C. Hege. Amira: A HighlyInteractive System for Visual Data Analysis. In The VisualizationHandbook, chapter 38, pages 749–767. 2005.

[32] C. Tietjen, B. Meyer, S. Schlechtweg, B. Preim, I. Hertel, andG. Strauss. Enhancing Slice-based Visualizations of MedicalVolume Data . In Proc. of Eurographics /IEEE VGTC Symposiumon Visualization (EuroVis), pages 123–130, 2006.

[33] C. Tietjen, B. Preim, I. Hertel, and G. Strauss. A Software-Assistantfor Pre-operative Planning and Visualization of Neck Dissections.In Proc. of CURAC, pages 176–177, 2006.

[34] P.-P. Vazquez, M. Feixas, M. Sbert, and W. Heidrich. ViewpointSelection using Viewpoint Entropy. In Proc. of Vision, Modeling,and Visualization (VMV) 2001, pages 273–280, 2001.

[35] I. Viola, M. Feixas, M. Sbert, and M. E. Groller. Importance-DrivenFocus of Attention. IEEE Transactions on Visualization and ComputerGraphics, 12(5):933–940, 2006.

[36] I. Viola and M. E. Groller. Smart Visibility in Visualization. In Proc.of EG Workshop on Computational Aesthetics Computational Aestheticsin Graphics, Visualization and Imaging, 2005.

[37] D. Weinstein, S. Parker, J. Simpson, K. Zimmerman, and G. Jones.Visualization in the scirun problem-solving environment. In TheVisualization Handbook, pages 615–632. 2005.

[38] M. Wohlfart and H. Hauser. Story Telling for Presentation inVolume Visualization. In Proc. of IEEE/Eurographics Symposium onVisualization (EuroVis), pages 91–98, 2007.

[39] I. Wolf, M. Vetter, I. Wegner, T. Bottger, M. Nolden, M. Schobinger,M. Hastenteufel, T. Kunert, and H.-P. Meinzer. The medicalimaging interaction toolkit. Medical Image Analysis, 9:594–604,2005.

15

Konrad Muhler received the diploma in com-puter science in 2005 from the University ofMagdeburg, Germany and is currently a Ph.D.student. He is working as a researcher and soft-ware developer at the Department of Simulationand Graphics at the University of Magdeburg.His research interests include medical anima-tions, user interface design, educational soft-ware, and software engineering methods to fa-cilitate a fast development of clinically usefulapplications. He is a student member of the

IEEE.

Christian Tietjen is a Ph.D. candidate in Com-puter Science at the University of Magdeburg,Germany. His research focuses on illustrativemedical visualization. In detail, he actively ex-plores the suitability of illustrative renderingtechniques for intervention planning and medicaleducation. The combination of rendering styles,the reliable approximation of curvature infor-mation, and surface smoothing techniques arethe ingredients for his 3D visualizations. SinceNovember 2008, he is working for Siemens

Healthcare, Forchheim, Germany.

Felix Ritter is head of visualization and humancomputer interaction at Fraunhofer MEVIS, Bre-men, Germany. His research interests includemedical visualization, perception, and humanfactors in visualization. Felix’s current work isfocused on the usability of medical workstationsand the development of intra-operative user in-terfaces. He has authored and co-authored var-ious papers in the field of medical visualizationand human computer interaction. He received aPh.D. in Computer Science from the University

of Magdeburg, Germany in 2005.

Bernhard Preim received the diploma in com-puter science in 1994 and a Ph.D. in 1998from the University of Magdeburg. In 1999 hejoined the staff of MeVis (Center for MedicalDiagnosis System and Visualization). In closecollaboration with radiologists and surgeons hedirected the work on ”‘computer-aided planningin liver surgery”’ and initiated several projectsin the area of computer-aided surgery. In June2002 he received the Habilitation degree (venialegendi) for computer science from the Univer-

sity of Bremen. Since March 2003 he is full professor for ”‘Visualization”’at the Computer Science Department at the University of Magdeburg,heading a research group which is focused on medical visualization andapplications in surgical education and surgery planning.