Collaborative Virtual Simulation Environment for Radiotherapy Treatment Planning

EUROGRAPHICS 2000 / M. Gross and F.R.A. Hopgood Volume 19 (2000), Number 3(Guest Editors)

© The Eurographics Association and Blackwell Publishers 2000. Published by BlackwellPublishers, 108 Cowley Road, Oxford OX4 1JF, UK and 350 Main Street, Malden, MA02148, USA.

Collaborative Virtual Simulation Environmentfor Radiotherapy Treatment Planning

Wenli Cai, Stefan Walter, Grigorios Karangelis and Georgios Sakas

Fraunhofer Institute for Computer Graphics, Rundeturmstrasse 6, D-64283 Darmstadt, Germanyemail: {wcai, walter, karangel, gsakas}@igd.fhg.de

Abstract

The simulation of Radiotherapy Treatment Planning (RTP) is a normal procedure in oncology clinics carriedout on a Simulator machine. The Virtual Simulation of RTP replaces the real Simulator machine with a vir-tual one by using the CT data sets of a patient instead of the real patient. In this paper, we present a collabo-rative virtual simulation environment of RTP, named EU-VIRTUOSO, which is based on volume renderingand telecommunication techniques.

The RTP procedure is visualised on a virtual patient, which is created by using the CT data of the patient.Different volume rendering and volume interaction techniques, such as DRR, MIP, gradient surface, and iso-surface, supply physicians with high quality rendering images to simulate the real working environment ofthe Simulator machine. In the collaborative environment, physicians distributed at different locations canwork together via network to plan the treatment or to validate the treatment plan on-line by a collaborativeapplication sharing approach. Both concepts virtualised planning and collaborative planning improve theefficiency and accuracy of a radiotherapy treatment while reducing the effort for an individual patient.

Keywords: Medical Visualisation, Volume Rendering, Collaborative Environment, Telemedicine

1. Introduction

Radiotherapy Treatment Planning (RTP) for percutan ra-diotherapy is a procedure to plan an irradiation treatment,which is usually simulated on a conventional machine,called Simulator, before applying it on the patient. The goalof RTP is to plan a treatment which will deliver adequateirradiation doses to a tumour location without causingsever damage to its surrounding normal, healthy tissues.The general clinical routine of a radiation treatment plan-ning consists of a loop of three steps, Simulator planning,CT planning, and Simulator verifying, see Figure 1.

First, the patient is placed on a Simulator machine,which has the same mechanism and coordinate systems asthe radiotherapy treatment machine [CEI96], to locate thetumour position (or called treatment target volume) and itsrelationship to the surrounding critical organs by using thetraditional X-ray fluoroscopy. Then, a primary treatmentplan is made.

Diagnostic Tools

Radiation Treatment

Simulator Verifying

Simulator Planning

CT Planing

Figure 1: Clinical Radiation Treatment Routine

Cai et al. / Collaborative Virtual Simulation Environment for Radiotherapy Treatment Planning

© The Eurographics Association and Blackwell Publishers 2000.

Second, the patient is placed on a CT scanner. Physi-cians analyse the primary treatment plan on CT slices byimagining the geometric relationship between the irradia-tion beam and the target volume or critical organs in 3Dspace. Then, the physicist calculates treatment parametersand doses.

Third, the patient is placed on the Simulator machineagain. Physicians simulate and verify the treatment plan byusing X-rays instead of irradiation beams. After the treat-ment plan is verified, the patient proceeds to the actualtreatment. Otherwise the first step is repeated.

1.1 Previous Works

According to the RTP evaluation report of Harma andPurdy[HaPu97], a small number of the dedicated VirtualSimulator (VS) systems, such as AcQsim[PICK], Advan-tage Sim[GE], Pro-Sim[Prowess], and PLATOVSS[Nucletron], have been developed or are under devel-opment. The major weaknesses or deficiencies of the cur-rent RTP systems come from their rendering methods,since most of them use surface rendering rather than directvolume rendering. All target objects (tumours) and othercritical organs are required to be segmented and modelledby interactive contouring slice by slice. This process is themost time consuming step in current RTP systems asMeyer and Purdy[MePu94] mentioned. Due to this heavycontouring work, usually only a few CT slices (ca. 10-20slices, compared to a normal CT study, ca. 50-100 slices)are involved and only the modelled objects are displayed.But in practice, physicians make the treatment plan directlyon an X-ray fluoroscopy which displays the anatomy of atumour location. Hence, several mentioned VS systemshave developed Digital Reconstruction Radiography(DRR) images to remedy the defect. But their quality (us-ing less than 40 CT slices) and speed (up to several minutesper frame) are far from the interactive requirements of avirtual simulation, compared to the x-ray images producedby a Simulator machine. The deficiency of interactivity andhigh quality DRR (X-ray) images, which physicians are fa-miliar to work with during RTP, restricts the usability ofcurrent RTP systems. As a result, physicists rather thanphysicians are the main users of the current RTP systems.

1.2 Motivations

The advances of CT scanner technology, especially thewidely availability of the spiral CT, make it possible to scanmore than 100 slices in a few minutes with the resolution of512×512 and a slice thickness up to 1mm. The largeamount of CT data offers the possibility to render highquality DRR images. Furthermore, the speedup of volumerendering techniques have stimulated the development of aVS system (software), which is a new generation of theRTP system and physician-oriented. The VS can simulateall the physical motions of the Simulator machine and dis-play interactively DRR images (X-ray images). Physicianscan interactively work on the DRR images, to define the ir-

radiation beam and blocks, to control the Simulator ma-chine, to investigate the geometric relationship between thebeam and the target volume. All of the planning steps canbe performed without the presence of the patient. Since theVS working environment is embedded to the clinical RTProutine, physicians will work in the same environment andin the same way as in the Simulator machine room.

Telecommunication techniques, such as Internet, ISDNor modem, supply a collaborative working environment forphysicians. The patient, the CT scanner, the physicians, andeven the treatment unit, could be distributed at differentplaces in the world and they are linked together via a com-munication network. The virtual patient and visualisedtreatment planning can be accessed from any remote siteswithout the presence of the real patient. Physicians can beconnected on-line over network to make and verify the ra-diation treatment plan by sharing the identical user interfaceof the application in real time.

Both concepts, virtualised planning and collaborativeplanning, are the basic motivation of our VS system. Inthis paper, we present a collaborative virtual simulation en-vironment of RTP, called EU-VIRTUOSO, which not onlysimulates the working environment of the Simulator ma-chine but also establishes the tele-consultation environmentfor physicians to make and validate the treatment plan.

2. System Design

The virtual simulation RTP routine, the system architec-ture, as well as the user interface in EU-VIRTUOSO aredesigned based on the clinical RTP routine,

2.1 Virtual Simulation RTP Routine

The VS RTP working procedure in EU-VIRTUOSO is de-fined as follows, see Figure 2.

1) Data Acquisition

The patient CT data is formatted as DICOM and stored onthe DICOM server. The CT scans of the patient are ac-quired in the sequence of a continuous equal distance tablemovement along the longitudinal direction.

2) Target Volume Contouring

According to the clinical RTP routine, physicians are re-sponsible for the target volume (or tumor site) contouringon the base of CT slices. The contouring procedure is visi-ble for the remote users when it is performed in an on-linetele-consulting session.

3) Treatment Planning

The physicians make the treatment plan interactively in theVS working environment, according to the DRR imagescontaining the target volume. In an on-line tele-consultingsession, the visualised plan is transferred among physicians



in real-time by sharing the same interface of the application(see Section 5).

4) Dose Calculation

Physicists are responsible to calculate the dose off-linebased on the plan finished in Step 3). Since the dose calcu-lation is beyond the scope of the Simulator working envi-ronment and already available by commercial software,EU-VIRTUOSO does not support this step in its workingenvironment.

5) Treatment Verification

The physicians verify the treatment plan finished in step 3)and the dose calculated by the physicists in step 4) in thesame VS working environment in step 3).

6) Treatment Delivery

The verified treatment plan will be sent to radiation treat-ment unit and be delivered to the patient.

Among the above steps, step 2), 3), 5) belong to thecollaborative working steps. At each collaborative step, thetreatment plan and other RTP data can be saved into thepatient file and sent to the remote partners over network.

2.2 System Architecture

The EU-VIRTUOSO system architecture is shown inFigure 3. Patient CT data is stored on the local DICOMserver. Two EU-VIRTUOSO workstations are linkedthrough network (ethernet or internet). The patientDICOM CT data, the EU-VIRTUOSO internal data, suchas contours and radiotherapy plans, as well as the EU-

VIRTUOSO collaborative application status can be ex-changed with remote partners on-line over network.

2.3 Virtual Working Environment

The virtual working environment in EU-VIRTUOSO, seeFigure 5*, is designed according to the international stan-dard of Radiotherapy Equipment [CEI96] and the practicalworking environment of the Simulator, see Figure 4. Allthe motion of the virtual machine are consistent with thereal Simulator device.

Instead of the presence of a physical patient a virtualpatient as well as a VS machine are displayed in a Simula-tor Window. Through this Simulator window, the physi-cians can investigate the movements of the machine as theyobserve it through the lead glass in their Simulator opera-tion room, see the Sim view direction in Figure 4.

The most important RTP imaging is the dynamic X-rayimage on the Simulator monitor, which is generated fromthe view point of the beam source, called Beam’s Eye View

Data Acquisition

Treatment Planning

Treatment Verification

Treatment Delivery

Tel

e-co

nsu

ltin

g a

nd

colla

bo

rati

ve w

ork

ing

ste

ps

Dose Calculation

Target Volume Contouring

Figure 2: Virtual Simulation RTP Routine

Site B

DICOMServer

EU-VIRTUOSO

DICOMServer

Site A

EU-VIRTUOSO

Figure 3: EU-VIRTUOSO System Architecture

Sim

BEV

OEV

Figure 4: Simulator Working Environment



(BEV) image (see the BEV view direction in Figure 4). InEU-VIRTUOSO the corresponding window, called theBEV window, displays the interactive DRR images.Through the BEV window, the physicians can investigatethe patient anatomy as they observe it on the Simulatormonitor.

In addition, another two windows are combined intothe EU-VIRTUOSO working environment. The slice win-dow displays the anatomy information in CT scans, includ-ing the beam and blocks geometry and target volume. TheObserver’s Eye View (OEV) window visualises the patientfrom the room’s eye view, which is the same view as thephysicians observes the patient in the Simulator room, seethe OEV view direction in Figure 4. All of the images inboth BEV and OEV windows are generated with patientCT scans by direct volume rendering.

3. Rendering Methodology

The main rendering pipeline of EU-VIRTUOSO is similarto the pipeline described by Sakas [Saka93]. The volumedata flow through three different stages, GeometricTransformation, Integration-in-Depth, and Mapping. Thevolume rendering contains two kinds of volume illumina-tion methods: DRR imaging [Cai99] and maximum inten-sity projection (MIP) [SaGS95], and two kinds of surfaceillumination methods: gradient surface and iso-surface[Levo88]. The system uses ray casting as the basic volume

rendering algorithm. Different volume attributes includingsome geometrical information (depth buffer) are evaluatedduring the ray traversal.

3.1 Realistic DRR Illumination Model

Compatible to the clinical routine of RTP, the X-ray imageis very important in the VS environment. DRR is a projec-tion of 3D CT scans that produces a ”radiograph image”,which is an X-ray casting through the CT data set to showall the anatomy hit by the X-ray beam. A realistic DRR im-age is the basic requirement of the RTP simulation.

Fourier projection slice theorem and the fast Fouriertransformation can compute the necessary integral veryrapidly [ToLe93, Malz93]. But the image quality is notsatisfactory and the memory consumption is very large. So,we decided to use ray casting for the interactive DRR vol-ume rendering.

DRR volume rendering (also called X-ray volume ren-dering) simulates X-ray passing through an absorbing me-dium, which is usually reconstructed from CT or other to-mographic data sets (physicians use CT scans in the mostcases of RTP). According to physics, when a beam ofphotons traversing through a medium, the main energy ex-tinction is attenuation. which is called “absorption only”optical model[Max95] because the attenuation of the inten-sity is the only optical phenomenon in the illuminationmodel.

Virtual

Simulator

CT

Slice

BEV’s

DRR

OEV’s

Navigator

Interactive

Window &

Control Panel

Figure 5*: The Virtual Simulator Working Environment, i.e. the User Interface of EU-VIRTUOSO



In physics, attenuation in any material is represented bythe linear attenuation coefficient (µ(E), E is the photon en-ergy) or the mass attenuation coefficient (µ(E, m), whereµ(E, m)=µ(E) / ρ, ρ is the mass density of material m).Different materials have different µ(E) and µ(E, m). At acertain photon energy, µ(E,m) is constant in one materialand does not depend on the densities of the material, butonly on its composition. The attenuation process for abeam of photons traversing a medium is an exponentialfunction of the form, see Figure 6.

)exp(1

0 ∑=

−=N

iis xII µ Eq. 1

Based on the Houndsfield number in a CT data, differ-ent tissues may be classified basically on the monotone in-creasing order of the CT number, where air is -1000, wateris 0, soft tissue is 10 to 100 and bone is 500 ~ 1000. Sup-posing that there are no more than two materials, of whichthe CT number distributions are overlapped, and their µ(m)is linear within the overlapping range, the material classifi-cation and mixture model [DrCH88] is a way to estimatethe material composition based on CT numbers. Sometypical values of µ(E, m), such as air, lung tissue, fat, wa-ter, soft tissue, and bone, can be looked up from the massattenuation coefficient tables [HuSe95].

Therefore, if we know the x-ray energy value of the CTdata ECT and the DRR image EDRR, the CT number g canbe converted to the corresponding linear attenuation coef-ficient in DRR rendering µ(EDRR, g) by Eq. 2, which iscalled the transfer function (TF) of attenuation coefficientin DRR volume rending

),(

1),(

1000

1000),(

)(),(),(

gCTCTgDRR

gDRRDRR

mEwaterE

gmE

gmEgE

µµµ

ρµµ

⋅⋅+⋅=

⋅=Eq. 2

where µ(EDRR, mg) and µ( ECT,, mg) are the mass attenua-tion coefficients of the CT number g at the energy EDRR

and ECT respectively.

Figure 7(a) shows a DRR image rendered by our X-rayillumination model. On the other hand, the µ(EDRR, m) ofdifferent materials in CT data can be adjusted in order toenhance some materials. Figure 7(b) shows the DRR imageincluding only soft tissue. In Figure 7(b) the tumour on the

left side of the breast can be distinguished clearly from thesurrounding tissues. This kind of DRR image is difficult toimplement physically in the real X-ray fluoroscopy.

3.2 Beam Projection on the Patient Body

Beam projection on the patient body shows the beam pro-jection area on the skin of a patient. Physicians detect thebeam size and position by directly marking and measuringthe beam projection area on the patient skin, a normal pro-cedure in the clinical RTP.

Considering that the beam is originated from the irra-diation source, the projection area of the beam on the pa-tient skin is the area which is directly irradiated by thebeam. In another word, the beam projection area is the in-tersection area between the beam volume and the patientskin surface, see Figure 8.

Because there is no geometric model of the patientbody in the system, it is very difficult to calculate the inter-section area between the beam volume and the patient skindirectly. We used a Z-buffer in OEV window and a Beam-buffer in BEV window to solve this problem.

Regarding the irradiation source as a light source, thebeam projection area is the surface which are visible in thebeam’s eye view (BEV). If we create a Z-buffer in the OEVrendering and a Beam-buffer in the BEV rendering, thebeam projection area is the voxel in the Z-buffer which isvisible in the Beam-buffer.

The Z-buffer (SurfPos, SurfCoord) of ray casting iscreated during skin gradient surface rendering in the OEVrendering, which stores the first visible voxel along the

µ1 µ2 µ3 µNI0

x

P0 P1

)exp(1

0 ∑=

−=N

iis xII µ

L

Figure 6: X-ray Attenuation Optical Model

(a) All Materials

(b) Only Soft Tissue

Figure 7: DRR Rendering Images



viewing direction, in which SurfPos is the Z-depth value,SurfCoord is the corresponding voxel coordinates in thevolume data.

The Beam-buffer size of the BEV window is createdand set the buffer accessibility by the collimator size andthe blocks . If the pixel is outside the collimator or coveredby the blocks, it is set to be inaccessible.

After evaluating the transformation matrix of the BEV(perspective projection), we project each visible voxel inthe Z-buffer to the collimator plane. If the correspondingpixel in the Beam-buffer is inaccessible, the voxel in Z-buffer is outside the beam projection area and need notfurther processing. Otherwise it is inside the irradiationbeam.

Because several voxels in the Z-Buffer may be pro-jected to the same pixel in the Beam-buffer, the visibility ofeach voxel is determined by its distance to the irradiationsource, called beam depth. If the beam depth of the currentvoxel is larger than the reserved value in the Beam-buffer,the current voxel is hidden by the voxel in the Beam-bufferalong the beam direction. Otherwise, the current voxel islocated in front of the voxel in the Beam-buffer.

After all the voxels in the Z-buffer are projected, theBeam-buffer keeps the voxels which are visible along theBEV. And the corresponding pixels of the visible voxels inthe Z- buffer are the projection area lightened by the irra-diation beam.

The basic algorithm suffers from the aliasing due to thefollowing fact:

� Voxel has a size in space: one voxel is possibly hit byseveral rays in the OEV rendering, called sharedvoxel. When a shared voxel is projected to the Beam-buffer, its projection covers the same position. Itmeans only one pixel instead of the all pixels of theshared voxel, is kept in the Beam buffer. It results inonly one pixel in OEV is lightened.

Collimator limits the size of the irradiation beam as a rec-tangle, while blocks refine the beam along the boundary ofa target volume, which in most of the cases is not a rectan-gle. Both of them can be rotated along the beam axis.

The solution is:

� The floating point sampling location instead of the in-teger one, is stored in Z-buffer, i.e. a more accuratesampling coordinate.

� An additional buffer of the same size of the OEV win-dow, called Shared Voxel buffer, is created to keepthe pixel list in Z-buffer, in which a list of pixels sharethe same voxel and projected to the same pixel in theBeam buffer.

The data structures of the three buffers in C notationare:

/* Z-Buffer (OEV window size): the sampling point involume (fixed point float */

struct {long SurfPos;long SurfCoord[3];

} Z_buffer;

/*Beam-Buffer (BEV window size) */struct {

short x, y; /* the coordinate of Z-buffer */long BeamDepth; /* distance to the irradiation

source */long VoxelCoord[3]; /* voxel position */

} Beam_buffer;

/* Shared-Voxel-buffer (OEV window size) : the nextpixel in Z-buffer which has the same beam depthand projected to the same pixel in Beam-buffer */

struct {short x, y;

} Shared_Voxel_buffer;

When comparing between the beam depth of the cur-rent voxel and the reserved beam depth in the Shadow-buffer, following steps are applied.

Patient Skin

CollimatorIrradiationSource

Figure 8: Beam Projection Area

Figure 9: Beam Projection Area on the Patient Skin



� If the beam depth of the current voxel < the beamdepth in the Beam-buffer; then the list in Shared-Voxel-buffer at (x, y) in Beam-buffer is deleted andthe new list is created by using the current Z-bufferpixel.

� If the beam depth of current voxel = the beam depthin the Beam-buffer; then the current pixel is insertedinto the list of Shared-Voxel-buffer.

After the comparison, the pixel lists in the Beam-bufferare the beam projection area in OEV window, see Figure 9.If the viewing angel of OEV is not changed, there is nonew 3D traversing in OEV window. The proposed algo-rithm is very efficient by only using the Z-buffer in OEVray casting. Its quality is also satisfactory.

3.3 Target Volume Contouring

The target volume is delineated by contouring in key slices.The contours among key slices are interpolated by thecontours on the preceding and succeeding key slices, seeFigure 10. The physicians are responsible for drawing thecontours in the key slices and validating the contours in theinterpolated slices. The contoured objects (or target vol-ume) are represented internally by a TargetVol volume.Different rendering methods can be applied to distinguishthe voxels inside or outside the target volume. Accumula-tion level intermixing with exclusive opacity and inclusiveopacity intermixing methods [CaSa99] are used to visualisethe target volume in both BEV and OEV images, seeFigure 11. Because bone is usually much more stable thansoft tissue or skin, the position of the target volume is oftencompared with the bone in DRR image, see Figure 11 (b).

4. Interactive Registration

In the RTP routine there are several coordinate systems tolocalise the patient in the world coordinate system (in dif-ferent rooms), which are the CT machine, the Simulator,and the treatment unit coordinate system. Three coordinatesystems must be consistent with each other in order to en-sure the identical patient position and machine movementin the treatment plan. This consistency coordinate system is

called the "fixed coordinate system" [CEI96], which is im-plemented physically by the laser positioning systems,which are installed in all machine rooms, and landmarksattached on the patient’s skin.

During a clinical RTP, the projection of the laser beamsas well as the beam projection area on the patient skin aremarked by using a water-insoluble marker on the patient’sskin. In the following steps, such as the CT scanning, theplanning verification, and the treatment delivery, the lasercoordinates in each operation room are required to matchthese landmarks on the patient’s skin so as to recover thepatient initial position while planning. The consistency ofthe coordinate systems is the basic requirement for a cor-rect treatment planning.

Before the CT scanning, the projection of the laserbeams on the patient skin are marked by three aluminiummarks (crosses) on the patient’s skin. These aluminiummarks are clearly visible on the CT slices, see the whitepoints on patient body in Figure 12*(b), as well as in theOEV rendering images, see Figure 12*(a). These threelandmarks define the reference point of the fixed coordi-nate system. All the treatment parameters of the VS are

Key Slice

Key Slice

Interpolated Slice

Figure 10: Contouring and Interpolation

(a) Target volume in OEV rendering

(b) Target Volume in BEV rendering

Figure 11: Target Volume Rendering



defined based on the registered reference point calculatedfrom the three landmarks.

The interactive landmarks registration is finished in theOEV window. Different viewing direction and scaling canbe selected in order to find a “clear” view of the landmarks,i.e. through interaction to find a rendering image which in-cludes a large and sharp landmark. The physician defineseach landmark by the manual selection with the mouseclicking at the centre of the landmark, see Figure 12(a).The corresponding Z-buffer value, (SurfPos, SurfCoord), isused to detect the voxel position in the CT slices, see thered box in slice window in Figure 12*(b). In order to avoidnoise, a 3×3 filter is used when detecting the voxel positionof the landmarks.

After the fixation of the three landmarks, i.e. LeftMark, Middle Mark and Right Mark, the reference point iscalculated by evaluating the cross point of the three land-marks in space.

� Calculating the coronal plane of the reference point byaveraging the coronal planes of Right Mark and LeftMark.

� Calculating the reference point by intersecting the linethrough the Middle Mark and the calculated coronalplane.

After calculating the reference point, the volume data ismoved to the calculated reference point as the originalpoint and all machine parameters are initialised. In Figure12*(c), a gradient surface image in the BEV is rendered toevaluate the patient position after registration. The middlelandmark is located at the centre of the collimator.

5. Collaborative Environment

The design of a collaborative working environment is an-other major objective of the EU-VIRTUOSO project. We

call this collaborative working scheme collaborative plan-ning. The application can be used as a stand-alone system,but the physicists can decide to perform a collaborativeplanning of the treatment when facing problems with a spe-cific simulation. In this case two applications are connectedto each other via a network. The basic requirement forsuch a collaborative environment is that the physicists usingEU-VIRTUOSO collaboratively see exactly the same viewof the data. The individual interactions with the application(e.g. drawing a contour) should be visible with the leastpossible latency on both sides and impact the internal datastructures of the respective application in the same way. Inthis system collaborative planning is implemented as inter-face sharing. A large number of collaborative applicationshas been introduced already [BeRS94] [CGSK96][AKBC95], so we here sum up the main properties of oursystem:

� First of all the application must be able to run in astand-alone mode, where no network connection isrequired.

� There is no external influence on the application thatchanges the data visualised by the application, such asdescribed in [BeRS94]). All changes of data are per-formed explicitly via the user interface, e.g. by draw-ing contours, the volume data itself is not manipu-lated.

� The physicians must have the identical view which re-quires a strict presentation level sharing collaborativescheme ("what you see is what I see") ([BeRS94]). Amore relaxed coupling of both user interfaces like in aview level sharing is not intended.

� The collaborative architecture must enable the de-signer of the EU-VIRTUOSO application to modifythe user interface easily according to the wishes of the

(a) Skin landmarks (OEV) (b) Iso-centre in axial slice (c) BEV after register

Figure 12*: Landmarks and Patient Registration



medical users which requires a more general solutionfor the collaboration.

5.1 Architecture

The underlying collaborative model of EU-VIRTUOSO ispeer-to-peer model, which means that a system in the col-laborative working environment has a connection to ex-actly one other system. The patient data and the data forsimulation is replicated at each side and is synchronised, re-spectively transferred from one side to the other, beforestarting the collaborative work. This architecture matchesthe star-topology of the EU-VIRTUOSO collaborativeplanning network of the involved clinics where the centralnode acts as a centre of expertise.

5.2 Interface sharing

Analysing the user interface and the elements that are usedfor interaction of the user with the application we distin-guish between - similar as the widespread X11 system[Nye92] does - two different main classes: widgets andwindows. Widgets are user interface elements such as but-tons, sliders, combo-boxes, edit fields, etc. Windows aremainly output windows of the application where computedimages and overlaid drawings are displayed. The interac-

tion of the user with windows is performed with the mouseor the keyboard, the user for example "clicks" with themouse into the display area of a window and "drags" themouse over it to draw a contour. The main property ofwidgets is that they are used at discrete times, and there-fore create events at discrete times. In opposite to thiswindows create a stream of events while the mouse isdragged over the window. Both kinds of events are han-dled by call-back functions which changes application-internal data values and create further events for control-ling the computation. We furthermore distinguish betweentwo classes of events that are created by the user interfaceelements: collaborative and non-collaborative events. Col-laborative events are transferred over the network to theremote application during a collaborative session, non-collaborative events remain local and influence only the lo-cal application. This differentiation is made because forsome of the widgets its local usage on one system is notintended to be transferred to a remote system, e.g. a buttonfor saving data to a local file might cause problems andunwanted side effects on the remote side. Collaborativewidgets and windows are registered during application in-terface initialisation.

Adaptation Layer

Sesssion Control

Data Transfer

Communication Layer (COM++)

Network (TCP/IP) Protocol)

Application LayerLocal Events

Non-collaborative Events

Collaborative Events

Network Events

Event Handler

User Display

Remote EU-VIRTUOSOApplication

Security Control

Figure 13: Architecture of the Collaborative Environment



The architecture of the EU-VIRTUOSO collaborativeenvironment consists of three layers, an Application Layer,an Adaptation Layer and a Communication Layer, see[Cai99a]. The Application Layer is responsible to handle allthe local events and events coming from a remote applica-tion to update the user interface. The Adaptation Layertransfers the collaborative events to the remote applicationby compiling different data packages according to a pro-tocol described below. All of the data packages are sent tothe remote system via the basic telecommunication layer.EU-VIRTUOSO uses the COM++ library in the communi-cation layer, which is developed in Fraunhofer-IGD[HaSc99].

Besides of the data created by the user interactionsthere are also a mechanisms for session control and binarydata transfer. We distinguish between three kinds of datapackages that are sent over a network to the remote appli-cation:

� Discrete events: such as button events or slidermovements. A later event has no correspondence withcurrent event.

� Stream events: such as the continuous mouse move-ments of the mouse in a window while contouring,and in navigation. A sequence of such events must behandled as a whole.

� Binary data: all the data in application. Mainly thisrefers to the volume data, but also other kinds of datasuch as images is contained in this kind of data.

The application additionally contains a master andslave collaborative scheme, where one side has full controlover the application and is allowed perform manipulationof all the data while the other is in a passive state. This wasmainly done to organise the collaborative session and guidethe collaborative work. The slave application can changeits role to the master role upon request.

According to the different events, the different proto-cols for collaboration environment are designed.

� Session protocol: connection and disconnection, role(master or slave) management, interruption of ongo-ing data transfer, exchange of information for syn-chronisation.

� Discrete event protocol: the action on the registeredwidget, including (Unique Widget ID, Parameter1,Parameter 2, Appendix).

� Stream events protocol: Only the current master in asession can generate continuous events, which have astart package and an end package to indicate the startpoint and the end point of the mouse movement in theregistered window. Within the start and end package,the continuous movement (x, y) is sampled according

to the frequency of the callback defined in the opera-tion system.

� File transfer protocol: All binary data is stored to a filebefore transfer to the remote site. A file header pack-age followed by a number of packages containing thebinary data is sent to the remote application, and re-constructed to a file after receipt. This transfermechanism also includes a LZW compression, a RSAbased encryption and a recovery for interruptedtransfers.

5.3 Synchronisation

At start up time of a collaborative session, directly after thenetwork connection is established, both applications syn-chronise their internal state and ensure that both have thesame volume data loaded. The patient data can be trans-ferred in an off-line session in advance or during the cur-rent session. Since the CT scans of the patient for VS ofRTP is very large, about 50-100MB of original, uncom-pressed data per patient, the off-line transfer is the usualcase. The identification of the patient data is confirmed byseveral patient and hospital dependent information, whichare stored in the patient DICOM CT file. In the secondstep the internal status of the application initiating the col-laborative session is stored to a file and transferred to theremote site so that both applications are synchronised.

To be able to have the system running on computingplatforms with different performance it is necessary to in-troduce an additional synchronisation mechanism for twoapplications. In the application’s loop for event handling asemaphore like mechanism was integrated where one appli-cation sends a signal to the remote application and waitsuntil itself receives the according signal from the remoteside before continuing with the event handling. Thismechanism serves to slow down the side with the higherperformance to the processing speed of the slower system.This synchronisation mechanism is a general solution forsuch kinds of synchronisation problems the alternative is totransfer the particular parameters of the user interface ele-ments.

6. Clinical Evaluation

The first EU-VIRTUOSO prototype system has been in-stalled in Offenbach (Germany) at the radiotherapy de-partment of the Stätische Kliniken Offenbach at the end of1998 for the clinical validity and accuracy test. All of theCT data sets used for our validation and accuracy meas-urements, produced from a SIEMENS Plus 4 spiral CTsystem. The Plus 4 system is able to produce one slice in0.75 sec. This means to scan an area of 300mm using a3mm slice thickness 75sec are need.

A cubic phantom size of 120×120mm, which wasavailable from the clinic, was initially used to validate theaccuracy of EU-VIRTUOSO. The phantom was scanned



using 3mm slice thickness. The total number of producedslices was 90, in the resolution of 512×512 pixels. Usingthis data set the interactive registration, the machinemovements, and all of the system algorithms have beentested. The system error concerning the reference point,the machine movements, and the rendered images was at±1mm. Other functionality tests, such as the automaticSource-to-Skin-Distance (SSD) setting, the automatic tar-get volume tracking, the automatic collimator size adapta-tion, and the automatic block adaptation gave an error of±1mm. Additionally different patients cases have beentested. The error measured in above was within ±3mm,which is below the error of ±5mm on the real Simulatormachine.

The second EU-VIRTUOSO prototype system hasbeen installed in Innsbruck (Austria) at the department ofradiotherapy and radiooncology of the Innsbruck Univer-sitätsklinik in September 1999. The collaborative environ-ment test between the developers in Darmstadt and Inns-bruck was done successful. The communication speed issatisfactory and there is no obvious latency during the col-laboration.

During the clinic evaluation, we have performed a sta-tistics to compare the time cost of patients, technicians andphysicians on each step of RTP by using the real Simulatorand virtual Simulator. Based on the statistics data in Table1, treatment planning using virtual Simulator greatly re-duced the time cost, especially the time that the patientmust be present.

Table 1: Time analysis of treatment planning procedure(unit: min)

REALSIMULATOR

VIRTUALSIMULATOR

Patient 104.0 12.2

Technicians 29.9 12.2

Physicians 30.0 10.0

EU-VIRTUOSO is implemented on SGI-IRIX work-stations. The above tests are done on Indigo2 worksta-tions. The rendering speed during continuous movement isabout 1~2 frames per second, the refresh speed after themovement is stopped is about 2~3 seconds per frame.

In the near future, another two clinical institutions willjoin into the EU-VIRTUOSO consortium for furtherevaluation.

7. Conclusions and Future Works

In this paper, we presented EU-VIRTUOSO, a collabora-tive virtual simulation environment for RTP. Compared to

the current medical visualisation systems and RTP systems,we concluded,

� EU-VIRTUOSO is a treatment rather than a diagnosissystem, a physicians-oriented Simulation system ratherthan only physicists-oriented RTP system .

� Direct volume rendering supplies a realistic workingenvironment for the physicians, especially the interac-tive and quality DRR images, rather than only somemodelled geometry objects.

� Interaction techniques based on the volume renderingimages enhances significantly the reality of VS work-ing environment, such as the beam projection on pa-tient skin, the interactive registration. Users can inter-act not only with the volume but with surface as well.

� The collaborative working environments greatly en-hance the consultation and validation for RTP amongdifferent clinical institutions.

� The concepts of visualised planning and collabora-tive planning can also be applied to other medical ap-plications as well as industry applications.

8. Acknowledgement

The work was supported by the European CommissionTelematics Application Projects, with the project numberHC 4024. The authors owe thanks to all the partners in theEU-VIRTUOSO project consortium.

9. References

[AKBC95] Ahlers, K., Kramer, A., Breen, D., Chevalier,P. C., et al. Distributed Augmented Reality for Collabora-tive Design Applications. Computer Graphics Forum (Eu-rographics’95), 14(3):3-14, , 1995

[BeRS94] Bentley, R., Rodden, T. and Sommerville. I. Ar-chitectural support for cooperative multiuser interfaces,IEEE Computer, pp 37-46, May 1994

[BeSh98] Bedford, J., and Shentall, G. A digital methodfor computing target margins in radiotherapy. MedicalPhysics, 25:224-231, 1998

[BPRG97] Belshi R., Pontvert D., Rosenwald J. C., Ga-boriaud G. Automatic three-dimensional expansion ofstructures applied to determination of the clinical targetvolume in conformal radiotherapy. Int. J. Radiation On-cology Biol. Phys., 37(3):689-696, 1997.

[Cai99] Cai, W. Transfer functions in DRR volume ren-dering. CARS’99, Paris, France, June 23-26, 1999

[Cai99a] Cai, W. EU-VIRTUOSO System Design, ProjectDeliverable D5.2, ZGDV, 1999



[CaSa99] Cai, W. and Sakas, G. Data intermixing andmulti-volume rendering. Computer Graphics Forum (Eu-rographics’99), 18(3):359-368,1999

[CEI96] CEI/IEC 1217: Radiotherapy Equipment: Coor-dinates, Movements, and Sacle.1996

[CGSK96] Coleman, J., Goettsch, A., Savchenko, A.,Kollmann, H., et al. TeleInViVo: towards collaborativevolume visualization environments. Computers & Graph-ics, 20(6): 801-811, 1996,

[DrCH88] Drebin, R, Carpenter, L. and Hanrahan,P. Vol-ume Rendering, ACM Computer Graphics(SIGGRAPH’88), 22(4):65-74, 1988

[GE] http://www.ge.com

[HaPu97] Harms B., and Purdy A. Evaluating commer-cially available three dimensional radiotherapy treatmentplanning system. Seminars in Radiation Oncology, 7(1):83-94, 1997

[HaSc99] Haley, M. and Schiffner, N. COM++ Frameworkfor Collaborative Application. FhG-IGD, 1999

[HuSe95] Hubbell, J. Seltzer, S. Tables of X-ray Mass At-tenuation Coefficients and Mass Energy-Absorption Coef-ficients 1KeV to 20 MeV for Elements Z=1 to 92 and 48Additional Substances to Dosimetric Interest, U.S. De-partment of Commerce, 1995

[KBWD98] Khoo S., Bedford L., Webb S., and DearnaleyP. Comparison of 2D and 3D algorithms for adding a mar-gin to the gross tumour volume in the conformal radiother-apy planning of prostate cancer. Int. J. Radiation OncologyBiol. Phys., 42(3): 673-679, 1998

[Levo88] Levoy, M. Display of surface from volume data.IEEE CG&A, 8(5), 1988

[Malz93] Malzbender, T. Fourier Volume Rendering, ACMTran. On Graphics, 12(3), 1993

[Max95] Max, N. Optical Models for Direct Volume Ren-dering. IEEE Trans on Visualization & Computer Graph-ics, 1(2):99-108, June 1995

[MePu94] Meyer J., Purdy J. (eds) 3-D Conformal radio-therapy. A new Era in the Irradiation of Cancer. KargerVerlag, 1994

[Nye92] Nye, A. Xlib Programming Manual, O’Reilly &Association Inc, 3rd Edition July, 1992

[Nucletron] http://www.nucletron.com/

[PICK] http://www.picker.com

[Prowess] http://www.prowess.com

[SaGS95] Sakas, G., Grimm, M. and Savopoulos, A. Op-timised maximum intensity projection (MIP), RenderingTechniques’95, Springer Verlag :51-63, 1995

[Saka93] Sakas, G. Interactive volume rendering of largefields. The Visual Computer, 9(8):425-438, August 1993

[ToLe93] Totsuka, T., and Levoy, M. Frequency DomainVolume Rendering, ACM Computer Graphics(SIGGRAPH’93), 1993



Virtual

Simulator

CT

Slice

BEV’s

DRR

OEV’s

Navigator

Interactive

Window &

Control Panel

Figure 5*: The Virtual Simulator Working Environment, i.e. the User Interface of EU-VIRTUOSO

(a) Skin landmarks (OEV) (b) Iso-centre in axial slice (c) BEV after register

Figure 12*: Landmarks and Patient Registration

Documents

Collaborative Virtual Simulation Environment for Radiotherapy Treatment Planning