1

Registration of 3D+t Coronary CTA and Monoplane2D+t X-Ray Angiography

Coert T. Metz∗, Michiel Schaap, Stefan Klein, Nora Baka, Lisan A. Neefjes, Carl J. Schultz, Wiro J. Niessen,and Theo van Walsum

Abstract—A method for registering preoperative 3D+t coro-nary CTA with intraoperative monoplane 2D+t X-ray angiog-raphy images is proposed to improve image guidance duringminimally invasive coronary interventions. The method uses apatient-specific dynamic coronary model, which is derived fromthe CTA scan by centerline extraction and motion estimation.The dynamic coronary model is registered with the 2D+t X-raysequence, considering multiple X-ray time points concurrently,while taking breathing induced motion into account. Evaluationwas performed on 26 datasets of 17 patients by comparing pro-jected model centerlines with manually annotated centerlines inthe X-ray images. The proposed 3D+t/2D+t registration methodperformed better than a 3D/2D registration method with respectto the accuracy and especially the robustness of the registration.Registration with a median error of 1.47 mm was achieved.

Index Terms—coronary, CTA, X-ray, percutaneous coronaryintervention, PCI, registration, motion modeling

I. INTRODUCTION

A. Background

PERCUTANEOUS coronary intervention (PCI) is com-monly applied to reopen coronary arteries which are

narrowed due to atherosclerosis. In PCI, a guidewire is in-serted into the femoral artery and advanced to the site ofthe lesion. Once the guidewire is in place, a hollow ballooncatheter is positioned over the guidewire and the vessel iswidened by inflation of the balloon. Often, at the same time,an expandable wire mesh tube (stent) is implanted to keepthe vessel open. The procedure is guided by X-ray imagingshowing the vessels and instruments. The procedural successrate of PCI is very high [1], but reduced success and highercomplication rates have been reported for complex vascularanatomies, bifurcating lesions and chronically totally occludedvessels [2]. Additional information from a preoperative CTAscan is potentially beneficial in these cases [3]. A vesselroadmap from CTA can be used to magnetically steer theguidewire through difficult branching points [4]. Likewise,information about the composition of the plaque, such as thelocation of calcium inside the lesion, may be useful whencrossing chronic total occlusions. To guide the surgeon, suchCTA-derived information can, for example, be overlaid on theintraoperative X-ray images.

To be able to use additional information from CTA, accurateregistration of the preoperative and intraoperative images isessential. This is a challenging task due to the highly dynamiccharacter of the coronary arteries and the ambiguity introduced

Copyright (c) 2013 IEEE. Personal use of this material is permitted.However, permission to use this material for any other purposes must beobtained from the IEEE by sending a request to pubs-permissions@ieee.org.

by the projective nature of monoplane X-ray imaging, whichis the modality commonly used in interventional cardiology.

B. Purpose and Contributions

The purpose of this work is to develop and evaluate amethod that achieves full-cycle alignment of the preoperativeCTA scan with the intraoperative monoplane 2D+t X-raysequence.

A common approach is to apply a 3D/2D registrationmethod for those X-ray images that correspond to the cardiacphase at which a high-quality diagnostic 3D CTA scan wasreconstructed. Such a gated approach has several drawbacks.First, accurate overlays of the CTA-derived information out-side the corresponding phases can not be achieved. Second,valuable information in the time-dimension of the imagingdata, which can potentially decrease the ambiguity introducedby the projective nature of monoplane X-ray imaging, is notexploited.

The key contribution of this work is the proposal of a3D+t/2D+t registration method to align preoperative 3D+tCTA data to intraoperative monoplane 2D+t X-ray images.The method takes into account multiple time points concur-rently, thereby employing information in the time dimensionof the data and achieving full cycle alignment. By usingretrospectively gated 3D+t CTA instead of 3D CTA, weare able to extract both the 3D geometry and the motionof the coronary arteries, which we subsequently use in theregistration procedure.

Given the 3D+t CTA image and the 2D+t X-ray sequence,both transformations in the temporal domain, to compensateheart rate differences, and the spatial domain, to compensatepose and shape differences, are taken into account. For thelatter, we propose a time-varying rigid transformation, whichcompensates for patient and breathing induced cardiac motion,mainly consisting of translations and rotations [5], [6]. Weaccount for cardiac motion by deriving a patient-specificmotion model from the preoperative 3D+t CTA image.

We quantitatively evaluate the performance of the methodon 26 clinical X-ray sequences of 17 patients and investigatethe influence of different temporal alignment and breathingmotion modeling approaches. Furthermore, we compare theaccuracy and robustness of the proposed 3D+t/2D+t methodwith a 3D/2D method. For more insight in the results, movies, error values and running times are made available at http://www.bigr.nl/movies/metz2012.

2

C. Related Work

An extensive overview of existing 3D/2D registration ap-proaches was provided by Markelj et al. [7]; here we focusspecifically on vascular applications.

Most previous work in this area focused on neurologicalapplications, where the vasculature can be considered as static[8]–[11]. Some work was presented on the rigid alignmentof preoperative cardiac data to intraoperative 2D imagingmodalities, for example to guide endovascular stent grafting[12], or for evaluating the applicability of dual-energy digitalradiography for calcium detection [13]. Only a few studieshave been dedicated to 3D/2D coronary artery registration.Turgeon et al. proposed a gated 3D/2D registration techniqueto align rotational angiography data to intraoperative X-rayangiography [14]. Their method was evaluated in a simulationstudy using both monoplane and biplane X-ray imaging.Ruijters et al. proposed a 3D/2D approach for registering a 3Dcoronary model, derived from CTA, and vessel enhancementfiltered 2D X-ray images [15]. We also performed ECG-gated3D/2D registration of the coronary arteries using an intensitybased approach and evaluated this approach on imaging dataof six patients [16].

Nonrigid approaches have been presented as well. Figlet al. [17] proposed to incorporate patient-specific cardiacmotion models in the registration of 3D+t coronary CTA andstereo endoscopic images using a photo-consistency measure.In the work of Gatta et al., 2D/2D nonrigid registration wasperformed to match simulated 2D projection images fromCTA-derived coronary segmentations with real X-ray images[18]. Serradell et al. generated a coronary deformation modelby randomly perturbing a CTA-derived coronary centerlinetree and using this deformation prior in the registrationprocedure [19]. In recent work of Rivest-Henault et al. aregistration method was proposed that iteratively optimizesa rigid, affine and local nonrigid deformation to match a3D coronary artery model to interventional X-ray images[20]. Their method was specifically developed for biplane X-ray angiography, which allows optimizing more degrees offreedom due to the additional information from the secondX-ray projection image. Nonrigid registration has also beenapplied to compensate breathing motion in the registration of3D CTA of the vasculature of the liver and digital subtractionangiography (DSA) images [21], [22].

We previously investigated the derivation of patient-specificcoronary motion models from 3D+t CTA scans and demon-strated their registration to 2D+t coronary X-ray images for asingle clinical case [23]. In [24], we enhanced the 3D+t/2D+tregistration approach and evaluated it on 12 clinical cases.In the current work, we concurrently optimize the temporaland spatial transformations, instead of using a separate ECG-based temporal alignment step, which was prone to noiseand differences in the ECG signals between the CTA acqui-sition and the intervention. Furthermore, we now investigatedifferent temporal alignment and breathing motion modelingapproaches and perform a more thorough evaluation using 26clinical cases.

0% 40% 70% 100%

Extract coronary tree4D registration

Transform coronary tree

Time-varying spatialtransformation Tspat

(Section II-C)

Sect

ion

II-F

Temporal transformationTtemp (Section II-B)

Optimizer(Section II-E)

Cost function(Section II-D)

X-ray images

Probable centerlinepoints X

Sect

ion

II-G

1 3 5 7 9

Patient-spec. dynamiccoronary model C

3D+t CTA scan

Fig. 1. An overview of the registration framework. See Section II-A for anexplanation.

II. METHOD

A. 3D+t/2D+t Registration Framework

An overview of the proposed 3D+t/2D+t registration methodis given in Fig. 1. The method performs the registration ona complete cardiac cycle. The transformation model consistsof both a temporal and spatial transformation. The temporaltransformation T temp corrects for differences in patient heartrate between the CTA acquisition and the intervention (SectionII-B). The spatial transformation T spat compensates for bothpatient and breathing motion (Section II-C). To estimate T temp

and T spat, a cost function is minimized that takes into accountmultiple time points concurrently (Section II-D).

Given the number of X-ray frames nx and CTA framesnc, we define integer time point indices 0 ≤ kx < nx and0 ≤ kc < nc for the X-ray and CTA images respectively.Furthermore, we define functions tx : N0 → R,

tx(kx) = kx/nx (1)

and tc : N0 → R,tc(kc) = kc/nc , (2)

that map the time point indices kx and kc to the continuousdomain [0, 1).

Cardiac motion is represented by a patient-specific dynamiccoronary model. The set of centerlines derived from the CTAimage at time point kc is denoted by Ckc and a continuousfunction C(tc) is defined to derive 3D centerline models at anytime point 0 ≤ tc < 1 by cubic spline interpolation. By usingthis continuous function, we also account for differences in the

3

temporal sampling rate between the CTA and X-ray sequences.Section II-F describes how Ckc is extracted.

A set of probable centerline points, derived from the X-rayimages, is denoted by

X = {Xkx : 0 ≤ kx < nx} (3)

with Xkxthe set of probable centerline points at X-ray time

point kx (Section II-G).The cost function in the proposed 3D+t/2D+t approach

defines the dissimilarity between C and X over all timepoints. In the gated 3D/2D approach, to which we comparethe proposed method in the experiments, the cost function isonly computed on C(tc(kc)) and Xkx , thus registering one3D coronary model to one X-ray image at the correspondingcardiac phase.

B. Temporal Transformation

The temporal transformation, T temp : [0, 1) → [0, 1), tocompensate heart rate differences and offsets between thecardiac cycles, maps the X-ray time point tx to its corre-sponding CTA time point tc. We investigate the applicationof two different transformation functions. The first is a lineartemporal transformation, having two degrees of freedom:

T temp-lµ (tx) = s+ (r + 1) tx , (4)

with µ = (s, r), and s and r parameters to handle the offsetand scaling between the selected X-ray and CTA cardiaccycles. Note that, for convenience, we use (r + 1), such thatµ = 0 results in the identity transform.

The second is a quadratic temporal transformation withthree degrees of freedom:

T temp-qµ (tx) = s+ (r + 1)

(−4at2x + (4a+ 1) tx

), (5)

with µ = (s, r, a) and a = 0 resulting in a linear transfor-mation function. This function enables to compensate for anonlinear relation between heart rate and cardiac phase. Sucha nonlinear relation between heart rate and the ECG signal has,for example, been reported by Bazett al. [25] As the electricalsignal initiates the contraction of the muscle cells, the timingof the contraction is also nonlinearly related to the heart rate.Figure 2 shows some examples for different s, r and a values.

Assuming a cyclic patient-specific dynamic coronary model,resulting time point values are mapped inside the [0,1) rangeusing:

tc = tc − btcc (6)

C. Time-Varying Rigid Transformation

To account for patient motion, and breathing motion duringthe cardiac cycle, we propose a time-varying rigid transforma-tion T spat

ν : R3 × [0, 1)→ R3:

T spatν (p, tx) = Rν(tx)p+mν(tx) , (7)

with p a three-dimensional position,Rν(tx) returning a matrixdescribing rotations around the axes of the CTA patient coor-dinate system at X-ray time point tx, and mν(tx) returning

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

X−ray time point

CT

A t

ime p

oin

t

µ= (0.0, 0.0, 0.0)µ= (0.1, 0.1, 0.1)µ= (0.3, 0.0,−0.1)µ= (0.1,−0.2,−0.1)

Fig. 2. Examples of quadratic temporal transformation Ttemp-qµ (t) for

different sets of µ. Values on the y-axis are mapped to the [0,1] domainaccording to (6).

a vector describing a 3-dimensional translation along theaxes of the CTA coordinate system for time point tx. Thistransformation is in principle not meant to compensate forrigid cardiac cycle related motion, as such motion is accountedfor by the patient-specific cardiac motion model. Vector νcontains the parameters describing the transformation thatmodels the breathing motion over time. Because the breathingcycle is slower than the cardiac cycle, we can approximatebreathing motion using a relatively simple model. We proposeboth the use of a linear and a quadratic function and compareboth, in the experiments, to not taking any breathing motioninto account. For the constant breathing function Rν(tx) andmν(tx) are constant over time and ν contains six elements:three Euler angles and three translations. For a linear breathingmodel, ν contains 12 parameters and Rν(tx) and mν(tx)are derived by linearly interpolating three rotation and threetranslation parameters at the start and end of the cardiac cycle.In the case of a quadratic breathing model, six additionaldegrees of freedom are introduced which define the rotationand translation of the model halfway the cardiac cycle andRν(tx) and mν(tx) are derived using quadratic interpolation.

D. Cost FunctionGiven the patient-specific centerline model C(tc) from the

CTA scan and the set of probable centerline points X in theX-ray sequence, the cost function S(µ,ν) is defined as:

S(µ,ν) =

nx−1∑kx=0

Skx(µ,ν) , (8)

with Skx(µ,ν) expressing the dissimilarity at X-ray frame

kx. To specify Skx(µ,ν), we first define C(µ,ν, tx), theprojection of the transformed centerline model on X-ray framekx:

C(µ,ν, tx) = P(T spatν

(C(T tempµ (tx)

), tx)), (9)

with tx = tx(kx), according to (1), and P (.) a function thatprojects points according to the geometry of the X-ray system,

4

0 5 10 150

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

5 10 15 30 65 125

d(mm)

h(d

)

Fig. 3. Examples of h(d) for different values of ha (indicated on the linesof the graph).

derived from the DICOM headers. Then, Skx(µ,ν) is defined

as the summed distance between the projected CTA centerlinesC(µ,ν, tx) and the set of probable centerline points in the X-ray images Xkx

:

Skx(µ,ν) =

1

|X|∑x∈Xkx

h(D(x, C (µ,ν, tx)

)), (10)

with D(x, C (µ,ν, tx)

)the shortest Euclidean distance of

pixel x to the projected centerline tree C, and |X| thetotal number of probable centerline points in X . Functionh(d) is a sigmoid-like function that controls the size of theneighborhood of the projected centerlines taken into account.It equals one when d is zero and decreases with increasing d:

h(d) = 2− 2

1 + exp (−d2/ha)(11)

with ha a parameter controlling the steepness of the function(see Fig. 3 for examples).

E. Optimization

Initially, the center of mass of the coronary centerline modelis positioned at the origin of the intraoperative coordinatesystem, i.e. at the iso-center of the X-ray system, and themodel is rotated according to the C-arm rotations as reportedin the DICOM header of the images.

The optimization procedure then determines those transfor-mation parameters that minimize the cost function (8):

(µ, ν) = arg min(µ,ν)

S(µ,ν) . (12)

Optimization is performed in a two-step approach. First,a grid search is performed to optimize the two translationparameters along the table direction, while all other parametersare kept to their initial values. A multi-resolution strategy isused in which ha is gradually decreased. The grid spacingand number of steps taken need to be configured for the firstresolution level. In subsequent levels, the search is continuednear the optimal values from the previous resolution level,

TABLE INUMBER OF PARAMETERS BEING OPTIMIZED FOR DIFFERENT

COMBINATIONS OF TEMPORAL AND BREATHING MOTION MODELS.

Temporal Breathing dim(µ) dim(ν) Total

Linear Constant 2 6 8Linear Linear 2 12 14Linear Quadratic 2 18 20

Quadratic Constant 3 6 9Quadratic Linear 3 12 15Quadratic Quadratic 3 18 21

using a grid extent of twice the previous grid spacing and thesame number of steps. In this part of the optimization, the twotranslation parameter values are kept identical for the completecardiac cycle, thus ignoring any possible breathing motion.

In the second part of the optimization approach, both µand ν are optimized using a Powell-Brent optimizer [26]–[28],which was proven to perform well for 3D/2D registration ofthe coronary arteries [20]. An advantage of this optimizationapproach is that derivatives of the cost function with respect tothe transformation parameters are not needed. Again a multi-resolution strategy is used in which ha is gradually decreased.

The dimensions of µ and ν and thus the total number ofparameters optimized depends on the chosen temporal andbreathing transformation model and are listed in Table I.

The magnitude of the influence of different parameters onthe cost function varies. For example, rotating the centerlinemodel with one radian has a larger influence on the costfunction than translating it with one millimeter, because thechange in appearance of the model in the X-ray images islarger. We therefore perform the optimization in terms ofrescaled parameters µ and ν, defined by

µi = µi/Zµi (13)

andνi = νi/Z

νi , (14)

with i the vector element index. We propose an automaticprocedure to estimate Zi before starting the optimization,in which we perturb µi or νi by a small value δ = 0.1.Subsequently, we derive C (µ,ν, tx) for both the zero anddeviated value of µi or νi, and define Zi as the averagedistance between these two sets of projected centerlines.

F. CTA-Derived Patient-Specific Dynamic Coronary Model

The patient-specific dynamic coronary model, Ckc, 0 ≤

kc < nc, is derived from a 3D+t CTA image, by first extractingthe centerlines of the arteries at the end-diastolic time point(i.e. at 70% of the cardiac cycle), and subsequently estimatingtheir motion from the complete image sequence (Fig. 1).

Centerline extraction in the 3D end-diastolic image is per-formed using a minimum cost path approach [23]. First, thestart and end points of all distinguishable coronary arteriesare manually annotated. Subsequently, initial centerlines areextracted. Resulting centerlines are visually checked in orthog-onal views of the CTA scan and in case of extraction failure,additional points along the coronary artery are annotated to

5

guide the extraction. In a second step, the centerline accuracyis improved by applying lumen segmentation [29], from whichsubsequently new centerlines are derived. Resulting centerlinesare resampled using spline interpolation to obtain a samplingdistance of 0.5 mm. To make sure that all tree segments areonly represented by one centerline, all centerline points fromdifferent centerlines which are closer than 0.5 mm apart aremerged into a single segment by averaging their coordinates.

Motion estimation is performed by a three-step registrationapproach using 3D+t nonrigid registration [30]. Histogramequalization was performed on the CTA images, because ofa positive effect on the registration results. The registrationprocedure determines the B-spline transformation that mini-mizes the variance of the intensity values over time. Trans-formations in the time-dimension of the data are not allowed,but smoothness in the time dimension is enforced using 4DB-splines. In the first registration step the cost function iscomputed from intensity values sampled throughout the wholeimage domain. Subsequently, the outer surface of the heart issegmented in the end-diastolic image (i.e. 70% of the cardiaccycle) using atlas-based segmentation [31] and propagated toall phases by application of the first-step registration result.In the second registration step, we determine the motion ofall cardiac structures. The cost function is therefore computedon intensity values sampled only inside the heart region1. Byusing this heart mask, the sliding motion of the heart alongthe lung surface, can be accounted for. As the motion of(especially the right) coronary arteries is very fast, we onlyfocus on the regions around the coronary arteries in the lastregistration step. In this step, a mask around the coronaryarteries is used, which is derived by dilating the extractedcoronary centerlines2 and propagating the 3D mask to the 3D+timage by application of the second-step registration result.

The transformation resulting from the final registration stepis used to propagate the centerlines from end-diastole to allother time points, thus generating a coronary model Ckc ateach time frame kc.

G. Detection of Probable Centerline Points in the X-RayImages

The detection of probable centerline points in the X-rayimages is based on computing a vesselness measure for eachframe. To prevent false positive responses of tubular non-vessel structures (e.g. edges of the spine, guide catheters, metalstitches), we first subtract a background image from the X-rayimages. This background image is derived by taking, for everyvoxel, the median value over time in the first B images of thecomplete X-ray sequence, assuming that in the majority ofthese images only non-coronary structures are visible.

A multi-scale vesselness filter is applied to the backgroundsubtracted X-ray image, to highlight vessel-like structures [32].The final set of probable centerline points X is derived bythinning a hysteresis thresholded version of this vesselness

1Dilated versions of the 3D+t cardiac segmentation generated using a kernelwith a radius of 13 and 8 voxels (∼10 mm and ∼6 mm respectively) are usedas masks to define valid sample locations and transformed sample locationsrespectively.

2A kernel with a radius of 20 voxels (∼27 mm) was used.

image. The hysteresis thresholding values (denoted by threshhand threshl) are determined by defining percentiles of the his-togram of the vesselness response values, to avoid dependencyon the actual contrast between the vessels and the background.An example of the detection procedure for one of the X-rayimages is shown in Fig. 4.

H. ImplementationCenterline detection and lumen segmentation were imple-

mented in C++ [23], [29]. 4D registration was performedusing elastix [30]. The 3D+t/2D+t registration frameworkwas implemented in MeVisLab [33]. Computation of the costfunction is implemented using CUDA, and for optimizationthe ITK Powell optimizer was used [26].

III. EXPERIMENTS AND RESULTS

A. Imaging DataWe collected X-ray angiography sequences of 17 patients

for which also retrospectively ECG-gated 3D+t CTA imageswere available. For two patients, X-ray images of differentintervention dates were available. For seven other patients,both sequences of the right and left coronary tree were avail-able. This resulted in a total of 26 different X-ray sequences.Patient and intervention information such as the coronary treeinvolved, the heart rate of the patient and the time intervalbetween the CTA and X-ray acquisition are listed in Table II.

X-ray images were acquired using a Siemens Axiom Artissystem. Acquisition took place between August 2007 andNovember 2009. The size of the X-ray images was 512×512pixels with a pixel size ranging from 0.22 mm to 0.35 mm.Although the number of time points used is in principlevariable to some extent, in this work we restrict ourselves toregistering one complete cardiac cycle. To this end, from everyX-ray sequence, we manually selected one contrast enhancedcardiac cycle for the experiments (nx = [9-19]).

CTA images were acquired using retrospective ECG-gatingwhile applying ECG-based dose modulation3 on a SiemensDefinition or Siemens Definition Flash scanner. Reconstruc-tions were made at every 5% of the cardiac cycle, resulting innc = 20 3D images per sequence. The matrix size was either256×256 or 512×512 and the in-plane voxel size ranged from0.35 to 0.76 mm. The slice thickness used for reconstructionwas either 1.5 or 3 mm and the z-voxel size ranged from 0.8to 1.5 mm.

Next to the X-ray data that was used to assess the perfor-mance of the method, an independent set of X-ray sequencesof 14 patients was collected for optimizing the parametersof the probable centerline point detection procedure. Thenumber of left and right trees was equally divided over thesesequences.

B. Parameter Settings for Extraction of Dynamic CoronaryModel and Probable Centerline Point Detection

Parameter settings for the centerline extraction in the end-diastolic CTA images (Section II-F) were adopted from [23]:

3The dose during the acquisition outside the end-diastolic phase is only 5%of the maximum dose used.

6

(a) (b) (c) (d) (e) (f)

Fig. 4. Example of X-ray probable candidate point detection: (a) Input X-ray frame, (b) Background image, (c) Background subtracted X-ray frame, (d)Vesselness image derived from (c), (e) Binary mask after hysteresis thresholding and (f) Final probable centerline point image derived by thinning the binarymask.

TABLE IIOVERVIEW OF IMAGING DATA USED. FOR PATIENT 2 AND 3, DATA OF DIFFERENT INTERVENTION DATES WAS AVAILABLE. FOR PATIENTS 6, 9, 10, 12, 13,

15 AND 16, BOTH IMAGES OF THE LEFT AND RIGHT CORONARY TREE WERE AVAILABLE. THE TABLE LISTS THE TIME INTERVAL BETWEEN THE CTAACQUISITION AND THE INTERVENTION, THE FRAME RATE AND THE NUMBER OF X-RAY IMAGES IN THE SELECTED CARDIAC CYCLE, AND THE HEART

RATE DURING BOTH THE CTA ACQUISITION AND THE INTERVENTION.

Patient 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Tree R L L/L R/R L L L/R R L R/L R/L L R/L R/L L R/L L/RInterval (days) 5 14 46/177 4/77 161 149 50 15 66 10 2 26 16 59 5 6 13

X-ray dataFrame rate (fps) 15 15 15 15 30 15 15 15 15 15 15 15 15 15 15 15 15# frames cycle (nx) 11 18 12/11 12/13 22 12 9/9 10 13 10/10 13/14 14 15/16 19/17 14 15/14 16/16

Heart rate (bpm)X-ray 82 50 75/82 75/69 82 75 100/100 90 69 90/90 69/64 64 60/56 47/53 64 60/64 56CTA 69 60 91 77 71 53 135 78 54 78 64 61 54 48 63 65 59Difference 13 -10 -16/-9 -2/-8 9 22 -35/-35 28 15 12 5/0 3 6/2 -1/5 1 -5/-1 -3

scales of 0.8, 1.3 and 2.0 mm, α = 0.5, β = 0.4, c = 230,w1 = 0.99, w2 = 0.1 and w3 = 0.1, for the vesselness filter,and as = −59 HU and bs = 4 HU for the sigmoid filter.The registration parameter settings were adopted from [30](cardiac case): B-spline spacing of 15 mm, spline order of 2,four resolution levels, 2000 samples for metric computationand 2000 iterations per resolution level. In four cases, wherethe registration approach was not able to correctly track thefast and large motion of the right coronary artery, a manualcorrection using thin plate spline interpolation was applied toguarantee correct dynamic coronary models.

For the probable centerline point detection in the X-rayimages (see Section II-G), the number of frames B used forbackground subtraction was set to 40. The vesselness filter wascomputed using three scales: 0.8, 1.26 and 2 mm, based on theexpected size of the vessels in the X-ray images, and parameterc was fixed to 5 as it correlates with the threshold parameters.Parameter β of the vesselness filter and parameter threshh andthreshl of the hysteresis thresholding were optimized using theX-ray images collected for optimization purposes. To this end,we manually segmented the vessels at two frames of eachsequence using a brush-like painting tool. Subsequently, theparameter space was explored exhaustively and the qualityof a segmentation was determined by calculating the Dicecoefficient between the automatic and manual segmentations.Parameter β was varied between 0.1 and 1.0 with steps of0.1, threshh was varied between 0.5 and 15 percentile withsteps of 0.5 percentile and threshl between the current valueof threshh and 30 percentile with steps of 0.5 percentile. Theoptimal parameter settings were determined by maximizing

the average Dice coefficient over all X-ray images and opti-mization was performed for the left and right tree individually.The following parameter values were found to yield the bestsegmentations and used in our experiments. For the left tree:β = 0.7, threshh = 2.5 percentile and threshl = 11.0percentile; for the right tree: β = 0.4, threshh = 1.0 percentileand threshl = 10.5 percentile.

C. Evaluation Methodology

Our evaluation metric was the projection error, whichindicates how well CTA-derived features, such as coronarycenterlines or lesion information, can be projected onto theX-ray images.

We manually annotated those coronary arteries that arepart of the patient-specific 3D+t coronary model in the X-ray images of the cardiac cycle selected for the experiments.The annotated centerlines were equidistantly resampled at 0.2mm. We denote the set of manually annotated centerline pointsin frame k of the X-ray sequence with Xkx

. The error for acertain time point kx was computed as the average centerlinedistance between the registered and projected CTA centerlinesC(µ, ν, tx) and the annotated 2D centerlines Xkx

:

Ek(µ, ν) =1

|Xkx|

∑p∈Xkx

D(p, C (µ, ν, tx)

), (15)

with D(p, C (µ, ν, tx)

)the minimum distance of point p to

the centerlines C (µ, ν, tx). The error for the complete cardiac

7

cycle was computed as the average error over time:

E(µ, ν) =1

nx

nx−1∑kx=0

Ek(µ, ν) . (16)

Furthermore, the maximum of the average centerline distanceover time, was computed as:

Emax(µ, ν) = max(E0(µ, ν), . . . , Enx−1(µ, ν)) . (17)

In the following paragraphs we use E and Emax to com-pare the different temporal alignment and breathing motionapproaches and E to evaluate the accuracy of the proposed3D+t/2D+t method. We use Ekx with kx the time point atwhich the 3D/2D registration was performed when compar-ing the proposed 3D+t/2D+t approach with the conventional3D/2D approach, as for the latter we only know the alignmentfor this specific time point.

For all boxplots in this paper, the whiskers are extendingto the most extreme data point that is not an outlier. Datapoints are considered outliers when their value is larger thanQ3+1.5(Q3-Q1) or smaller than Q1-1.5(Q3-Q1), with Q1 andQ3 the 25th and 75th percentiles (lower and upper extent ofthe box), respectively. The numbers between brackets in theboxplots indicate the number of data points outside the scopeof the y-axis.

D. Evaluation of Temporal and Breathing Motion Modelsusing Synthetic Probable Centerline Point Images

The purpose of the first experiment was to evaluate thetemporal alignment and breathing motion models (SectionII-B and II-C). In this experiment, instead of using the X-rayimages, we generated images with non-zero pixel values at thelocation of the manual annotations, and zero pixel values atall other locations. Herewith, we ensure that the registrationswere not hampered by false positive responses of the X-ray segmentation procedure, allowing a better assessment ofthe effect of the temporal alignment and breathing motionmodeling. For each sequence, registration was performed usingthree resolution levels and a start value for ha of 32 mm inboth the grid and Powell-Brent optimization steps. In eachnext resolution level this value was divided by four, resultingin a value of 2 mm for ha in the final resolution level. Inthe first resolution level of the grid search, the translationparameters were varied from -50 to 50 mm with steps of5 mm. The Powell-Brent optimizer used a maximum of 20iterations, a value tolerance of 0.001, a step tolerance of 0.03and a maximum step length of 0.1.

The accuracy was determined per sequence; a boxplot ofthese error values is shown in Fig. 5. The different columnsshow both the average and maximum error over time usingdifferent combinations of temporal alignment and breathingmotion models.

Numerical values are given in Table III. It lists the medianof the pairwise differences in the error values (E) betweenevery combination of methods (left method - top method). Alldifferences were found to be statistically significant (Wilcoxon

L/C L/L L/Q Q/C Q/L Q/Q0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Method (temporal/breathing)

[1] [1]

Av

era

ge

/ma

xim

um

ce

nte

rlin

e d

ista

nc

e E

/Em

ax (

mm

)

E Em

ax

E Em

ax

E Em

ax

E Em

ax

E Em

ax

E Em

ax

Fig. 5. Comparison of different temporal alignment and breathing motionmodels using synthetic probable centerline point images. The X-axis shows themodels used (temporal L or Q/breathing C, L or Q): (C)onstant (no motion),(L)inear or (Q)uadratic. The first column shows the initial error. The numberbetween brackets shows the number of points outside the scope of the y-axisof the plot.

TABLE IIIMEDIAN OF PAIRWISE DIFFERENCES (LEFT - UPPER METHOD) (MM). ALL

DIFFERENCES WERE STATISTICALLY SIGNIFICANT (WILCOXON SIGNEDRANK TEST, P < 0.05). THE FIRST ROW AND COLUMN SHOW THE MODELSUSED (TEMPORAL/BREATHING): (C)ONSTANT, (L)INEAR, (Q)UADRATIC.

L/C L/L L/Q Q/C Q/L Q/Q

Initial 6.41 6.74 6.86 6.44 6.73 6.76

L/C 0.15 0.23 0.04 0.20 0.25

L/L 0.06 -0.09 0.02 0.09

L/Q -0.19 -0.04 0.02

Q/C 0.13 0.20

Q/L 0.07

signed rank test, p<0.05). The first row shows that the best re-sults are achieved using a linear temporal model and quadraticbreathing motion model, having a median error of 1.12 mm.When fixing the breathing motion model and comparing thetemporal alignment transformation models, the differences arevery small.

E. Registration Accuracy

1) Accuracy 3D+t/2D+t Registration: In this experiment,we assessed the accuracy of the method when using lineartemporal alignment and quadratic breathing motion modeling.We used the real X-ray images and the segmentation approachdescribed in Section II-G. Registration was performed for all26 X-ray sequences. As the registration in this case is influ-enced by false positive responses of the X-ray segmentationapproach, we used two instead of three resolution levels. Thevalue of ha therefore ranged from 8 mm to 2 mm. All otherparameters values were the same as in the previous experiment.

Results are presented in Fig. 6, showing the initial error,the error after optimizing the translations along the tabledirections, and the error after registration, for both coronarytrees and individually for the left and right tree. The firstcolumn of Table IV lists the corresponding median values.

8

Initial Intermed. Final0

2

4

6

8

10

12

14

16

Both

[4] [1] [1]A

vera

ge c

en

terl

ine d

ista

nce E

(m

m)

Initial Intermed. FinalLeft

[2]

Initial Intermed. FinalRight

[2] [1] [1]

Fig. 6. Error of 3D+t/2D+t registration before registration (Initial), after gridsearch optimization (Intermed.) and after registration (Final), for all X-raysequences and for the sequences of the left and right coronary tree separately.

[0,10) [10,20) [20,30) [30,40) [40,50) [50,60) [60,70) [70,80) [80,90) [90,100]0

1

2

3

4

5

6

7

Percentage of cardiac cycle

[5] [3] [3] [3] [3] [2] [3] [3] [4] [6]

Cen

terl

ine d

ista

nce E

kx

(m

m)

Fig. 7. Error of 3D+t/2D+t registration over time. Values are binned oncardiac phase.

Fig. 7 shows the error over time over all X-ray sequencestemporally binned in 10% intervals. Differences between theinitial errors and the errors after registration were statisticallysignificant (Wilcoxon signed rank test, p<0.05). For both trees,the registration improved the alignment considerably; the mostaccurate results were achieved for the left tree, which also hadsmaller initial misalignment. No trends can be observed wheninspecting the error over time. Examples of alignment beforeand after registration are given in Fig. 8.

Movies and error values for all individual X-ray sequencesare available at http://www.bigr.nl/movies/metz2012.

TABLE IVMEDIAN ERROR AFTER INITIALIZATION, AND AFTER 3D+T/2D+T AND

3D/2D REGISTRATION (MM).

All frames Sys./dias. frames

Initial alignment 8.19 8.23

Automatic 3D+t/2D+t initialization 2.88 2.68Automatic 3D+t/2D+t registration 1.47 1.44

Automatic 3D/2D initialization n.a. 2.75Automatic 3D/2D registration n.a. 1.55

3D/2D 3D+t/2D+t0

2

4

6

8

10

Both

[2] [1]

Av

era

ge

ce

nte

rlin

e d

ista

nc

e E

(m

m)

3D/2D 3D+t/2D+tLeft

3D/2D 3D+t/2D+tRight

[2] [1]

Fig. 9. Comparison of proposed 3D+t/2D+t and 3D/2D method. Error valueswere only computed on the time point at which the 3D/2D registration wasperformed. A boxplot of the per-patient error after registration is shown forall X-ray sequences and for the sequences of the left and right coronary treeseparately. The median error before registration (not shown in plot) was 8.23mm, 7.86 mm and 10.10 mm for both trees and the individual left and righttree, respectively.

2) Comparison 3D+t/2D+t and 3D/2D Registration: Thegoal of this experiment is to evaluate the added value of in-corporating the time dimension into the registration procedure.Next to the 3D+t/2D+t registrations, we performed 3D/2Dregistrations at both 33% and 66% of the cardiac cycle andoptimizing six degrees of freedom. Results are shown in aboxplot in Fig. 9 for both coronary trees and individually forthe left and right tree. The second column of Table IV lists thecorresponding median values. When considering both trees,the median error is 0.11 mm lower for the 3D+t/2D+t method,but the differences were not statistically significant (Wilcoxonsigned rank test, p<0.05). However, the lower third quartilevalues for the 3D+t/2D+t approach indicate more successfullregistrations for the proposed approach.

F. Capture Range and Registration Robustness

1) Capture Range 3D+t/2D+t Registration: In this exper-iment, we investigated how the precision of the first step ofthe optimization procedure, i.e. the grid search for the optimaltranslation parameter values along the table directions, influ-ences the performance of the proposed 3D+t/2D+t registrationapproach. To this end, we generated 40 random initializationsof the model by randomly perturbing the resulting two param-eter values of the grid search optimization results of SectionIII-E1. All other parameter values were kept at zero. Therandom values were determined by drawing a distance valuebetween 0 and 20 mm and an angle value between 0 and 360degrees from uniform distributions. The x and z parameterperturbations were derived from this distance and angle valueand added to the parameter values as resulting from the gridsearch optimization step. The distance values were drawn in abinned fashion using a bin size of 2 mm, making sure that forevery patient 4 different initializations per bin were generated.For every initialization we then performed the Powell-Brentoptimization step to align the dynamic coronary model withthe X-ray images.

9

0% 25% 50% 75%

Initi

aliz

atio

n3D

+t/2

D+t

regi

stra

tion

Initi

aliz

atio

n3D

+t/2

D+t

regi

stra

tion

Initi

aliz

atio

n3D

+t/2

D+t

regi

stra

tion

Fig. 8. Examples of registration results. CTA centerlines are projected on top of the X-ray images. From left to right, the image at 0%, 25%, 50% and 75%of the cardiac cycle is shown. The first, third and fifth row show the alignment after manual initialization; the second, fourth and last row the alignment after3D+t/2D+t registration. Movies, showing the results for all sequences can be found at http://www.bigr.nl/movies/metz2012/.

10

0 2 4 6 8 10 12 14 16 18 20

2

4

6

8

10

12

14

16

18

20

2D Euclidian distance from grid initialization parameters (mm)

Av

era

ge

ce

nte

rlin

e d

ista

nc

e E

(m

m)

10

20

30

40

50

60

70

80

90

100

Pe

rce

nta

ge

of re

gis

tratio

ns

ha

vin

g E

< 2

mm

.

Fig. 10. Robustness with respect to position of the model along the tabledirections. At 0 mm, the model is positioned according to the resuls of the gridoptimization step performed in Section III-E1. The red dashed line indicatesthe threshold used to define the success rate. The two black lines show thesuccessrate of the method: the dotted line is for all registrations (grey crossesand black circles), the solid one for those registrations that were successfulin Section III-E1 (only black circles).

Figure 10 shows the results of this experiment. The x-axisof the scatter plot shows the 2D Euclidean distance betweenthe original and perturbed initialization position and the left y-axis the average centerline distance after registration. The twoblack lines show the percentage of successful registrations perbin (right y-axis). A registration was defined to be successful ifthe error after registration was smaller than 2 mm (red dashedline). The dotted line shows this percentage when consideringall registrations (both the results that are represented by thegrey crosses and the black circles). The solid line showsthis percentage when only considering those registrations thatwere successful for the unperturbed registrations as resultingfrom the experiment described in Section III-E1 (only resultsrepresented by black circles). It can be seen that the successrate of the registration method decreases approximately lin-early with increasing perturbation magnitude. Up to 4 mmperturbations along the table, most registrations that wereinitially successful, remain successful.

2) Comparison of Robustness Between 3D+t/2D+t and3D/2D Registration: The purpose of this experiment wasto compare the robustness of the 3D+t/2D+t and 3D/2Dmethod against the initial positioning of the coronary motionmodel with respect to the intraoperative imaging system (bothtranslation orthogonal to table and initial rotations of themodel). To this end, we generated 20 random initializationsper patient, 10 at 33% and 10 at 66% of the cardiac cycle, bysimultaneously perturbing the translation parameter orthogonalto the table (y) and the three rotation parameters (α, β and γ)from their initial values. Random values were sampled froma uniform distribution between -5 and 5 mm and between -10 and 10 degrees. For the two translation parameters alongthe table directions, the grid search optimization results ofSection III-E1 were used. The 6 resulting rigid parameterinitializations were used for the complete cardiac cycle, thusignoring any possible breathing motion in the initialization

x y z alpha beta gamma

0

5

10

15

Parameter

[2] [5] [2] [3] [1] [3] [2]

Std

. p

er

ph

as

e a

nd

pa

tie

nt

(mm

or

de

gre

es

)

Fig. 11. Boxplot of the standard deviations of resulting parameter values forthe 3D/2D approach (left) and 3D+t/2D+t approach (right) and ten differentinitializations. Results are given for x, y and z translation, and α, β, γ rotation.

phase. The temporal parameters were all initialized to zero.Registration was performed as described in III-E2 using all

these random initializations. From each individual registrationresult, the rotation and translation of the coronary model atthe initialization phase (33% or 66% of the cardiac cycle)was determined. In the 3D/2D case, these values are directlyresulting from the optimization procedure; in the 3D+t/2D+tcase, these values were interpolated using (5). In the ideal case,all random initializations lead to the same end results, and thetranslation and rotation values should therefore be identicalper cardiac phase. To derive a measure for the spread of theparameter values, we computed the standard deviation of eachset of 10 values. Figure 11 shows a boxplot of these standarddeviation values for both methods. Standard deviations of thetwo approaches were statistically significantly different for allparameters (Wilcoxon signed rank test, p<0.05).

G. Running Time of 3D+t/2D+t Registration

The running time for the probable centerline points detec-tion in the X-ray images was approximately 1.25 seconds perX-ray frame on a single core of an Intel Xeon E5520 2.27Ghz CPU. The running time of the 3D+t/2D+t optimizationprocedure (as performed in Section III-E1) ranged from 3.5to 15.9 seconds (mean±std: 9.0±3.3 s) on a GeForce 9800GTX (128 CUDA cores), and was approximately linear in thetotal number of probable X-ray centerline points over all X-ray frames. Running times for the individual sequences can befound online (http://www.bigr.nl/movies/metz2012/).

IV. DISCUSSION

The proposed method was shown to be successful for full-cycle alignment of preoperative coronary CTA and intraoper-ative X-ray images. A median error of 1.47 mm was achievedand results were in general best for the left coronary tree(Section III-E1). The error values after performing the firststep of the optimization were already much smaller than afterinitialization, which is largely owing to the cardiac motionmodel used and applying the initial rotations of the C-armsystem.

11

We focused on concurrently aligning one complete cardiaccycle, and in the experiments we manually selected one cycleof interest. In the clinical situation, we would distinguishbetween two situations: i) initial alignment using one completecardiac cycle, and ii) maintaining alignment during the rest ofthe imaging session. Regarding both, the ECG signal couldbe used to automatically determine the phase of the heart.With respect to ii), we foresee the use of a sliding windowapproach, in which incoming images replace the previouslyused images. The rigid parameters that model the patientand breathing motion could be automatically initialized byinterpolation and extrapolation of the previous results andthe temporal transformation can gradually adopt to heart ratevariations. The initial and sliding window sizes could inprinciple be chosen freely, but are limited in size, due to thetemporal transformation and breathing motion models used.Implementing this clinical approach is subject of future work.

Currently, for creating the dynamic coronary model, user-interaction is required to define the start and end points of thecoronary arteries, and in some cases to correct the estimatedmotion for the right coronary artery. The time required forthese interactions is limited (only a few minutes) and doesnot limit clinically applicability, as these models are createdpre-operatively.

Two transformations were involved in the registration pro-cedure: a temporal transformation, and a time-dependent rigidspatial transformation. We showed that, when using an ap-propriate breathing model, the differences between a linearand quadratic temporal transformation were very small, anda linear model with less degrees of freedom may thereforebe preferred. For modeling the time-dependency of the rigidtransformation, and thus the breathing motion, a quadraticfunction yielded the most accurate results (Section III-D). Thisis in line with our expectations, as it is better able to adapt tothe sinusoidal shape of the respiratory cycle. The differencesin the average errors of the different breathing motion modelswere small. Breathing motion is, however, often only affectingthe alignment in parts of the cardiac cycle and the differencesin the maximum errors over time were therefore larger (Figure5). Furthermore, not all sequences exhibit breathing motion, orneed the highest order motion model to accurately compensateit.

The optimization procedure consists of two steps, a gridsearch to initialize the translation parameters along the tabledirection and a Powell-Brent optimization of all parameters.We investigated the influence of the results of the first op-timization step on the performance of the method, whichindicates the precision with which the grid search needs tobe performed. Up to an initial Euclidean distance of 4 mmfrom the optimal initialization point, no large effect wasdemonstrated.

Comparing the accuracy of the 3D+t/2D+t method to a3D/2D method, we saw lower third quartile values for theproposed method (Fig. 9). We also investigated the robustnessof both approaches with respect to variations in the initial valueof the translation parameter orthogonal to the table, and thethree rotation parameters, which are automatically initializedusing the center of mass of the coronary centerline model

and the C-arm rotation parameters derived from the DICOMheaders of the imaging data. This showed better robustnessfor the proposed method, indicating more reliable results andshowing the benefit of incorporating the time dimension intothe registration procedure.

The running time of the current implementation of thealgorithm is on average 9.0 seconds and linear in the numberof probable centerline points detected in the X-ray images.As the current implementation is parallelized over the X-raycenterline points, a 12 fold speed increase should be possibleusing an up-to-date GPU (1526 CUDA cores). A strategyto decrease the running time further, could be to randomlyselect a subset of probable centerline points to use in theoptimization approach or to increase the sampling distanceof the centerline model. The running time of the single-coreCPU implementation of the probable centerline point detectionwas approximately 1.25 seconds per X-ray frame. Makingthis detection real-time should also be possible using GPUacceleration.

It is our experience that, first, the quality of the dynamiccoronary model is important. The 3D+t CTA images shouldideally be free from breathing and irregular heart beat artifacts,which was not the case for one of our images. Furthermore,a reasonably small slice thickness (i.e. in the order of thez-resolution of the imaging system) should be used in thereconstruction procedure. For some of our images the slicethickness was 3 mm, which makes centerline extraction andmotion estimation very challenging. Second, the orientationused to generate the X-ray images should ideally minimizeambiguity with respect to the alignment procedure. Whichconfigurations yield the best results remains to be investigated.Third, in case of complete occlusions, fully automaticallyregistering the right tree turned out to be extra challenging, dueto the small number of vessel branches. Occlusion often leadsto a very limited visibility of the vessels, which introducesambiguity in the first step of the optimization approach. Inthose cases it may be beneficial to initialize the registrationmanually, or to explicitly track the guide catheter in the X-ray images and initialize the model using the catheter tip.Fourth, the robustness of the method can possibly improvewhen minimizing false positive responses in the detection ofprobable centerline points in the X-ray images. Currently, adifference of 0.35 mm in the median error of the experimentusing synthetic probable centerline images and the experimentusing the real X-ray images, was observed. Explicit coronarycenterline extraction, as for example proposed by Schneiderand Sundar [34], might therefore be beneficial. Finally, differ-ences between the preoperative and intraoperative situation re-main, due to, for example, nonrigid breathing induced cardiacmotion or the use of stiff guide wires. These differences canpotentially be addressed by including an additional nonrigidstep in the registration procedure, for example, by employingone of the methods proposed by Groher et al. [22] or Rivest-Henault et al. [20]. However, it remains to be investigated towhat extent these approaches can be applied for monoplanecoronary X-ray imaging.

Our registration approach currently depends on 3D+t CTAdata. Although these scans were acquired using ECG-based

12

dose modulation, lowering the dose to 5% of its maximumvalue outside the end-diastolic phase, the trend is towardsprospective ECG-gating techniques. Even though only a 3Dscan will be available in this case, the proposed method canstill be applied. The patient-specific prior could, for example,be replaced by population based motion models, such as themean cardiac motion over a population or a shape-dependentmotion estimate [35], [36].

In the current work, we evaluated the projection error ofthe registered dynamic coronary models. This metric indicateshow accurately CTA-derived information can be projectedon top of the X-ray images. We considered including realpoint correspondences in the metric, but found the numberof landmarks that can be sufficiently reliably annotated in theX-ray images to be too small due to overprojection issues atbifurcation points. The error metric may therefore be slightlyoptimistic, as it does not take into account misalignment alongthe centerline direction. However, we expect this effect to besmall, since moving the model along the vessel direction willcause misalignment at some locations, due to the curvednessof the vessels. To give the reader complete insight in theaccurateness of the method, we published movies, individualerror values and running times for all registration results online(http://www.bigr.nl/movies/metz2012/)

For integrated 3D visualization, in which the position of theguide wire is visualized with respect to the CTA image, 3Devaluation is necessary. This can for example be achieved bythe application of biplane X-ray data. While registering on oneof the X-ray views, the projection error can be evaluated inboth X-ray views, providing an indication of the 3D accuracy.A prerequisite for this approach is the availability of biplane X-ray data and accurate calibration data, which we are planningto collect for future research.

V. CONCLUSIONS

We investigated the alignment of preoperative 3D+t coro-nary CTA with intraoperative monoplane 2D+t X-ray an-giography images. Our approach is based on the derivationof patient-specific dynamic coronary models from the CTAscan and multiple time points are taken into account inthe registration procedure. This approach enables full-cyclealignment of the preoperative and intraoperative imaging data.Various temporal alignment and breathing motion modelingapproaches were investigated, and quantitatively compared.Best results were obtained using quadratic breathing motionmodeling in combination with linear temporal alignment. Onreal X-ray data, we showed a projection error of 1.47 mm,more accurate and especially more robust results compared toa gated 3D/2D approach.

ACKNOWLEDGMENT

This work was financially supported by ITEA project 09039,Mediate.

REFERENCES

[1] K. Tsuchida, A. Colombo, T. Lefèvre, K. G. Oldroyd, V. Guetta,G. Guagliumi, W. von Scheidt, W. Ruzyllo, C. W. Hamm, M. Bressers,H.-P. Stoll, K. Wittebols, D. J. Donohoe, and P. W. Serruys, “TheClinical Outcome of Percutaneous Treatment of Bifurcation Lesions inMultivessel Coronary Artery Disease With the Sirolimus-Eluting Stent:Insights From the Arterial Revascularization Therapies Study Part II(ARTS II).” Eur. Heart J., vol. 28, no. 4, pp. 433–442, 2007.

[2] A. Hoye, R. T. van Domburg, K. Sonnenschein, and P. W. Serruys,“Percutaneous Coronary Intervention for Chronic Total Occlusions: TheThoraxcenter Experience 1992-2002.” Eur. Heart J., vol. 26, no. 24, pp.2630–2636, 2005.

[3] M. Magro, C. Schultz, C. Simsek, H. M. Garcia-Garcia, E. Regar,K. Nieman, N. Mollet, P. W. Serruys, and R.-J. van Geuns, “Computedtomography as a tool for percutaneous coronary intervention of chronictotal occlusions.” EuroIntervention, vol. 6 Suppl G, pp. G123–G131,2010.

[4] S. Ramcharitar, M. S. Patterson, R. J. van Geuns, C. van Meighem, andP. W. Serruys, “Technology Insight: Magnetic Navigation in CoronaryInterventions.” Nat. Clin. Pract. Cardiovasc. Med., vol. 5, no. 3, pp.148–156, 2008.

[5] G. Shechter, C. Ozturk, J. R. Resar, and E. R. McVeigh, “RespiratoryMotion of the Heart From Free Breathing Coronary Angiograms.” IEEETrans. Med. Imaging, vol. 23, no. 8, pp. 1046–1056, 2004.

[6] G. Shechter, J. R. Resar, and E. R. McVeigh, “Displacement and Velocityof the Coronary Arteries: Cardiac and Respiratory Motion.” IEEE Trans.Med. Imaging, vol. 25, no. 3, pp. 369–375, 2006.

[7] P. Markelj, D. Tomaževic, B. Likar, and F. Pernuš, “A Review of3D/2D Registration Methods for Image-Guided Interventions.” Med.Image Anal., vol. 16, no. 3, pp. 642–661, 2012.

[8] E. Bullitt, A. Liu, S. R. Aylward, C. Coffey, J. Stone, S. K. Mukherji,K. E. Muller, and S. M. Pizer, “Registration of 3D Cerebral VesselsWith 2D Digital Angiograms: Clinical Evaluation.” Acad. Radiol., vol. 6,no. 9, pp. 539–546, 1999.

[9] J. Hipwell, G. Penney, R. McLaughlin, K. Rhode, P. Summers, T. Cox,J. Byrne, J. Noble, and D. Hawkes, “Intensity-Based 2-D-3-D Regis-tration of Cerebral Angiograms.” IEEE Trans. Med. Imaging, vol. 22,no. 11, pp. 1417–1426, 2003.

[10] R. A. McLaughlin, J. Hipwell, D. J. Hawkes, J. A. Noble, J. V. Byrne,and T. C. Cox, “A Comparison of a Similarity-Based and a Feature-Based 2-D-3-D Registration Method for Neurointerventional Use.” IEEETrans. Med. Imaging, vol. 24, no. 8, pp. 1058–1066, 2005.

[11] M. Groher, T. F. Jakobs, N. Padoy, and N. Navab, “Planning and In-traoperative Visualization of Liver Catheterizations: New CTA Protocoland 2D-3D Registration Method.” Acad. Radiol., vol. 14, no. 11, pp.1325–1340, 2007.

[12] H. Imamura, N. Ida, N. Sugimoto, S. Eiho, S. i. Urayama, K. Ueno,and K. Inoue, “Registration of Preoperative CTA and IntraoperativeFluoroscopic Images for Assisting Aortic Stent Grafting,” in Proc. Int.Conf. Med. Image Comput. Comput. Assist. Interv., ser. Lect. NotesComput. Sci., vol. 2489, 2002, pp. 477–484.

[13] X. Chen, R. Gilkeson, and B. Fei, “Automatic 3D-To-2D Registration forCT and Dual-Energy Digital Radiography for Calcification Detection.”Med. Phys., vol. 34, no. 12, pp. 4934–4943, 2007.

[14] G.-A. Turgeon, G. Lehmann, G. Guiraudon, M. Drangova,D. Holdsworth, and T. Peters, “2D-3D Registration of CoronaryAngiograms for Cardiac Procedure Planning and Guidance.” Med.Phys., vol. 32, no. 12, pp. 3737–3749, 2005.

[15] D. Ruijters, B. M. ter Haar Romeny, and P. Suetens, “Vesselness-Based2D-3D Registration of the Coronary Arteries.” Int. J. Comput. Assist.Radiol. Surg., vol. 4, no. 4, pp. 391–397, 2009.

[16] C. Metz, M. Schaap, S. Klein, A. Weustink, N. Mollet, C. Schulz,R. Geuns, P. Serruys, T. van Walsum, and W. Niessen, “GPU AcceleratedAlignment of 3-D CTA With 2-D X-Ray Data for Improved Guidancein Coronary Interventions,” in Proc. IEEE Int. Symp. Biomed. Imaging,2009, pp. 959–962.

[17] M. Figl, D. Rueckert, D. Hawkes, R. Casula, M. Hu, O. Pedro, D. P.Zhang, G. Penney, F. Bello, and P. Edwards, “Image Guidance ForRobotic Minimally Invasive Coronary Artery Bypass.” Comput. Med.Imaging Graph., vol. 34, no. 1, pp. 61–68, 2010.

[18] C. Gatta, S. Baloccoa, V. Martin-Yuste, R. Leta, and P. Radevab, “Non-rigid Multi-modal Registration of Coronary Arteries Using SIFTflow,”in Proc. Pat Recogn and Image Anal, 2011, pp. 159–166.

[19] E. Serradell, A. Romero, R. Leta, C. Gatta, and F. Moreno-Noguer,“Simultaneous correspondence and non-rigid 3D reconstruction of the

13

coronary tree from single X-ray images,” in Proc. Int. Conf. Comput.Vision, 2011.

[20] D. Rivest-Henault, H. Sundar, and M. Cheriet, “Non-rigid 2D/3Dregistration of coronary artery models with live fluoroscopy for guidanceof cardiac interventions.” IEEE Trans Med Imaging, vol. 31, no. 8, pp.1557–1572, 2012.

[21] J. Jomier, E. Bullitt, M. V. Horn, C. Pathak, and S. R. Aylward, “3D/2DModel-to-Image Registration Applied to TIPS Surgery.” in Proc. Int.Conf. Med. Image Comput. Comput. Assist. Interv., ser. Lect. NotesComput. Sci., vol. 3023, 2006, pp. 662–669.

[22] M. Groher, D. Zikic, and N. Navab, “Deformable 2D-3D Registrationof Vascular Structures in a One View Scenario.” IEEE Trans. Med.Imaging, vol. 28, no. 6, pp. 847–860, 2009.

[23] C. Metz, M. Schaap, S. Klein, L. Neefjes, E. Capuano, C. Schultz, R. vanGeuns P.W. Serruys T. van Walsum, and W. Niessen, “Patient Specific4D Coronary Models From ECG-gated CTA Data for Intra-operativeDynamic Alignment of CTA With X-ray Images,” in Proc. Int. Conf.Med. Image Comput. Comput. Assist. Interv., ser. Lect. Notes Comput.Sci., vol. 5761, 2009, pp. 369–376.

[24] C. Metz, M. Schaap, S. Klein, P. Rijnbeek, L. Neefjes, N. Mollet,C. Schultz, P. Serruys, W. Niessen, and T. van Walsum, “Alignmentof 4D Coronary CTA with Monoplane X-Ray Angiography,” in Proc.MICCAI workshop: Augmented Environments for Computer AssistedInterventions, 2011, pp. 106–116.

[25] H. Bazett, “An Analysis of the Time Relationships of Electrocardio-grams,” Heart, vol. 7, pp. 353–370, 1920.

[26] T. Yoo, M. Ackerman, W. Lorensen, W. Schroeder, V. Chalana, S. Ayl-ward, D. Metaxes, and R. Whitaker, “Engineering and Algorithm Designfor an Image Processing API: A Technical Report on ITK - The InsightToolkit,” 2002.

[27] M. Powell, “An efficient method for finding the minimum of a functionof several variables without calculating derivatives,” The computerjournal, vol. 7, no. 2, pp. 155–162, 1964.

[28] W. Press, B. Flannery, S. Teukolsky, and W. Vetterling, Numericalrecipes. Cambridge university press, 2007.

[29] M. Schaap, T. van Walsum, L. Neefjes, C. Metz, E. Capuano,M. de Bruijne, and W. Niessen, “Robust shape regression for supervisedvessel segmentation and its application to coronary segmentation inCTA.” IEEE Trans Med Imaging, vol. 30, no. 11, pp. 1974–1986, 2011.

[30] C. T. Metz, S. Klein, M. Schaap, T. van Walsum, and W. J. Niessen,“Nonrigid registration of dynamic medical imaging data using nD + tB-splines and a groupwise optimization approach.” Med Image Anal,vol. 15, no. 2, pp. 238–249, 2011.

[31] H. Kirisli, M. Schaap, S. Klein, L. Neefjes, A. Weustink, T. van Walsum,and W. Niessen, “Fully Automatic Cardiac Segmentation From 3D CTAData: A Multiatlas Based Approach,” in Proc. SPIE Med Imaging: ImageProcess., 2010, pp. 762 305–1–762 305–9.

[32] A. Frangi, W. Niessen, K. Vincken, and M. Viergever, “MultiscaleVessel Enhancement Filtering,” in Proc. Int. Conf. Med. Image Comput.Comput. Assist. Interv., ser. Lect. Notes Comput. Sci., vol. 1496, 1998,pp. 130–137.

[33] MeVisLab, “http://www.mevislab.de,” Software for Medical Image Pro-cessing and Visualization.

[34] M. Schneider and H. Sundar, “Automatic global vessel segmentationand catheter removal using local geometry information and vector fieldintegration,” in Proc. IEEE Int. Symp. Biomed. Imaging, 2010, pp. 45–48.

[35] C. T. Metz, N. Baka, H. Kirisli, M. Schaap, S. Klein, L. A. Neefjes,N. R. Mollet, B. Lelieveldt, M. de Bruijne, W. J. Niessen, and T. vanWalsum, “Regression-based cardiac motion prediction from single-phaseCTA.” IEEE Trans Med Imaging, vol. 31, no. 6, pp. 1311–1325, 2012.

[36] N. Baka, C. Metz, C. Schultz, L. Neefjes, R. van Geuns, B. Lelieveldt,W. Niessen, M. de Bruijne, and T. van Walsum, “3D+t/2D+t CTA-XA registration using population-based motion estimates,” in MICCAI-Workshop on Computer Assisted Stenting Proceedings, MICCAI-STENT’12, 2012.