Co-registration framework for histology-registration-based validation of fused multimodality prostate cancer imaging

Eli Gibson1*, Cathie Crukley1;2, Vaishali Karnik1, Mena Gaed1;2, José Goméz3, Madeleine Moussa3,Joseph L. Chin4, Glenn Bauman2;5, Aaron Fenster1;2;5;6, and Aaron D. Ward2;6

1Robarts Research Institute; 2Lawson Health Research Institute; Departments of 3Pathology, 4Urology, 5Oncology, and 6Medical Biophysics,The University of Western Ontario

London, Canadae-mail: egibson@imaging.robarts.ca

Abstract—The correlation of multiple in vivo imaging modalities with graded whole-mount histology ideally requires co-registration with appropriately quantified accuracy. This work presents a framework for the co-registration of multiple in vivo prostate images with histology as a composition of pairwise registrations. This work also proposes a methodology for the measurement of the accuracy of these registrationsbased on a composition of target registration errors calculated from homologous landmarks in registered pairs of images.Preliminary results from the implementation of components of the framework are reported.

Keywords - prostate cancer, registration, image fusion, validation

I. INTRODUCTION

Prostate cancer is the most common non-cutaneous cancer in North American men. While sensitive screening tests are in place in many regions, including Canada, USA, France and Austria (1), it has been estimated that 50% of patients in screened populations that are diagnosed with cancer have indolent cancers (2); these non-aggressive cancers grow so slowly that there is no clinical need to treat them. Since cancer treatment has substantial negative side effects, the ability to identify these indolent cancers before treatment is of great clinical interest. Furthermore, while the multi-focal nature of prostate cancer has been well described; the majority of cancers are characterized by a dominant focus that is the key driver of the aggressiveness of the cancer (3). Thus, focal therapy (treatment of cancer foci which spares healthy prostate tissue) may be feasible if these aggressive cancer foci can be located and ablated.

Aggressiveness of prostate cancer foci is currentlyassessed using the Gleason grade, an aggressiveness marker,from histology of biopsy samples. Unfortunately, this technique cannot reliably identify cancer grade, with over-and underestimation of Gleason grade in 40% of patients (4).Several imaging modalities have shown promise for grading prostate cancer. The apparent diffusion coefficient computed from diffusion weighted (DW) magnetic resonance imaging (MRI) is negatively correlated with Gleason grade (5).Elevated choline signal and (choline+creatine)/citrate ratio computed from MR spectroscopic imaging (MSRI) arepositively correlated with histologic grade (6,7). However, itis not yet well understood how to reliably grade prostate cancer from these images. Other imaging modalities, such as dynamic contrast enhanced (DCE) MRI and 18F-choline

positron emission tomography (PET), have been shown to correlate with prostate cancer (8,9), but no relationshipbetween imaging and Gleason grade has yet been identified.In contrast, Gleason grade can be reliably assessed on whole-mount histopathology of resected prostate specimens.

Thus, grading of prostate cancer from in vivo images can be validated by registering images to graded whole-mount prostate histology. In order to directly compare multiple modalities and to look at combinations of modalities for prostate cancer grading, each subject must undergo multiple imaging sessions, and the resulting images must all be co-registered with graded prostate histology images.

This work presents a framework for the co-registration ofin vivo imaging modalities with graded histology: T2-weighted MRI, DW MRI, DCE MRI, functional maps derived from DCE computed tomography (CT), 18F-choline PET, and radiofrequency (RF) time series transrectal ultrasound (TRUS). A graph of pairwise registrations was constructed allowing any two images to be registered by composition of multiple registrations, and a methodology for assessing the accuracy of the co-registration was defined.

II. MATERIALS AND METHODS

A. MaterialsIn this study, 66 patients scheduled for radical

prostatectomy have undergone (N=28) or will undergo(N=38) a series of in vivo imaging sessions.

B. ImagingA subset of acquired images is shown in Fig. 1. Imaging

modalities to be validated and additional images to facilitate registration were collected over several imaging sessions:

In vivo MRI without endorectal coil: T2W images were acquired on a Discovery MR750 (GE Healthcare, Waukesha, WI, USA) at 3T using either a torso coil (GE Healthcare) or a cardiac coil (USA Instruments, Aurora, OH, USA).

In vivo MRI with endorectal coil: Images were acquiredin the same session as above after repositioning, using an endorectal (ER) receive coil (Prostate eCoil, Medrad, Inc., Warrendale, PA, USA) filled with a barium sulfate suspension (9). Acquisitions included MRSI, T2W, T1WDCE and DW MRI protocols.

PET/CT: PET/CT images were acquired on a Discovery LS (GE Healthcare) using an 18F-choline tracer and dual phase protocol incorporating a time series of PET images immediately after injection and a delayed total body scan.

2011 International Conference on Intelligent Computation and Bio-Medical Instrumentation

978-0-7695-4623-0/11 $26.00 © 2011 IEEE

DOI 10.1109/ICBMI.2011.51

93

DCE CT: Images were acquired on a HD750 VCT (GE Healthcare) at 120 kVp and 125 mA, comprising a time series of images before and after contrast injection. Functional maps, including blood flow, blood volume, vascular permeability, and mean transit time were computed from these images (11).

3D Ultrasound: Images were acquired on a SonixTouch with a BPL9-5/55 side-firing transrectal probe (Ultrasonix,Richmond, BC, Canada), and comprised B-mode images and a time series of RF image acquisitions.

Ex vivo MRI: Images were acquired on a Discovery MR750 using an unfilled ER coil placed posterior to the specimen and high resolution T1W and T2W protocols.

Histology: After resection, the specimens were marked to facilitate registration (12), sliced into 4.4 mm tissue blocks and processed for standard whole-mount histopathology. A 4µm thick section was cut from each tissue block, stained with hematoxylin and eosin (H&E), and digitized on a ScanScope GL (Aperio Technologies, Vista, CA, USA) bright field slide scanning system with a 0.5 µm pixel size.

A subset of these modalities were collected for facilitating registration and measuring registration accuracy:ex vivo MRI, in vivo MRI without ER coil, B-mode ultrasound, and the CT imaging of PET/CT.

All in vivo MRI was performed in one session 1-6 weeksbefore surgery. Ultrasound imaging was performed in one session on the same day as in vivo MR imaging. PET/CT and diffusion CT were each performed in one session on the same day, 1-6 weeks before surgery. Ex vivo MRI was performed 2-10 days after surgery. Histological processing was performed 2-5 days after ex vivo MRI. Due to changes in protocol, some imaging modalities were not collected for the early subjects: RF time series (21/66), PET/CT (27/66), in vivo MRI without ER coil (14/66).

C. RegistrationsCo-registration of multiple modalities requires

comparison of images whose image content represents

different tissue properties, and compensation of distortions from multiple sources. At the same time, the number of degrees of freedom in such co-registrations is high, makingoptimization challenging. To mitigate these challenges, intermediate registrations between pairs of more similar (in terms of image content and/or deformations) images can be composed to yield co-registration of dissimilar images.

A registration graph (Fig. 2) was constructed such that each vertex is a set of images taken without repositioning the subject and each edge is a pairwise registration. Images taken without repositioning the subject were taken to be co-registered. Any image pair can be registered via the composed registrations along the path between the images.

Only a subset of possible intermediate registrations is necessary to ensure a path between any two images. Intermediate registrations used in the registration graph were selected for the following properties:1. source and target images have low expected deformation 2. results are independent of image appearance due to

anatomical variation 3. source and target images have similar image content

Each intermediate registration may use different similarity metrics and different classes of deformation, because the image content and expected deformations between image pairs vary. Expected distortions and appropriate transformation classes are summarized in Table 1. Assumptions underlying each of the chosen intermediate registrations are outlined in the paragraphs below.

Histology to ex vivo MRI: These imaging modalities are deformed relative to each other due to physical and chemical treatments during the histological processing, and have very different imaging content, with MR images depicting magnetic resonance of protons, and histology imagesdepicting chemical affinity for H&E stains. However, because both modalities are ex vivo imaging, non-anatomical extrinsic fiducial markers can be added that enable a fiducial-based registration that is independent of the appearance of prostate tissue on the images. This registration component has been developed and validated (13).

Ex vivo MRI to in vivo MRI without ER coil: These imaging modalities are deformed relative to each other due to the effects of resection and tissue shrinkage due to formalin fixation. Preliminary data suggest that large anisotropic or non-linear deformations are not necessary at this stage, although this has not been formally investigated. Due to formalin-induced cross-linking of proteins, the soft tissue contrast of the ex vivo image is reduced; however, many anatomical structures are visible in both modalities.

In vivo MRI without ER coil to with ER coil: Due to the pressure of the filled ER coil on the posterior wall of the prostate, ER coil images have substantial non-linear deformation relative to images acquired without the coil.However, the T2W images from the two sessions contain similar underlying image content without and with the ERcoil, although resolutions and noise levels differ.

In vivo MRI with ER coil to ultrasound: Because both in vivo MRI with ER coil and transrectal ultrasound have endorectal pressure on the posterior wall, we suspect distortion will be smaller than between ultrasound and other

Figure 1. A subset of images collected in this study: (a) 3D TRUS, (b) histology, (c) T2W in vivo MRI with ER coil, (d) ex vivo MRI

a bc

c d

94

modalities, although this has not been formally investigated. The image content of these modalities differs substantially, although prostate boundaries and some internal anatomy can be distinguished on both modalities.

In vivo MRI without ER coil to DCE CT: Because neither of these modalities have endorectal equipment, the expected deformations are smaller than other registrations considered, and must only compensate for patient pose and inter-session prostate motion and deformation. The image content of these modalities differs substantially, although prostate boundaries can be distinguished on these modalities.

DCE CT to PET/CT: Structural CT images with similar image content are available for both imaging sessions. As in the in vivo MRI to DCE CT registration above, the deformation must only compensate for patient pose and inter-session prostate motion and deformation.

D. Accuracy quantificationThe accuracy of registration directly impacts the analyses

that can be performed on the data and the conclusions that can be drawn about the correlation between grade on histology and on in vivo imaging. Thus, quantified accuracy measurement is a critical component of the framework.

The accuracy of registrations in this framework can be calculated as the target registration error (TRE), the post-registration misalignment between homologous anatomical points on imaging modalities. Previous work (13) has shown that homologous landmarks, specifically 1 mm diameter ducts and cysts, can be identified on histology and ex vivoMR imaging. Based on preliminary inspection of imaging modalities used in this framework, calcifications that can be identified on B-mode US, in vivo MR and CT images also show promise for accuracy assessments. Where no point landmarks are identifiable, artificial point landmarks can be defined as centroids of larger structures.

Ideally, identical homologous points would be identified on all modalities; however, because different imaging modalities show different underlying tissue properties, this is not generally possible. An alternative is to evaluate theregistration accuracy of each modality to histology images,using pairs of homologous points identified on image pairs.However, due to the small volume of tissue in histology sections, differences in resolution, and different image content, there may be few or no identifiable homologous features for some pairs. A third alternative is to evaluate registration error between pairs of images that are registered to each other, and to combine these error measurements. If the TRE of two composed registrations are independent, we can combine the error in quadrature. In this framework, we propose to evaluate the registration using a combination of the two latter methods: where pairs of homologous landmarks are identifiable in images from histology and another imaging modality, such as in histology and ex vivo

MR imaging, the registration error will be evaluated directly; where no such homologous landmark pairs are identifiable, such as in histology and PET images, the registration error will be evaluated as the combination in quadrature of errors along the shortest path in the registration graph (Fig. 2) to a modality where TRE can be directly evaluated.

III. PRELIMINARY RESULTS

Localization of homologous landmarks has been performed for two intermediate registrations: for histology to ex vivo MR images, 3-7 homologous landmark pairs per histological section for 3-5 sections per specimen from 9 specimens (total 184 landmarks, 34 sections) were identified; for in vivo MR images without ER coil to those with ER coil, 5-11 homologous landmark pairs per specimen from 11specimens (total 81 homologous landmarks) were identified.

Initial registrations of histology to ex vivo MR images have been performed. The mean±std TRE was found to be 0.71±0.38. Fig. 3 shows one result of these registrations. (14)

IV. DISCUSSION AND CONCLUSIONS

This work proposes a framework for the co-registration of multiple in vivo imaging modalities with digital histology for prostate cancer imaging validation, outlining the registrations necessary to correct for the expected deformations, and the validation needed.

The implementation of this framework is ongoing work in our laboratory; however, preliminary results are available showing sub-millimeter accuracy in the registration of digital histology images to ex vivo MRI.

One limitation of this framework is that the registration graph is constructed based on assumptions and preliminary investigations. Further work is needed to validate these assumptions. In particular, verifying the consistent presenceof homologous landmarks in different pairs of modalities is critical for the validation of the resulting registrations.

Another limitation of this framework is that it focuses on independent pairwise registrations. An alternative that has shown promise (15) is to construct a fused vector-valued image from co-registered images after each intermediate registration, and use these fused images in subsequent intermediate registrations. For example, in the registration to DCE CT, the post-registration fusion of in vivo MR images with and without the ER coil would have common image content (contrast diffusion information from DCE CT and DCE MR) but due to co-registration would not have the

TABLE I. DISTORTION AND TRANSFORMATION CLASSES FOR INTERMEDIATE REGISTRATIONS

Modalities Causes of expected Distortions Transformation ClassHistology Ex vivo MR Histological processing, histological sectioning Affine

Ex vivo MR In vivo MR w/o ER Coil formalin fixation, resection, different external pressure SimilarityIn vivo MR w/o ER Coil In vivo MR w/ ER Coil Pressure from ER coil, patient pose DeformableIn vivo MR w/o ER Coil US B-mode/ RF Time Series Pressure from US probe, patient pose DeformableIn vivo MR w/o ER Coil DCE CT/PET Patient pose, inter-session variability Rigid

Figure 2. Registration graph. Boxes denotes sets of images collected without repositioning. Intermediate registrations are denoted by

arrows. Alternate registrations are denoted by dotted lines.

95

deformation caused by the ER coil. This framework can be extended to incorporate this adaptation by imposing an order on intermediate registrations and constructing fused images.

In conclusion, we have presented a framework for the co-

registration of multiple imaging modalities from multiple imaging sessions with histology for validation of prostate cancer imaging. We have outlined a methodology for measuring the accuracy of these co-registrations and presented a quantitative accuracy measurement for the implementation of one component of the framework.

ACKNOWLEDGEMENTS

This work was supported by the National Sciences and Engineering Research Council, Cancer Care Ontario, the Ontario Institute for Cancer Research and the Canadian Institutes of Health Research [CTP 87515].

REFERENCES

[1] Fisher A. Cancer Screening: What tests can reduce deaths from malignant disease? Canadian Health. Canadian Medical Association; Jan/Feb 2008. p. 30-32.

[2] Draisma G, Boer R, Otto SJ, et al. Lead times and overdetection due to prostate-specific antigen screening: estimates from the European Randomized Study of Screening for Prostate Cancer. J Natl Cancer Inst 2003;95(12):868-878.

[3] Mouraviev V, Villers A, Bostwick DG, et al. Understanding the pathological features of focality, grade and tumour volume of early-stage prostate cancer as a foundation for parenchyma-sparing prostate cancer therapies: active surveillance and focal targeted therapy. BJU Int. 2011. [Epub ahead of print]

[4] Moreira Leite KR, Camara-Lopes LH, Dall'Oglio MF, et al. Upgrading the Gleason score in extended prostate biopsy: implications for treatment choice. Int J Radiat Oncol Biol Phys 2009;73(2):353-356.

[5] Vargas HA, Akin O, Franiel T, et al. Diffusion-weighted endorectal MR imaging at 3 T for prostate cancer: tumor detection and assessment of aggressiveness. Radiology 2011;259(3):775-784.

[6] Vigneron DB, Males R, Noworolski S, et al. 3D MRSI of prostate cancer: correlation with histologic grade ISMRM. Berkeley, California; 1998. p. 488.

[7] Zakian KL, Sircar K, Hricak H, et al. Correlation of proton MR spectroscopic imaging with gleason score based on step-section pathologic analysis after radical prostatectomy. Radiology 2005;234(3):804-814.

[8] Engelbrecht MR, Barentsz JO, Jager GJ, et al. Prostate cancer staging using imaging. BJU Int 2000;86 Suppl 1:123-134.

[9] Beheshti M, Imamovic L, Broinger G, et al. 18F choline PET/CT in the preoperative staging of prostate cancer in patients with intermediate or high risk of extracapsular disease: a prospective study of 130 patients. Radiology 2010;254(3):925-933.

[10] Rosen Y, Bloch BN, Lenkinski RE, Greenman RL, Marquis RP, Rofsky NM. 3T MR of the prostate: Reducing susceptibility gradients by inflating the endorectal coil with a barium sulfate suspension. Magn Reson Med 2007;57(5):898-904.

[11] Lee TY, Purdie TG, Stewart E. CT imaging of angiogenesis. Q J Nucl Med 2003;47(3):171-187.

[12] Ward A, Crukley C, McKenzie C, et al. Registration of in vivo prostate magnetic resonance images to digital histopathology images. In: MICCAI Prostate Cancer Workshop. LNCS 6367: Springer; 2010. p. 66-76.

[13] Gibson E, Crukley C, Gómez JA, et al. Validation of direct registration of whole-mount prostate digital histopathology to ex vivo MR images. MICCAI Prostate Cancer Imaging Workshop. LNCS 6963: Springer; 2011. p. 134.

[14] Gibson E, Crukley C, Gómez JA, et al. Registration of prostate histology images to ex vivo MR images via strand-shape fiducials. Journal of Magnetic Resonance Imaging 2011:(Submitted).

[15] Chappelow J, Bloch BN, Rofsky N, et al. Elastic registration of multimodal prostate MRI and histology via multiattribute combined mutual information. Medical Physics 2011;38(4):2005-2018.

a b cFigure 3. Co-registered histology (a), T1W ex vivo MR (b) and T2W ex vivo MR (c) images.

96