Prospective Blinded Evaluation of Computed TomographicColonography for Screen Detection of Colorectal Polyps

C. DANIEL JOHNSON,* WILLIAM S. HARMSEN,‡ LYNN A. WILSON,* ROBERT L. MACCARTY,*TIMOTHY J. WELCH,* DUANE M. ILSTRUP,‡ and DAVID A. AHLQUIST§

*Department of Radiology, Mayo Clinic Rochester; ‡Department of Health Sciences Research, Section of Biostatistics, Mayo ClinicRochester; and §Division of Gastroenterology and Hepatology, Department of Internal Medicine, Mayo Clinic Rochester,Rochester, Minnesota

See editorial on page 608.

Background & Aims: This study used a low lesion prev-alence population reflective of the screening setting toestimate the sensitivity and specificity of computerizedtomographic (CT) colonography for detection of colorec-tal polyps. Methods: This prospective, blinded studycomprised 703 asymptomatic persons at higher-than-average risk for colorectal cancer who underwent CTcolonography followed by same-day colonoscopy. Two of3 experienced readers interpreted each CT colonographyexamination. Results: Overall lesion prevalence for ade-nomas >1 cm in diameter was 5%. Seventy percent ofall lesions were proximal to the descending colon. Withcolonoscopy serving as the gold standard, CT colonog-raphy detected 34%, 32%, 73%, and 63% of the 59polyps >1 cm for readers 1, 2, 3, and double-reading,respectively; and 35%, 29%, 57%, and 54% of the 94polyps 5–9 mm for readers 1, 2, 3, and double-reading,respectively. Specificity for CT colonography rangedfrom 95% to 98% and 86% to 95% for >1 cm and5–9-mm polyps, respectively. Interobserver variabilitywas high for CT colonography with � statistic valuesranging from �0.67 to 0.89. Conclusions: In a low prev-alence setting, polyp detection rates at CT colonographyare well below those at colonoscopy. These rates areless than previous reports based largely on high lesionprevalence cohorts. High interobserver variability war-rants further investigation but may be due to the lowprevalence of polyps in this cohort and the high impacton total sensitivity of each missed polyp. Specificity,based on large numbers, is high and exhibits excellentagreement among observers.

Colorectal cancer (CRC) exacts significant morbidityand mortality, especially in industrialized nations. It

is the third most common cancer and the second leadingcause of malignant death in the United States with anestimated 148,300 new CRC cases and 56,600 CRCdeaths in 2002.1 The average lifetime incidence of CRC

is 6% and is even higher in persons with a family historyof colorectal neoplasia or with other well-establishedCRC risk factors.2 Because the natural history of CRCpermits the recognition and curative treatment of pre-cursor adenomas, there is an enormous opportunity toprevent CRC through effective screening programsbroadly applied to a general population. Indeed, evidencenow exists from prospective trials,3–5 case-control stud-ies,6–11 and predictive models12–14 to support a benefit byvarious screening interventions in reducing CRC-specificincidence and mortality. However, the potential efficacyand practicality of such a screening effort are compro-mised by limitations in the performance, comfort, andexpense of available screening tests. Despite recommen-dations for widespread CRC screening by any of theestablished modalities,15 compliance rates remain low.16,17

Better tools are needed to screen more effectively forcolorectal neoplasia.

Computerized tomographic (CT) colonography, aminimally invasive new imaging tool, uses advancedvisualization technology to produce 2- and 3-dimen-sional images that permit evaluation of the entire colo-rectal structure. Several promising reports of the perfor-mance of CT colonography have been published in theliterature.18–28 However, most of these have been basedon selected patient groups with high lesion prevalence,such as symptomatic patients or those with known orsuspected colorectal neoplasms. Because of selection bias,it may not be appropriate to extrapolate such earlyoutcomes to the screening setting. It is imperative thatthe performance of CT colonography is critically evalu-ated in large asymptomatic populations before a wide-spread screening application can be justified. Its compar-

Abbreviations used in this paper: CRC, colorectal cancer; CT, com-puterized tomographic.

© 2003 by the American Gastroenterological Association0016-5085/03/$30.00

doi:10.1016/S0016-5085(03)00894-1

GASTROENTEROLOGY 2003;125:311–319

ative performance with other established screening toolsin this setting is also needed to guide practice.

The purpose of this study was to estimate the sensi-tivity, specificity, and interobserver agreement at CTcolonography for detection of colorectal polyps in a low-prevalence population and to compare such estimateswith colonoscopy as the gold standard.

Patients and MethodsSeven hundred three outpatients, 50 years of age or

older, who were prescheduled for colonoscopy and at higher-than-average risk for developing a colorectal neoplasm (a priorhistory of colorectal neoplasia, a strong family history of colo-rectal cancer, or new onset of asymptomatic iron deficiencyanemia) were recruited for CT colonography. Signed informedconsent was obtained. The Institutional Review Board ap-proved the study. Exclusion criteria comprised melena, hema-tochezia, inflammatory bowel disease, and known familialpolyposis. The study was conducted between January 27,1998, and February 13, 2001.

CT Colonography

All patients had CT colonography prior to same-daycolonoscopy. Preparation for the examination included theGoLyte oral lavage and bisacodyl tablets (10 mg) preparationin 681 of 703 (96.9%) patients, magnesium citrate (300 mL)and bisacodyl tablets (20 mg) in 4 of 703 (0.6%) patients, andPhospho-Soda (90 mL) in 18 of 703 (2.5%) patients. Sixhundred thirty-five patients (90%) received glucagon, 1 mgsubcutaneously 10 minutes prior to CT acquisition, unlesscontraindicated or refused by the patient; 68 (10%) patientsdid not receive glucagon. Patients were placed in either decu-bitus position for enema tip insertion and slow manual insuf-flation of approximately 2 liters of carbon dioxide (until thepatient verbally indicated air administration had reached max-imal tolerance). Both supine and prone data acquisitions wereobtained, and additional carbon dioxide was added (as toler-ated by the patient) before scanning in the prone position tocompensate for any lost during position changes.

All examinations were performed using either a G.E.HiSpeed Advantage (120 of 703 [16%]) or Lightspeed (4 rowmultidetector) helical CT scanner (583 of 703 [83%]) (GEMedical Systems, Milwaukee, WI). Following colon insuffla-tion, a breath-hold AP scout was obtained to assess luminaldistention and to prescribe axial slices through the entire largebowel. Images were acquired using the single-slice helical CTscanner with 5-mm collimation, table speed of 6.5 mm/s(pitch of 1.3), 3-mm reconstruction intervals, matrix 512 �512, field-of-view to fit, 70 mAs, 120 kVp, standard recon-struction algorithm, and 20-second breath holds. Three to 4breath holds were performed within 2 minutes, 150 axialimages (per acquisition). Three-centimeter acquisition overlapbetween breath holds was obtained. The multislice scannerused 5-mm collimation, pitch of 15 mm/s, 3-mm reconstruc-tion intervals, 80 mAs, 120 kVp, standard reconstruction

algorithm, and a single approximately 20-second breath hold.The 80-mAs setting was chosen to match the noise of thesingle slice scanner using 70 mAs.

The image data were networked to an off-line workstation(SUN Microsystems, San Jose, CA) using customized softwaredeveloped and extensively tested28 within our laboratory. Theaxial images were magnified and reviewed using both lung andsoft-tissue window settings. Suspected abnormalities were fur-ther evaluated using multiplanar reformatted images and 3-Dendoluminal images. Lesion location, size, and observer confi-dence were noted for each abnormality. Lesion size was deter-mined from 2-D images. Observer confidence for each lesionwas rated on a scale of 1 to 5 (doubtful to definite). Each dataset (supine, prone) was evaluated sequentially, and suspiciousregions were compared between the 2 data sets using simul-taneous, synchronized views (axial, 2-D MPR, 3-D endolumi-nal) of both.

Diagnostic review of each study was performed by 2 of 3experienced radiologists (Board certified abdominal radiolo-gists with more than 10 years of practice experience and over150 CT colonography examination interpretations with endo-scopic correlation) in a blinded fashion. Examinations forinterpretation were randomly assigned to each radiologist.Results are reported for each individual reviewer and fordouble reading (the combined reports of the 2 individualreaders). Reviewers were instructed to ignore polyps �5 mmin diameter. Results were not disclosed after each case wasread.

Colonoscopy

Colonoscopy was performed in the standard fashion toexamine the entire colorectum. Examinations were videotapedduring withdrawal of the instrument. All of the examinationswere performed by staff gastroenterologists or colorectal sur-geons experienced with this technique. Colonoscopists wereblinded to the results at CT colonography. Forty-seven pa-tients had a right hemicolectomy but were judged completebecause the colonoscopist visualized the ileocolonic anastomo-sis. Incomplete studies in 13 of 703 examinations (1.8%) werecomplete to the sigmoid (1), descending (2), splenic flexure (1),transverse (3), hepatic flexure (3), and ascending colon (3). Thefindings in these patients were compared between CT colonog-raphy and colonoscopy only in the segments of the colon thatwere visualized endoscopically.

Lesions detected at CT colonography were matched manu-ally with those found at colonoscopy using both the colonos-copy and pathologic reports. Lesion size was determined by thepathology report unless the lesion was removed in pieces. Inthese cases, the size estimate at colonoscopy was used. If thelesion was within 1 colonic segment of the colonoscopic loca-tion, it was considered a match. The colon was considered tohave 8 segments: cecum, ascending, hepatic, transverse,splenic, descending, sigmoid, and rectum. Videotaped exam-inations were reviewed if a convincing lesion (�1 cm) wasidentified at CT colonography and not reported on the colonos-copy report. In only 3 cases was a 5–9-mm polyp seen in

312 JOHNSON ET AL. GASTROENTEROLOGY Vol. 125, No. 2

retrospect. These cases were then recoded as true positivefindings.

Analysis

Colonoscopy was considered the “gold standard,” ortruth, with respect to presence or absence of polyps. Thesensitivities were compared with McNemar’s test or with thesign test. Only polyps �5 mm in diameter were considered inthe analysis. Double reading included all detections from bothindependent interpretations. No discussion of individual find-ings or consensus reporting occurred. Receiver operating char-acteristic (ROC) curves were generated and area under thecurves calculated separately for each of the 3 reviewers as wellas for the double reading.29 In this analysis, the judgment ofpolyp presence was graded by the radiologists as follows: notpossible, doubtful, somewhat doubtful, conservatively confi-dent, moderately confident, and highly confident. For polypsnot detected by a radiologist, an answer of “not possible” wasassumed. For the double reading, the higher of the 2 reviewerconfidences was used. Agreement between the 2 reviewingradiologists for each patient was estimated by the � statistic,along with 95% confidence intervals (CI). Specificity wasdetermined on a per patient basis and reported for 2 groups ofpatients: those without a proven lesion �5 mm and thosewithout a �10-mm lesion.

For the 153 polyps, in 103 patients, confirmed by colonos-copy, several polyp characteristics (polyp size, location, histol-ogy, and morphology) potentially associated with missedpolyp detection were assessed. Univariate logistic regressionmodels were used to predict missed detection by both review-ing radiologists. The odds ratios (OR) derived from thesemodels were adjusted to approximate risk ratios (RR) ofmissed detection.30 The generalized estimating equation(GEE) method was used to adjust the estimated standard errorsof the regression model coefficients to account for withinpatient correlation. A working correlation matrix for polypswithin a patient was assumed to have an exchangeable corre-lation structure. Two-sided � levels of 0.05 were used toindicate statistical significance.

ResultsA total of 703 patients were recruited for the

study. Gender mix included 442 (62.9%) males and 261(37.1%) females. Ethnic backgrounds included whites(681), Native Americans (1), Asians (4), blacks (4), andHispanics (5). The mean age was 64 years (SD 7) with arange of 50–84 years. A first-degree relative with ahistory of colorectal cancer accounted for 178 of 703(25.4%) patients, prior personal history of a polyp orcancer for 512 of 703 (72.6%), and the new onset ofanemia in 13 of 703 (2.0%).

A total of 153 polyps 5 mm in diameter or larger werepresent in 103 patients, 94 polyps 5–9 mm were presentin 69 patients, and 59 polyps in 47 patients were �1 cm

in diameter. Median size of the 153 polyps �5 mm was0.8 cm (range of 0.5–5.0 cm). Mean size was 1.1 cm (SD0.9 cm). Ninety-nine of the 153 polyps (64.7%) wereadenomas in 73 patients. There were 43 adenomas �1cm in 35 patients. Median and mean size of these largerpolyps was 1.5 cm (range, 1.0–5.0 cm) and 1.8 cm (SD1.0 cm), respectively. The prevalence of at least 1 adeno-matous polyp �1 cm in diameter was 5.0%. Seventypatients had 1 polyp �5 mm, 21 patients with 2 polyps,11 patients with 3 polyps, and 1 patient with 8 polyps.

Colorectal lesions 5–9 mm had histologic findings asfollows: adenoma, 51 (54.3%); hyperplastic, 34 (36.2%);mixed, 5 (5.3%); and none reported (lesion lost at thetime of removal), 4 (4.3%). Lesions �1.0 cm had thefollowing histologic findings: adenoma, 37 (62.7%); car-cinoma, 3 (5.1%); hyperplastic, 13 (22.0%); lipoma 3,(5.1%); other, 1 (1.7%); and mixed (adenomatous andhyperplastic features), 2 (3.4%).

Distribution of lesions 5–9 mm was as follows: rec-tum, 12 (12.8%); sigmoid, 16 (17.0%); descending, 8(8.5%); splenic flexure, 5 (5.3%); transverse, 15 (16.0%);hepatic flexure, 8 (8.5%); ascending, 22 (23.4%); andcecum, 8 (8.5%). Lesions �1.0 cm in diameter had thefollowing distribution: rectum, 5 (8.5%); sigmoid, 9(15.3%); descending, 4 (6.8%); splenic flexure, 2 (3.4%);transverse, 8 (13.6%); hepatic flexure, 7 (11.9%); ascend-ing, 13 (22%); and cecum, 11 (18.6%).

Overall examination quality was judged globally forthe presence of excessive fluid, stool, or collapse. Residualfluid in the colon was judged to be none in 76 (11%)patients, mild in 536 (76%), moderate in 88 (13%), andnondiagnostic in 3 (4%). Residual stool in the colon wasjudged to be none in 315 (45%) patients, mild in 306(44%), moderate in 75 (11%), and nondiagnostic in 7(1%). Colon distention was judged to be optimal in 483(69%) patients, suboptimal in 1 segment in 205 (29%),and suboptimal in �1 segment in 15 (2.1%). Breath-hold artifacts were judged as none in 588 (84%) patients,mild in 107 (15%), and moderate in 8 (1%). Twenty-three patients had a nondiagnostic examination by eitherfluid, stool, or suboptimal distention result. Of these, 2patients had a total of 4 polyps, 3 sized 0.5–0.9 cm and1 sized �1.0 cm. Because of their small numbers, theywere included in the study analysis.

Per Polyp Assessment

The sensitivity for the detection of polyps 5–9mm was 35%, 29%, 57%, and 54% for readers 1, 2, 3,and double reading, respectively. For the detection ofpolyps �1.0 cm, the sensitivity was 34%, 32%, 73%,and 63% for readers 1, 2, 3, and double reading, respec-tively (Table 1). There was no significant difference in

August 2003 PROSPECTIVE EVALUATION OF CT COLONOGRAPHY 313

diagnostic detection rates when the detection of adeno-matous polyps was compared with the detection rateof nonadenomatous polyps (Table 1). This was thecase for all 3 readers, as well as the double reading forpolyps sized 0.5–0.9 cm and �1.0 cm (P � 0.2).Inter-reader agreement of sensitivity varied substantially(Table 2).

The median size of polyps �5 mm that were eitherdetected or missed were 0.8 cm and 0.7 cm, respectively.There was a protective, although nonsignificant, associ-ation between increasing polyp size and missed detection(P � 0.14; RR � 0.8 per 1-mm increase, 95% CI:0.6–1.1). Dichotomizing size as 5–9 mm vs. �10 mm,the relative risk of missed detection for polyps sized 5–9mm (relative to �10 mm) was RR � 1.2, 95% CI:0.8–1.6 (P � 0.26; Table 3). There was no significantassociation between missed detection and polyp location.The risk of missed detection for each colon segment(relative to the rectum) was as follows: for polyps in thesigmoid, P � 0.10 (RR � 0.5, 95% CI: 0.2–1.1); indescending, P � 0.88 (RR � 0.9, 95% CI: 0.3–1.7); insplenic flexure, P � 0.24 (RR � 0.5, 95% CI: 0.2–1.4);in transverse, P � 0.58 (RR � 0.8, 95% CI: 0.3–1.5); inhepatic flexure, P � 0.51 (RR � 0.7, 95% CI: 0.2–1.6);in ascending, P � 0.93 (RR � 1.0, 95% CI: 0.4–1.6);and in the cecum, P � 0.34 (RR � 0.6, 95% CI:0.2–1.4) (Table 4). There was a borderline association fornonadenomatous polyps to be undetected more fre-quently than adenomatous polyps, P � 0.08 (RR � 1.4,

95% CI: 0.9–1.7). Lesion morphology was significantlyassociated with missed detection. The risk of misseddetection was greater for both sessile polyps, P � 0.02(RR � 1.9, 95% CI: 1.2–2.2) and for flat polyps, P �0.04 (RR � 1.9, 95% CI: 1.1–2.2), relative to thepedunculated polyps (Table 5).

Per Patient Assessment

The sensitivity for detecting patients with at least1 polyp 5–9 mm ranged from 41% to 69%, with adouble reading sensitivity of 65% (Table 6). Specificityfor the same polyps ranged from 88% to 95%, with adouble reading specificity of 86%. Areas under the ROCwere 0.672 (SE 0.043), 0.664 (SE 0.054), 0.826 (SE0.043), and 0.759 (SE 0.035) for reviewers 1, 2, 3, anddouble reading, respectively. For polyps �1.0 cm, thesensitivity on a per patient basis ranged from 35% to72%, with a double reading sensitivity of 64%. Speci-ficity for these larger polyps ranged from 97% to 98%,with a double reading specificity of 95%. Areas underthe ROC curve were 0.674 (SE 0.051), 0.665 (SE 0.069),0.858 (SE 0.050), and 0.799 (SE 0.043) for readers 1, 2,3, and double reading, respectively.

The distribution of lesions 5–9 mm that were presentin the randomly assigned cases to each pair of radiologistreaders is as follows: readers 1 and 2, 31 polyps; readers1 and 3, 49 polyps; readers 2 and 3, 14 polyps. � Statisticvalues to evaluate interobserver variability ranged be-tween .34 and .62 for all polyps 5–9 mm in diameter.

Table 1. Sensitivity at CT Colonography on a Per Polyp Basis

Per polyp analysis Any polyp Percentage 95% CI Adenomatous Percentage 95% CI

Reviewer 15–9 mm 28/80 35.0 24.7–46.5 18/41 43.9 28.5–60.3�10 mm 18/53 34.0 21.5–48.3 13/36 36.1 20.8–53.8

Reviewer 25–9 mm 13/45 28.9 16.4–44.3 12/30 40.0 22.7–59.4�10 mm 9/28 32.1 15.9–52.4 8/23 34.8 16.4–57.3

Reviewer 35–9 mm 36/63 57.1 44.1–69.5 18/31 58.1 39.1–75.5�10 mm 27/37 73.0 55.9–86.2 17/23 73.9 51.6–75.5

Double read5–9 mm 51/94 54.3 43.7–64.6 31/51 60.8 46.1–74.2�10 mm 37/59 62.7 49.2–75.0 26/41 63.4 46.9–77.9

Table 2. Interobserver Variability for Polyp Detection at CTColonography (� Statistics)

Polyp diameter(mm)

Readers

1 and 2 1 and 3 2 and 3

5–9 0.62 0.41 0.34�10 0.89 0.21 �0.67

NOTE. Reader’s values are � statistic values.

Table 3. Polyp Detection by Size

5–9 mm(n � 94)

�10 mm(n � 59)

Missed by both readers (%) 43 (46) 22 (37)Detected by 1 reader (%) 25 (27) 20 (34)Detected by both readers (%) 26 (28) 17 (29)Detected by 1 or both readers (%)a 51 (54) 37 (63)

aThe sum of detected by 1 plus detected by both readers.

314 JOHNSON ET AL. GASTROENTEROLOGY Vol. 125, No. 2

These values also ranged from �0.67 to 0.89 for allpolyps �10 mm in diameter (Table 2).

Of the 94 polyps between 5–9 mm in diameter, 26(28%) were detected by both observers, 25 (27%) by 1observer, and 43 (46%) by neither. Of the 59 polyps �1cm in diameter, 17 (29%) were detected by both observ-ers, 20 (34%) by 1 observer, and 22 (37%) by neither.

DiscussionThis study prospectively evaluated the perfor-

mance of CT colonography in a large asymptomaticpopulation with a low disease prevalence. As such, thestudy population represents a sample of patients likely toundergo CT colonography screening. The overall preva-lence of polyps in this study was similar to a recentlypublished report in a screening population.31 CTcolonography was less sensitive in detecting polyps thanwould have been predicted from earlier studies in cohortswith a higher prevalence of polyps. Technical and percep-tual errors accounted for polyp misses by CT colonography.

CT colonography has received considerable attentionas a possible screening test since it was introduced in1994.32 Most early studies comparing colonoscopy andCT colonography have been in selected groups of pa-tients, often symptomatic and with high disease preva-lence.18–28 Generally, these reports affirm that CTcolonography will be successful as a full structural colo-rectal cancer-imaging tool. We recruited only asymp-tomatic individuals �50 years of age that had beenprescheduled for colonoscopic screening. Higher-than-average risk patients were recruited for the study so thatthe cost of colonoscopy was covered.

Results from the present low lesion prevalence study(5% of patients had polyps �1 cm) differ from reports ofother medical centers and from our own previous re-sults.28,33 Fenlon et al.24 reported on 100 patients (in-cluding patients recruited with positive flexible sigmoid-oscopy and fecal occult blood tests) with a prevalence for�1 cm polyps of 19%. Interpretation included both afull axial and endoluminal image review with consensusreporting by 2 radiologists. The sensitivity and specific-ity for detecting these larger polyps were 91% and 96%,respectively. Yee et al.27 reported on 300 patients, ofwhich two thirds were evaluated for symptoms. Overall,prevalence for �1 cm polyps was 16%, with a per polypsensitivity of 90% and specificity of 72%; however, inthe subset of patients that were asymptomatic, the perpolyp sensitivity was only 72%. Our group28 reported on180 patients, the majority with known or suspectedcolorectal neoplasms and an exceedingly high prevalencefor �1 cm polyps of 67%. The sensitivity and specificityin this group was 75% and 93%, respectively. Differ-ences between the present and past studies, in part, canbe attributed to the substantially lower prevalence ofpolyps in this screening group. Our reading methods alsodiffered from those reported by Fenlon et al.24 and Yee etal.,27 in that we used 3-D endoluminal views only inshort segments of the colon (identified by the initial axialreview) to problem solve and improve the observers’confidence. Both Fenlon et al. and Yee et al. reviewed theentire data set using both axial and 3-D endoluminalimages. McFarland et al.,34 testing different display tech-niques in a data set of 30 colonic segments, did not finda statistical difference in polyp detection comparing 2-D

Table 4. Polyp Detection by Colon Segment

Cecum(%)

Ascending(%)

Hepaticflexure (%)

Transverse(%)

Splenicflexure (%)

Descending(%)

Sigmoid(%)

Rectum(%)

Missed by both readers, n � 65 7 (37) 18 (51) 6 (40) 10 (43) 2 (29) 6 (50) 7 (28) 9 (53)Detected by 1 reader, n � 45 7 (37) 8 (23) 3 (20) 11 (48) 4 (57) 3 (25) 6 (24) 3 (18)Detected by both readers, n � 43 5 (26) 9 (26) 6 (40) 2 (9) 1 (14) 3 (25) 12 (48) 5 (29)Detected by 1 or both readers, n � 88a 12 (14) 17 (19) 9 (10) 13 (15) 5 (6) 6 (7) 18 (20) 8 (9)

aThe sum of detected by 1 plus detected by both readers.

Table 5. Polyp Detection by CT Colonography Based on Lesion Morphology

Sessile (n � 60) Pedunculated (n � 21) Flat (n � 22) Not reporteda (n � 50)

Missed by both readers (%) 27 (45) 3 (14) 10 (45) 25 (50)Detected by 1 reader (%) 19 (32) 6 (29) 9 (41) 11 (22)Detected by both readers (%) 14 (23) 12 (57) 3 (14) 14 (28)Detected by 1 or both readers (%)b 33 (55) 18 (86) 12 (55) 25 (50)

aFifty polyps had no reported morphologic comments on either the colonoscopy or pathology reports.bThe sum of detected by 1 plus detected by both readers.

August 2003 PROSPECTIVE EVALUATION OF CT COLONOGRAPHY 315

multiplanar views, 3-D endoluminal views, or 3-D thickslab reformations.

Causes of error are multifactorial. Both perceptive andtechnical causes of CT colonography error occurred.Technical errors were defined as polyps missed by bothobservers. These errors accounted for 43 of 94 (46%) ofthe polyps 5–9 mm in diameter and 22 of 59 (37%)polyps �1 cm. Because perceptual errors by both ob-servers may occur, these estimates of “technical error” arelikely high. Possible causes of technical error includeinadequate colon preparation, suboptimal CT technique,and software capability. Overall, colon preparations werevery good because only 23 of 703 (3.3%) were consideredto be nondiagnostic because of poor preparation. In re-gard to CT technique, the 5-mm-slice collimation usedin this study is not likely responsible for the lower thanexpected sensitivity. Recent reports confirm that thinnercollimation acquisitions are associated with improveddetection of diminutive polyps (�5 mm) but not forpolyps �5 mm.35,36 The usefulness of thinner collima-tion for improved detection of flat polyps is currentlyunknown. Similarly, recent reports also confirm thatlow-dose techniques, utilized in this study, do not ad-versely affect polyp detection.37,38 In addition, a largemajority of the examinations were performed on multi-slice (4 slice) spiral CT scanners. Our customized soft-ware has been extensively tested and shown in trials witha patient population with a higher disease prevalence toperform at a high level (competitive with other re-ports).18,19,28,33

If a lesion was identified by one observer and missedby the other, these were classified as perceptive errors.Perceptive errors accounted for 25 of 94 (27%) polyps5–9 mm in diameter and 20 of 59 (34%) polyps �1 cmin diameter.

Polyp morphology, but not location, influenced detec-tion rates. In this study, polyp size did not affect detec-tion rates, but there were few large polyps. Large lesions

that were missed simulated regions of lumen collapse orpartially distended colon. Others infiltrated the wall ofthe colon while maintaining a normal but thickened foldcontour. Some small polyps were easy to detect becausethey protruded into the lumen and created an obviouscontour abnormality. The location of a polyp in the colonwas not related to a detection error. The rectum ac-counted for the colon segment with the highest errorrate, but this was not significantly higher than error ratesin other colon segments. Lesion morphology was associ-ated with significant detection differences. Pedunculatedpolyps are detected most easily, probably because theycause the greatest contour abnormality in the colon atCT. Flat and sessile polyps, with the least contour dif-ference, are significantly more difficult to detect thanpedunculated polyps.

The low frequency of polyp occurrence in the presentstudy may have contributed to detection difficulties be-cause of reader fatigue and data overload. The readingmethod employed in this study required the reader toexamine a huge set of data for each patient. In a standard150 image supine data set, our method involved readingall of these images from the rectum to the cecum (150)and reverse (150) using lung windows and again usingsoft-tissue windows forward and backward (300). Thisprocess was repeated for the prone images (600). There-fore, a minimum of 1200 images was reviewed for eachpatient. Given the low lesion prevalence, over 13,000images had to be reviewed to find a single polyp �1 cm.

The wide variation in polyp detection rates (32%–73%) between observers in the present study was diffi-cult to explain. All of the examinations were performedin a blinded fashion by endoscopic and radiologic inves-tigators with similar levels of experience. During thestudy, radiologists met to enhance uniformity of practice.The interpretation protocol has been accepted by mostexperts as clinically relevant and “state-of-the-art.” Be-cause all of the readers were experienced radiologists and

Table 6. Sensitivity and Specificity at CT Colonography for Polyp Detection on a Per Patient Basis

Per patient analysis Sensitivity Percentage 95% CI Specificity Percentage 95% CI

Reviewer 15–9 mm 26/56 46 33.0–60.3 465/530 88 84.6–90.4�10 mm 16/42 38 23.6–54.4 524/543 97 94.6–97.9

Reviewer 25–9 mm 15/37 41 24.8–57.9 333/360 93 89.3–95.0�10 mm 8/23 35 16.4–57.3 367/374 98 96.2–99.2

Reviewer 35–9 mm 31/45 69 53.4–81.8 358/378 95 92.0–96.7�10 mm 21/29 72 52.8–87.3 387/394 98 96.4–99.3

Double read5–9 mm 45/69 65 52.8–76.3 542/634 86 82.5–88.1�10 mm 30/47 64 48.5–77.3 625/656 95 93.4–96.8

316 JOHNSON ET AL. GASTROENTEROLOGY Vol. 125, No. 2

trained in CT colonography, it is unlikely that experi-ence and training are causative factors. Interpretationtimes were not recorded but appeared similar based onimpressions among the 3 readers. Identical software wasused for all interpretations.

Only 3 prospective patient series have assessed inter-observer variability, all much smaller in sample size and1 from our early experience. Hara et al.33 reported on 70patients with 115 polyps (15 polyps �1 cm). Sensitivityfor the 2 readers using 3-D endoluminal views was 67%and 73%. Dachman et al. reported 22 polyps in 44patients (6 polyps �8 mm) with nearly identical sensi-tivity for the 2 readers at 83%.21 McFarland et al.39

reported interobserver variability among experienced ra-diologists independently reviewing 70 CT colonographyexaminations. Per polyp sensitivity for polyps 6–9 mmand �10 mm in diameter ranged from 20% to 44% and60% to 78%, respectively.39 Pescatore et al. reportedinterobserver agreement in 24 patients with 65 polyps.40

� Statistic values were 0.72 for polyps �1 cm and 0.56for polyps of any size.

High variability of interobserver sensitivities may berelated to the case mix of the study cohort. In thispopulation, very few large polyps were present amonghundreds of cases. The cost of each missed polyp had alarge effect on reader sensitivity, translating into highinterobserver variability. In contrast, very high specific-ities are based on much larger numbers and demonstrateexcellent interobserver agreement. This latter fact is en-couraging for CT colonography as a future highly com-petitive colorectal examination.

Even colonoscopy is subject to interpreter variability.Studies of back-to-back colonoscopy examinations havereported error rates of 15%–25% for adenoma detec-tion,41,42 although the majority of these errors are polyps�1 cm. Agreement is high for polyps �1 cm and forcancers. Complete examination of the colon at colonos-copy can also vary considerably, ranging in reports from80% to 99%.43–46

Interobserver variability is recognized with proceduresother than CT colonography, and the causes are oftenpoorly understood. Alluisi47 has shown that individualspredictably overlook some targets when they are randomand infrequent events under experimental conditions.Beam et al.48 reported on the variability at screeningmammography by sampling 108 radiologists at 50 ac-credited centers within the United States using a set ofmammograms stratified by proven disease status. Radi-ologists referred from 47% to 100% of women withcancer to biopsy, 0% to 87% of women with benignbreast disease to biopsy, and 0% to 65% of women with

normal breast tissue to biopsy. Up to 30% of womenwith cancer were referred for routine follow-up. Mc-Donald49 has shown that clinical protocol errors areprobably caused by the limited human capacity to pro-cess data rather than to correctable perceptive tech-niques.

Several measures may counter perceptual errors andinterobserver variability in high-volume CT colonogra-phy settings. First, double reading appears to be a safe-guard that would likely double interpretation time butnot require additional technology. The feasibility of dou-ble reading may not be possible in every busy radiologypractice. We found that double reading increased sensi-tivity 19%–29% but at the expense of a modest drop inspecificity. Second, more reader-efficient image displayssuch as virtual pathology, by reducing the number ofimages to be reviewed by the radiologist, could poten-tially also improve reader performance.50 Third, the er-rors and variability encountered at CT colonography maybe correctable in the future with the assistance ofcomputer-aided diagnosis (CAD). Several reports of CADfor the detection of colorectal polyps have been publishedwith promising preliminary results.51–55

The distribution of lesions in this cohort demonstrateda significant right-sided shift, with nearly 70% of polyps1 cm or larger proximal to the descending colon. Screen-ing strategies in this cohort relying on flexible sigmoid-oscopy would have detected less than 30% of all largepolyps (only 24% were located in the rectum and sig-moid colon).

CT colonography as a full structural examination ofthe colorectum has several additional test characteristicsthat may be desirable for screening. These include safety(there have been no reported deaths or colon perfora-tions), patient acceptance,56 minimal discomfort withoutthe need for sedation,56 ability to return to work imme-diately following the examination, assessment of abdom-inal and pelvic organs,57 and capability of performingcolonoscopy immediately following the procedure if alesion is discovered (no additional preparation required).Because a full bowel preparation is still required, thediscomfort and inconvenience associated with the prepara-tion is similar for CT colonography and colonoscopy.56

Developments are underway to obviate the catharticbowel preparation for CT colonography examination us-ing a stool-tagging method, and promising preliminaryresults have been reported.58,59

Based on observations in this screening-like setting ofan asymptomatic population with low prevalence of colo-rectal neoplasia, the accuracy of CT colonography forpolyp detection may not be as high as suggested in

August 2003 PROSPECTIVE EVALUATION OF CT COLONOGRAPHY 317

earlier reports from smaller selected patient groups en-riched with pathology. The perceptive and technicalerror inherent in reading large complex data sets likelycontributes to our findings. Technologic refinements,such as computer-aided diagnosis, should be explored tominimize reading error. Specificities, based on largenumbers, are high and exhibit excellent agreementamong observers.

References1. Cancer Facts and Figures 2002: American Cancer Society, 2002.2. Burt RW. Colon cancer screening. Gastroenterology 2000;119:

837–853.3. Mandel JS, Bond JH, Church TR, Snover DC, Bradley GM, Schu-

man LM, Ederer F. Reducing mortality from colorectal cancer byscreening for fecal occult blood. N Engl J Med 1993;328:1365–1371.

4. Winawer SJ, Flehinger BJ, Schottenfeld D, Miller DG. Screeningfor colorectal cancer with fecal occult blood testing and sigmoid-oscopy. J Natl Cancer Inst 1993;85:1311–1318.

5. Winawer SJ, Zauber AG, Ho MN, O’Brien MJ, Gottlieb LS, Stern-berg SS, Waye JD, Schapiro M, Bond JH, Parish JF. Prevention ofcolorectal cancer by colonoscopic polypectomy. N Engl J Med1993;329:1977–1981.

6. Muller AD, Sonnenberg A. Protection by endoscopy against deathfrom colorectal cancer. A case-control study among veterans.Arch Intern Med 1995;155:1741–1748.

7. Muller AD, Sonnenberg A. Prevention of colorectal cancer byflexible endoscopy and polypectomy: a case control study of32,702 veterans. Ann Intern Med 1995;123:904–910.

8. Selby JV, Friedman GD, Quesenberry CP, Weiss NS. A case-control study of screening sigmoidoscopy and mortality fromcolorectal cancer. N Engl J Med 1992;326:653–657.

9. Selby JV, Friendman GD, Quesenberry CP, Weiss NS. Effect offecal occult blood testing on mortality from colorectal cancer: acase control study. Ann Intern Med 1993;118:1–6.

10. Newcomb PA, Norfleet RG, Storer BE, Surawicz TX, Marcus PM.Screening sigmoidoscopy and colorectal cancer mortality. J NatlCancer Inst 1992;84:1572–1575.

11. Lazovich D, Weiss NS, Stevens NG, White E, McKnight B, WagnerEH. A case-control study to evaluate efficacy of screening forfaecal occult blood. J Med Screen 1995;2:84–89.

12. Eddy DM, Nugent TW, Eddy JF, Coller J, Gilbertsen V, Gottlieb LS,Rice R, Sherlock P, Winawer SJ. Screening for colorectal cancerin a high-risk population: results of a mathematical model. Gas-troenterology 1987;92:682–692.

13. Eddy DM. Screening for colorectal cancer. Ann Intern Med 1990;113:373–384.

14. Wagner JL, Tunis S, Brown M, Ching A, Almeda R. The cost-effectiveness of colorectal cancer screening in average riskadults. In: Young G, Levin B, eds. Prevention and early detectionof colorectal cancer. Philadelphia: Saunders, 1996.

15. Smith RA, von Eschenbach AC, Wender R, Levin B, Byers T,Rothenberger D, Brooks D, Creasman W, Chohen C, Runowicz C,Saslow D, Cokkinides V, Eyre H. American Cancer Society guide-lines on screening and surveillance for the early detection ofadenomatous polyps and colorectal cancer. CA Cancer J Clin2001;51:44–54.

16. Anderson L, May D. Has the use of cervical, breast, and colorec-tal cancer screening increased in the United States? Am J PublicHealth 1995;85:840–842.

17. Brown M, Protsky A, Thompson G. The knowledge and use ofscreening tests for colorectal and prostate cancer: data from the

1987 National Health Interview Survey. Prev Med 1990;19:562–574.

18. Hara AK, Johnson CD, Reed JE, Ahlquist DA, Nelson H, EhmanRL, McCollough CH, Ilstrup DM. Detection of colorectal polyps bycomputed tomographic colography: feasibility of a novel tech-nique. Gastroenterology 1996;110:284–290.

19. Hara AK, Johnson CD, Reed JE, Ehman RL, Ilstrup DM. Colorectalpolyp detection using computed tomographic colography: two-versus three-dimensional techniques. Radiology 1996;200:49–54.

20. Royster AP, Fenlon HM, Clarke PD, Nunes DP, Ferrucci JT. CTcolonoscopy of colorectal neoplasms: two-dimensional andthree-dimensional virtual-reality techniques with colonoscopiccorrelation. AJR Am J Roentgenol 1997;169:1237–1242.

21. Dachman AH, Kuniyoshi JK, Boyle CM, Samara Y, Hoffmann K,Rubin DT, Hanan I. CT colonography with three-dimensional prob-lem solving for detection of colonic polyps. AJR Am J Roentgenol1998;171:989–995.

22. Macari M, Berman P, Dicker M, Milano A, Megibow A. Usefulnessof CT colonography in patients with incomplete colonoscopy. AJRAm J Roentgenol 1999;173:561–564.

23. Fenlon H, Clarke PD, Ferrucci JT. Virtual colonoscopy: imagingfeatures with colonoscopic correlation. AJR Am J Roentgenol1998;170:1303–1309.

24. Fenlon H, McAneny DB, Nunes DP, Clarke PD, Ferrucci JT. Occlu-sive colon carcinoma: virtual colonoscopy in the preoperativeevaluation of the proximal colon. Radiology 1999;210:423–428.

25. Morrin MM, Kruskal JB, Farrell RJ, Goldberg SN, McGee JB,Raptopoulos V. Endoluminal CT colonography after an incompleteendoscopic colonoscopy. AJR Am J Roentgenol 1999;172:913–918.

26. Chen SC, Lu DSK, Hecht JR, Kadell BM. CT colonography: valueof scanning in both the supine and prone positions. AJR Am JRoentgenol 1999;172:595–599.

27. Yee J, Akerkar GA, Hung RK, Sreinauer-Gebauer AM, Wall SD,McQuaid K. Colorectal neoplasia: performance characteristics ofCT colonography for detection in 300 patients. AJR Am J Roent-genol 2001;219:685–692.

28. Fletcher JG, Johnson CD, Welch TJ, MacCarty RL, Ahlquist DA,Reed JE. Optimization of CT colonography technique: a prospec-tive trial in 180 patients. Radiology 2000;216:704–711.

29. Hanley J, McNeil BJ. The meaning and use of the area under areceiver operating characteristic (ROC) curve. Radiology 1982;143:29–36.

30. Zhang J, Yu K. What’s the relative risk? JAMA 1998;280:1690–1691.

31. Lieberman DA, Weiss DG, Bond JH, Ahnen DJ, Garewal H, ChejfecG, Harford WV, Provenzale D, Sontag S, Schnell T, Durbin TE,Nelson DB, Ewing SL, Triadafilopoulos G, Ramirez FC, Lee JG,Collins JF, Fennerty MB, Johnston TK, Corless CL, McQuaid KR,Sampliner RE, Morales TG, Fass R, Smith R, Maheshwari Y. Useof colonoscopy to screen asymptomatic adults for colorectalcancer. N Engl J Med 2000;343:162–168.

32. Vining DJ, Gelfand DW. Noninvasive colonoscopy using helical CTscanning, 3D reconstruction, and virtual reality. Presented at the23rd Annual Meeting and Postgraduate Course of the Society ofGastrointestinal Radiologists, Maui, Hawaii, 1994.

33. Hara AK, Johnson CD, Reed JE, Ahlquist DA, Nelson H, MacCartyRL, Harmsen WS, Ilstrup DM. Detection of colorectal polyps withCT colography: initial assessment of sensitivity and specificity.Radiology 1997;205:59–65.

34. McFarland HG, Brink JA, Pilgram TK, Heiken JP, Balfe DM, HirseljDA, Weinstock L, Littenberg B. Spiral CT colonography: readeragreement and diagnostic performance with two- and three-di-mensional image-display techniques. Radiology 2001;218:375–383.

35. Macari M, Lui Y, Israel GM, Babb JS, Bini E. Thin versus thick

318 JOHNSON ET AL. GASTROENTEROLOGY Vol. 125, No. 2

section CT colonography for colorectal polyp detection (abstr).Radiology 2002;225:585.

36. Taylor SA, Halligan S, Bartram CI, Atkin I. Multislice CT colonog-raphy: effect of collimation, pitch and orientation in polyp detec-tion determined using a human colectomy specimen (abstr).Radiology 2002;225:584.

37. Cohnen MG, Vogt C, Beck A, Aurich V, Haeussing D, Moedder U.Low-dose multi-slice CT colonography: first results (abstr). Radi-ology 2002;225:584.

38. Venema HW, Van Gelder RE, Determann RM, Lameris JS, StokerJ. Determination of the technique factors for CT colonography:adjustment of mAs value to the size of the patient for constantimage quality (abstr). Radiology 2002;225:584.

39. McFarland HG, Pilgrim TK, Brink JA, McDermott RA, Santillan CV,Brady PW, Heiken JP, Balfe DM, Weinstock LB, Thyssen EP,Littenberg B. CT colonography: multiobserver diagnostic perfor-mance. Radiology 2002;225:380–390.

40. Pescatore P, Glucker T, Delarive J, Meuli R, Pantoflickova D,Duvoisin B, Schnyder P, Blum AL, Dorta G. Diagnostic accuracyand interobserver agreement of CT colonography (virtual colonos-copy). Gut 2000;47:126–130.

41. Rex DK, Cutler CS, Lemmel GT, Rahmani E, Clark DW, Helper DJ,Lehman GA, Mark DG. Colonoscopic miss rates of adenomasdetermined by back-to-back colonoscopies. Gastroenterology1997;112:24–28.

42. Hixson LJ, Fennerty MB, Sampliner RE, McGee D, Garewal H.Prospective study of the frequency and size distribution of polypsmissed by colonoscopy. J Natl Cancer Inst 1990;82:1769–1772.

43. Godreau CJ. Office-based colonoscopy in a family practice. FamPract Res J 1992;12:313–320.

44. Anderson ML, Heigh RI, McCoy GA, Parent K, Muhm JR, McKeeGS, Eversman WG, Collins JM. Accuracy of assessment of theextent of examination by experienced colonoscopists. Gastroin-test Endosc 1992;38:560–563.

45. Cass OW, Freeman ML, Peine CJ, Zera RT, Onstad GR. Objectiveevaluation of endoscopy skills during training. Ann Intern Med1993;118:40–44.

46. Winawer SJ, Stewart ET, Zauber AG, Bond JH, Ansel H, Waye JD,Hall D, Hamlin JA, Schapiro M, O’Brien MJ, Sternberg SS, GottliebLS. A comparison of colonoscopy and double-contrast bariumenema for surveillance after polypectomy. National Polyp StudyWork Group. N Engl J Med 2000;342:1766–1772.

47. Alluisi EA. Attention and vigilance as mechanisms of response.In: Bilodeau E, ed. Acquisition of skill. New York: AcademicPress, 1966:201–213.

48. Beam CA, Layde PM, Sullivan DC. Variability in the interpretationof screening mammograms by US radiologists: findings from anational sample. Arch Intern Med 1996;156:209–213.

49. McDonald CJ. Protocol-based computer reminders, the quality ofcare and the non-perfectability of man. N Engl J Med 1976;295:1351–1355.

50. Fletcher JG, Johnson CD, Reed JE, Garry J. Feasibility of planarvirtual pathology: a new paradigm in volume-rendered CT colonog-raphy. J Comput Assist Tomogr 2001;25:864–869.

51. Yoshida H, Masutani Y, MacEneaney P, Rubin DT, Dachman AH.Computerized detection of colonic polyps at CT colonography onthe basis of volumetric features: pilot study. Radiology 2002;222:327–336.

52. Summers RM, Beaulieu CF, Pusanik LM, Malley JD, Jeffrey RB Jr,Glazer DI, Napel S. Automated polyp detector for CT colonogra-phy: feasibility study. Radiology 2000;216:284–290.

53. Paik DS, Beaulieu CF, Jeffrey RB, Karadi C, Napel S. Detection ofpolyps in CT colonography: a comparison of a computer-aideddetection algorithm to 3D visualization methods (abstr). Radiol-ogy 1999;213:193.

54. Vining DJ, Ge Y, Ahn DK, Stelts DR. Virtual colonoscopy withcomputer-assisted polyp detection. In: Doi K, MacMahon H, GigerM, Hoffmann K, eds. Computer-aided diagnosis in medical imag-ing. Amsterdam, The Netherlands: Elsevier Science, 1999:445–452.

55. Summers RM, Johnson CD, Pusanik LM, Malley JD, Youssef AM,Reed JE. Automated polyp detection at CT colonography: feasi-bility assessment in a human population. Radiology 2001;219:51–59.

56. Gluecker TM, Johnson CD, Harmsen WS, Offord KP, Harris AM,Wilson LA, Ahlquist DA. Colorectal cancer screening by CTcolonography, colonoscopy, and double-contrast barium enema:prospective assessment of patient perceptions and preferences.Radiology 2003;227:378–384.

57. Hara AK, Johnson CD, MacCarty RL, Welch TJ. Incidental extra-colonic findings at CT colonography. Radiology 2000;215:353–357.

58. Callstrom MR, Johnson CD, Fletcher JG, Reed JE, Ahlquist DA,Harmsen WS, Tait K, Wilson LA, Corcoran KA. CT colonographywithout cathartic preparation: feasibility study. Radiology 2001;219:693–698.

59. Zalis ME, Hahn PF. Digital subtraction bowel cleansing in CTcolonography. AJR Am J Roentgenol 2001;176:646–648.

Received February 14, 2003. Accepted May 1, 2003.Address requests for reprints to: C. Daniel Johnson, M.D., Depart-

ment of Radiology, Mayo Clinic Rochester, 200 First Street S.W.,Rochester, Minnesota 55905.

Supported by NIH grant 2R01 CA75333.Dr. Johnson holds a software license for G.E. Medical Systems.

August 2003 PROSPECTIVE EVALUATION OF CT COLONOGRAPHY 319