Current StatusSteven D. Wexner, M.D., Editor

Quality Assessment and Improvement inColon and Rectal SurgeryAlan Cook, M.D., Neil Hyman, M.D.

Department of Surgery, University of Vermont College of Medicine, Burlington, Vermont

[Key words: Quality; Colon and rectal surgery; Outcomes]

“. . . Quality isn’t method. It’s the goal towardwhich the method is aimed.”1

Quality belongs in medicine as an expression of theintegrity and dignity of caring for a fellow humanbeing. Quality assessment and improvement havecaptured the attention of the medical industry and arenow the very goals toward which a considerableamount of effort is made. We briefly review the evo-lution of quality assessment and improvement inmedicine and surgery. We give examples of the stateof the art and propose a view of the future in whichquality assessment and improvement are tangible anduseful concepts applied daily to the practice of colonand rectal surgery.

EVOLUTION OF QUALITY IN MEDICINE

In recent years, the amount of attention directedtoward quality assessment and improvement seems tobe expanding at an exponential rate. The origin ofoutcomes measurement and quality improvement canbe traced to the mid 19th century. Florence Nightin-gale is cited as the first pioneer in the application ofthis discipline in medicine. She persuaded surgeons

and hospitals to assume accountability for factual rec-ord keeping and reporting. She recorded patients’ re-sponses to interventions and developed statisticalmethods to track this information. Her efforts resultedin a decrease in the mortality rate of British soldiersduring the Crimean War.2–5

Sixty years later, an American surgeon became thenext to advocate outcome measurement in medicalpractice. E. A. Codman, a surgeon practicing at theMassachusetts General Hospital (MGH), published“The Product of a Hospital” in surgery, gynecology,and obstetrics in 1914. In it he sought “. . . to stimulatethought on and discussion of the standardization ofhospitals.” He believed that hospitals should be heldaccountable for the results of their therapies. To thisend he posed the question, “What happens to thecases?”6 He advocated that outcomes be scientificallymeasured on a patient-by-patient basis and that hos-pitals should publish outcomes data in a uniformmanner to allow for comparison. He practiced whathe preached. He kept an “end result card” on everypatient he treated and proposed that the same bedone on a hospital-wide basis. He also developed aclassification scheme for errors related to adverse out-comes. Codman chaired the hospital standardizationcommittee formed in 1912 by the Clinical Congress ofthe Surgeons of North America (the forerunner to theAmerican College of Surgeons).7

The idea of measuring and scrutinizing outcomeswas not unique to medicine. In 1925, Shewhart pub-lished, “The Application of Statistics as an Aid in Main-taining Quality of a Manufactured Product.”8 In this,he explained how statistical methods help in estimat-ing the significance of observed trends, thus assistingin the maintenance of the quality of the product. The

Reprints are not available.

Correspondence to: Neil Hyman, M.D., Fletcher 301, MCHVCampus, Department of Surgery, University of Vermont College ofMedicine, Burlington, Vermont 05401, e-mail: Neil.Hyman@vtmednet.org

Dis Colon Rectum 2005; 47: 2195–2201DOI: 10.1007/s10350-004-0742-5© The American Society of Colon and Rectal SurgeonsPublished online: 26 October 2004

2195

theory of industrial quality improvement was furtherexpanded by Juran and Deming who asserted that thekey to quality improvement came in gathering andunderstanding data about the manufacturing processitself. These theories came to be collective known as“Total Quality Management” or “Continuous QualityImprovement.” These theories of industrial processmeasurement and improvement became the core ofthe Japanese industrial revolution and later gainedfavor in American industry.9 Laffell and Blumenthal10

have illustrated that health care organizations couldapply the techniques of industrial quality manage-ment science in an effort to provide optimal qualityhealth care. The definition of quality as it applied tothe practice of medicine was evolving during thissame time.

Donabedian11 offered a definition of quality inmedical care and a conceptual framework for qualityassessment. He defined the quality of care as “. . . thatkind of care which is expected to maximize an inclu-sive measure of patient welfare, after one has takenaccount of the balance of expected gains and lossesthat attend the process of care in all its parts.”12 Medi-cal outcomes, he argued, remain the ultimate valida-tors of the effectiveness and quality of medical care.He wrote that quality of medical care could be as-sessed by gathering information that could be classi-fied into three categories: structure, process, and out-come. Structure, as defined in this context, is thesetting in which care occurs, including material andhuman resources, and organizational structure. Pro-cess describes not only the acts of diagnosis and treat-ment, but also the patient’s seeking of care and fol-lowing medical advice. Finally, outcome denotes theeffects of care on the health status of individuals andpopulations including the patient’s degree of satisfac-tion with care. This approach is based on the logicthat “. . . good structure increases the likelihood ofgood process, and good process increases the likeli-hood of a good outcome.”13

In 1990, the Institute of Medicine (IOM) refined thedefinition of quality health care: “Quality of care is thedegree to which health services for individuals andpopulations increase the likelihood of desired healthoutcomes and are consistent with current professionalknowledge.”14 Other theorists in the field of medicalquality assurance had previously proposed methodsfor monitoring and evaluating medical care. Rutsteinand coworkers15 argued that because there were noeasily measured quantitative definitions of variousstates of health, a quantitative index of sentinel ad-

verse health events could be used to identify areas ofneeded improvement. Williamson16 offered a deci-sion-making process for the selection of quality im-provement priorities. He described a process of struc-tured group judgment that, he wrote, was especiallysuited for planning performance evaluation projectsby organizations like the Joint Commission on theAccreditation of Hospitals, the predecessor of JCAHO.

While these concepts were developing in the 1970sand 1980s, it seemed that little was being accom-plished toward actually improving the quality ofmedical care. A concurrent development was theemergence of managed care as a presence in thehealth care industry. Managed care evolved in re-sponse to employers’ demands to contain medicalcosts. The focus of health care institutions shifted toefficiency to maintain viability as hospitals and clinicsfelt the effects of cost containment. Practices by healthcare plans, such as denial of care and a perceivedintrusion into the physician-patient relationship,raised concern that managed care might represent athreat to the quality of American health care.17 In1995, the IOM convened the National Roundtable onHealth Care Quality. After a series of six meetingsduring two years, they reached the consensus that“. . . Serious widespread problems exist throughoutAmerican medicine, and that quality of care was theproblem, not managed care.”14

After this consensus statement, the IOM issued thecontroversial and often debated report, “To Err Is Hu-man: Building a Safer Health System” in 2000.18 Theyconcluded that tens of thousands of Americans dieeach year from errors in care, and hundreds of thou-sands of nonfatal, preventable injuries occur. Healeyand coworkers19 prospectively tracked complicationsin 4,743 surgical inpatients at a single institution. Al-though the morbidity and mortality rates comparedfavorably with national benchmarks, one-half of theadverse events were judged contemporaneously bypeers to be avoidable. This was contrary to the argu-ments that the IOM report overestimated rates of com-plication.20,21 The harm done to patients, the in-creased financial costs to institutions and payers, theloss of trust of the patients and community, and theloss of morale and job satisfaction among health careworkers form the collective burden of medical errorlisted in “To Err is Human.” In 2001, the IOM thenpublished, “Crossing the Quality Chasm: A NewHealth System for the 21st Century.” In this, they de-clare that quality is a property of the system of healthcare delivery and that to improve the quality of medi-

2196 COOK AND HYMAN Dis Colon Rectum, December 2004

cal care, systems of care will have to be changed.Among their recommendations they state, “. . . with-out ongoing tracking to assess progress . . . healthprofessionals will be unable to determine progress orunderstand where improvement efforts have suc-ceeded and where further work is most needed.”22

QUALITY ASSESSMENT IN SURGERY

The practice of surgery provides the ideal oppor-tunity for medical quality assessment. It is generally adiscreet intervention to address a defined problemwith a measurable, expected outcome. The surgicalmorbidity and mortality conference has long been afixture in the tradition of surgery. It is perhaps, thefirst effort of surgeons to confront and learn from er-ror, although quality assessment and improvement gobeyond the cathartic, anecdotal nature of the tradi-tional morbidity and mortality conference. Codman,as previously described, championed the systematicgathering, analysis, and reporting of what would nowbe known as outcome data. Several factors importantto the quality and success of reporting systems havebeen identified by current investigators. They includeconfidentiality or data de-identification, independentoutsourcing of report collection and analysis by peerexperts, rapid and meaningful feedback, ease of re-porting and sustained leadership support.23 Addition-ally, Ireson and Schwartz4 emphasized that outcomemeasurement should include the patient’s satisfactionwith care, functional status, health-related quality oflife, and that these measures should be assessed atseveral points across the trajectory of illness.

Two current collaborative surgical quality initiativeshave received a significant amount of attention in themedical literature and serve as models for quality im-provement in surgery: the Northern New EnglandCardiovascular Disease Study Group (NNE),24 andthe National Surgical Quality Improvement Programof the Veterans Administration Hospital System(NSQIP).25 The NNE was established in 1987 to de-termine whether an organized, multi-institutional in-tervention could improve the hospital mortality ratesassociated with coronary artery bypass surgery. Thegroup consisted of all five hospitals in the northernNew England region that performed coronary arterybypass surgery. The components of the interventionincluded prospective data collection, feedback of out-come data, training in continuous quality improve-ment techniques, and site visits to other participating

medical centers. The most striking result of this proj-ect was a 24 percent reduction in the hospital mortal-ity rate after the nine-month intervention. Of equalsignificance was the fact that the NNE project createdan infrastructure for the collection and sharing of data,consensual development of performance metrics,study and reporting of variability in outcomes, en-couragement of cross-institutional visitation and ob-servation of care processes, linking of clinical pro-cesses to cause-specific outcomes, and continuousmonitoring of processes and outcomes.24

The NSQIP was initiated in 1991 with the goal ofdeveloping and validating risk-adjusted models forthe prediction of surgical outcome, and for the com-parative assessment of the quality of major surgicalcare among the Veterans Administration (VA) surgicalcenters. This initiative was undertaken in response tocongressional criticism that surgical care at the VAmedical centers was substandard compared with thatprovided in the private sector. The principals of theNSQIP felt the criticism of VA surgical quality unfairlyneglected consideration of case mix differences be-cause of the inherently higher risk profile of the VApatient population. To accomplish their goal, twotasks were undertaken. The first was to develop areliable clinical database of the patients’ relevant pre-operative risk factors and postoperative outcomes.This was accomplished by a trained clinical nurse re-viewer at each of the 44 participating VA medical cen-ters. Data for risk and 30-day outcome variables werecollected prospectively on all major noncardiac cases.The second task was to develop analytic tools forproper risk adjustment and to account for randomevents. After comparing 11 morbidity scoringschemes, the final models were developed. Operativecomplexity was accounted for using a complexityscore created by a panel of six expert surgeons foreach of 3,000 CPT codes contained in the VA data-base.26 The National Veterans Affairs Surgical RiskStudy applied the risk adjustment model to the par-ticipating hospitals and found that risk adjustment hadan appreciable impact on the rank ordering of hospi-tals, thus demonstrating that the raw outcome datadid not accurately reflect the quality of care deliveredby any one individual medical center.25 In 1994, theNSQIP model was expanded to all 133 VA medicalcenters. Feedback to surgeons includes risk-adjustedoutcomes, patient risk profiles, and length of stay datathat is benchmarked against national and peer groupdata. Additionally, NSQIP disseminates the best prac-tices, identified from the institutions with the best per-

2197QUALITY ASSESSMENT IN COLORECTAL SURGERYVol. 47, No. 12

formance and institutions that have made the greatestimprovement in a performance area.26 The VA 30-daysurgical morbidity and mortality rates decreased by 30and 9 percent, respectively, within three years of em-barking on this landmark effort.27

The NNE and NSQIP are examples of effective, sur-geon-driven initiatives to improve the quality of sur-gical outcomes. Several characteristics common toNNE and NSQIP are worth highlighting. Data collec-tion for both programs is accomplished by nurse re-viewers, in a prospective manner, using predeter-mined criteria. The NNE gathers preoperative clinicaland in-hospital mortality data, whereas NSQIP gatherspreoperative, intraoperative, postoperative, and 30-day mortality data.24,26 Both programs use risk-adjustment models based on data gathered from theirrespective patient cohorts. Site visits were performedby NNE and NSQIP teams to compare structures andprocesses of care. Best practices were shared amongall participating members in each program. Perhapsmost importantly, participants were given feedbackon their outcomes. In the NNE, the surgeon receivesthree reports: their own data, that of their institution,and regional data.24 NSQIP reports hospital-specificdata, including workload, risk-adjusted outcomes, pa-tient risk profiles, and length of stay, with all datacompared to national averages and peer group data.In contrast to the NNE, NSQIP discourages generationof surgeon-specific risk-adjusted outcomes for twoimportant reasons. First, they believe that the typicalVA surgeon does not perform enough operations toprovide statistically significant sample sizes for analy-sis, and second, they believe that it is not possible toseparate the performance of the individual surgeonfrom the system in which they work.26

With the success of the NSQIP program in the VAsystem, a pilot study was performed in 1999 to test theability of the NSQIP models in the nonfederal system.The general and vascular surgery services of threeacademic medical centers participated in the study.Despite the greater variability in patient populations,the NSQIP model has demonstrated validity in non-federal institutions.28

WHY MEASURE QUALITY?

We have discussed the evolution of quality assess-ment and improvement in medicine and two initia-tives that have demonstrated the value of collabora-tive surgical quality improvement programs. At this

point, an important question must be asked: whyshould surgeons commit the resources and availthemselves to the tracking and reporting of their out-comes? First, and perhaps foremost, NNE and NSQIPhave demonstrated that quality assessment improvessurgical outcomes. Moreover, quality measurementmay be valuable to surgeons for the following fouradditional reasons: 1) document the value added of asurgical specialty; 2) compel third-party payers tocontract with specialists based on improved out-comes; 3) prevent other interested parties (e.g., gov-ernment agencies, collective payer groups) from im-posing ill-conceived metrics of quality based onpolitical expediency; and 4) provide accountability toan increasingly interested public.

Quality measurement and reporting can inform theliterature of the value of specialty training and dis-seminate best practices. The evidence for the addedvalue of colorectal surgeons to high-quality care isaccumulating in the medical literature. Rosen and col-leagues,29 using a state-legislated database, demon-strated that board-certified colorectal surgeons had alower in-hospital mortality rate and lower averagelength of stay for colorectal procedures comparedwith other institutional surgeons in the database.Smedh and coworkers30 conducted a study to auditthe results for rectal cancer surgery performed in anewly created colorectal unit led by a colorectal sur-geon. Compared with historical controls, the out-comes of the colorectal unit were superior, includingan eightfold decrease in the mortality rate, decreasesin the overall complication rate, relaparotomy rate,and postoperative length of stay. Data from both ofthese studies are vulnerable to criticism because oftheir retrospective nature and prospective validationis needed.

A recent prospective study demonstrated the feasi-bility of a surgeon initiated quality assessment andimprovement project in a rural state. The data for thisproject were submitted voluntarily, under peer-review protection, for 364 consecutive patients under-going elective surgery for colorectal cancer in 11 of 13acute care hospitals in Vermont, including the tertiarycare medical center. In addition to demonstrating thefeasibility of a collaborative quality improvement ef-fort in colorectal cancer surgery, the project alsoshowed that colorectal cancer care in Vermont is safeand adherence to national standards for the use ofadjuvant chemotherapy and radiation was outstand-ing.31 A prospective New England regional qualityinitiative is presently underway, sponsored by the Re-

2198 COOK AND HYMAN Dis Colon Rectum, December 2004

search Foundation of The American Society of Colo-rectal Surgeons.

Quality assurance and improvement are becomingtangible entities in the business realm of medical prac-tice. A number of large employers are forming collec-tive organizations intent on forwarding the movementfor quality in medicine. Their approach includes cre-ating contractual requirements aimed at addressingspecific issues related to improving medical quality.The Leapfrog Group is the prototype of such an or-ganization. Their initial efforts have targeted patientsafety by advocating for “evidence-based hospital re-ferral,” computerized physician order entry systems,and intensivist staffing in intensive care units. Theevidence-based hospital referral initiative channelspatients to certain hospitals for conditions or proce-dures based on hospital teaching status and volumefor those conditions and procedures.2,32 The creationof contractual requirements or payment of incentivesbased on quality measures argue for the value of qual-ity assurance and improvement in the marketplace.Given this trend toward the involvement of payers asstakeholders in the quality movement, one couldforesee the advantage or even the necessity of indi-vidual surgeons measuring outcomes and participat-ing in quality improvement in their daily practice.

The need for surgeons to become constructive par-ticipants in the movement of quality improvement isbecoming a matter of increasing urgency. The debateregarding the purpose and validity of quality mea-sures is composed of different groups of stakeholders,including health care providers, payers, patients, andmedical institutions. Each group has individual con-cerns regarding the quality of healthcare and the wayit is measured, although a lack of valid and reliablemeasures are a clear hindrance to effective initia-tives.33 In the absence of prospectively gathered in-dicators developed on a diagnosis-specific or proce-dure-specific basis, the selection of measurements foruse by payers or consumers may, by default, bedriven by administrative databases with their inherentlimitations.

Several investigators have shown that administra-tive databases are vulnerable to coding bias and ques-tions have been raised about the appropriateness ofhealth care policy and public reporting based on suchdata.2,33–38 For example, Finlayson and Birkmeyer,39

using Medicare data, demonstrated increased life ex-pectancy associated with colon cancer operationsperformed at “high-volume hospitals.” Additionally,Schrag et al.,40 after an analysis of Survey of Epide-

miology and End Results-Medicare linked data, sug-gested that 350 postoperative deaths and 770 deathsfive years after colon cancer surgery could be poten-tially avoided annually if patients treated for coloncancer at low-volume hospitals were rerouted tohigh-volume hospitals. The National Inpatient Samplewas used by Birkmeyer et al.41 to analyze the poten-tial effects of implementation of the Leapfrog Group’srequirement of hospitals caring for their employees tomeet volume standards for five high-risk surgical pro-cedures. They estimate that 2,581 lives would besaved. However, when prospective data were used,Meyerhardt and coworkers42 found that there was nosignificant relationship between hospital procedurevolume and the risk for cancer recurrence or cancer-specific mortality. Panageas and associates43 reana-lyzed the Survey of Epidemiology and End Results-Medicare linked database to assess the impact ofsurgeon and hospital volume on outcomes and ap-plied appropriate statistical techniques to correct forinterrelated variables; they found that the statisticalsignificance of the volume-outcome relationship wasattenuated substantially. The literature is repletewith statistical analyses of administrative databasesclaiming improved outcomes based on hospital vol-ume,39–41,44,45 surgical volume,45 years of experi-ence,46 specialty training or board certification29,46,47;it is becoming increasingly clear that these relation-ships are complex and that using these administrativedatabases to create healthcare policy (e.g., regional-ization of surgical services) is highly questionable.

ONE VIEW OF THE FUTURE OF QUALITYASSESSMENT AND IMPROVEMENT IN

COLON AND RECTAL SURGERY

The movement of quality assessment and improve-ment is becoming established as a presence in theenvironment of health care. Nelson et al.48 outlinedthe following eight principles for incorporating mea-surement and data collection into daily medical prac-tice: 1) seek usefulness, not perfection, in the mea-surement; 2) use a balanced set of process, outcome,and cost measures; 3) keep measurement simple; 4)use qualitative and quantitative data; 5) write downthe operational definitions of measures; 6) measuresmall, representative samples; 7) build measurementinto daily work; and 8) develop a measurement team.

How can colon and rectal surgeons participate inand gain the benefits of quality assessment and im-

2199QUALITY ASSESSMENT IN COLORECTAL SURGERYVol. 47, No. 12

provement in the practice of our profession? Weshould not wait until the “perfect” system has beendeveloped before beginning a collaborative effort tomeasure and improve outcomes. The process couldresemble the NSQIP model in which peer-review,protected-data is gathered by physician surrogatesand a risk-adjustment method is applied to allow formeaningful reporting and comparison. The surgeonwould enter a limited amount of data of particularvalue and significance in a manner that is convenientand reliable. Certain highly relevant nuances of anoperation or disease process are not easily expressedin categoric terms by anyone but the operating sur-geon (e.g., “difficult” vs. “easy” pelvis in a rectal dis-section, anatomy of an anal fistula). Finally, somemeasure of patient quality of life and satisfactionwould be included to allow for improvement whereindicated. A surgeon’s results would be compiled andadded to a regional or national database such as thatdescribed by Gunnarsson and colleagues.38 Feedbackwould include individual and hospital data with re-gional or national comparison after confidential peerreview. It would allow for performance measurementusing meaningful indices and risk-adjustment.

CONCLUSIONS

Continuous benchmarking and definition of “bestpractices” clearly improves outcomes and quality.This must occur in a nonpunitive background, en-couraging surgeons to provide the data required toassess where there are opportunities for improve-ment. Reliable peer-review protection is critical be-cause medicolegal considerations are the clear andmenacing threat to the reporting that is required toimprove patient safety. Surgeons must participate andembrace quality assessment and improvement; ourpatients deserve no less.

REFERENCES

1. Pirsig RM. Zen and the art of motorcycle maintenance.New York: Bantam Doubleday Dell Publishing Group,1981:380.

2. Ireson CI, Ford MA, Hower JM, Schwartz RW. Outcomereport cards: a necessity in the health care market. ArchSurg 2002;137:46–51.

3. Osler T, Horne L. Quality assurance in the surgical in-tensive care unit: where it came from and where it’sgoing. Surg Clin North Am 1991;71:887–904.

4. Ireson CL, Schwartz RW. Measuring outcomes in surgi-cal patients. Am J Surg 2001;181:76–80.

5. Wangensteen OH, Wangensteen SD. The rise of sur-gery: from empiric craft to scientific discipline. Minne-apolis: University of Minnesota Press, 1978:380.

6. Codman EA. The product of a hospital. Surg GynecolObstet 1914;18:491–6.

7. Passaro E, Organ CH, Ernest A. Codman: the improperBostonian. Bull Am Coll Surg 1999;84:16–22.

8. Shewhart WA. The application of statistics as an aid inmaintaining quality of a manufactured product. J AmStat Assoc 1925;20:546–8.

9. Berwick DM. Continuous improvement as an ideal inhealth care. N Engl J Med 1989;320:53–6.

10. Laffel G, Blumenthal D. The care for using industrialquality management science in health care organiza-tions. JAMA 1989;262:2869–73.

11. Donabedian A. Evaluating the quality of medical care.Millbank Memorial Fund Quarterly 1966;44:166–206.

12. Donabedian A. Explorations in quality assessment andmonitoring: the definition of quality and approaches toits assessment. Vol. 1. Ann Arbor: Health AdministrationPress, 1980:163.

13. Donabedian A. The quality of care: how can it be as-sessed? JAMA 1988;260:1743–8.

14. Chassin MR, Galvin RW. National roundtable on healthcare quality. The urgent need to improve health carequality. Institute of Medicine National Roundtable onHealth Care Quality. JAMA 1998;280:1000–5.

15. Rutstein DD, Berenberg W, Chalmers TC, Child CG,Fishman AP, Perrin EB. Measuring the quality of medi-cal care: a clinical method. N Engl J Med 1976;294:582–8.

16. Williamson JW. Formulating priorities for quality assur-ance activity: description of a method and its applica-tion. JAMA 1978;239:631–7.

17. Sekhri NK. Managed care: the U.S. experience. BullWorld Health Organ 2000;78:830–44.

18. Committee on Quality of Health Care in America. To erris human: building a safer health system. In: Kohn LT,Corrigan JM, Donaldson MS, eds. Washington, D.C.: Na-tional Academy Press, 2000:287.

19. Healey MA, Shackford SR, Osler TM, Rogers FB, BurnsE. Complications in surgical patients. Arch Surg 2002;137:611–8.

20. Hayward RA, Hofer TP. Estimating hospital deaths dueto medical errors: preventing is in the eye of the re-viewer. JAMA 2001;286:415–20.

21. McDonald CJ, Weiner M, Hui SL. Deaths due to medicalerrors are exaggerated in Institute of Medicine report.JAMA 2000;284:93–5.

22. Committee on Quality of Health Care in America. Cross-ing the quality chasm: a new health system for the 21stcentury. Washington, D.C.: National Academy Press,2001:337.

23. Barach P, Small SD. Reporting and preventing medicalmishaps: lessons from non-medical near miss reportingsystems. BMJ 2000;320:759–63.

2200 COOK AND HYMAN Dis Colon Rectum, December 2004

24. O’Connor GT, Plume SK, Olmstead EM, et al. A regionalintervention to improve the hospital mortality associ-ated with coronary artery bypass graft surgery. JAMA1996;275:841–6.

25. Khuri SF, Daley J, Henderson WG, et al. Risk adjustmentof the postoperative mortality rate for the comparativeassessment of the quality of surgical care: results of thenational Veterans Affairs surgical risk study. J Am CollSurg 1997;185:325–38.

26. Khuri SF. Quality, advocacy, healthcare policy, and thesurgeon. Ann Thorac Surg 2002;74:641–9.

27. Khuri SF, Daley J, Henderson WG, et al. The Depart-ment of Veterans Affairs’ NSQIP: the first national, vali-dated, outcome-based, risk-adjusted, and peer-controlled program for the measurement andenhancement of the quality of surgical care. Ann Surg1998;228:491–507.

28. Fink AS, Campbell DA, Mentzer RM, et al. The nationalsurgical quality improvement program in non-VeteransAdministration hospitals: initial demonstration of feasi-bility. Ann Surg 2002;236:344–54.

29. Rosen L, Stasik JJ, Reed JF, Olenwine JA, Aronoff JS,Sherman D. Variations in colon and rectal surgical mor-tality: comparison of specialties with a state-legislateddatabase. Dis Colon Rectum 1996;39:129–35.

30. Smedh K, Olsson L, Johansson H, Aberg C, Anderson M.Reduction of postoperative morbidity and mortality inpatients with rectal cancer following the introduction ofa colorectal unit. Br J Surg 2001;88:273–7.

31. Hyman N, Labow SB. The Vermont colorectal cancerproject: self-portrait. Arch Surg 2002;137:413–6.

32. Bodenheimer T. The American health care system: themovement for improved quality in health care. N Engl JMed 1999;340:488–92.

33. Grol R. Improving the quality of medical care: buildingbridges among professional pride, payer profit and pa-tient satisfaction. JAMA 2001;286:2578–85.

34. Naylor CD. Public profiling of clinical performance.JAMA 2002;287:1323–5.

35. Jencks SF, Daley J, Draper D, Thomas N, Lenhart G,Walker J. Interpreting hospital mortality data: the role ofclinical risk adjustment. JAMA 1988;260:3611–6.

36. Green J, Wintfeld N. How accurate are hospital dis-charge data for evaluating effectiveness of care. MedCare 1993;31:719–31.

37. Romano PS, Mark DH. Bias in the coding of hospitaldischarge data and its implications for quality assess-ment. Med Care 1994;32:81–90.

38. Gunnarsson U, Seligsohn E, Jestin P, Pahlman L. Regis-tration and validity of surgical complications in colorec-tal cancer surgery. Br J Surg 2003;90:454–9.

39. Finlayson EV, Birkmeyer JD. Effects of hospital volumeon life expectancy after selected cancer operations inolder adults: a decision analysis. J Am Coll Surg 2003;196:410–7.

40. Schrag D, Cramer LD, Bach PB, Cohen AM, Warren JL,Begg CB. Influence of hospital procedure volume onoutcomes following surgery for colon cancer. JAMA2000;284:3028–35.

41. Birkmeyer JD, Finlayson EV, Birkmeyer CM. Volumestandards for high-risk surgical procedures: potentialbenefits of the Leapfrog initiative. Surgery 2001;130:415–22.

42. Meyerhardt JA, Catalano PJ, Schrag D, et al. Associationof hospital procedure volume and outcomes in patientswith colon cancer at high risk for recurrence. Ann In-tern Med 2003;139:649–57.

43. Panageas KS, Schrag D, Reidel E, Bach PB, Begg CB.The effect of clustering of outcomes on the associationof procedure volume and surgical outcomes. Ann In-tern Med 2003;138:658–65.

44. Birkmeyer JD, Siewers AF, Finlayson EV, et al. Hospitalvolume and surgical mortality in the United States. NEngl J Med 2002;346:1128–37.

45. Holm T, Johansson H, Cedermark B, Ekelund G,Rutqvist LE. Influence of hospital- and surgeon-relatedfactors on outcome after treatment of rectal cancer withor without preoperative radiotherapy. Br J Surg 1997;84:657–63.

46. Prystowsky JB, Bordage G, Feinglass JM. Patient out-comes for segmental colon resection according to sur-geon’s training, certification, and experience. Surgery2002;132:663–72.

47. Singh KK, Barry MK, Ralston P, et al. Audit of colorectalcancer surgery by non-specialist surgeons. Br J Surg1997;84:343–7.

48. Nelson EC, Splaine ME, Batalden PB, Plume SK. Build-ing measurement and data collection into medical prac-tice. Ann Intern Med 1998;128:460–6.

2201QUALITY ASSESSMENT IN COLORECTAL SURGERYVol. 47, No. 12