ORIGINAL ARTICLES

A Validation Study of a Retrospective Venous ThromboembolismRisk Scoring Method

Vinita Bahl, DMD, MPP,* Hsou Mei Hu, PhD,* Peter K. Henke, MD,† Thomas W. Wakefield, MD,†Darrell A. Campbell, Jr., MD,‡ and Joseph A. Caprini, MD§

Objectives: Validate a retrospective venous thromboembolism (VTE) riskscoring method, which was developed at the University of Michigan HealthSystem and based on the Caprini risk assessment model, and assess theconfounding effects of VTE prophylaxis.Background: Assessing patients for risk of VTE is essential to initiatingappropriate prophylaxis and reducing the mortality and morbidity associatedwith deep vein thrombosis and pulmonary embolism.Methods: VTE risk factors were identified for 8216 inpatients from theNational Surgical Quality Improvement Program using the retrospectivescoring method. Logistic regression was used to calculate odds ratios (OR)for VTE within 30 days after surgery for risk factors and risk level. Abivariate probit model estimated the effects of risk while controlling foradherence to prophylaxis guidelines.Results: Distribution of the study population by risk level was highest,52.1%; high, 36.5%; moderate, 10.4%; and low, 0.9%. Incidence of VTEwithin 30 days was overall 1.4%; by risk level: highest, 1.94%; high, 0.97%;moderate, 0.70%; low, 0%. Controlling for length of hospitalization (�2 d)and fiscal year, pregnancy or postpartum (OR � 8.3; 1.0–68, P � 0.05),recent sepsis (4.0; 1.4–10.9, P � 0.01), malignancy (2.3; 1.5–3.3, P � 0.01),history of VTE (2.1; 1.1–4.1, P � 0.05), and central venous access (1.8;1.1–3.0, P � 0.05) were significantly associated with VTE. Risk level wassignificantly associated with VTE (1.9; 1.3–2.6, P � 0.01). The bivariateprobit demonstrated significant correlation between the probability of VTEand lack of adherence to prophylaxis guidelines (� � 0.299, P � 0.013).Conclusion: The retrospective risk scoring method is valid and supports useof individual patient assessment of risk for VTE within 30 days after surgery.

(Ann Surg 2010;251: 344–350)

Venous thromboembolism (VTE), a disease which includes deepvein thrombosis (DVT) and pulmonary embolism (PE), is a

common complication among hospitalized patients, where reportedincidence ranges from 10% to 40% in medicine and general surgerypopulations in the absence of appropriate thrombosis prophylaxis.VTE results in significant mortality and morbidity; approximately10% of hospital deaths are attributed to PE, and patients who do notdie have an increased risk of postthrombotic syndrome, pulmonaryhypertension, or recurrent thrombosis.1

There is substantial evidence that primary thromboprophy-laxis prevents VTE. According to the Agency for Healthcare Re-search and Policy, VTE prevention for at-risk patients presents themost significant opportunity to improve patient safety in hospitalsamong 79 patient safety practices because of its efficacy, cost-effectiveness, and benefit-risk ratio.2 As a consequence, improve-ments in VTE prevention have been targeted by the Centers forMedicare and Medicaid Services, the National Quality Forum, andmost recently, the Surgeon General of the United States.3–5

However, despite strong evidence that VTE prophylaxis re-duces the incidence of DVT and PE, numerous studies show thatprophylaxis is underutilized or inadequate for at-risk patients.6–8

Accurate assessment of patient VTE risk is critical to improvingcompliance with prophylaxis guidelines,7 but risk assessment can becomplex because a variety of VTE risk factors must be considered,each risk factor confers a different relative risk, and the effect ofmultiple risk factors is cumulative.9,10

Several risk assessment models that stratify patients accord-ing to their risk of VTE have been published, the most notable beingthose developed by Caprini, Cohen, and Kucher.11–13 These riskassessment models consist of a list of exposing risk factors (pre-senting illness or procedure) and predisposing risk factors (geneticand clinical characteristics), each with an assigned relative riskscore.11,14 The scores are summed to produce a cumulative score,which is used to classify the patient to 1 of 3 or 4 risk levels anddetermine the onset, intensity, type, and duration of prophylaxis.These models have been criticized for being cumbersome andbecause very few have been validated.1,15 Consequently, the impactof the weight and interaction of individual risk factors on cumulativerisk has not been tested.

As an alternative, the American College of Chest Physiciansadvocate a simpler approach, assigning risk according to the patientgroup to which an individual belongs. The patient group describesthe primary reason the patient was hospitalized, such as majorgeneral surgery or major orthopedic surgery, and each was tested inrandomized clinical trials of thromboprophylaxis. Though somepatient-specific, predisposing risk factors may be considered, thismethod does not promote an individualized approach to risk assess-ment and thromboprophylaxis.1

In 2005, the University of Michigan Health System (UMHS)adopted the Caprini risk assessment model11 for medicine andsurgery patients and prescribed a prophylaxis regimen for each risklevel. To measure compliance with UMHS prophylaxis guidelines,we developed a retrospective VTE risk scoring method based on theCaprini model, which uses data from electronic sources, includinghospital billing data, the operating room information system, andclinical data repository. A test of the reliability of this method waspreviously conducted.16 The goal of this study was to validate ourscoring method.

An important step toward encouraging assessment of individ-ual patient risk for VTE at UMHS and improving compliance withprophylaxis guidelines is to demonstrate the link between individualpatient risk assigned by the Caprini model and VTE outcomes.17

The specific objectives of this study were to: (1) validate the

From the Departments *Clinical Information & Decision Support Services, Officeof Clinical Affairs, University of Michigan, Ann Arbor, MI; †Section ofVascular Surgery, Department of Surgery, University of Michigan, AnnArbor, MI; ‡Office of Clinical Affairs, University of Michigan, Ann Arbor,MI; and §Department of Surgery, Evanston Northwestern Healthcare, EvanstonIL and Feinberg School of Medicine, Northwestern University, Chicago, IL.

Institutional Review Board Study ID: HUM00023501.Reprints: Vinita Bahl, DMD, MPP, Clinical Information & Decision Support

Services, Office of Clinical Affairs, University of Michigan Health System,300 North Ingalls 7A10, Ann Arbor, MI 48109–5485. E-mail: vbahl@umich.edu.

Copyright © 2010 by Lippincott Williams & WilkinsISSN: 0003-4932/10/25102-0344DOI: 10.1097/SLA.0b013e3181b7fca6

Annals of Surgery • Volume 251, Number 2, February 2010344 | www.annalsofsurgery.com

internally-developed retrospective VTE risk scoring method that isbased on the Caprini risk assessment model for a population ofgeneral, vascular, and urologic surgery patients discharged fromUMHS and (2) assess the confounding effects of provider adherenceto UMHS VTE prophylaxis guidelines on the results of the valida-tion study.

METHODSA total of 8216 general, vascular, and urologic surgery

inpatients from the UMHS National Surgical Quality Improvement(NSQIP) Program discharged between July 2001 and January 2008were selected for study. The NSQIP is a program that measures andpromotes improvement in surgical outcomes and is well-docu-mented in the surgery literature. Nurses, who receive in-depthtraining to ensure data reliability, collect preoperative risk factors,operative variables, and 30-day postoperative mortality and morbid-ity outcomes, including VTE, in surgical patients.18

For 30-day VTE outcomes, nurses review patient medicalrecords for evidence of a DVT or PE from diagnostic test results andphysician notes. A DVT is noted when confirmed by duplex,venogram, or CT scan or evidence of treatment with anticoagulantsor insertion of a vena cava filter. A PE is confirmed by V-Q scan,pulmonary arteriogram, or CT angiogram.

The Caprini risk assessment model was introduced at UMHSin 2005 to improve compliance with VTE prophylaxis guidelines formedicine and surgery patients and uses a point-scoring system;the relative scores for individual risk factors are summed toproduce a cumulative risk score that defines the patient’s risklevel (low, moderate, high, or highest risk) and associated pro-phylaxis regimen (Fig. 1).

The VTE cumulative risk score and risk level was assigned toeach patient in the study population using an internally-developedretrospective scoring method that is based on the Caprini model.Data for each VTE risk factor were obtained from electronic

FIGURE 1. UMHS VTE patient risk assessment model and prophylaxis guidelines.

Annals of Surgery • Volume 251, Number 2, February 2010 Validation of a VTE Risk Scoring Method

© 2010 Lippincott Williams & Wilkins www.annalsofsurgery.com | 345

sources. Hospital billing data were used to obtain patient age,ICD-9-CM diagnoses present at time of hospitalization or within amonth or year prior to hospitalization, and ICD-9-CM proceduresperformed during the hospitalization. The operating room informa-tion system was used to define major and minor surgery based onsurgical duration and the clinical data repository to calculate bodymass index (BMI). Risk factors for each patient were scored andsummed to determine the cumulative VTE risk and associated risklevel. A test of the reliability of this retrospective scoring method wasconducted for a population of 1470 general surgery inpatients whoseVTE risk was prospectively assigned by physician assistants during thepatients’ preoperative history and physical. The weighted kappa coef-ficient comparing these 2 approaches was 0.572 (0.525�0.618, P �0.001), indicating acceptable agreement.16

A descriptive analysis of the percentage of the study popula-tion with each risk factor was conducted. Of the patients with eachrisk factor, the differences in the percentage who did and did notdevelop VTE were compared using a �2 test. The distribution of theincidence rate of VTE by risk level and cumulative risk score andthe significance of differences was reported using a �2 test.

Logistic regression was used to calculate odds ratios for VTEof individual risk factors and of the 4 risk levels.19 The model isspecified to include dichotomous risk factors as the explanatoryvariables, and the outcome variable of VTE. Odds ratios and the95% confident intervals were reported. Ten risk factors whosecoefficient estimates were insignificant due to small size (no morethan 0.2% of the study sample), and which did not contribute to themodel fit were excluded from the final model. A second logisticregression model was identified for the outcome of VTE regressedon 4 risk levels, as an ordinal categorical variable, coded 1 to 4 forlow, moderate, high, and highest risk groups. Both models weretested for its fit using Hosmer-Lemeshow goodness-of-fit test, andc-statistic was reported for its discriminative power.

In both models, 2 control variables were included. Length ofhospitalization of 2 days or longer was added to control for unob-served factors associated with short stays on VTE outcomes. Fiscalyear was used to control for unobserved changes in clinical practice,medical record documentation, and ICD-9-CM diagnosis and pro-cedure coding practices.

Because VTE prophylaxis protects patients from acquiringVTE, the rates of pharmacologic prophylaxis by risk level werereported and the confounding effects of adherence to UMHS pro-phylaxis guidelines were studied. Unfortunately, information aboutactual use of pharmacologic and mechanical prophylaxis were notreadily available without manual abstraction of medication admin-istration records and nursing notes, so drug orders and hospitalbilling codes for sequential compression devices that were availableelectronically were used instead. The percentage of the populationthat received pharmacologic prophylaxis and recommended prophy-laxis by risk level was reported.

A control variable for VTE prophylaxis was not included inthe regression models because of its correlation with the indepen-dent variable of risk category. VTE prophylaxis protects patientsfrom acquiring VTE and, according to various studies, is more likelyto be given to higher risk patients. Because of this simultaneous andopposite effect of prophylaxis and risk category on the probability ofacquiring a VTE, a bivariate probit model was applied to estimatethe effects of the risk level while accounting for provider adherenceto UMHS prophylaxis guidelines.20–22 Finally, because it is difficultto interpret the magnitude of probit model coefficients, we calcu-lated the marginal change in the probability of acquiring VTEassociated with a change in the level of risk, after accounting forVTE prophylaxis.

The study was approved by the institutional review board ofthe University of Michigan Medical School.

RESULTSOf the 8216 inpatients in the study population, 67% were

general surgery, 16% were vascular surgery, and 17% were urologicsurgery patients. Roughly, 88% of patients underwent a majorsurgical procedure, defined as an operation of more than 45 minutesin duration and 35% had a malignancy (Table 1).

The majority (52.1%) of the study population was classifiedto the highest risk level; 36.5% were classified as high-risk, 10.4%as moderate risk, and 0.9% as low risk.

The overall incidence of acquired VTE within 30 days was1.44%. The incidence was associated with an increase in risk level;of the patients in the highest risk level, 1.94% acquired a VTE; ofthe high risk patients, 0.97%; moderate risk patients, 0.70%; andlow-risk patients, 0%. The difference between high and highest risklevels was statistically significant (P � 0.001), but the differencebetween low and moderate and between moderate and high risklevels was not statistically significant (Fig. 2).

Further disaggregating the patients in the highest risk cate-gory, the growth in incidence of acquired VTE appears to accelerateaccording to their cumulative risk score (Fig. 3). The increase wasstatistically significant for cumulative risk scores of 7 to 8 andgreater than 8.

The logistic regression identified recent pregnancy or post-partum (OR � 8.3; 1.0–68, P � 0.05), recent sepsis (4.0; 1.4–10.9,P � 0.01), malignancy (2.3; 1.5–3.3, P � 0.01), history of VTE(2.1; 1.1–4.1, P � 0.05), and central venous access (1.8; 1.1–3.0,P � 0.05) as significantly associated with VTE while age, varicoseveins, and positive Factor V Leiden were marginally significant(P � 0.1). Both control variables, length of hospitalization of 2 daysor longer and fiscal year, were significant (Table 2).

Controlling for length of hospitalization (�2 d) and fiscalyear, the 4 risk levels were significantly associated with the likeli-hood of acquiring a VTE within 30 days (1.9; 1.3–2.6, P � 0.01).With an increase in risk of 1 level, the odds of acquiring a VTEincreased 90% (Table 3).

The percentage of patients for whom orders were written forpharmacologic prophylaxis increased with risk level, though thepercentage of patients for whom orders for pharmacologic and/ormechanical prophylaxis that satisfied UMHS guidelines decreasedas risk level increased (Table 4).

The bivariate probit model demonstrated that there was asignificant correlation between the probability of acquiring VTE andlack of adherence to UMHS prophylaxis guidelines (� � 0.299, P �0.013) (The positive value of � indicated that patients who did notreceive recommended prophylaxis had higher probability of devel-oping VTE within 30 days following surgery). However, a test of thesignificance of the protective effect of adherence to UMHS prophy-laxis guidelines showed that it was not statistically significant.

After accounting for the effect of recommended prophylaxis,the estimated mean probability of acquiring VTE was 0.28% formoderate risk patients and 0.90% for high risk patients. The mar-ginal effect of an increase in risk level from moderate to high on themean probability of VTE was an increase of 0.37 percentage points(P � 0.0001); the marginal effect of an increase from high to highestrisk was a growth of 0.90 percentage points (P � 0.0001).

DISCUSSIONThis study demonstrates that the retrospective VTE risk

scoring method based on the Caprini model is valid and underscoresthe importance of assessing individual patient risk for VTE. And,because the retrospective risk scoring method uses electronic

Bahl et al Annals of Surgery • Volume 251, Number 2, February 2010

346 | www.annalsofsurgery.com © 2010 Lippincott Williams & Wilkins

sources of data to assess patient risk rather than data collectedthrough labor-intensive medical record abstraction, it is also aneconomical method. As such, the method can be the basis of a usefulmeasure of adherence to VTE prophylaxis guidelines for general,vascular, and urologic surgery patients.

This retrospective cohort study tested a practical method ofrisk stratifying a large population of surgical inpatients. By contrast,prior evaluations of VTE risk assessment methods were conductedas small retrospective studies of VTE cases versus controls, whereresults were highly dependent on selection of adequate controls.15,17

Our findings validate the retrospective risk scoring method;the 4 VTE risk levels were predictors of VTE outcomes andincreases in risk level and cumulative risk score were accompanied

by increases in the incidence rate of VTE within 30 days aftersurgery.

The incidence of VTE within 30 days was 1.44% in thisstudy, which was higher than the 0.64% reported in a study ofpredictors of VTE in NSQIP general and vascular surgery patientsfrom 128 Veterans Affairs and private sector hospitals by Rogers etal.23 The higher incidence can be explained by the fact that thisstudy includes only inpatients, whereas the other included bothinpatients and outpatients.

An increase in incidence of VTE associated with an increasein risk was similarly reported by O’Shaughnessy et al who appliedthe risk assessment model from Kucher to 5692 patients from adatabase of patients with a suspected VTE where 33% had a

TABLE 1. VTE Risk Factors of Study Population and Bivariate Analysis

RelativeRisk Score Pct Observed Pct VTE P

Age, �41 (not a risk factor) — 19.28% 0.88% —

Age, 41–60 1 39.59% 1.41%

Age, 61–74 2 28.40% 1.67%

Age, 75� 3 12.73% 1.82%

Acute myocardial infarction* 1 0.22% 0.00% 0.608

Heart failure 1 3.93% 1.55% 0.863

Varicose veins 1 0.11% 11.11% 0.015

Obesity (BMI �25) 1 24.71% 1.63% 0.409

Inflammatory bowel disease 1 2.98% 1.22% 0.777

Sepsis (�1 mo) 1 0.85% 7.14% �0.000

COPD or abnormal pulmonary function 1 7.57% 1.77% 0.469

Severe lung disease, including pneumonia (�1 mo) 1 0.68% 3.57% 0.178

Oral contraceptives or hormone replacement therapy* 1 0.00% —

Pregnancy or postpartum (�1 mo) 1 0.19% 6.25% 0.105

Hx of unexpected stillborn infant, recurrent spontaneousabortion (�3), premature birth with toxemia orgrowth-restricted infant*

1 0.01% 0.00% 0.904

Medical patient currently at bed rest† 1 — — —

Minor surgery planned 1 5.66% 1.94% 0.352

Swollen legs (current)† 1 — — —

Central venous access 2 7.64% 3.50% �0.000

Arthroscopic surgery* 2 0.00% — —

Major surgery (�45 min) 2 88.16% 1.41% 0.561

Malignancy (present or previous) 2 34.98% 2.09% 0.0003

Laparoscopic procedure �45 min 2 6.18% 1.38% 0.909

Patient confined to bed (�72 h)† 2 — — —

Immobilizing plaster cast (�1 mo)† 2 — — —

History of DVT/PE 3 3.47% 4.21% �0.000

Positive Factor V Leiden; positive prothrombinG20210A; Positive Lupus anticoagulant

3 0.56% 6.52% 0.004

Other congenital or acquired thrombophilia* 3 0.02% 0.00% 0.864

Heparin-induced thrombocytopenia (HIT) 3 0.13% 9.09% 0.033

Family history of VTE† 3 — — —

Elevated anticardiolipin antibodies† 3 — — —

Stroke (�1 mo)* 5 0.15% 0.00% 0.676

Multiple trauma (�1 mo)* 5 0.04% 0.00% 0.834

Elective major lower extremity arthroplasty* 5 0.01% 100.00% �0.000

Hip, pelvis, or leg fracture (�1 mo)* 5 0.00% — —

Acute spinal cord injury (paralysis) (�1 mo)* 5 0.00% — —

*Not included in the logistic regression of independent predictors of VTE.†Not included in retrospective scoring method.

Annals of Surgery • Volume 251, Number 2, February 2010 Validation of a VTE Risk Scoring Method

© 2010 Lippincott Williams & Wilkins www.annalsofsurgery.com | 347

confirmed VTE and 67% did not.24 Although the percentage ofpatients with a VTE in both studies increased almost linearly withrisk, our study exhibited greater specificity for patients classified aslow risk (a cumulative risk score of 0 or 1). For low-risk patients, wereported a 0% incidence of VTE, whereas O’Shaughnessy et alidentified 20% with confirmed VTE. The difference may be attrib-uted to the Kucher risk assessment model, which is based on only 8risk factors (compared with more than 30 risk factors in the Caprinimodel); patients classified as low risk may have had one or more riskfactors that were not identified by the model.13

Our observation that the rate of acquired VTE within 30 daysappeared to accelerate for patients within the highest risk level ascumulative risk score increased may be useful for identifying thosewho would benefit the most from long-term prophylaxis, but furtherstudy is needed of the relationship between the cumulative risk scoreand the length of time the patient continues to be at-risk.25,26

Although pharmacologic prophylaxis increased as patient riskincreased, adherence to UMHS prophylaxis guidelines actually fell.The bivariate probit model showed that adherence to UMHS pro-phylaxis guidelines protected patients from VTE, but the effect wasnot statistically significant. This result may be due to the fact that weincluded the type and frequency but not the duration of prophylaxis

in the model. Still, after adjusting for prophylaxis, the mean prob-ability of VTE doubled when risk level increased from moderate tohigh and from high to highest.

Seruya et al stratified plastic surgery patients into VTE risklevels by applying the Caprini model preoperatively. Of 1156

FIGURE 2. Incidence rate of VTE by risk level.

FIGURE 3. Incidence rate of VTE for low, moderate, andhigh risk levels and by cumulative risk score within the high-est risk level.

TABLE 2. Independent Predictors of VTE

Odds Ratio 95% CI

Age group (�41, 41–60, 61–74, 75�) 1.197* (0.968–1.479)

Major surgery (�45 min) 0.613 (0.303–1.241)

Laparoscopic procedure (�45 min) 0.906 (0.325–2.525)

Heart failure 0.699 (0.274–1.783)

Varicose veins 7.611* (0.771–75.151)

Obesity (BMI �25) 1.303 (0.793–2.139)

Inflammatory bowel disease 0.801 (0.247–2.597)

Sepsis (�1 mo) 3.965† (1.443–10.893)

COPD or abnormal lung function 0.932 (0.485–1.792)

Severe lung disease, includingpneumonia (�1 mo)

1.409 (0.320–6.196)

Pregnancy or postpartum (�1 mo) 8.315‡ (1.020–67.754)

Central venous access 1.837‡ (1.115–3.028)

Malignancy (present or previous) 2.259† (1.532–3.331)

History of DVT/PE 2.105‡ (1.086–4.081)

Positive Factor V Leiden 3.392* (0.905–12.715)

Heparin-induced thrombocytopenia (HIT) 3.413 (0.405–28.776)

Inpatient stay 2 d or longer 10.638† (3.892–29.073)

Fiscal yr 0.896* (0.789–1.017)

C-statistic � 0.741.Hosmer and Lemeshow goodness-of-fit test.�2 statistic � 7.033; Degree of freedom � 8; P � 0.533.*P � 0.1.†P � 0.01.‡P � 0.05.CI indicates confidence interval.

TABLE 3. Risk Level as a Predictor of VTE

Odds Ratio 95% CI

Risk level (4 groups: low,moderate, high, highest)

1.851† (1.337–2.563)

Inpatient stay 2 d or longer 11.330† (4.173–30.768)

Fiscal yr 0.948* (0.857–1.050)

C-statistic 0.698

Hosmer and Lemeshowgoodness-of-fit test

�2 statistic 7.286

Degree of freedom 9

P 0.607

*P � 0.1.†P � 0.01.CI indicates confidence interval.

TABLE 4. Thromboprophylaxis by Risk Level

Risk Level Low Moderate High Highest

Pct Pts w pharmacologicprophylaxis

8% 15% 25% 32%

Pct Pts w adherence to UMHSprophylaxis guidelines

100% 79% 25% 27%

Bahl et al Annals of Surgery • Volume 251, Number 2, February 2010

348 | www.annalsofsurgery.com © 2010 Lippincott Williams & Wilkins

patients, 120 (10%) were classified as highest risk. Despite a varietyof prophylaxis measures, 9 of the 120 patients (7.5%) suffered aVTE. Because of their high postoperative incidence of VTE, highestrisk patients may be candidates for extended out-of-hospital prophy-laxis. In general, the study highlights the value of scoring to identifypatients at-risk and of scoring patient populations that have not beenstudied with randomized clinical trials.27

The results of this study show that, controlling for length ofhospitalization (�2 d) and fiscal year, recent pregnancy or postpar-tum, recent sepsis, malignancy, history of VTE, and central venousaccess were associated with VTE. Age, varicose veins, and Factor VLeiden were marginally significant.

These results are well-supported by the literature. Andersonand Spencer report that while major general surgery is a strong riskfactor with an odds ratio of more than 10, other factors conferadditional and substantial risk with odds ratios between 2 and 9,including arthroscopic knee surgery, central venous lines, chemo-therapy, congestive heart failure or respiratory failure, hormones,malignancy, oral contraceptives, paralytic stroke, pregnancy post-partum, previous VTE, and thrombophilia. The weakest risk factorswith odds ratios less than 2 were age, bed rest over 3 days,immobility due to casts, prolonged sitting, or long car or airplanerides, laparoscopic surgery, obesity, varicose veins, or antepartumpregnancy.9 Petralia and Kakkar, in a recent report on the status ofVTE prophylaxis in general surgery patients, identified a long list ofrisk factors and highlighted the patient-related risk factors, malig-nant disease and its treatment, older age, history of VTE, increasedBMI, varicose veins, and use of estrogens.28

Though identified in previous studies of factors associatedwith increased incidence of VTE, body mass index (BMI) �25,inflammatory bowel disease, chronic obstructive pulmonary disease,and heart failure in this study were not statistically significant inpredicting VTE outcomes. However, it is possible that if BMI hadbeen represented as 2 or 3 ordinal categories in the version of theCaprini model adopted by UMHS, patients with higher BMI mighthave been significantly associated with VTE.

In addition, the odds ratios of some factors, notably preg-nancy or postpartum, varicose veins, and sepsis, were higher thantheir assigned relative weight. The relative weights in the Caprinirisk assessment model were established and revised based on resultsof randomized clinical trials reported in the literature over a periodof 25 years, so updates that reflect our findings will only be madewhen corroborating evidences from other clinical trials and valida-tion studies are published.29

The results of this study strongly support individual assess-ment of exposing and predisposing risk factors with relative weight-ing and application of a cumulative risk score and level.

• Almost all patients in the study population underwent majorsurgery (88%) but a large portion had 1 or more predisposing riskfactors—35% had a cancer diagnosis, 40% were age 61 or older,and 25% had a BMI of greater than 25. Although the percentageof patients with other predisposing risk factors was fairly small,several were significant predictors of VTE within 30 days aftersurgery and conferred substantial risk (pregnancy/postpartum,history of VTE, varicose veins, and Factor V Leiden).

• The 4 VTE risk levels were significantly associated with thelikelihood of acquiring a VTE. Incidence of VTE grew with anincrease in risk level and appeared to accelerate in the highest risklevel with an increase in cumulative risk score.

• Stratification by risk level effectively discriminated patient riskand thromboprophylaxis; the majority of the study population wasclassified to the highest risk level (52.1%) with a recommendationof pharmacologic and mechanical prophylaxis and a small, butsignificant percentage, was classified to moderate (10.4%) or low

risk (0.9%) levels with a recommendation of mechanical prophy-laxis or no prophylaxis, respectively.

This study has several limitations. First, it was conductedusing a retrospective risk scoring method that cannot identify allpatient risk factors. Although the reliability study showed thatagreement between the retrospective scoring method and the PAreported risk level was acceptable, the PA reported results cannot beconsidered the gold standard. Still, the retrospective scoring modelmay understate the risk level and thromboprophylaxis for a minorityof patients, roughly 5%.16

Second, this study was conducted at 1 academic medicalcenter; a similar test of the retrospective scoring method and theassociation between VTE risk and outcomes should be conducted atother centers that participate in the NSQIP. A broader study wouldhave the added benefit of enhancing the previous analysis of pre-dictors of VTE from preoperative data collected for NSQIP byexamining known VTE risk factors.23 But, such a study can only beaccomplished if the data sources that are needed to apply theretrospective VTE risk scoring method are reliable.

Third, VTE outcomes were reported for a period of 30-dayspost surgery, whereas studies have shown that the impact of hospi-talization on development of VTE can exceed 30 days.26,27 In addition,the NSQIP method of reporting VTE outcomes may not identifyperioperative DVTs that are clinically silent. So, the reported incidenceof VTE in the study population may be understated.23

Ultimately, a multicenter study is needed to create and vali-date an individual prospective VTE scoring system for surgicalpatients, based on 90-day VTE outcomes.

Finally, this study was conducted for general, vascular, andurologic surgery populations. Similar studies for other surgery andfor medicine populations should be conducted to validate the riskscoring method and demonstrate the importance of individual riskassessment for a broader range of patients. After conducting suchstudies, the scoring method could be adopted as a very practicalmeans to measure adherence to prophylaxis guidelines more broadlyand fill the gap in current VTE measure sets promulgated by theCenters for Medicare and Medicaid Services and the NationalQuality Forum.3,4

In conclusion, an internally-developed UMHS retrospectiveVTE risk scoring system that is based on the Caprini risk assessmentmodel, coupled with data about the patient length of hospitalization,produced a valid method for identifying patients at risk for DVT orPE within 30 days after surgery. This study demonstrates theimportance of assessing individual patient VTE risk factors todetermine overall risk and appropriate thromboprophylaxis. Finally,since the scoring method is based on readily available electronicdata sources, it can be widely used to stratify patients by risk of VTEand provide the basis for a practical and economical means tomeasure and improve compliance with VTE prophylaxis guidelines.

REFERENCES1. Geerts WH, Bergqvist D, Pineo GF, et al. Prevention of venous thromboem-

bolism: American College of Chest Physicians evidence-based clinical prac-tice guidelines (8th edition). Chest. 2008;133:381–453.

2. Shojania KG, Duncan BW, McDonald KM, et al. Making health care safer:a critical analysis of patient safety practices. Evid Rep Technol Assess(Summ). 2001:i–x, 1–668.

3. Surgical Care Improvement Project (SCIP). �MedQIC Web site.� Partnershipand hospital participation information. Available at: http://www.qualitynet.org. Accessed January 29, 2008.

4. National Quality Forum (NQF) consensus standards for prevention and careof venous thromboembolism. May 18, 2006. Available at: http://www.qualityforum.org/pdf/news/DVT5–18–06.pdf. Accessed January 6, 2009.

5. The US Department of Health and Human Services. The surgeon general’scall to action to prevent deep vein thrombosis and pulmonary embolism.

Annals of Surgery • Volume 251, Number 2, February 2010 Validation of a VTE Risk Scoring Method

© 2010 Lippincott Williams & Wilkins www.annalsofsurgery.com | 349

2008. Available at: http://www.surgeongeneral.gov/topics/deepvein/calltoaction/call-to-action-on-dvt-2008.pdf. Accessed January 6, 2009.

6. Tapson VF, Hyers TM, Waldo AL, et al. Antithrombotic therapy practices inan era of practice guidelines. Arch Intern Med. 2005;165:1458–1464.

7. Caprini JA, Hyers TM. Compliance with antithrombotic guidelines: currentpractice, barriers, and strategies for improvement. Manag Care. 2006;15:49–66.

8. Holley AB, Moores LK, Jackson JL. Provider preferences for DVT prophy-laxis. Thromb Res. 2006;117:563–568.

9. Anderson FA, Spencer FA. Risk factors for venous thromboembolism.Circulation. 2003;107:I-9–I-16.

10. Mommertz G, Sigala F, Glowka TR, et al. Differences of venous thrombo-embolic risks in vascular general and trauma surgery patients. J CardiovascSurg. 2007;48:727–733.

11. Caprini JA. Thrombosis risk assessment as a guide to quality patient care. DisMon. 2005;51:70–78.

12. Cohen AT, Alikhan R, Arcelus, et al. Assessment of venous thromboembo-lism risk and the benefit of thromboprophylaxis in medical patients. ThrombHaemost. 2005;94:750–759.

13. Kucher N, Koo S, Quiroz R, et al. Electronic alerts to prevent venous thromboem-bolism among hospitalized patients. N Engl J Med. 2005;352:969–977.

14. Caprini JA, Arcelus JI, Reyna JJ. Effective risk stratification of surgical andnon-surgical patients for venous thromboembolic disease. Semin Hematol.2001;38(suppl 5):12–19.

15. Zakai NA, Wright J, Cushman M. Risk factors for venous thrombosis inmedical inpatients: validation of a thrombosis risk score. J Thromb Haemost.2004;2:2156–2161.

16. Bahl V, Hu HM, Henke PK, et al. A reliability study of a venous thrombo-embolism risk scoring method using electronic data sources. Submitted forpublication.

17. McCaffrey R, Bishop M, Adonis-Rizzo M, et al. Development and testing ofa DVT risk assessment tool: providing evidence of validity and reliability.Worldviews Evid Based Nurs. 2007;4:14–20.

18. Khuri SF, Henderson WG, Daley J, et al. The patient safety in surgery study:background, study design, and patient populations. J Am Coll Surg. 2007;204:1089–1102.

19. SAS Software. Version 9.1. Cary, NC: SAS Institute Inc; 2002–2003.

20. Stata Statistical Software. Release 9. College Station, TX: StataCorp; 2005.

21. Greene WH. Bivariate and Multivariate Probit Models, Econometric Analy-sis. Upper Saddle River, NJ: Prentice Hall; 2000:849–855.

22. Newhouse JP, McClellan M. Econometrics in outcomes research: the use ofinstrumental variables. Annu Rev Public Health. 1998;19:17–34.

23. Rogers SO, Kilaru RK, Hosokawa P, et al. Multivariable predictors ofpostoperative venous thromboembolic events after general and vascularsurgery: results from the Patient Safety in Surgery study. Am Coll Surg.2007;204:1211–1221.

24. O’Shaughnessy D, Rose P, Pressley F, et al. Evaluating the performance of apreviously reported risk score to predict venous thromboembolism: a VER-ITY Registry Study �ASH Annual Meeting Abstracts�. Blood. 2006;108:Abstract 716.

25. Huber O, Bounameaux H, Borst F, et al. Postoperative pulmonary embolismafter hospital discharge. An underestimated risk. Arch Surg. 1992;127:310–313.

26. Yale SH, Medlin SC, Liang H, et al. Risk assessment model for veno-thromboembolism in post-hospitalized patients. Int Angiol. 2005;24:250 –254.

27. Seruya M, Venturi ML, Iorio ML, et al. Efficacy and safety of venousthromboembolism prophylaxis in highest risk plastic surgery patients. PlastReconstr Surg. 2008;122:1701–1708.

28. Petralia GA, Kakkar AK. Venous thromboembolism prophylaxis for thegeneral surgery patient: where do we stand? Semin Respir Crit Care Med.2008;29:83–89.

29. Wakefield TW, Caprini JA, Comerota AJ. Thromboembolic diseases. CurrProbl Surg. 2008;45:844–899.

Bahl et al Annals of Surgery • Volume 251, Number 2, February 2010

350 | www.annalsofsurgery.com © 2010 Lippincott Williams & Wilkins