ORIGINAL REPORT

A systematic review of validated methods for identifying acuterespiratory failure using administrative and claims data

Natalie Jones, Gary Schneider*, Sumesh Kachroo, Philip Rotella, Ruzan Avetisyan and Matthew W. Reynolds

United BioSource Corporation, Lexington, MA, USA

ABSTRACTPurpose The Food and Drug Administration’s (FDA) Mini-Sentinel pilot program initially aims to conduct active surveillance to refine safetysignals that emerge for marketed medical products. A key facet of this surveillance is to develop and understand the validity of algorithmsfor identifying health outcomes of interest (HOIs) from administrative and claims data. This paper summarizes the process and findings of thealgorithm review of acute respiratory failure (ARF).Methods PubMed and IowaDrug Information Service searches were conducted to identify citations applicable to the anaphylaxis HOI. Level 1abstract reviews and Level 2 full-text reviews were conducted to find articles using administrative and claims data to identify ARF, includingvalidation estimates of the coding algorithms.Results Our search revealed a deficiency of literature focusing on ARF algorithms and validation estimates. Only two studies provided codesfor ARF, each using related yet different ICD-9 codes (i.e., ICD-9 codes 518.8, “other diseases of lung,” and 518.81, “acute respiratory failure”).Neither study provided validation estimates.Conclusions Research needs to be conducted on designing validation studies to test ARF algorithms and estimating their predictive power,sensitivity, and specificity. Copyright © 2012 John Wiley & Sons, Ltd.

key words—acute respiratory failure; administrative and claims data; Mini-Sentinel; coding algorithm

INTRODUCTION

Mini-Sentinel is the Food and Drug Administration’s(FDA) pilot program that aims to conduct active sur-veillance of automated health care data. The initialgoal is to refine safety signals that emerge for mar-keted medical products. Essential components ofthis exercise are (i) to identify administrative andclaims data—friendly algorithms used to detect vari-ous health outcomes of interest (HOIs)—and (ii) toidentify the performance characteristics of these algo-rithms as measured within the studies in which theywere used. In this article, we describe the algorithmreview process and findings for 1 of the 20 HOIsselected for review by the FDA: acute respiratoryfailure (ARF).The respiratory system can be divided into two

parts: the lung, which is responsible for gas exchange,

and the “pump” (the chest wall, central nervous system[CNS] respiratory controllers, and the nerves thatconnect the two), which ventilates the lung.1,2 ARFis a condition in which one or both of these parts fail.Hypoxemia (type I respiratory failure) occurs when thelung fails and gas exchange becomes dysfunctional;failure of the pump causes ventilator failure, whichmanifests as hypercapnia (type II respiratory failure).2

Type I respiratory failure is generally caused by lungdisease, including pneumonia, pulmonary embolism,and acute respiratory distress syndrome (ARDS). TypeII respiratory failure is caused by anatomical andfunctional defects of the CNS, impairment of neuro-muscular transmission, and mechanical defects of theribcage, as well as conditions leading to fatigue ofthe respiratory muscles.2 Acute hypercapnia can alsooccur during acute exacerbations of chronic pulmo-nary conditions such as chronic obstructive pulmonarydisease (COPD), although it is important to differenti-ate it from chronic-onset or insidious-onset hypercap-nia, which is most frequently seen in COPD and isassociated with a particularly poor prognosis.2

*Correspondence to: G. Schneider, Epidemiology and Database Analytics,United BioSource Corporation, 430 Bedford St., Suite 300, Lexington, MA02420, USA. E-mail: gary.schneider@unitedbiosource.com

Copyright © 2012 John Wiley & Sons, Ltd.

pharmacoepidemiology and drug safety 2012; 21(S1): 261–264Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/pds.2326

Patients with ARF vary in terms of demographicsand disease etiology, severity, and prognosis. Treat-ment can occur at a range of settings, including emer-gency departments, hospital wards, or the intensivecare unit (ICU).3 These characteristics, as well as alack of a consensus definition for the condition, makeARF difficult to study in the community. Therefore,most studies of the epidemiology of ARF focus onits most severe forms. A US study defined ARF asthe condition for which inpatients have dischargerecords of acute respiratory distress or failure, mechan-ical ventilation, and at least 24 h of hospitalization;using that definition, the incidence of ARF was esti-mated to be 137.1 hospitalizations per 100 000 USresidents aged ≥5 years.4 Studies from Berlin5 andScandinavia6 defined ARF as the condition whereinpatients are intubated and mechanically ventilated for≥24 h in ICUs, and these studies reported ARF inci-dences of 88.6 and 77.6 cases per 100 000 populationper year, respectively. The incidence of ARF increasessubstantially with age and is especially high amongpersons 65 years of age and older.1,4 ARF requiringmechanical ventilation is associated with poor sur-vival: in-hospital mortality ranges from 35.9%4 to42.7%,5 with lower survival influenced by older ageand the degree of comorbid multi-organ failure.3

METHODS

The general search strategy originated from prior workby the Observational Medical Outcomes Partnershipand its contractors and was modified slightly for the20 HOIs selected for review.Details of the methods for these systematic reviews

can be found in the accompanying manuscript byCarnahan and Moores.7 In brief, the base PubMedsearch was combined with the following terms torepresent the HOI: “respiratory insufficiency,” “respi-ratory” AND “insufficiency,” “respiratory failure” AND“respiratory” AND “failure.”To identify other relevant articles that were not

found in the PubMed search, the Iowa Drug Informa-tion Service Web (IDIS/Web) was searched using asimilar search strategy. Both the PubMed and IDISsearches were conducted on 10 May 2010. An addi-tional PubMed search was conducted on 6 July 2010to amend the original search strategy with additionaldatabases. All searches were restricted to articles pub-lished in 1990 or later. The details of these searchescan be found in the full report on the Mini-Sentinelwebsite: http://mini-sentinel.org/foundational_activities/related_projects/default.aspx.

The search results were compiled, and duplicateresults were eliminated. The results were then outputand provided to organizations contracted to conductthe literature reviews. Mini-Sentinel collaboratorswere also asked to help identify relevant validationstudies.The abstract of each citation identified was reviewed

by two investigators. When either investigator selectedan article for full-text review, the full text wasreviewed by both investigators. Agreement on whetherto review the full text or include the article in the evi-dence table was calculated via Cohen’s kappa statistic.A single investigator abstracted each study for the finalevidence table; data included in the table were con-firmed by a second investigator for accuracy. A clini-cian or topic expert was consulted to review the resultsof the evidence table and discuss how they comparedwith diagnostic methods currently used in clinicalpractice. This included whether certain diagnosticcodes used in clinical practice were missing from thealgorithms, and the appropriateness of the validationdefinitions compared with diagnostic criteria currentlyused in clinical practice.

RESULTS

The total number of citations identified from the com-bined searches was 242 (PubMed: 170, IDIS: 69, addi-tional PubMed: 3); with the exclusion of overlaps, thenumber of unique citations was 207. Mini-Sentinelcollaborators provided no additional reports of valida-tion studies.Of the 207 abstracts reviewed, we accepted six for

full-text review. The straightforward inclusion criteria,consisting of (i) examination of the HOI of interest, (ii)use of administrative and claims databases, and (iii)study conducted in the USA or Canada, enabled per-fect agreement between the two reviewers on accep-tance/rejection status, although there was substantialvariation in the reasons for rejection.Of the six full-text articles reviewed, two were

excluded for not being administrative and claims data-base studies and two for not focusing on the HOI. Thetwo remaining studies did not report validation of theARF coding algorithm directly in the article, norwithin a reference cited in the article. Perfect agree-ment between reviewers on inclusion versus exclusionof full-text articles was achieved.

Summary of algorithms

We identified only two studies containing ARF algo-rithms, neither of which had corresponding validation

n. jones et al.262

Copyright © 2012 John Wiley & Sons, Ltd. Pharmacoepidemiology and Drug Safety, 2012; 21(S1): 261–264DOI: 10.1002/pds

estimates. Wu et al.8 used ICD-9 code 518.8 (otherdiseases of lung) to identify respiratory failure casesamong Medicare beneficiaries. ICD-9 code 518.81(acute respiratory failure) was used by Dransfieldet al.9 to identify hospitalized patients with a primarydiagnosis of ARF. Information on the study popula-tions, outcomes, and algorithms used in each of thesestudies is presented in Table 1.

DISCUSSION

In practice, one would expect that ICD-9 code 518.81would be commonly used to identify ARF cases, as itexplicitly defines ARF. One would also expect thiscode to have high positive predictive value and highspecificity.The sensitivity of ICD-9 code 518.81 may be more

questionable. It is possible that patients with ARFmay be coded with other conditions in administrativeand claims data. Likely examples are chronic respira-tory failure (CRF; ICD-9 code 518.83) and ARDS(ICD-9 code 518.82). CRF differs from ARF in termsof presentation, treatment, and interventions3; there-fore, examination of procedure codes may provebeneficial in discerning these two distinct types ofrespiratory failure. By contrast, treatments and proce-dures for ARDS are similar to ARF; in fact, ARDSis a possible cause of ARF.10 Thus, examination ofprocedural codes may be inadequate to differentiatebetween these conditions. Administrative and claimsdata with corresponding laboratory results, however,may prove viable in distinguishing between ARF andARDS. There is also an ICD-9 code specifying bothconditions (i.e., ICD-9-CM code 518.84, Acute andchronic respiratory failure); examination of proceduralcodes may prove useful here as well.

Much of what is known about ARF in the USA isfrom a study using data from the 1994 NationwideInpatient Sample.4 As these are not administrativeand claims data, this article did not meet the inclusioncriteria of the present study. Nevertheless, it providesan example of an ARF algorithm that implements bothdiagnostic and procedure codes. Diagnostic codes foracute respiratory distress or failure (ICD-9-CM 518.5[Pulmonary insufficiency following trauma and sur-gery], 518.81, or 518.82) combined with a procedurecode for continuous mechanical ventilation (ICD-9-CM 96.7) were used, thereby identifying severeARF cases. Via this definition, the oft-cited US ARFincidence of 137.1 hospitalizations per 100 000 USresidents aged ≥5 years4 was estimated.It should be noted that in both the medical literature

and clinical practice, ARF is usually described as sec-ondary to other diseases/abnormalities or trauma.3,4 Ifinterest lies in ARF as a result of a specific condition,we suggest that algorithm development incorporatedisease/condition-specific codes combined with ICD-9 code 518.81 (or an alternative ARF algorithm).

CONCLUSION

The most recent estimates are that approximately330 000 ARF hospitalizations occur annually inthe USA, with 31-day hospital mortality at 31.4%.4

Despite this epidemiology, our current search high-lights a scarcity of literature providing validated ornon-validated algorithms for ARF that can be appliedto administrative and health care data. Research needsto be conducted on designing validation studies to testARF algorithms and estimating their predictive power,sensitivity, and specificity.

Table 1. Acute respiratory failure coding algorithms

Citation Study population and study periodDescription of outcome

studied Algorithm

Dransfield et al. 20089 Patients admitted to University of Alabama Hospital whose discharge or deathsummaries indicated a primary diagnosis of acute exacerbation of chronicobstructive lung disease (International Classification of Diseases, Ninth Edition[ICD-9] code 491.21) or a primary diagnosis of acute respiratory failure (518.81)and a secondary diagnosis of acute exacerbation were identified. Patients with adiagnosis of asthma (493) were excluded; 825 patients met the inclusion criteria forthe study, of which 410 were males and 415 were females. The mean age of patientswas 66.5 years. The study period was 1 October 1999–30 September 2006.

In-hospital mortalityafter use of b-blockers.

Acute respiratoryfailure: 518.81.

Wu et al. 20038 Patients over the age of 65 years who underwent total hip arthroplasty in the Medicaredatabase were identified from the part B data using CPT codes (n=23136; 8180 malesand 14956 females). Patients were eligible to be included if the procedure wasperformed by an orthopedic surgeon (part B) with an accompanying inpatient record forthe same procedure (part A). Major morbidity counts at 7 and 30days after the procedurewere obtained from part B based on ICD-9 diagnosis codes, one of which corresponded torespiratory failure (ICD-9 code 518.8). The study period was 1994–1999.

Morbidity and deathat 7 and 30days afterhip replacement surgery.

Respiratoryfailure: 518.8.

detection of acute respiratory failure in claims 263

Copyright © 2012 John Wiley & Sons, Ltd. Pharmacoepidemiology and Drug Safety, 2012; 21(S1): 261–264DOI: 10.1002/pds

KEY POINTS• There is limited literature focusing on acute res-piratory failure that provides administrative andclaims data-based coding algorithms and valida-tion estimates.

• Additional research is needed regarding the useof administrative and claims data-based codingalgorithms to identify acute respiratory failure.

CONFLICT OF INTEREST

The authors declare no conflict of interest. This is notproduct-specific or privately funded research. Theviews expressed in this document do not necessarilyreflect the official policies of the Department of Healthand Human Services, nor does mention of tradenames, commercial practices, or organizations implyendorsement by the US government.

ACKNOWLEDGEMENT

This work was supported by the Food and DrugAdministration (FDA) through Department of

Health and Human Services (HHS) contract numberHHSF223200910006I.

REFERENCES

1. Delerme S, Ray P. Acute respiratory failure in the elderly: diagnosis and progno-sis. Age Ageing 2008; 37: 251–257.

2. Roussos C, Koutsoukou A. Respiratory failure. Eur Respir J Suppl 2003; 47: 3s–14s.3. Markou NK, Myrianthefs PM, Baltopoulos GJ. Respiratory failure: an overview.

Crit Care Nurs Q 2004; 27: 353–379.4. Behrendt CE. Acute respiratory failure in the United States: incidence and 31-day

survival. Chest 2000; 118: 1100–1105.5. Lewandowski K, Metz J, Deutschmann C, et al. Incidence, severity, and mortal-

ity of acute respiratory failure in Berlin, Germany. Am J Respir Crit Care Med1995; 151: 1121–1125.

6. Luhr OR, Antonsen K, Karlsson M, et al. Incidence and mortality after acute res-piratory failure and acute respiratory distress syndrome in Sweden, Denmark,and Iceland. The ARF Study Group. Am J Respir Crit Care Med 1999; 159:1849–1861.

7. Carnahan RM, Moores KG. Mini-Sentinel’s systematic reviews of validatedmethods for identifying health outcomes using administrative data: methodsand lessons learned. Pharmacoepidemiol Drug Saf 2012; 21(S1): 82–89.

8. Wu CL, Anderson GF, Herbert R, Lietman SA, Fleisher LA. Effect of postoper-ative epidural analgesia on morbidity and mortality after total hip replacementsurgery in Medicare patients. Reg Anesth Pain Med 2003; 28: 271–278.

9. Dransfield MT, Rowe SM, Johnson JE, Bailey WC, Gerald LB. Use of betablockers and the risk of death in hospitalised patients with acute exacerbationsof COPD. Thorax 2008; 63: 301–305.

10. Viegi G. Epidemiology of acute respiratory failure. Rassegna di Patologiadell’Apparato Respiratorio 2008; 23: 335–338.

n. jones et al.264

Copyright © 2012 John Wiley & Sons, Ltd. Pharmacoepidemiology and Drug Safety, 2012; 21(S1): 261–264DOI: 10.1002/pds