ORIGINAL REPORT

A systematic review of validated methods for identifying pulmonaryfibrosis and interstitial lung disease using administrative and claimsdata

Natalie Jones, Gary Schneider*, Sumesh Kachroo, Philip Rotella, Ruzan Avetisyan and Matthew W. Reynolds

United BioSource Corporation, Lexington, MA, USA

ABSTRACTPurpose The Food and Drug Administration’s Mini-Sentinel pilot program initially aimed to conduct active surveillance to refine safetysignals that emerge for marketed medical products. A key facet of this surveillance is to develop and understand the validity of algorithmsfor identifying health outcomes of interest (HOIs) from administrative and claims data. This paper summarizes the process and findings ofthe algorithm review of pulmonary fibrosis and interstitial lung disease.Methods PubMed and Iowa Drug Information Service Web searches were conducted to identify citations applicable to the pulmonaryfibrosis/interstitial lung disease HOI. Level 1 abstract reviews and Level 2 full-text reviews were conducted to find articles usingadministrative and claims data to identify pulmonary fibrosis and interstitial lung disease, including validation estimates of the codingalgorithms.Results Our search revealed a deficiency of literature focusing on pulmonary fibrosis and interstitial lung disease algorithms and validationestimates. Only five studies provided codes; none provided validation estimates. Because interstitial lung disease includes a broad spectrumof diseases, including pulmonary fibrosis, the scope of these studies varied, as did the corresponding diagnostic codes used.Conclusions Research needs to be conducted on designing validation studies to test pulmonary fibrosis and interstitial lung diseasealgorithms and estimating their predictive power, sensitivity, and specificity. Copyright © 2012 John Wiley & Sons, Ltd.

key words—pulmonary fibrosis; interstitial lung disease; administrative and claims data; Mini-Sentinel; coding algorithm

INTRODUCTION

Mini-Sentinel is the Food and Drug Administration’s(FDA) pilot program that aimed to conduct activesurveillance of administrative and health care claimsdata. The initial goal is to refine safety signals thatemerge for marketed medical products. Essentialcomponents of this exercise are as follows: (i) toidentify administrative and claims data—friendlyalgorithms used to detect various health outcomes ofinterest (HOIs) and (ii) to identify the performancecharacteristics of these algorithms as measured withinthe studies in which they were used. In this article, wedescribe the algorithm review process and findingsfor 1 of the 20 HOIs selected for review by the

Mini-Sentinel Protocol Core: pulmonary fibrosis (PF)and interstitial lung disease (ILD).Interstitial lung diseases, also called diffuse paren-

chymal lung diseases, are a diverse group of pulmo-nary disorders with similar clinical, radiographic,physiologic, and/or pathologic features.1 As delineatedby a joint consensus statement from the AmericanThoracic Society and European Respiratory Society,ILD can be grouped into four main categories: disordersof known causes (e.g., collagen vascular disease or lungdisorders associated with drug, occupational, or environ-mental exposure), idiopathic interstitial pneumonias(idiopathic pulmonary fibrosis [IPF] and others, e.g.,nonspecific interstitial pneumonia, desquamative inter-stitial pneumonia, respiratory bronchiolitis-associatedILD, acute interstitial pneumonia, cryptogenic organiz-ing pneumonia, and lymphocytic interstitial pneumo-nia), granulomatous lung disorders (e.g., sarcoidosis),and other forms of ILD of unknown cause(e.g., lymphangioleiomyomatosis [LAM], pulmonary

*Correspondence to: G. Schneider, Epidemiology and Database Analytics,United BioSource Corporation, 430 Bedford St., Suite 300, Lexington, MA,02420 USA. E-mail: gary.schneider@unitedbiosource.com

Copyright © 2012 John Wiley & Sons, Ltd.

pharmacoepidemiology and drug safety 2012; 21(S1): 256–260Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/pds.2338

Langerhans cell histiocytosis/histiocytosis X [HX],and eosinophilic pneumonia).2

The incidence of ILD is approximately 30 per100 000; about half of these cases are classified asPF.1 Treatment and prognosis of ILD/PF depend onthe specific diagnosis and its histopathologic features.Corticosteroid treatment can be beneficial for somepatients, including those with nonspecific interstitialpneumonia, cryptogenic organizing pneumonia, anddesquamative interstitial pneumonia.1,2 However, forother disorders such as IPF and LAM, corticosteroidtreatment is generally ineffective; in such disorders,mortality is high and lung transplantation may be theonly viable treatment option.1

METHODS

The general search strategy originated from prior workby the Observational Medical Outcomes Partnershipand its contractors, and was modified slightly byMini-Sentinel investigators for the 20 HOIs selectedfor review.Details of the methods for these systematic reviews can

be found in the accompanying manuscript by Carnahanand Moores (on page 82 in this issue). In brief, the basePubMed search was combined with the following termsto represent the HOI: “Lung Diseases, Interstitial,”“Pulmonary Fibrosis,” “idiopathic pulmonary fibrosis,”and “interstitial” AND (“lung” OR “pulmonary”).To identify other relevant articles that were not

found in the PubMed search, the Iowa Drug Informa-tion Service Web (IDIS/Web) was also searched usinga similar search strategy. Both the PubMed and IDIS/Web searches were conducted on 10 May 2010. Thedetails of these searches can be found in the full reporton the Mini-Sentinel website: http://mini-sentinel.org/foundational_activities/related_projects/default.aspx.The search results from different databases were

compiled and duplicate results eliminated by Univer-sity of Iowa investigators, who used a citation managerprogram. The results were then output and provided toorganizations contracted to conduct the literaturereviews. Mini-Sentinel collaborators were also askedto help identify relevant validation studies.The abstract of each citation identified was reviewed

by two investigators. When either investigator selectedan article for full-text review, the full text wasreviewed by both investigators. Agreement on whetherto review the full text or include the article in theevidence table was calculated using the Cohen’s kappastatistic.A single investigator abstracted each study for the

final evidence table. The data included in the table

were confirmed by a second investigator for accuracy.A clinician or topic expert was consulted to review theresults of the evidence table and discuss how theycompared with diagnostic methods currently usedin clinical practice. This included whether certaindiagnostic codes used in clinical practice were missingfrom the algorithms, and the appropriateness ofthe validation definitions compared with diagnosticcriteria currently used in clinical practice.

RESULTS

The PubMed and IDIS/Web searches identified 203and 7 citations, respectively. The total numberof unique citations from the combined searcheswas 210. Mini-Sentinel collaborators provided noadditional published or unpublished reports of valida-tion studies.Of the 210 abstracts reviewed, we accepted only 20

for full-text review. The straightforward inclusioncriteria—consisting of (i) examination of the HOI,(ii) use of administrative and claims database, and(iii) study conducted in the United States orCanada—enabled perfect agreement between the tworeviewers on acceptance/rejection status, althoughthere was variation in the reasons for rejection.Exclusion criteria for full-text review consisted of

the following: (i) poorly described HOI identificationalgorithm and (ii) no validation of the outcomedefinition or reporting of validity estimates. Bothreviewers agreed that none of the 20 full-text articlesreviewed fulfilled this second criterion; therefore,five studies identified as fulfilling all other criteriawere reviewed.

SUMMARY OF ALGORITHMS

We identified five studies that provided codes for PFor ILD. Suissa et al. used ICD-9 codes 515, 516.3,516.8, and 516.9 to identify spontaneous reports ofILD among patients with rheumatoid arthritis.3 Raghuet al. used ICD-9 code 516.3 as the basis of their broadand narrow IPF definitions.4 Ehrlich et al. used ICD-9codes 515 and 516.3 to identify PF in patientswith and without diabetes.5 Pinheiro et al. used theICD-10 code J84.1 to identify occupational risks forIPF.6 Finally, ICD-9 code 501 and ICD-10 code J61were used by Gan et al. to identify asbestosis, a specialtype of ILD occurring from asbestos exposure.7

Information on the study populations, outcomes, andalgorithms used in each of these studies is presentedin Table 1.

detection of pulmonary fibrosis in claims 257

Copyright © 2012 John Wiley & Sons, Ltd. Pharmacoepidemiology and Drug Safety, 2012; 21(S1): 256–260DOI: 10.1002/pds

DISCUSSION

Interstitial lung diseases are a diverse group of pulmo-nary disorders classified together because of similarclinical, physiologic, or pathologic features.1 Thediversity of the diseases classified under ILD is wellillustrated by Raghu et al.,4 who used a total of 36

ICD-9 codes in their IPF algorithm; 1 to define IPF(ICD-9-CM 516.3) and 35 additional codes that areconsistent with other ILDs and were used as exclusioncriteria.Despite this vast collection of ICD-9-CM codes

consistent with ILD, we found that papers that studiedILD used only five ICD-9-CM codes, specifically 515,

Table 1. Pulmonary fibrosis and interstitial lung disease coding algorithms

Citation Study population and time period Description of outcome studied Algorithm

Ehrlichet al.20105

Kaiser Permanente Medical Care program inNorthern California. The study cohort(n= 121 886) was drawn from a population of1 811 228 members aged <18 years as of 1January 1996.

Incidence of asthma, chronic obstructivepulmonary disease, pulmonary fibrosis,pneumonia, and lung cancer in patients withand without a diagnosis of diabetes.

Pulmonary fibrosis, ICD-9 515 (chronicpostinflammatory) and 516.3 (idiopathic).

Gan etal. 20097

British Columbia Linked Health Databasedata, individuals ≥15 years of age. The studycohort included 1170 new asbestosis cases(1121 men, 49 women) identified usingworkers’ compensation records (n= 271),hospitalization records (n= 562), andoutpatient records (n= 582) from 1992 to2004.

Population-based surveillance of asbestosisusing multiple health data sources.

ICD-9 code 501 (asbestosis) and ICD-10 codeJ61 (pneumoconiosis due to asbestos andother mineral fibers) to identify asbestosiscases.

Pinheiroet al.20086

United States National Institute forOccupational Safety and Health mortalitysurveillance system for respiratory diseases ofoccupational interest; multiple cause-of-deathdata compiled by the National Center forHealth Statistics for US residents aged15 years and older, from 1999 to 2003.

Idiopathic pulmonary fibrosis mortality rateand occupational risks.

The term “IPF” refers here to the group ofdiseases classified under ICD-10 code J84.1,comprising “Other interstitial pulmonarydiseases with fibrosis, including fibrosingalveolitis (cryptogenic), Hamman–Richsyndrome, and idiopathic pulmonaryfibrosis.” Cases were defined as thosedecedents whose death certificates mentionedICD-I0 code J84.1 (i.e., IPF) as the underlyingor contributing cause of death and did notmention any other type or cause of interstitiallung disease.

Raghu etal. 20064

Unspecified data source. Data were obtainedfrom the health care claims processing systemof a large US health plan (1996–2000) thatconsisted of claims for service facilities (e.g.,hospitals), health care professionals (e.g.,physicians), and retail pharmacies andprovided services through health maintenanceorganizations, preferred providerorganizations, Medicare Risk, and indemnityproducts to approximately three millionpersons residing in 20 states. The studysample consisted of all persons 18 years orolder who were eligible for comprehensivehealth benefits for at least 1 day in calendaryear (CY) 2000.

Annual incidence and prevalence ofidiopathic pulmonary fibrosis in the UnitedStates.

Algorithms with “broad” and “narrow” casedefinitions of IPF.IPF (“broad case definition”) was identified as(i) one or more medical encounters with adiagnosis code for IPF (ICD-9-CM 516.3) and(ii) no medical encounters with a diagnosiscode for any other type of ILD on or after thedate of their last medical encounter with adiagnosis of IPF.IPF (“narrow case definition”) was identifiedas (i) satisfaction of the broad case definitionand (ii) one or more medical encounters witha procedure code for surgical lung biopsy(ICD-9-CM 33.28, 34.21; CPT-4 32095,32100–32160, 32602), transbronchial lungbiopsy (ICD-9-CM 33.27; CPT-4 31628,31629), or computed tomography of thethorax (ICD-9-CM 87.41; CPT-4 71250,71260, 71270) on or before the date of theirlast medical encounter with a diagnosis ofIPF.

Suissa etal. 20063

PharMetrics Patient-Centric Database, 1September 1998–31 December 2003. Subjectswere 18 years of age or older at cohort entry.

Risk of ILD in patients with rheumatoidarthritis treated with leflunomide.

Cases of probable drug-related ILD wereidentified from inpatient encounters as allsubjects who were hospitalized with a first-time primary diagnosis of postinflammatorylung fibrosis (ICD-9 code 515), idiopathicfibrosing alveolitis (code 516.3), or other/unspecified alveolar pneumonopathies (codes516.8 and 516.9).

n. jones et al.258

Copyright © 2012 John Wiley & Sons, Ltd. Pharmacoepidemiology and Drug Safety, 2012; 21(S1): 256–260DOI: 10.1002/pds

516.3, 516.8, 516.9, and 501 (note that Raghu et al.4

studied IPF, a subtype of ILD). ICD-9-CM codes516.8 and 516.9 were used to broadly define ILD.ICD-9-CM codes 515 and 516.3 were used to definePF, whereas IPF, a subtype of PF, was defined byICD-9-CM code 516.3. ICD-9-CM code 501 was usedto identify asbestosis, which can be considered a sub-group of ILD. The limited breadth of codes used forILD identification in the reviewed literature, despitethe recognition of 36 ILD-consistent ICD-9-CMcodes, suggests that the general literature search usedby Mini-Sentinel was inadequate for this HOI.Regardless of the diagnostic codes used for PF/ILD

algorithms, fine tuning of algorithms may be possiblevia relevant procedural codes, such as those for lungbiopsies, and relevant imaging techniques (e.g., high-resolution computed tomography). Such strategieswere applied by Raghu et al.,4 who narrowed theirIPF algorithm by requiring cases to have one or moremedical encounters with a procedure code for surgicallung biopsy, transbronchial lung biopsy, or computedtomography of the thorax. These procedural codesadded specificity to the algorithm but reduced thenumber of IPF patients identified by approximatelythreefold.4 There are, however, limiting factors thatmust be considered when using procedural codes fromadministrative and claims data. Most notably, informa-tion on results and diagnosis confirmation is generallynot available.Algorithm development may be hindered further by

potential differential coding resulting from the special-ization of health care providers and the settings ofhealth care. For example, we speculate that the codeschosen may be associated with the perceived certaintyof the diagnosis, which may vary between specialistsand primary care providers. Likewise, there may bedifferences in inpatient and outpatient settings. Theseall potentially result in various diagnostic codes beingcaptured in administrative and claims health caredatabases and may influence algorithm development.

CONCLUSION

There are almost certainly definitional problemspertaining to each of the codes used within theidentified PF/ILD algorithms. However, because noneof these algorithms provided validation, the extent ofthese problems cannot be known. We suspect thatICD-9-CM codes 515 and 516.3 are most likely toonarrow to identify all PF/ILD cases. By contrast,the ICD-9-CM codes 516.8 and 516.9 have more-extensive definitions and are perhaps used when thereis uncertainty about a specific PF/ILD diagnosis. There

are also many diagnostic codes consistent with ILDthat were not incorporated into the reviewed studies,suggesting that even these more-extensive diagnosticcodes may be inadequate to capture all ILD cases.Therefore, using the codes identified in this literaturereview for case identification, even in combinationwith procedural codes, may not provide the desiredlevels of sensitivity and specificity. The scarcity ofliterature providing validated or non-validatedalgorithms for PF/ILD suggests the need for additionalresearch focused on identifying and designing valida-tion studies that test PF/ILD algorithms and estimatetheir predictive power, sensitivity, and specificity.

CONFLICTS OF INTEREST

The authors declare no conflict of interest. This is notproduct-specific or privately funded research. Theviews expressed in this document do not necessarilyreflect the official policies of the Department of Healthand Human Services, nor does this document mentiontrade names, commercial practices, or organizationsimply endorsement by the US government.

KEY POINTS• There is limited literature focusing on pulmonaryfibrosis and interstitial lung disease that providesadministrative and claims data-based codingalgorithms and validation estimates.

• The broad spectrum of diseases under theumbrella of interstitial lung disease may compli-cate algorithm development for common sub-types, such as pulmonary fibrosis.

• Additional research is needed regarding the useof administrative and claims data-based codingalgorithms to identify pulmonary fibrosis andinterstitial lung disease.

ACKNOWLEDGEMENTS

This work was supported by the Food and DrugAdministration (FDA) through Department of Healthand Human Services (HHS) Contract NumberHHSF223200910006I.

REFERENCES

1. King TE, Jr. Clinical advances in the diagnosis and therapy of the interstitial lungdiseases. Am J Respir Crit Care Med 2005; 172: 268–279.

2. American Thoracic Society/European Respiratory Society International Multidis-ciplinary Consensus Classification of the Idiopathic Interstitial Pneumonias. This

detection of pulmonary fibrosis in claims 259

Copyright © 2012 John Wiley & Sons, Ltd. Pharmacoepidemiology and Drug Safety, 2012; 21(S1): 256–260DOI: 10.1002/pds

joint statement of the American Thoracic Society (ATS) and the EuropeanRespiratory Society (ERS) was adopted by the ATS board of directors in June2001 and by the ERS Executive Committee in June 2001. Am J Respir Crit CareMed 2002; 165: 277–304.

3. Suissa S, Hudson M, Ernst P. Leflunomide use and the risk of interstitial lungdisease in rheumatoid arthritis. Arthritis Rheum 2006; 54: 1435–1439.

4. Raghu G, Weycker D, Edelsberg J, Bradford WZ, Oster G. Incidence andprevalence of idiopathic pulmonary fibrosis. Am J Respir Crit Care Med 2006;174: 810–816.

5. Ehrlich SF, Quesenberry CP, Jr., Van Den Eeden SK, Shan J, Ferrara A. Patientsdiagnosed with diabetes are at increased risk for asthma, chronic obstructive pul-monary disease, pulmonary fibrosis, and pneumonia but not lung cancer. DiabetesCare 2010; 33: 55–60.

6. Pinheiro GA, Antao VC, Wood JM, Wassell JT. Occupational risks for idiopathicpulmonary fibrosis mortality in the United States. Int J Occup Environ Health2008; 14: 117–123.

7. Gan WQ, Demers PA, McLeod CB, Koehoorn M. Population-based asbestosissurveillance in British Columbia. Occup Environ Med 2009; 66: 766–771.

n. jones et al.260

Copyright © 2012 John Wiley & Sons, Ltd. Pharmacoepidemiology and Drug Safety, 2012; 21(S1): 256–260DOI: 10.1002/pds