1
Development and Validation of an Illness Severity Scoring System for Viral Respiratory Tract Infections in Children Parisa Mortaji, Orrin Myers, Megan Woslager, Alfonso Belmonte, Annalisa Behnken, Anjali Subbaswamy, Hemant Agarwal, Teresa Vigil, Francine Caffey, Martha Muller, Darrell Dinwiddie, Walter Dehority Department of Pediatric Infectious Disease, University of New Mexico School of Medicine Introduction Viral respiratory infections are universal illnesses caused by many different viral strains and are highly prevalent in young children. Novel investigative tools, such as whole genome sequencing, offer promise of identifying gene determinants of disease severity, which help characterize host and viral genetic contributions to severity of illness. A validated scoring system for quantifying illness severity is needed to contextualize results. Limitations of existing scoring systems in the literature: Do not reflect novel changes in clinical care over time, such as early discharge rates, access to home health care, improved ICU care, and medical technological advancement Limited to only one viral season Few to no raters of disease severity Exclusion of children with underlying comorbidities Development of a scoring system for a specific viral strain (e.g. RSV patients only) Limited age range of subjects and small sample sizes Frequently cited studies used as gold standards in scoring system development presented many of the aforementioned limitations Overall: limited generalizability and practicality, invalidated, underpowered, and outdated A new scoring system that is generalizable and reflects modern medical practice is needed. We sought to develop such a scoring system to quantify severity of illness in children hospitalized with viral respiratory infections. Methods Children hospitalized with viral respiratory infections were prospectively enrolled over a 2-year period from March 2015 to March 2017. Children were required to present with 2 of the following clinical findings to be eligible: rhinorrhea or nasal congestion, cough, fever, increased work of breathing, hypoxia or sore throat, and no obvious alternative etiology. 51 clinical variables from each subject were extracted from the medical record and entered into a REDCap database. Seven variables that significantly correlated with each other (p<0.001) were combined into a disease severity index: (duration of hospital/ICU stay, oxygen and high flow nasal cannula (HFNC) use, intubation, maximum nasal cannula and HFNC oxygen requirement). For validation comparison, sampling algorithms selected a subset of 96 patients whose disease severity would be randomly assessed by 8 pediatricians in blocks of 12. Severity was scored on a scale of 1-10. Mixed model regression analyses compared clinician- scored disease severity with the scoring system. Akaike Information criteria (AIC) and coefficients of determination (R 2 ) ranked severity indices. Conclusions We developed a novel, validated scoring system for pediatric viral respiratory infections. To our knowledge, this is the first such scoring index and the largest dataset comprising 445 pediatric patients with viral respiratory infections. The scoring index using 7 variables and 7 levels of exposure produced an R 2 of 0.7 at a p-value of <0.0001. The correlation coefficient of 0.7, given the inherent subjectivity and difficulty involved in defining illness severity, performed and correlated well with our clinician-assessed gold standard. The patients had variable characteristics regarding age and comorbidities, making the results more generalizable toward a wider population. In addition, we included multiple respiratory seasons to allow for varied viral virulence by year, as well as multiple different viruses and strains which, again, increases generalizability. Limitations: Pediatric inpatients only Single institution in one geographic region of the country allows for potentially undetected confounders and differing viral serovars based on climate, weather, and socioeconomic factors In summary, we developed a 7-variable validated universal scoring index for pediatric viral respiratory infections for use by clinicians and researchers. References Bezerra PGM, Britto MCA, Correia JB, et al. Viral and Atypical Bacterial Detection in Acute Respiratory Infection in Children Under Five Years. PLoS ONE. 2011;6(4): e18928. Chonmaitree T, Revai K, Grady JJ, et al. Viral Upper Respiratory Tract Infection and Otitis Media Complication in Young Children. Clinical Infectious Diseases. 2008;46(6)815–823. Duarte-Dorado, D. M., Madero-Orostegui, D. S., Rodriguez-Martinez, C. E., et al. Validation of a scale to assess the severity of bronchiolitis in a population of hospitalized infants. J Asthma. 2013;50(10):1056-1061. Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)-a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. Lazar I, Weibel C, Dziura J, et al. Human metapneumovirus and severity of respiratory syncytial virus disease. Emerg Infect Dis. 2004;10(7):1318-1321. Martinello RA, Chen MD, Weibel C, et al. Correlation between respiratory syncytial virus genotype and severity of illness. J Infect Dis. 2002;186:839- 842. McConnochie KM, Hall CB, Walsh EE, et al. Variation in severity of respiratory syncytial virus infections with subtype. J Pediatr. 1990;117:52-62. Mejias A, Dimo B, Suarez NM, et al. Whole blood gene expression profiles to assess pathogenesis and disease severity in infants with respiratory syncytial virus infection. PLoS Med. 2013;10(11):1-19. Mella C, Suarez-Arrabal MC, Lopez S, et al. Innate immune dysfunction is associated with enhanced disease severity in infants with severe respiratory syncytial virus bronchiolitis. J Infect Dis. 2013;207:564-573. Midulla F, Scagnolari C, Bonci E, et al. Respiratory syncytial virus, human bocavirus and rhinovirus bronchiolitis in infants. Arch Dis Child. 2010;95:35- 41. Miyaji Y, Kobayashi M, Sugai K, et al. Severity of respiratory signs and symptoms and virus profiles in Japanese children with acute respiratory illness. Microbiol Immunol. 2013;57:811-821. Monto AS, Malosh RE, Petrie JG, et al. Frequency of acute respiratory illnesses and circulation of respiratory viruses in households with children over 3 surveillance seasons. J Infect Dis. 2014;210(11):1792-1799. Tal A, Bavilski C, Yohai D, et al. Dexamethasone and salbutamol in the treatment of acute wheezing in infants. Pediatrics. 1983;71(1):13-21. Taylor JA, Weber WJ, Martin ET, et al. Development of a symptom score for clinical studies to identify children with a documented viral upper respiratory tract infection. Pediatr Res. 2010;68(3):252-257. Wainwright C, Altamirano L, Cirujano M, et al. A multi-center, randomized, double-blind, controlled trial of nebulized epinephrine in infants with acute bronchiolitis. N Engl J Med. 2003;349:27-35. Walsh EE, McConnochie KM, Long CE, et al. Severity of respiratory syncytial virus infection is related to virus strain. J Infect Dis. 1997;175:814-820. Abstract Novel investigative tools help characterize host and viral genetic contributions to disease severity in pediatric viral respiratory infections. However, a validated scoring system for quantifying illness severity is needed to properly contextualize results. Existing scoring systems are outdated, invalidated and underpowered. We sought to develop such a scoring system to quantify severity of illness in children hospitalized with viral respiratory infections. Such a scoring system may be useful to clinicians in assessing disease severity and researchers requiring a quantitated assessment of illness severity. Children hospitalized with viral respiratory infections were prospectively enrolled over two years (n=445), with 51 clinical variables extracted from the medical record. Seven variables that significantly correlated with each other were combined into a disease severity index. For a validation comparison, disease severity of a random subset of patients was assessed by 8 pediatricians. Mixed model regression analyses compared clinician-scored disease severity with the scoring system. Akaike Information criteria and coefficients of determination (R 2 ) ranked severity indices. Clinician scores of disease severity averaged 6.2 (SD=2.2, range 1-10). A scoring system using 7 variables with 7 levels of exposure for each variable produced the lowest AIC (0.00, R 2 = 0.70 for predicting clinician-scored disease severity after adjustment for rater effects, p<0.0001). A 7-variable scoring system quantifying disease severity in pediatric viral respiratory infections correlates well with clinician assessment, and may advance the study of such infections. Results Out of 579 potential subjects approached, 445 were enrolled (76.9% consent rate), of which 250 (56.2%) were male and 195 (43.8%) were female. The mean age of subjects was 2.40 years (median=1.2 years, inter-quartile range=0.4–3.2). A majority (65.8%) of subjects had no past medical history. The most frequent medical problems were asthma (12.8%), prematurity (9.7%, defined as birth between 24-34 weeks’ gestation), and chronic lung disease (7.6%). Clinician scores of disease severity averaged 6.2 (SD = 2.2, range 1-10). A scoring system using 7 variables with 7 levels of exposure per variable produced the lowest AIC (0.00, R 2 = 0.70 for predicting clinician-scored disease severity after adjustment for rater effects, p<0.0001). 0 5 10 15 20 25 30 35 40 45 Figure 1: Distribution of Viral Isolates (2015 – 2017) Percentage of Isolates Table 1: Racial Demographics (n = 445) Race Number Percent White/Anglo 289 64.9 American Indian/Alaskan Native 95 21.3 Black/African American 15 3.4 Asian 7 1.6 Hispanic/Latino 1 0.2 Other 3 0.7 Decline to answer 35 7.9 Table 2: Distributions of clinical variables in total sample (n = 445) Frequency of positive clinical values Distribution of positive clinical values N (%) Median Min Max Days with HFNC (d) 196 (44.0) 3.0 0.1 18.9 Duration of Hospitalization (d) 445 (100.0) 3.7 0.1 71.8 Duration of Intubation (d) 18 (4.0) 4.3 0.3 29.3 Duration of Oxygen Use in Hospital (d) 409 (91.9) 3.3 0.1 71.8 Max HFNC Requirement (L) 200 (44.9) 12.0 3.0 35.0 Max Nasal Cannula Requirement (L) 395 (88.8) 2.0 0.1 20.0 PICU Stay (d) 143 (32.1) 3.0 0.5 29.3 Table 3: Results Categories per clinical variable r2 r2 adjusted for raters p-value 3 0.499 0.671 <0.0001 4 0.461 0.662 <0.0001 5 0.486 0.680 <0.0001 6 0.483 0.684 <0.0001 7 0.509 0.700 <0.0001 8 0.491 0.687 <0.0001 9 0.480 0.685 <0.0001 10 0.484 0.691 <0.0001

Development and Validation of an Illness Severity Scoring

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Development and Validation of an Illness Severity Scoring

DevelopmentandValidationofanIllnessSeverityScoringSystemforViralRespiratoryTractInfectionsinChildren

ParisaMortaji,OrrinMyers,MeganWoslager,AlfonsoBelmonte,AnnalisaBehnken,AnjaliSubbaswamy,HemantAgarwal,TeresaVigil,FrancineCaffey,MarthaMuller,DarrellDinwiddie,WalterDehority

DepartmentofPediatricInfectiousDisease,UniversityofNewMexicoSchoolofMedicine

Introduction• Viralrespiratoryinfectionsareuniversalillnessescausedbymanydifferentviralstrainsandare

highlyprevalentinyoungchildren.• Novelinvestigativetools,suchaswholegenomesequencing,offerpromiseofidentifyinggene

determinantsofdiseaseseverity,whichhelpcharacterizehostandviralgeneticcontributionstoseverityofillness.

• Avalidatedscoringsystemforquantifyingillnessseverityisneededtocontextualizeresults.• Limitationsofexistingscoringsystemsintheliterature:

• Donotreflectnovelchangesinclinicalcareovertime,suchasearlydischargerates,accesstohomehealthcare,improvedICUcare,andmedicaltechnologicaladvancement

• Limitedtoonlyoneviralseason• Fewtonoratersofdiseaseseverity• Exclusionofchildrenwithunderlyingcomorbidities• Developmentofascoringsystemforaspecificviralstrain(e.g.RSVpatientsonly)• Limitedagerangeofsubjectsandsmallsamplesizes• Frequentlycitedstudiesusedasgoldstandardsinscoringsystemdevelopmentpresented

manyoftheaforementionedlimitations• Overall:limitedgeneralizabilityandpracticality,invalidated,underpowered,andoutdated

• Anewscoringsystemthatisgeneralizableandreflectsmodernmedicalpracticeisneeded.Wesoughttodevelopsuchascoringsystemtoquantifyseverityofillnessinchildrenhospitalizedwithviralrespiratoryinfections.

Methods• Childrenhospitalizedwithviralrespiratoryinfectionswereprospectivelyenrolledovera2-year

periodfromMarch2015toMarch2017.• Childrenwererequiredtopresentwith2ofthefollowingclinicalfindingstobeeligible:

rhinorrheaornasalcongestion,cough,fever,increasedworkofbreathing,hypoxiaorsorethroat,andnoobviousalternativeetiology.

• 51clinicalvariablesfromeachsubjectwereextractedfromthemedicalrecordandenteredintoaREDCap database.

• Sevenvariablesthatsignificantlycorrelatedwitheachother(p<0.001)werecombinedintoadiseaseseverityindex:(durationofhospital/ICUstay,oxygenandhighflownasalcannula(HFNC)use,intubation,maximumnasalcannulaandHFNCoxygenrequirement).

• Forvalidationcomparison,samplingalgorithmsselectedasubsetof96patientswhosediseaseseveritywouldberandomlyassessedby8pediatriciansinblocksof12.

• Severitywasscoredonascaleof1-10.Mixedmodelregressionanalysescomparedclinician-scoreddiseaseseveritywiththescoringsystem.

• Akaike Informationcriteria(AIC)andcoefficientsofdetermination(R2)rankedseverityindices.

Conclusions• Wedevelopedanovel,validatedscoringsystemforpediatricviralrespiratoryinfections.• Toourknowledge,thisisthefirstsuchscoringindexandthelargestdatasetcomprising445

pediatricpatientswithviralrespiratoryinfections.• Thescoringindexusing7variablesand7levelsofexposureproducedanR2 of0.7atap-value

of<0.0001.Thecorrelationcoefficientof0.7,giventheinherentsubjectivityanddifficultyinvolvedindefining illnessseverity,performedandcorrelatedwellwithourclinician-assessedgoldstandard.

• Thepatientshadvariablecharacteristicsregardingageandcomorbidities,makingtheresultsmoregeneralizabletowardawiderpopulation.Inaddition,weincludedmultiplerespiratoryseasonstoallowforvariedviralvirulencebyyear,aswellasmultipledifferentvirusesandstrainswhich,again,increasesgeneralizability.

• Limitations:• Pediatricinpatientsonly• Singleinstitutioninonegeographicregionofthecountryallowsforpotentially

undetectedconfoundersanddifferingviralserovars basedonclimate,weather,andsocioeconomicfactors

• Insummary,wedevelopeda7-variablevalidateduniversalscoringindexforpediatricviralrespiratoryinfectionsforusebycliniciansandresearchers.

References• Bezerra PGM,Britto MCA,Correia JB,etal.ViralandAtypicalBacterialDetectioninAcuteRespiratoryInfectioninChildrenUnderFiveYears.PLoS

ONE.2011;6(4):e18928.• Chonmaitree T,Revai K,GradyJJ,etal.ViralUpperRespiratoryTractInfectionandOtitisMediaComplicationinYoungChildren.ClinicalInfectious

Diseases.2008;46(6)815–823.• Duarte-Dorado,D.M.,Madero-Orostegui,D.S.,Rodriguez-Martinez,C.E.,etal.Validationofascaletoassesstheseverityofbronchiolitisina

populationofhospitalizedinfants.JAsthma.2013;50(10):1056-1061.• HarrisPA,TaylorR,Thielke R,etal.Researchelectronicdatacapture(REDCap)-ametadata-drivenmethodologyandworkflowprocessforproviding

translationalresearchinformaticssupport.JBiomedInform.2009;42(2):377-381.• LazarI,Weibel C,Dziura J,etal.Humanmetapneumovirus andseverityofrespiratorysyncytialvirusdisease.Emerg InfectDis.2004;10(7):1318-1321.• Martinello RA,ChenMD,Weibel C,etal.Correlationbetweenrespiratorysyncytialvirusgenotypeandseverityofillness.JInfectDis.2002;186:839-

842.• McConnochie KM,HallCB,WalshEE,etal.Variationinseverityofrespiratorysyncytialvirusinfectionswithsubtype.JPediatr.1990;117:52-62.• Mejias A,Dimo B,SuarezNM,etal.Wholebloodgeneexpressionprofilestoassesspathogenesisanddiseaseseverityininfantswithrespiratory

syncytialvirusinfection.PLoS Med.2013;10(11):1-19.• Mella C,Suarez-Arrabal MC,LopezS,etal.Innateimmunedysfunctionisassociatedwithenhanceddiseaseseverityininfantswithsevererespiratory

syncytialvirusbronchiolitis.JInfectDis.2013;207:564-573.• Midulla F,Scagnolari C,Bonci E,etal.Respiratorysyncytialvirus,humanbocavirus andrhinovirusbronchiolitisininfants.ArchDisChild.2010;95:35-

41.• MiyajiY,KobayashiM,Sugai K,etal.SeverityofrespiratorysignsandsymptomsandvirusprofilesinJapanesechildrenwithacuterespiratoryillness.

Microbiol Immunol.2013;57:811-821.• MontoAS,Malosh RE,PetrieJG,etal.Frequencyofacuterespiratoryillnessesandcirculationofrespiratoryvirusesinhouseholdswithchildrenover3

surveillanceseasons.JInfectDis.2014;210(11):1792-1799.• TalA,Bavilski C,Yohai D,etal.Dexamethasoneandsalbutamolinthetreatmentofacutewheezingininfants.Pediatrics.1983;71(1):13-21.• TaylorJA,WeberWJ,MartinET,etal.Developmentofasymptomscoreforclinicalstudiestoidentifychildrenwithadocumentedviralupper

respiratorytractinfection.Pediatr Res.2010;68(3):252-257.• WainwrightC,Altamirano L,Cirujano M,etal.Amulti-center,randomized,double-blind,controlledtrialofnebulizedepinephrineininfantswith

acutebronchiolitis.NEngl JMed.2003;349:27-35.• WalshEE,McConnochie KM,LongCE,etal.Severityofrespiratorysyncytialvirusinfectionisrelatedtovirusstrain.JInfectDis.1997;175:814-820.

AbstractNovelinvestigativetoolshelpcharacterizehostandviralgeneticcontributionstodiseaseseverityinpediatricviral

respiratoryinfections.However,avalidatedscoringsystemforquantifyingillnessseverityisneededtoproperlycontextualizeresults.Existingscoringsystemsareoutdated,invalidatedandunderpowered.Wesoughttodevelopsuchascoringsystemtoquantifyseverityofillnessinchildrenhospitalizedwithviralrespiratoryinfections.Suchascoringsystemmaybeusefultocliniciansinassessingdiseaseseverityandresearchersrequiringaquantitatedassessmentofillnessseverity.

Childrenhospitalizedwithviralrespiratoryinfectionswereprospectivelyenrolledovertwoyears(n=445),with51clinicalvariablesextractedfromthemedicalrecord.Sevenvariablesthatsignificantlycorrelatedwitheachotherwerecombinedintoadiseaseseverityindex.Foravalidationcomparison,diseaseseverityofarandomsubsetofpatientswasassessedby8pediatricians.Mixedmodelregressionanalysescomparedclinician-scoreddiseaseseveritywiththescoringsystem.AkaikeInformationcriteriaandcoefficientsofdetermination(R2)rankedseverityindices.Clinicianscoresofdiseaseseverityaveraged6.2(SD=2.2,range1-10).Ascoringsystemusing7variableswith7levelsofexposureforeachvariableproducedthelowestAIC(0.00,R2 =0.70forpredictingclinician-scoreddiseaseseverityafteradjustmentforratereffects,p<0.0001).A7-variablescoringsystemquantifyingdiseaseseverityinpediatricviralrespiratoryinfectionscorrelateswellwithclinicianassessment,andmayadvancethestudyofsuchinfections.

Results• Outof579potentialsubjectsapproached,445wereenrolled(76.9%consentrate),ofwhich

250(56.2%)weremaleand195(43.8%)werefemale.Themeanageofsubjectswas2.40years(median=1.2years,inter-quartilerange=0.4–3.2).

• Amajority(65.8%)ofsubjectshadnopastmedicalhistory.Themostfrequentmedicalproblemswereasthma(12.8%),prematurity(9.7%,definedasbirthbetween24-34weeks’gestation),andchroniclungdisease(7.6%).

• Clinicianscoresofdiseaseseverityaveraged6.2(SD=2.2,range1-10).Ascoringsystemusing7variableswith7levelsofexposurepervariableproducedthelowestAIC(0.00,R2 =0.70forpredictingclinician-scoreddiseaseseverityafteradjustmentforratereffects,p<0.0001).

05

1015202530354045

Figure1:DistributionofViral Isolates(2015– 2017)

PercentageofIsolates

Table1:RacialDemographics(n=445)

Race Number PercentWhite/Anglo 289 64.9

AmericanIndian/AlaskanNative 95 21.3

Black/AfricanAmerican 15 3.4

Asian 7 1.6

Hispanic/Latino 1 0.2

Other 3 0.7

Declinetoanswer 35 7.9

Table2:Distributionsofclinicalvariablesintotalsample(n=445)

Frequencyofpositiveclinical

values

Distributionofpositiveclinicalvalues

N (%) Median Min Max

DayswithHFNC(d) 196 (44.0) 3.0 0.1 18.9

DurationofHospitalization(d) 445 (100.0) 3.7 0.1 71.8

DurationofIntubation(d) 18 (4.0) 4.3 0.3 29.3

DurationofOxygenUseinHospital(d)

409 (91.9) 3.3 0.1 71.8

MaxHFNCRequirement(L) 200 (44.9) 12.0 3.0 35.0

MaxNasalCannulaRequirement(L) 395 (88.8) 2.0 0.1 20.0

PICUStay(d) 143 (32.1) 3.0 0.5 29.3

Table3:Results

Categoriesperclinicalvariable

r2 r2adjustedforraters p-value

3 0.499 0.671 <0.0001

4 0.461 0.662 <0.0001

5 0.486 0.680 <0.0001

6 0.483 0.684 <0.0001

7 0.509 0.700 <0.0001

8 0.491 0.687 <0.0001

9 0.480 0.685 <0.0001

10 0.484 0.691 <0.0001