Upload
carlos-poveda
View
218
Download
0
Embed Size (px)
Citation preview
8/18/2019 Criterios Clinicos de Sepsis Seymour
1/13
Copyright 2016 American Medical Association. All rig hts reserved.
Assessment of Clinical Criteria for Sepsis
For the Third International Consensus Definitions
for Sepsis and Septic Shock (Sepsis-3)
Christopher W. Seymour, MD, MSc; Vincent X. Liu, MD, MSc; Theodore J.Iwashyna,MD, PhD; FrankM. Brunkhorst, MD;Thomas D.Rea,MD, MPH;
AndréScherag, PhD;Gordon Rubenfeld,MD, MSc;JeremyM. Kahn, MD,MSc; Manu Shankar-Hari, MD,MSc; Mervyn Singer, MD,FRCP;
CliffordS. Deutschman, MD, MS; Gabriel J.Escobar, MD;Derek C.Angus, MD, MPH
IMPORTANCE The Third InternationalConsensus Definitions Task Force defined sepsis
as “life-threatening organ dysfunction due to a dysregulated hostresponse to infection.”
Theperformanceof clinical criteria forthis sepsisdefinition is unknown.
OBJECTIVE To evaluate the validity of clinical criteria to identify patients with suspected
infectionwho are at risk of sepsis.
DESIGN, SETTINGS, AND POPULATION Among1.3 million electronichealth record encounters
from January 1, 2010, to December 31,2012, at 12 hospitalsin southwesternPennsylvania, we
identified those with suspected infectionin whom to compare criteria. Confirmatory analyses
were performed in 4 data sets of 706 399 out-of-hospital andhospital encounters at 165 US
andnon-US hospitalsrangingfrom January 1, 2008, until December 31,2013.
EXPOSURES Sequential[Sepsis-related] OrganFailure Assessment(SOFA)score, systemic
inflammatory response syndrome(SIRS) criteria,LogisticOrgan Dysfunction System (LODS)
score,and a newmodelderived usingmultivariablelogisticregressionin a split sample, thequick
Sequential[Sepsis-related] OrganFailure Assessment(qSOFA) score(range,0-3 points, with1
pointeach forsystolichypotension [100mm Hg], tachypnea[22/min],or altered mentation).
MAIN OUTCOMESAND MEASURES For constructvalidity, pairwise agreementwas assessed.
For predictive validity, the discrimination for outcomes (primary: in-hospital mortality;
secondary:in-hospital mortality or intensive care unit [ICU] length of stay3 days) more
common in sepsis thanuncomplicated infectionwas determined. Results were expressed as
thefold change in outcome over deciles of baseline risk of death andarea under thereceiver
operating characteristic curve (AUROC).
RESULTS In theprimary cohort, 148907 encounters hadsuspected infection(n = 74 453
derivation; n = 74454 validation), of whom 6347 (4%) died. Among ICUencounters in the
validation cohort (n = 7932 withsuspected infection, of whom1289 [16%] died),the predictive
validityfor in-hospital mortalitywas lowerfor SIRS (AUROC = 0.64; 95% CI, 0.62-0.66) and
qSOFA (AUROC = 0.66; 95% CI, 0.64-0.68) vs SOFA (AUROC = 0.74; 95%CI, 0.73-0.76;
P < .001 for both) or LODS(AUROC = 0.75; 95%CI, 0.73-0.76; P < .001 for both). Among
non-ICU encounters in thevalidationcohort (n = 66 522withsuspectedinfection, of whom
1886 [3%]died),qSOFA had predictive validity (AUROC = 0.81; 95%CI, 0.80-0.82) thatwas
greater thanSOFA (AUROC = 0.79; 95% CI, 0.78-0.80;P < .001) and SIRS (AUROC = 0.76; 95%
CI, 0.75-0.77; P < .001). Relativeto qSOFA scoreslowerthan 2, encounters with qSOFAscores of
2 or higherhad a 3- to 14-fold increase in hospitalmortality across baseline risk deciles. Findingswere similar in external data sets andfor thesecondary outcome.
CONCLUSIONS AND RELEVANCE Among ICU encounters with suspected infection, the
predictive validity for in-hospital mortality of SOFA was not significantly different than the
more complex LODS butwas statisticallygreater than SIRS andqSOFA, supporting itsuse in
clinical criteria for sepsis. Among encounters with suspectedinfection outside of theICU, the
predictive validity for in-hospital mortality of qSOFA was statistically greater thanSOFA and
SIRS, supporting itsuse as a promptto consider possible sepsis.
JAMA. 2016;315(8):762-774. doi:10.1001/jama.2016.0288
Editorial page 757
Author AudioInterview at
jama.com
Related articles pages775 and
801
Supplementalcontent at
jama.com
Author Affiliations: Authoraffiliations arelisted at theendof this
article.
Corresponding Author: Christopher
W.Seymour, MD,MSc, Departments
of Critical Care Medicineand
Emergency Medicine,Universityof
PittsburghSchool of Medicine,
Clinical Research,Investigation,and
Systems Modeling of AcuteIllness
(CRISMA) Center,3550 TerraceSt,
ScaifeHall, Ste639, Pittsburgh, PA
15261 ([email protected]).
Research
Original Investigation | CARING FOR THE CRITICALLY ILL PATIENT
762 (Reprinted) jama.com
Copyright 2016 American Medical Association. All rig hts reserved.
wnloaded From: http://jama.jamanetwork.com/ by Jose Vergara on 02/29/2016
http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0288&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0288&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0290&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0288&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www./http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0287&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0289&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0288&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288mailto:[email protected]:[email protected]:[email protected]://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288mailto:[email protected]://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0288&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0289&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0287&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www./http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0288&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0290&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0288&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288
8/18/2019 Criterios Clinicos de Sepsis Seymour
2/13
Copyright 2016 American Medical Association. All rig hts reserved.
Although common and associated with high morbidity
and mortality,, sepsis and related terms remain diffi-
cult to define. Two international consensus confer-
encesin andusedexpertopinion togenerate thecur-
rentdefinitions.,However,advancesin theunderstanding of
thepathobiologyand appreciation that elements of thedefini-
tions may be outdated, inaccurate, or confusing prompted
the European Society of
Intensive Care Medicine
and the Society of Critical
Care Medicine to convene
a Third International Con-
sensus Task Force to re-
examine the definitions.
Like many syndromes,
there is no “gold stan-
dard” diagnostic test for
sepsis. Therefore, the task
force chose several meth-
odsto evaluatethe useful-
ness of candidate clinical criteria, including clarity, reliability
(consistency and availability), content validity (biologic ratio-
nale and face validity),constructvalidity (agreementbetween
similar measures), criterion validity (correlation with estab-
lished measures and outcomes), burden, and timeliness. Un-
like prior efforts, the task force used systematic literature re-
views and empirical data analyses to complement expert
deliberations.
Based on clarity and content validity and after literature
review and expert deliberation, the task force recommended
elimination of the terms sepsis syndrome, septicemia, and se-
vere sepsis and instead defined sepsis as “life-threatening or-
gan dysfunction due to a dysregulated host response to
infection.” Ofnote,the task forcedid notattempt to redefine
infection. Rather, it next sought to generate recommenda-tions for clinical criteria that could be used to identify sepsis
among patients with suspected or confirmed infection. The
purposeof thisstudywas to inform this stepby analyzingdata
from several large hospitaldatabases to explore the construct
validity andcriterionvalidityof existingand novel criteriaas-
sociated with sepsis.
Methods
This study was approved with waiver of informed consent by
the institutional review boardsof theUniversity of Pittsburgh,
Kaiser Permanente Northern California (KPNC), Veterans Ad-ministration (VA) Ann Arbor Health System, Washington State
Department of Health, King County Emergency Medical Ser-
vices (KCEMS), University of Washington, and Jena University
Hospital.
Study Design, Setting, and Population
A retrospective cohortstudywas performedamong adult en-
counters (age ≥ years) with suspected infection. The pri-
mary cohortwas all hospital encounters from to at
communityand academic hospitalsin theUPMChealth care
systemin southwesternPennsylvania. The cohortincluded all
medical and surgical encounters in the emergency depart-
ment, hospital ward, and intensive care unit (ICU). We cre-
ated a random split sample (/) fromthe UPMC cohort,the
derivation cohort for developing new criteria, and the valida-
tion cohort for assessment of new and existing criteria.
We also studied external data sets: () all inpatient
encounters at KPNC hospitalsfrom to ; ()all en-
counters in hospitals in the United States’ VA system
from to ; () all nontrauma, nonarrest emergency
medical services records from advanced life support agen-
ciesfrom - transported to hospitalswith commu-
nityinfectionin KingCounty, Washington(KCEMS);and()all
patients from - at German hospital enrolled with
hospital-acquired infection in the ALERTS prospective cohort
study.These cohorts were selected becausethey included pa-
tient encounters fromdifferent phases of acutecare (outof hos-
pital, emergency department, hospital ward) and countries
(United States and Germany) with different types of infection
(community and nosocomial). The UPMC, KPNC, and VA data
were obtained fromthe electronic healthrecords (EHRs)of the
respective health systems;KCEMSdata wereobtainedfrom the
administrative out-of-hospital record; and ALERTS data were
collected prospectively by research coordinators.
Defining a Cohort With Suspected Infection
For EHR data (UPMC, KPNC, and VA), the first episode of sus-
pected infection wasidentifiedas thecombination of antibiot-
ics(oral or parenteral) andbodyfluidcultures(blood, urine,ce-
rebrospinal fluid, etc). We required thecombination of culture
and antibiotic start time to occur within a specifictime epoch.
If theantibioticwas givenfirst,the culturesamplingmusthave
beenobtained within hours.If theculturesamplingwas first,
theantibioticmust have beenordered within hours. The“on-
set” of infection was defined as the time at which the first of these events occurred (eAppendix in the Supplement). For
non-EHR data in ALERTS, patients were included who met
US Centers for Disease Control and Prevention definitions or
clinical criteria for hospital-acquired infection more than
hoursafter admission asdocumentedbyprospectivescreening.
For non-EHR data in KCEMS, administrative claims identified
infection present on admission (Angus implementation of in-
fectionusing International Classification of Diseases, Ninth Re-
vision, Clinical Modification ( ICD--CM ) diagnosis codes).
Determining Clinical Criteriafor SepsisUsing ExistingMeasures
In UPMC derivation and validation data, indicators were gen-
erated for each component of the systemic inflammatory re-sponse syndrome (SIRS) criteria; the Sequential [Sepsis-
related] Organ Failure Assessment (SOFA) score; and the
Logistic Organ Dysfunction System (LODS) score, a weighted
organ dysfunction score (Table ). We used a modifiedversion
of the LODS score that did not contain urine output (because
ofpooraccuracy inrecording onhospital ward encounters), pro-
thrombin, or urea levels. The maximum SIRS criteria, SOFA
score, and modified LODS score were calculated for the time
window from hoursbeforeto hours after the onset ofin-
fection, aswell as on each calendar day. This window wasused
EHR electronic healthrecord
GCS GlasgowComa Scale
ICU intensivecare unit
LODS LogisticOrgan Dysfunction
System
qSOFA quickSequential
[Sepsis-related]Organ Function
Assessment
SIRS systemic inflammatory
response syndrome
SOFA Sequential [Sepsis-related]
OrganFunctionAssessment
Assessment of Clinical Criteria for Sepsis Original Investigation Research
jama.com (Reprinted) JAMA February23,2016 Volume 315, Number8 763
Copyright 2016 American Medical Association. All rig hts reserved.
wnloaded From: http://jama.jamanetwork.com/ by Jose Vergara on 02/29/2016
http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0288&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0288&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0288&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288
8/18/2019 Criterios Clinicos de Sepsis Seymour
3/13
Copyright 2016 American Medical Association. All rig hts reserved.
forcandidate criteriabecause organ dysfunction in sepsismay
occur prior to, near the moment of, or after infection is recog-
nized by clinicians or when a patient presents for care. More-
over, the clinical documentation, reporting of laboratory val-
ues in EHRs, and trajectory of organ dysfunction are
heterogeneous acrossencounters andhealth systems. In a post
hocanalysis requestedby thetask force,a change inSOFA score
was calculated of points or more from up to hours before
to up to hours after the onset of infection.
Deriving Novel Clinical Criteria for Sepsis
In the derivation cohort (UPMC),new, simple criteriawere de-
veloped accordingto theTransparent Reportingof a Multivari-able Prediction Model for Individual Prognosis or Diagnosis
(TRIPOD) recommendations.This entailed steps:() assess-
ingcandidatevariablequality andfrequencyof missingdataand
()developinga parsimoniousmodel andsimplepointscore.,,
Becauseof thesubjectivenature andcomplexityof variablesin
existing criteria, we soughta simple model that could easily be
used by a clinician at the bedside.
Based on the assumptionthat hospital mortality would be
farmorecommonin encounterswithinfectedpatients whohave
sepsis than in those who do not, all continuous variables were
dichotomized by defining their optimal cutoffsusing themini-
mum/ distanceon thearea under thereceiver operating char-
acteristic curve(AUROC)forin-hospitalmortality.
Cutoffswererounded to the nearest integer, and standard single-value im-
putation was used, withnormal value substitution if variables
were missing. The latter approach is standard in clinical risk
scores,, and mirrors how clinicians would use the score at
the bedside. Multiple logistic regression was used withrobust
standard errors and forward selection of candidate variables
using theBayesianinformation criterion to developthe “quick
SOFA” (qSOFA) model. The Bayesian information criterion is a
likelihood-based stepwise approach that retainsvariables that
improve the model’s overall ability to predict the outcome of
interest while incorporating a penalty for including too many
variables. Favoring simplicity over accuracy, a point score of
was assigned to each variable in the final model, irrespective
of the regression coefficients. Model calibration was assessed
by comparing clinically relevant differences in observed vs ex-
pected outcomes, as the Hosmer-Lemeshow test may be sig-
nificant due to large sample sizes.
Assessments of Candidate Clinical Criteria
Thetest:retestor interraterreliability ofindividualelementswas
not assessed, in part because most elements have known reli-
ability. However, the frequency of missing data was deter-
mined for each element because more common missing datafor individual elements will potentially affect the reliability of
integratedscoressuch asthe SOFA score. Constructvaliditywas
determined by examining the agreement between different
measures analogous to the multitrait-multimethod matrixap-
proach of Campbell and Fiske, using the Cronbach α to mea-
sureagreementor commonality.,Confidenceintervalswere
generated withthe bootstrap method ( replications).
Criterion validity was assessed using the predictive valid-
ity of the candidate criteriawith outcomes (primary outcome:
in-hospital mortality; secondary outcome: in-hospital mortal-
ityorintensivecareunit[ICU]lengthofstay≥days).Theseout-
comes are objective, easily measured across multiple hospi-
tals in US/non-US cohorts, and are more likely to be present inencounters with patients with sepsis than those with uncom-
plicatedinfection.To measurepredictivevalidity, a baselinerisk
model wascreated forin-hospital mortality based on preinfec-
tion criteria using multivariable logistic regression. The base-
linemodel included age(as a fractional polynomial),sex,race/
ethnicity ( black, white, or other), and the weighted Charlson
comorbidity score (as fractional polynomial) as a measure of
chronic comorbidities., Race/ethnicity was derived from
UPMC registration system data using fixed categories consis-
tent with the Centers for Medicare & Medicaid Services EHR
Table 1. Variables for CandidateSepsisCriteria Among Encounters WithSuspected Infection
SystemicInflammatoryResponse Syndrome(SIRS) Criteria(Range, 0-4 Criteria)
Sequential[Sepsis-related] Organ FailureAssessment (SOFA)(Range, 0-24 Points)
Logistic Organ DysfunctionSystem (LODS)(Range, 0-22 Points)a
Quick Sequential[Sepsis-related] Organ FailureAssessment (qSOFA)(Range, 0-3 Points)
Respiratory rate,breaths per minute
PaO2/FiO2 ratio PaO2/FiO2 ratio Respiratory r ate, b reathsper minute
White bloodcell
count, 109/L
Glasgow ComaScalescore Glasgow ComaScalescore Glasgow ComaScalescore
Bands,% Meanarterialpressure, mmHg Systolicbloodpressure, mm Hg Systolicbloodpressure,mmHg
Heartrate, beatsper minute
Administration of vasopressorswithtype/dose/rateof infusion
Heart rate, beatsper minute
Temperature, °C Serum creatinine, mg/dL,or urineoutput, mL/d
Serum creatinine,mg/dL
Arterial carbondioxide tension,mmHg
Bilirubin, m g/dL Bilirubin, m g/dL
Plateletcount, 109/L Platelet count, 109/L
White bloodcell count,109/L
Urine output, L/d
Serumurea, mmol/L
Prothrombin time,% of standard
Abbreviation:FiO2, fraction of
inspired oxygen.
a Measurement unitsfor LODS
variables per original descriptionby
LeGall etal.9
Research Original Investigation Assessment of Clinical Criteria for Sepsis
764 JAMA February23,2016 Volume315,Number 8 (Reprinted) jama.com
Copyright 2016 American Medical Association. All rig hts reserved.
wnloaded From: http://jama.jamanetwork.com/ by Jose Vergara on 02/29/2016
http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288
8/18/2019 Criterios Clinicos de Sepsis Seymour
4/13
Copyright 2016 American Medical Association. All rig hts reserved.
meaningful use data set. Race/ethnicity was included in the
baseline modelbecauseof itsdescribedassociation withthe in-
cidence and outcomes of sepsis.
Encounterswerethen dividedinto decilesof baseline risk.
Within each decile, the rate of in-hospital mortality ± ICUlength of stay of days or longer was determined comparing
encounters with infection with or more SIRS, SOFA, LODS,
andqSOFA points vs encounterswith lessthan criteriaof the
same score (threshold of points was determined a priori).
Model discrimination was assessed with the AUROC for each
outcome using the continuous score(s) alone, then added to
the baseline risk model. Analyses were separately performed
in ICUencountersand non-ICU encounters at theonsetof in-
fection. New, simple criteria in external data sets were as-
sessed in both ICU and non-ICU encounters.
Because serum lactate is widely used as a screening tool
in sepsis,howitsmeasurement would improvepredictiveva-
lidity of newcriteriawas assessedin posthoc analyses. Evalu-ation included qSOFA modelsthatdid and didnot include se-
rum lactate at thresholds of ., ., and . mmol/L (, ,
and mg/dL)and as a continuousvariable.OnlyKPNC data
were used for these analyses because an ongoing quality im-
provement program promoting frequent serum lactate mea-
surement acrossthe healthsystem minimized confoundingby
indication.
Several sensitivity analyses were performed to assess ro-
bustness of the findings. These included a variety of restric-
tions to thecohort, more rigorous definitions of suspected or
presumedinfection,alternative ways to measureclinicalvari-
ables (such as altered mentation in theEHR), andmultiple im-
putation analyses for missing data. There are many possible
time windowsfor criteria aroundthe onset of infection. A va-
riety of windows differing from the primary analysis weretested,including () hours before to hours after;() hours
before to hours after;and () restricting to only the hours
after the onset of infection. Detailed descriptions are in the
Supplement.
All analyses were performed with STATA software, ver-
sion . (Stata Corp). All tests of significance used a -sided
P ≤ .. We considered AUROCs to be poor at . to ., ad-
equate at . to ., good at . to ., andexcellent at . or
higher.
Results
Cohorts and Encounter Characteristics
At hospitals in US and non-US data sets between
and (Table), encounterswere studied.In the
primarycohort of records(UPMCderivationand vali-
dation; Figure ), encounters had suspected infec-
tion, most often presenting outside of the ICU (n =
[%]). Asshownin Table, first infection wascommonlysus-
pected within hours of admission (%), most often pre-
senting in the emergency department (%) compared with
theward (%) or ICU(%), andmortality waslow (%).The
Table 2. Summary of Data Sets
Characteristics UPMCa KPNC VA ALERTS KCEMS
Years of cohort 2010-2012 2009-2013 2008-2010 2011-2012 2009-2010
No. of hospitals 12 20 130 1 14
Total No. of encounters 1 309 025 1 847 165 1 640 543 38 098 50 727
Data sourceand study design
Retrospective studyof EHRs
Retrospective study ofEHRs
Retrospective studyof EHRs
Prospective cohortstudy
Retrospective studyof administrative records
Setting Integrated healthsystem in southwesternPennsylvania
Integrated healthsystem in northernCalifornia
All hospitals in the USVA system
Single universityhospital, Jena,Germany
Out-of-hospital recordsfrom integratedemergency medicalservices system in KingCounty, Washington
Definition of suspectedinfection
Combination of bodyfluid culture andnonprophylacticantibiotic administrationin the EHRb
Combination of bodyfluid culture andnonprophylacticantibiotic administrationin the EHRb
Combination of bodyfluid culture andnonprophylacticantibiotic administrationin the EHRb
CDC criteriafor hospital-acquiredinfectionsc
ICD-9-CM codesfor infection, withpresent-on-admissionindicatorsd
No. with suspectedinfection (% of total)
148 907 (11) 321 380 (17) 377 325 (23) 1186 (3) 6508 (13)
Location at onset ofinfection, No. (%) infected
Intensive care unit 15 768 (11) 7031 (2) 73 264 (19) 300 (25) 0
Outside of intensivecare unit
133 139 (89) 314 349 (98) 304 061 (81) 886 (75) 6508 (100)
In-hospital mortality,No. (%) infectede
6347 (4) 16 092 (5) 22 593 (6) 210 (18) 700 (11)
Abbreviations: KCEMS, King County EmergencyMedical Services; KPNC,Kaiser
Permanente Northern California;EHR, electronic health record; ICD-9-CM,
International Classificationof Diseases, NinthRevision,ClinicalModification;
VA,VeteransAdministration.
a Referredto as theprimary cohort,further dividedintoderivation (n = 74453)
and validation (n = 74454) cohorts.
b SeetheeAppendix in theSupplementfor detailsabouttimewindows
specified between bodyfluid cultures and antibioticadministration.
c Patientswere enrolled in ALERTSif thein-hospitalstaywas longerthan
48 hoursand in-person prospectivescreeningrevealedhospital-acquired
infection criteria according to Centers for Disease Control and Prevention
(CDC)guidelines.7
d Required Angusimplementation ICD-9-CM code for infection accompanied
by present-on-admissionindicator, as previously validated.6
e AmongUPMC encounters,28 286 (19%)had in-hospitalmortality plus
intensivecareunit lengthof stay of 3 days or longer.
Assessment of Clinical Criteria for Sepsis Original Investigation Research
jama.com (Reprinted) JAMA February23,2016 Volume 315, Number8 765
Copyright 2016 American Medical Association. All rig hts reserved.
wnloaded From: http://jama.jamanetwork.com/ by Jose Vergara on 02/29/2016
http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0288&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0288&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288
8/18/2019 Criterios Clinicos de Sepsis Seymour
5/13
Copyright 2016 American Medical Association. All rig hts reserved.
mediantimefrom thestart ofthe encounteruntil theonset of
suspected infection(defined as cultureor antibiotics order) was
. hours (interquartile range, .-. hours). In KPNC hos-
pitals(eTable in the Supplement), firstsuspected infections
occurred outside theICU (%)with similar mortality(%)and
proportion identifiedwithin hours of admission(%). Se-
rum lactate was measured in % of suspected infection en-
counters in KPNC hospitals compared with less than % in
the other cohorts. In VA hospitals, encounters with sus-
pected infectionhadsimilarmortality(%)but weremore likely
to be firstidentified in the ICU(%).A minority of firstinfec-
tion episodes occurred following surgery, and positive blood
cultures were found in %to % of encounters. In the base-line risk model, using only demographics and comorbidities,
there was a -fold variation for in-hospital mortality across
deciles of baseline risk, ranging from .% to %(eFigure in
the Supplement).
Frequency of Missing Data Among Clinical
and Laboratory Variables
In theUPMC derivationcohort, SIRS criteriaand selectedlabo-
ratory tests in SOFA andLODS were variably measured in the
EHR near the onset of infection (eFigure in the Supple-
ment). Tachycardia, tachypnea, and hypotension, although
present in less than % of encounters, were the most com-
monclinical abnormalities. Encounters in theICU were morelikely to have SIRS and SOFA variables measured and values
were morelikely tobe abnormal. For encountersoutsideof the
ICU, laboratory data were less available, with total bilirubin,
ratioofPaO to fractionof inspiredoxygen,and platelet counts
absent in %, %, and %of encounters, respectively.
Performance of Existing Criteria in the ICU
in the UPMC Cohort
Among ICUencounters withsuspected infectionin theUPMC
validation cohort (n = [%]), most had or more LODS
points (%), SOFA points (%), or SIRS criteria (%) near
thetime of suspected infection, withmortalityratesof %for
allscoresat thisthreshold (Figure andeFigureinthe Supple-
ment). SOFA and LODShad greater statistical agreement with
each other (α = .; % CI, .-.) but lower with SIRS
(α = . [% CI,.-.] for SOFA; α = .[% CI,.-
.]forLODS)(Figure).EncountersintheICUwithormore
vslessthan SIRS criteriawerecomparedwithindecileof base-
line risk and observed a - to -fold increased rate of hospital
mortality compared with a - to -fold increase in mortality
comparing those with or more vs less than SOFA points
(Figure).Thefoldchangein the LODS scorewasevengreater
than that for SOFA.In the ICU, the predictive validity for hospital mortality
using SOFA (AUROC = .; % CI, .-.) and LODS
(AUROC = .; % CI, .-.; P = .) were not statisti-
cally different but were statistically greater than that of SIRS
(AUROC = .; % CI, .-.; P < . for either LODS
or SOFA vs SIRS) (Figure and eFigure and eTable in the
Supplement).Resultsfor a change in SOFA of points or more
weresignificantlygreatercompared with SIRS(AUROC = .;
% CI, .-.; P < . vs SIRS criteria). The SOFA score
was or more in %of decedents (% CI,%-%); among
survivors,the SOFA score wasless than in %(%CI, %-
%). These proportions were similar for a LODS threshold of
or (eTable in the Supplement). Among decedents, ormore SIRS criteriawerepresent in % (% CI,%-%). Re-
sults were consistent for the combined outcome (eFigures
and inthe Supplement).
Performance of Existing Criteria Outside the ICU
in the UPMCCohort
For encounters with suspected infection outside of the ICU
(n = [% of cohort]), (%) had no SIRS crite-
ria, (%) had no SOFA points, and (%) had
no LODSpoints(Figure ).Agreementfollowed a patternsimi-
Figure 1. Accrual of Encounters for Primary Cohort
1309025 Patient encounters at 12 UPMChospitals in 2010-2012
148907 With suspected infection in ED,ICU, ward, step-down unit, orPACU included in primary cohort
1160118 Excluded
1109402 No infection present
2117 Error in encounter start time
45628 Aged
8/18/2019 Criterios Clinicos de Sepsis Seymour
6/13
8/18/2019 Criterios Clinicos de Sepsis Seymour
7/13
Copyright 2016 American Medical Association. All rig hts reserved.
Figure 2. Distribution of PatientEncounters Over SIRSCriteria and SOFA, LODS, andqSOFA Scores Among ICU Patientsand Non-ICU Patients
WithSuspected Infectionin theUPMC Validation Cohort (N = 74 454)
ICU encounters (n = 7932) Non-ICU encounters (n = 66 522)
50
40
30
20
10
0
E n c o u n t e r s ,
%
qSOFA Score
0 1 32
50
40
30
20
10
0
E n c o u n t e r s ,
%
qSOFA Score
0 1 32
qSOFA scoreD
50
40
30
20
10
0
E n c o u n t e r
s ,
%
LODS Score
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
50
40
30
20
10
0
E n c o u n t e r
s ,
%
LODS Score
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
LODS scoreC
50
40
30
20
10
0
E n c o u n t e r s , %
SOFA Score
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
50
40
30
20
10
0
E n c o u n t e r s , %
SOFA Score
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
SOFA scoreB
50
40
30
20
10
0
E n c o u n t e r s ,
%
SIRS Criteria
0 1 2 3 4
50
40
30
20
10
0
E n c o u n t e r s ,
%
SIRS Criteria
0 1 2 3 4
SIRS criteriaA
ICU indicates intensivecare unit;LODS,Logistic OrganDysfunction System;
qSOFA, quickSequential [Sepsis-related]Organ Function Assessment; SIRS,
systemic inflammatory response syndrome;SOFA, Sequential[Sepsis-related]
OrganFunctionAssessment. Thex-axis is the scorerange, withLODS truncated
at 14points(of 22points) andSOFA truncatedat 16points(of 24points) for
illustration.
Research Original Investigation Assessment of Clinical Criteria for Sepsis
768 JAMA February23,2016 Volume315,Number 8 (Reprinted) jama.com
Copyright 2016 American Medical Association. All rig hts reserved.
wnloaded From: http://jama.jamanetwork.com/ by Jose Vergara on 02/29/2016
http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288
8/18/2019 Criterios Clinicos de Sepsis Seymour
8/13
Copyright 2016 American Medical Association. All rig hts reserved.
The discrimination of hospital mortality using SOFA
(AUROC = .;% CI,.-.),LODS (AUROC = .; %
CI, .-.), or change in SOFA (AUROC = .;%CI, .-
.) scores was significantly greater comparedwith SIRScri-
teria(AUROC = .; %CI, .-.; P < . forall) (Figure
andeFigureandeTableinthe Supplement). Sixty-eight per-cent (% CI, %-%) of decedents had or more SOFA
points and %(%CI, %-%)of survivorshad less than
SOFA points. In comparison, only % (% CI, %-%)
of decedents had or more SIRS criteria,whereas % of sur-
vivorshad less than SIRS criteria (% CI,%-%) (eTable
in the Supplement). Results were consistent for the com-
bined outcome (eFigures and in the Supplement).
Performance of New, Simple Criteria
The final qSOFA model included Glasgow Coma Scale (GCS)
scoreoforless,systolicbloodpressureofmmHgorless,
and respiratory rate of /min or more ( point each; score
range, -) (Table ). Most encounters with infection (%-%) hadless than qSOFA points,and mortality rangedfrom
%to % over the scorerange (eFigure in the Supplement).
Calibration plots showed similar observed vs expected pro-
portionof deathsacross qSOFA scores(eFigure in theSupple-
ment). The qSOFA agreed reasonably well with both SOFA
(α = .;% CI,.-.) andLODS (α = .;% CI,.-
.) and, unlike SOFA andLODS, also agreedmore with SIRS
(α = .; % CI, .-.) (Figure ). The % of encoun-
terswithinfection with or qSOFA points accountedfor %
of deaths, % of deaths or ICU stays of days or longer.
In the ICU, the predictive validity for hospital mortality of
qSOFA above baseline risk (AUROC = .; % CI, .-
.) was statistically greater than SIRS criteria ( P = .) but
significantly less than SOFA ( P < .) (Figure and eFigure
and eTable in the Supplement). Outside of the ICU, there
was a - to -fold increase in the rate of hospital mortalityacross the entire range of baseline risk comparing those with
or more vs less than qSOFA points (Figure ). The predic-
tive validity of qSOFA was good for in-hospital mortality
(AUROC = .; % CI, .-.), was not statistically dif-
ferent from LODS ( P = .) and was statistically greater than
SOFA or change in SOFA score ( P < . for both) (Figure ,
Figure, and eFigure and eTable inthe Supplement). Sev-
enty percent (% CI, %-%) of decedents had or more
qSOFA points and % (% CI, %-%) of survivors had
less than qSOFA points (eTable in the Supplement).
Results were consistent for the combined outcome (eFigures
and in the Supplement).
Among encounterswith or moreqSOFA points,% alsohad ormoreSOFA points (eFigure inthe Supplement). This
proportion was greater among decedents (%) and ICU en-
counters (%) and increased as the time window for evalu-
ationwas extendedto hours (%) and hours (%) af-
ter the onset of infection.
External Data Sets
The qSOFA was tested in external data sets comprising
patient encounters at hospitals in out-of-
hospital(n = ),non-ICU(n = ), andICU (n = )
Figure 3. AreaUnder theReceiver OperatingCharacteristicCurve and 95%Confidence Intervals forIn-Hospital Mortalityof CandidateCriteria
(SIRS, SOFA, LODS, and qSOFA) Among SuspectedInfection Encounters in theUPMC Validation Cohort (N = 74 454)
ICU encounters (n = 7932)A
SIRS 0.64
(0.62-0.66)
0.43
(0.41-0.46)
0.41
(0.38-0.43)
0.46
(0.43-0.48)
SOFA
8/18/2019 Criterios Clinicos de Sepsis Seymour
9/13
Copyright 2016 American Medical Association. All rig hts reserved.
settings (eTable in theSupplement). Among encounterswith
communityinfection(KCEMS) or hospital-acquiredinfection
(ALERTS), qSOFA had consistent predictive validity
(AUROC = . and ., respectively) (Table and eFigure
in the Supplement). Results were similar in the VA data set
(AUROC = .), in which no GCS data were available.
Serum LactateDuring model building in UPMC data, serum lactate did not
meet prespecifiedstatistical thresholdsfor inclusionin qSOFA.
In KPNC data,the post hocaddition of serum lactate levelsof
. mmol/L( mg/dL)or more toqSOFA (revised toa -point
score with added point forelevatedserumlactate level)sta-
tisticallychanged thepredictive validityof qSOFA (AUROCwith
lactate = .; % CI, .-. vs AUROC without lac-
tate = .; % CI, .-.; P < .) (eFigure A in the
Supplement).AsshownineTableintheSupplement,thiswas
consistent for higher thresholds of lactate (. mmol/L
[mg/dL], .mmol/L[ mg/dL])or using a continuousdis-
tribution ( P < .).However,the clinicalrelevance wassmall
as theratesof in-hospital mortalitycomparingencounters with
or more vs lessthan points across deciles of risk were nu-
merically similar whether or not serum lactate was included
in qSOFA (eFigure B in the Supplement).
Among encounters with qSOFA point but also a serum
lactate level of .mmol/L or more,in-hospitalmortality washigher than that for encounters with serum lactate levels of
lessthan. mmol/L acrossthe range of baseline risk. Therate
of in-hospitalmortality wasnumerically similar to that foren-
counters with qSOFA points using the model without se-
rumlactate(eFigureinthe Supplement). Becauseserumlac-
tate levels arewidely used for screening at many centers, the
distribution of qSOFA scores over strata of serumlactatelevel
was investigated. The qSOFA consistently identified higher-
risk encounters even at varying serum lactate levels (eFigure
in the Supplement).
Figure 4. FoldChangein Rateof In-Hospital Mortality(Log Scale) Comparing Encounters With≥2 vs
8/18/2019 Criterios Clinicos de Sepsis Seymour
10/13
Copyright 2016 American Medical Association. All rig hts reserved.
Time Windows for Measuring qSOFA Variables
When qSOFA variables were measured in the time window
from hours before/after or hours before/after the onset of
infection in KPNC data (eTable in the Supplement), results
were notsignificantly differentfrom theoriginalmodel( P
= .for hours and P = . for hours). When qSOFA variables
were restricted to only the -hour period after the onset of
infection,the predictive validityfor in-hospital mortalitywas
significantly greater (AUROC = .; % CI, .-.;
P < .) compared with the primary model.
Additionalsensitivityanalyses areshown ineTable in the
Supplement. The predictive validityof qSOFA wasnot signifi-
cantly differentwhen using moresimple measures,such as any
altered mentation (GCSscore
8/18/2019 Criterios Clinicos de Sepsis Seymour
11/13
Copyright 2016 American Medical Association. All rig hts reserved.
tion, lowsystolic blood pressure, andelevated respiratoryrate
hadstatisticallygreater predictivevalidity thanthe SOFA score
(Figure ).The predictive validityof qSOFA wasrobust toevalu-
ation under varied measurementconditions,in academic and
community hospitals, in international locations of care, for
community and hospital-acquired infections, and after mul-
tiple imputation for missing data. It was, however, statisti-
cally inferior compared with SOFA for encounters in the ICU
and has a statistically lower content validity as a measure of
multiorgan dysfunction. Thus, the task force recommended
use of a SOFA score of pointsor morein encounterswithin-
fection as criteria for sepsisand useof qSOFA in non-ICU set-
tings to consider the possibility of sepsis.
Criteria Outside of the ICU
For infected patients outside of the ICU, there is an increasing
focus on early recognitionof sepsis. Potentialcriteria fororgan
dysfunctionlike SOFA or LODSrequired clinical andlaboratory
variables that maybe missing anddifficultto obtain in a timely
manner.These characteristics mayincreasemeasurement bur-
denfor clinicians.In comparison, a simplemodel (qSOFA)uses
clinical variables, hasno laboratorytests,and hasa predictive
validity outside of the ICUthat is statistically greater than the
SOFA score ( P < .). TheqSOFA andSOFA scoresalso hadac-
ceptable agreement in the majority of encounters.
However, potentially controversial issues are worth not-
ing.First, qSOFA wasderived andtestedamongpatient encoun-
tersin whichinfection wasalreadysuspected. TheqSOFA is not
analertthatalone willdifferentiate patients withinfection from
those without infection. However, at least in many US and
European hospital settings, infection is usually suspected
promptly, as evidenced by rapid initiation of antibiotics.,
Second,mentalstatus is assessed variablyin different set-
tings, which may affect the performance of the qSOFA. Al-
though the qSOFA appeared robust in sensitivity analyses toalternative GCS cut points, further work is needed to clarify
itsclinical usefulness. In particular,the modelevaluated only
whether mental status was abnormal, not whether it had
changed from baseline, which is extremely difficult to opera-
tionalize and validate, both in the EHR and as part of routine
charting.An alternative to theGCS (eg, Laboratory andAcute
PhysiologyScore, version, in KPNCencounters)foundsimi-
lar results.
Third, serumlactate levels, which have been proposed as
a screening tool for sepsis or septic shock, were not retained
in the qSOFA during model construction. One reason may be
because serum lactatelevels were not measuredcommonly in
the UPMC data set. When serum lactate levels were added toqSOFA post hocin theKPNC healthsystem data set, in which
measurement of lactatelevels wascommon, thepredictive va-
lidity wasstatistically increasedbut withlittledifference inhow
encounters were classified. This analysis assessed only how
serumlactate levels at differentthresholdscontributed above
and beyondthe qSOFA model. However,among intermediate-
riskencounters(qSOFA score = ),the additionof a serum lac-
tate level of . mmol/L( mg/dL)or higheridentified those
with a risk profile similar to those with qSOFA points. Thus,
areas for further inquiry include whether serum lactate lev-
els could be used for patients with borderline qSOFA values
or as a substitute for individual qSOFA variables (particularly
mental status,given theinherentproblemsdiscussed above),
especially in health systems in which lactate levels are reli-
ably measured at low cost and in a timely manner.
Criteria in the ICU
Among ICU encounters, the diagnosis of sepsis may be chal-
lenging because of preexisting organ dysfunction, treatment
priorto admission,and concurrent organ support.In thisstudy,
as others have reported in a distinct geographic region and
health care system, traditional tools such as the SIRS crite-
ria have poor predictive validity among patients who are in-
fected. Yet in our study, SOFA and LODS scores had superior
predictive validity in the ICU and greater agreement, perhaps
because more variables were likely to be measured, abnor-
mal, and independent of ongoing interventions. These re-
sults areconsistent withpriorstudiesof SOFA andLODS in the
ICU., On average, only of infected decedents in the
ICU had a SOFA orLODSscoreof lessthan .TheqSOFA score
had statisticallyworse predictive validity in the ICU, likely re-
lated to theconfoundingeffectsof ongoingorgansupport(eg,
mechanical ventilation, vasopressors).
Advances Using EHRs
The data from these analyses provided the Third Interna-
tional Consensus Task Force with evidence aboutclinical cri-
teria for sepsis using EHRs from large health systems with
both academic and community hospitals. More than % of
US nonfederal, acute care hospitals(and all US federal hospi-
tals) now use advanced EHRs. Adoption of EHRs has in-
creased -fold since in the United States and will con-
tinue to increase. The EHR may present hospitals with an
opportunity to rapidly validate criteria for patients likely to
have sepsis, to testprompts or alerts among infected patientswithspecificEHR signaturessuggestive of sepsis,and to build
platforms for automated surveillance. In addition, criteria
such as in theqSOFA canbe measured quickly and easily and
assessed repeatedly over timein patientsat riskof sepsis,per-
haps even in developing countries without EHRs.
Limitations
Thisinvestigationhas severallimitations.First,we studiedonly
patients in whom infection was already suspected or docu-
mented. We did not addresshow to diagnose infectionamong
those in whomlife-threateningorgandysfunctionwas theini-
tialpresentation. Therefore, these dataalone do not mandate
thathospitalized patients with SOFA or qSOFA pointsbe evalu-ated for the presence of infection.
Second, we chose to develop simple criteria that clini-
cians could quickly use at the bedside, balancing timeliness
and content validity with greater criterion validity. We ac-
knowledge that predictive validity would be improved with
more complex models thatinclude interaction terms or serial
measurements over time.,, We tested how the change in
SOFA score over time would perform, andalthough similar to
the maximum SOFA score, the optimal time windows over
which change should be measured are not known.
Research Original Investigation Assessment of Clinical Criteria for Sepsis
772 JAMA February23,2016 Volume315,Number 8 (Reprinted) jama.com
Copyright 2016 American Medical Association. All rig hts reserved.
wnloaded From: http://jama.jamanetwork.com/ by Jose Vergara on 02/29/2016
http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288
8/18/2019 Criterios Clinicos de Sepsis Seymour
12/13
Copyright 2016 American Medical Association. All rig hts reserved.
Third, no organ dysfunction measurements evaluated in
this study distinguish between chronic and acute organ dys-
function, assess whether the organ dysfunction has an expla-
nation other than infection, or attribute dysfunction specifi-
cally to a dysregulated host response. For example, a patient
with dementiawithan abnormal GCS scoreat baselinewill al-
ways have qSOFA pointbut may notbe aslikely to have sep-
sisas a patientwith a normalbaseline sensorium. Assuch, we
illustrated the predictive validity of various criteria across a
fullrange of underlyingriskdeterminedfrom comorbidity and
demographics.
Fourth, we chose outcomes associated morecommonly
with sepsis than with uncomplicated infection. These out-
comeshavehigh contentvalidityand weregeneralizableacross
data sets, but there are certainly alternative choices.
Fifth, we compared predictive validity with tests of infer-
ence that may be sensitive to sample size. We found that sta-
tistically significant differencesin AUROC wereoften present,
yet these resulted in differences in classification with debat-
able clinical relevance. We reconciledthese databy reporting
the fold change in outcome comparing encounters of differ-
ent scores to provide more clinical context.
Sixth,the acute,life-threatening organdysfunctionin sep-
sis may also occur at different times in different patients
(before, during, or afterinfectionis recognized).Results were
unchanged overa variety of timewindows, includingboth long
(-hour)and short(-hour) windows around the onsetof in-
fection. Prospective validation in other cohorts, assessment
in low- to middle-income countries, repeated measurement,
and the contribution of individual qSOFA elements to predic-
tive validity are important future directions.
Conclusions
Among ICU encounters with suspected infection, the predic-
tive validityfor in-hospital mortalityof SOFA wasnot signifi-
cantly different than the more complex LODS but was statis-
tically greater than SIRS and qSOFA, supporting its use in
clinical criteria for sepsis. Among encounters with suspected
infection outside of the ICU, the predictive validity for in-
hospital mortality of qSOFA was statisticallygreaterthanSOFA
and SIRS, supporting its use as a prompt to consider possible
sepsis.
ARTICLE INFORMATION
Author Affiliations: Departmentof Critical Care
Medicine,Universityof PittsburghSchool of
Medicine, Pittsburgh, Pennsylvania (Seymour,
Kahn,Angus); Clinical Research,Investigation, and
Systems Modeling of AcuteIllness (CRISMA) Center,
Pittsburgh,Pennsylvania(Seymour, Kahn, Angus);
Divisionof Research,Kaiser Permanente, Oakland,
California (Liu);Department of Internal Medicine,
Universityof Michigan, Ann Arbor (Iwashyna,
Escobar); VeteransAffairs Center for Clinical
Management Research,Ann Arbor, Michigan
(Iwashyna,Escobar); Australia and New Zealand
IntensiveCare Research Centre, Departmentof Epidemiologyand PreventiveMedicine, Monash
University, Melbourne,Victoria, Australia
(Iwashyna,Escobar); Center for Clinical Studies,
JenaUniversityHospital,Jena, Germany
(Brunkhorst); Division of General Internal Medicine,
Universityof Washington, Seattle (Rea); Research
Group Clinical Epidemiology, IntegratedResearch
and Treatment Center,Center for SepsisControl
and Care,Jena UniversityHospital, Jena,Germany
(Scherag);Trauma,Emergency, and Critical Care
Program, Sunnybrook Health Sciences Centre;
InterdepartmentalDivision of Critical Care,
Universityof Toronto, Toronto,Ontario,Canada
(Rubenfeld); Critical Care Medicine,Guy’s and St
Thomas’ NHS FoundationTrust,London, England
(Shankar-Hari);BloomsburyInstituteof Intensive
Care Medicine,UniversityCollege London, London,England (Singer); FeinsteinInstitute for Medical
Research,Hofstra–North Shore–LongIsland Jewish
Schoolof Medicine, Steven and Alexandra Cohen
Children’s Medical Center,New HydePark,
New York (Deutschman).
Author Contributions: DrSeymour hadfull access
toall ofthedata inthestudyandtakes
responsibility forthe integrityof thedataand the
accuracy of thedataanalysis.
Study concept and design: Seymour,Iwashyna,
Rubenfeld,Kahn, Shankar-Hari, Deutschman,
Escobar,Angus.
Acquisition, analysis, or interpretation of data: Liu,
Iwashyna, Brunkhorst,Rea, Scherag, Kahn,Singer,
Escobar, Angus.
Drafting of themanuscript:Seymour, Singer,
Deutschman, Angus.
Critical revision of themanuscriptfor important
intellectual content: Liu, Iwashyna,Brunkhorst,Rea,
Scherag, Rubenfeld,Kahn, Shankar-Hari, Singer,
Deutschman, Escobar,Angus.
Statistical analysis: Seymour, Liu, Iwashyna,
Scherag.
Obtained funding: Escobar.
Administrative, technical, or material support:
Brunkhorst,Rea, Scherag, Deutschman, Escobar,Angus.
Study supervision: Deutschman, Escobar.
Conflict of Interest Disclosures: All authors have
completedand submittedtheICMJEFormfor
Disclosure of PotentialConflicts of Interest.Dr
Seymourreportsreceipt of personal fees from
Beckman Coulter.Dr Singer reports board
membershipswith InflaRx, Bayer, Biotest, and
Merck. Dr Deutschmanreports holding patents on
materials unrelatedto this work andreceiptof
personal fees fromtheCenters forDiseaseControl
and Prevention, theWorld Federationof Societies
of Intensive and Critical Care,the Pennsylvania
Assembly of Critical CareMedicine,the Society of
Critical Care Medicine,the Northern Ireland Society
of Critical Care Medicine,the International Sepsis
Forum, Stanford University, theAcute DialysisQuality Initiative,and the European Society of
Intensive Care Medicine.Dr Escobar reports receipt
of grantsfrom theNational Institutes of Health, the
Gordonand BettyMoore Foundation,Merck,
Sharpe& Dohme, and AstraZeneca-MedImmune.
No otherdisclosureswere reported.
Funding/Support: This work wassupported
in partby theNational Institutes of Health
(grants K23GM104022 and K23GM112018),
theDepartmentof VeteransAffairs (grantHSR&D
11-109),the Permanente Medical Group,
andthe Centerof SepsisControl andCare,funded
bythe GermanFederalMinistryof Educationand
Research(grant01 E01002/01E0 1502).
Roleof the Funder/Sponsor:Thefunding sources
hadno rolein thedesign andconduct of thestudy;
collection, management, analysis, and
interpretation of thedata; preparation,review, or
approval of themanuscript; anddecisionto submit
the manuscriptfor publication.
Disclaimer: Thisarticle does not necessarily
representthe view of theUS government or
Departmentof VeteransAffairs. Dr Angus,
Associate Editor, JAMA, had noroleinthe
evaluationof or decision to publishthisarticle.
Additional Contributions: We acknowledge the
European Society of IntensiveCare Medicine and
Society of Critical CareMedicine fortheirpartial
administrative supportof thiswork. We
acknowledgethe contributions of the 2016Third
International Consensus SepsisDefinitions Task
Forcemembers,who were notcoauthors, fortheir
reviewof themanuscript: John C.Marshall,MD,
Universityof Toronto, Toronto,Ontario,Canada;
DjilalliAnnane,MD, PhD, Critical Care Medicine,
Schoolof Medicine, Universityof Versailles,France;
Greg S.Martin, MD, Emory University Schoolof
Medicine,Atlanta, Georgia; Michael Bauer, MD,
Center for SepsisControl and Care,University
Hospital, Jena, Germany; Steven M. Opal,MD,
Infectious Disease Section, BrownUniversitySchool
of Medicine,Providence, RhodeIsland; RinaldoBellomo, MD,Australianand NewZealand Intensive
CareResearch Centre, Schoolof PublicHealth and
PreventiveMedicine, MonashUniversity,University
of Melbourne,and AustinHospital,Melbourne,
Victoria, Australia; GordonR. Bernard, MD,
VanderbiltInstitute for Clinical and Translational
Research,Vanderbilt University, Nashville,
Tennessee;Jean-Daniel Chiche, MD,PhD,
RéanimationMédicale-Hôpital Cochin, Descartes
University, CochinInstitute, Paris, France; CraigM.
Coopersmith,MD, Emory Critical Care Center,
Emory UniversitySchool of Medicine, Atlanta,
Assessment of Clinical Criteria for Sepsis Original Investigation Research
jama.com (Reprinted) JAMA February23,2016 Volume 315, Number8 773
Copyright 2016 American Medical Association. All rig hts reserved.
wnloaded From: http://jama.jamanetwork.com/ by Jose Vergara on 02/29/2016
http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288
8/18/2019 Criterios Clinicos de Sepsis Seymour
13/13
Georgia; Tomvan derPoll,MD,Academisch
Medisch Centrum, Amsterdam,the Netherlands;
Richard S. Hotchkiss,MD, WashingtonUniversity
Schoolof Medicine,St Louis,Missouri;Jean-Louis
Vincent, MD,PhD, UniversitéLibre de Bruxelles,
and Departmentof Intensive Care,Erasme
UniversityHospital, Brussels, Belgium;
andMitchellM. Levy, MD, Division of
Pulmonary and Critical Care Medicine,
BrownUniversitySchool of Medicine,Providence,
RhodeIsland. Thesecontributions wereprovided
without compensation.
REFERENCES
1. Liu V, EscobarGJ,Greene JD, etal. Hospital
deathsin patientswith sepsisfrom 2 independent
cohorts. JAMA. 2014;312(1):90-92.
2. Angus DC, Linde-Zwirble WT, Lidicker J,
Clermont G, CarcilloJ, PinskyMR. Epidemiology of
severesepsis in theUnitedStates:analysis of
incidence,outcome,and associatedcosts of care.
Crit Care Med . 2001;29(7):1303-1310.
3. Levy MM,FinkMP, Marshall JC,et al;
SCCM/ESICM/ACCP/ATS/SIS. 2001
SCCM/ESICM/ACCP/ATS/SIS International Sepsis
Definitions Conference. Crit Care Med . 2003;31(4):
1250-1256.
4. BoneRC, Balk RA,CerraFB, et al;ACCP/SCCM
Consensus ConferenceCommittee; American
Collegeof ChestPhysicians/Society of Critical Care
Medicine.Definitionsfor sepsisand organfailure
andguidelinesfor theuse of innovative therapiesin
sepsis. Chest . 1992;101(6):1644-1655.
5. Shankar-HariM, Phillips GS,LevyML, et al;
SepsisDefinitions Task Force. Developinga new
definition and assessing new clinical criteria for
septicshock: for theThird International Consensus
Definitionsfor Sepsis and Septic Shock(Sepsis-3).
JAMA. doi:10.1001/jama.2016.0289 .
6. Seymour CW, Rea TD, Kahn JM,WalkeyAJ, Yealy
DM,Angus DC.Severe sepsisin pre-hospital
emergencycare: analysis of incidence,care, and
outcome. Am J Respir Crit Care Med . 2012;186(12):
1264-1271.
7. HagelS, Ludewig K, FrosinskiJ, etal.
Effectiveness of a hospital-wide educational
programmefor infectioncontrol to reducethe rate
of health-care associated infections and related
sepsis(ALERTS)—methods and interim results
[in German]. DtschMedWochenschr . 2013;
138(34-35):1717-1722.
8. Vincent JL,Moreno R, Takala J,et al.The SOFA
(Sepsis-RelatedOrgan Failure Assessment) scoreto
describe organdysfunction/failure. IntensiveCare
Med . 1996;22(7):707-710.
9. LeGall JR,Klar J,Lemeshow S,et al;ICU Scoring
Group.The Logistic OrganDysfunction System:
a newwayto assessorgandysfunctionin the
intensivecare unit. JAMA. 1996;276(10):802-810.
10. CollinsGS, Reitsma JB,Altman DG,Moons KG.
Transparent Reporting of a MultivariablePrediction
Modelfor Individual Prognosis or Diagnosis
(TRIPOD): theTRIPOD statement. Ann InternMed .
2015;162(1):55-63.
11. Rangel-FraustoMS, Pittet D,Costigan M, Hwang
T, DavisCS, Wenzel RP. Thenaturalhistory of the
systemicinflammatoryresponse syndrome (SIRS):
a prospectivestudy. JAMA. 1995;273(2):117-123.
12. PerkinsNJ, Schisterman EF.The inconsistency
of “optimal”cutpointsobtained usingtwo criteria
basedon thereceiver operating characteristic
curve. Am J Epidemiol . 2006;163(7):670-675.
13. Seymour CW, Kahn JM,CookeCR, Watkins TR,
Heckbert SR,Rea TD. Predictionof critical illness
duringout-of-hospital emergencycare. JAMA.
2010;304(7):747-754.
14. KnausWA, ZimmermanJE, Wagner DP, Draper
EA, Lawrence DE. APACHE—AcutePhysiologyand
Chronic HealthEvaluation:a physiologically based
classification system. Crit Care Med . 1981;9(8):591-
597.
15. KramerAA, ZimmermanJE. Assessingthe
calibrationof mortality benchmarksin critical care:
theHosmer-Lemeshow testrevisited. Crit Care Med .
2007;35(9):2052-2056 .
16. Bland JM,Altman DG.Cronbach’sα. BMJ . 1997;
314(7080):572.
17. CampbellDT,Fiske DW. Convergent
and discriminantvalidationby the
multitrait-multimethod matrix. Psychol Bull .
1959;56(2):81-105.
18. Quan H,SundararajanV,Halfon P, etal. Coding
algorithms for defining comorbiditiesin ICD-9-CM
and ICD-10 administrative data. MedCare . 2005;43
(11):1130-1139.
19. Royston P, Sauerbrei W.Building multivariable
regression modelswith continuouscovariatesin
clinical epidemiology—withan emphasis on
fractional polynomials.Methods InfMed . 2005;44
(4):561-571.
20. Centers for Medicaid and Medicare Services.
Eligible Professional MeaningfulUse CoreMeasures:
Measure7 of13. 2014. https://www.cms.gov
/Regulations-and-Guidance/Legislation
/EHRIncentivePrograms/downloads/7_Record
_Demographics.pdf.Accessed January 1, 2016.
21. Barnato AE, Alexander SL, Linde-Zwirble WT,Angus DC.Racial variationin theincidence,care,
andoutcomesof severesepsis: analysis of
population, patient, and hospital characteristics.
Am J Respir Crit Care Med . 2008;177(3):279-284.
22. Dellinger RP, Levy MM,Rhodes A, etal;
Surviving SepsisCampaignGuidelinesCommittee
Including thePediatricSubgroup. Surviving Sepsis
campaign:international guidelinesfor management
ofseveresepsis andseptic shock:2012. Crit CareMed .
2013;41(2):580-637.
23. CecconiM, DeBacker D,Antonelli M,et al.
Consensus on circulatory shockand hemodynamic
monitoring: task force of theEuropeanSocietyof
IntensiveCare Medicine. IntensiveCare Med . 2014;
40(12):1795-1815.
24. LiuV,MorehouseJW, Soule J,Whippy A,Escobar GJ.Fluid volume, lactate values, and
mortality in sepsispatientswith intermediate
lactate values. Ann Am Thorac Soc . 2013;10(5):466-
473.
25. HanleyJA, McNeilBJ. The meaning anduse of
thearea undera receiver operating characteristic
(ROC)curve. Radiology . 1982;143(1):29-36.
26. FerrerR, Martin-Loeches I,Phillips G,et al.
Empiricantibiotictreatment reduces mortality in
severesepsis andseptic shockfrom thefirsthour:
results from a guideline-basedperformance
improvement program. Crit Care Med . 2014;42(8):
1749-1755.
27. Houck PM,Bratzler DW, NsaW,Ma A,
Bartlett JG. Timing of antibioticadministration and
outcomes for Medicare patients hospitalizedwith
community-acquired pneumonia. Arch InternMed .2004;164(6):637-644.
28. EscobarGJ,Gardner MN, GreeneJD,Draper D,
KipnisP. Risk-adjustinghospital mortality usinga
comprehensive electronicrecord in an integrated
health caredelivery system. MedCare . 2013;51(5):
446-453.
29. KaukonenKM, BaileyM, Pilcher D,Cooper DJ,
Bellomo R. Systemic inflammatoryresponse
syndrome criteria in defining severe sepsis. NEnglJ
Med . 2015;372(17):1629-1638.
30. VincentJL, deMendonçaA, CantraineF, etal.
Useof theSOFA score to assessthe incidenceof
organdysfunction/failure in intensivecare units:
results of a multicenter,prospectivestudy. CritCare
Med . 1998;26(11):1793-1800.
31. Ferreira FL,Bota DP, Bross A, Mélot C,Vincent JL.Serial evaluationof theSOFA score to
predictoutcomein critically ill patients. JAMA.
2001;286(14):1754-1758.
32. Charles D,Gabriel M,FurukawaMF. Adoption of
ElectronicHealthRecord SystemsamongUS
Non-Federal AcuteCare Hospitals: 2008-2013.
Washington, DC: Officeof the National Coordinator
for Health InformationTechnology;2014.ONC Data
Brief 16.
33. Rhee C, MurphyMV, Li L, PlattR, Klompas M;
Centers for Disease Controland Prevention
EpicentersProgram. Comparison of trends in sepsis
incidenceand codingusing administrative claimsvs
objective clinical data.Clin Infect Dis. 2015;60(1):
88-95.
34. ShapiroNI, Wolfe RE,MooreRB, SmithE,BurdickE, BatesDW.Mortalityin Emergency
DepartmentSepsis (MEDS) score:a prospectively
derived and validated clinical prediction rule.Crit
Care Med . 2003;31(3):670-675.
35. OpalSM. Concept of PIRO as a newconceptual
framework to understandsepsis. Pediatr CritCare
Med . 2005;6(3)(suppl):S55-S60.
36. Iwashyna TJ,Odden A, Rohde J,et al.
Identifyingpatientswith severe sepsisusing
administrative claims:patient-level validation of the
Angusimplementationof theInternational
Consensus Conferencedefinitionof severe sepsis.
MedCare . 2014;52(6):e39-e43.
37. Kellum JA,KongL, Fink MP, etal; GenIMS
Investigators. Understanding theinflammatory
cytokine response in pneumonia and sepsis: resultsof theGeneticand InflammatoryMarkers of Sepsis
(GenIMS) study. Arch InternMed . 2007;167(15):
1655-1663.
Research Original Investigation Assessment of Clinical Criteria for Sepsis
774 JAMA February23,2016 Volume315,Number 8 (Reprinted) jama.com
http://www.ncbi.nlm.nih.gov/pubmed/24838355http://www.ncbi.nlm.nih.gov/pubmed/24838355http://www.ncbi.nlm.nih.gov/pubmed/24838355http://www.ncbi.nlm.nih.gov/pubmed/11445675http://www.ncbi.nlm.nih.gov/pubmed/11445675http://www.ncbi.nlm.nih.gov/pubmed/11445675http://www.ncbi.nlm.nih.gov/pubmed/12682500http://www.ncbi.nlm.nih.gov/pubmed/12682500http://www.ncbi.nlm.nih.gov/pubmed/12682500http://www.ncbi.nlm.nih.gov/pubmed/12682500http://www.ncbi.nlm.nih.gov/pubmed/1303622http://www.ncbi.nlm.nih.gov/pubmed/1303622http://www.ncbi.nlm.nih.gov/pubmed/1303622http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0289&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0289&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0289&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.ncbi.nlm.nih.gov/pubmed/23087028http://www.ncbi.nlm.nih.gov/pubmed/23087028http://www.ncbi.nlm.nih.gov/pubmed/23087028http://www.ncbi.nlm.nih.gov/pubmed/23087028http://www.ncbi.nlm.nih.gov/pubmed/23934590http://www.ncbi.nlm.nih.gov/pubmed/23934590http://www.ncbi.nlm.nih.gov/pubmed/23934590http://www.ncbi.nlm.nih.gov/pubmed/23934590http://www.ncbi.nlm.nih.gov/pubmed/8844239http://www.ncbi.nlm.nih.gov/pubmed/8844239http://www.ncbi.nlm.nih.gov/pubmed/8844239http://www.ncbi.nlm.nih.gov/pubmed/8844239http://www.ncbi.nlm.nih.gov/pubmed/8769590http://www.ncbi.nlm.nih.gov/pubmed/8769590http://www.ncbi.nlm.nih.gov/pubmed/8769590http://www.ncbi.nlm.nih.gov/pubmed/25560714http://www.ncbi.nlm.nih.gov/pubmed/25560714http://www.ncbi.nlm.nih.gov/pubmed/25560714http://www.ncbi.nlm.nih.gov/pubmed/25560714http://www.ncbi.nlm.nih.gov/pubmed/7799491http://www.ncbi.nlm.nih.gov/pubmed/7799491http://www.ncbi.nlm.nih.gov/pubmed/7799491http://www.ncbi.nlm.nih.gov/pubmed/16410346http://www.ncbi.nlm.nih.gov/pubmed/16410346http://www.ncbi.nlm.nih.gov/pubmed/16410346http://www.ncbi.nlm.nih.gov/pubmed/20716737http://www.ncbi.nlm.nih.gov/pubmed/20716737http://www.ncbi.nlm.nih.gov/pubmed/20716737http://www.ncbi.nlm.nih.gov/pubmed/20716737http://www.ncbi.nlm.nih.gov/pubmed/7261642http://www.ncbi.nlm.nih.gov/pubmed/7261642http://www.ncbi.nlm.nih.gov/pubmed/7261642http://www.ncbi.nlm.nih.gov/pubmed/7261642http://www.ncbi.nlm.nih.gov/pubmed/17568333http://www.ncbi.nlm.nih.gov/pubmed/17568333http://www.ncbi.nlm.nih.gov/pubmed/17568333http://www.ncbi.nlm.nih.gov/pubmed/17568333http://www.ncbi.nlm.nih.gov/pubmed/9055718http://www.ncbi.nlm.nih.gov/pubmed/9055718http://www.ncbi.nlm.nih.gov/pubmed/9055718http://www.ncbi.nlm.nih.gov/pubmed/9055718http://www.ncbi.nlm.nih.gov/pubmed/13634291http://www.ncbi.nlm.nih.gov/pubmed/13634291http://www.ncbi.nlm.nih.gov/pubmed/13634291http://www.ncbi.nlm.nih.gov/pubmed/13634291http://www.ncbi.nlm.nih.gov/pubmed/16224307http://www.ncbi.nlm.nih.gov/pubmed/16224307http://www.ncbi.nlm.nih.gov/pubmed/16224307http://www.ncbi.nlm.nih.gov/pubmed/16224307http://www.ncbi.nlm.nih.gov/pubmed/16342923http://www.ncbi.nlm.nih.gov/pubmed/16342923http://www.ncbi.nlm.nih.gov/pubmed/16342923http://www.ncbi.nlm.nih.gov/pubmed/16342923https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/7_Record_Demographics.pdfhttps://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/7_Record_Demographics.pdfhttps://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/7_Record_Demographics.pdfhttps://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/7_Record_Demographics.pdfhttp://www.ncbi.nlm.nih.gov/pubmed/17975201http://www.ncbi.nlm.nih.gov/pubmed/17975201http://www.ncbi.nlm.nih.gov/pubmed/17975201http://www.ncbi.nlm.nih.gov/pubmed/23353941http://www.ncbi.nlm.nih.gov/pubmed/23353941http://www.ncbi.nlm.nih.gov/pubmed/23353941http://www.ncbi.nlm.nih.gov/pubmed/23353941http://www.ncbi.nlm.nih.gov/pubmed/25392034http://www.ncbi.nlm.nih.gov/pubmed/25392034http://www.ncbi.nlm.nih.gov/pubmed/25392034http://www.ncbi.nlm.nih.gov/pubmed/25392034http://www.ncbi.nlm.nih.gov/pubmed/24004068http://www.ncbi.nlm.nih.gov/pubmed/24004068http://www.ncbi.nlm.nih.gov/pubmed/24004068http://www.ncbi.nlm.nih.gov/pubmed/24004068http://www.ncbi.nlm.nih.gov/pubmed/7063747http://www.ncbi.nlm.nih.gov/pubmed/7063747http://www.ncbi.nlm.nih.gov/pubmed/7063747http://www.ncbi.nlm.nih.gov/pubmed/24717459http://www.ncbi.nlm.nih.gov/pubmed/24717459http://www.ncbi.nlm.nih.gov/pubmed/24717459http://www.ncbi.nlm.nih.gov/pubmed/24717459http://www.ncbi.nlm.nih.gov/pubmed/15037492http://www.ncbi.nlm.nih.gov/pubmed/15037492http://www.ncbi.nlm.nih.gov/pubmed/15037492http://www.ncbi.nlm.nih.gov/pubmed/15037492http://www.ncbi.nlm.nih.gov/pubmed/23579354http://www.ncbi.nlm.nih.gov/pubmed/23579354http://www.ncbi.nlm.nih.gov/pubmed/23579354http://www.ncbi.nlm.nih.gov/pubmed/23579354http://www.ncbi.nlm.nih.gov/pubmed/25776936http://www.ncbi.nlm.nih.gov/pubmed/25776936http://www.ncbi.nlm.nih.gov/pubmed/25776936http://www.ncbi.nlm.nih.gov/pubmed/25776936http://www.ncbi.nlm.nih.gov/pubmed/9824069http://www.ncbi.nlm.nih.gov/pubmed/9824069http://www.ncbi.nlm.nih.gov/pubmed/9824069http://www.ncbi.nlm.nih.gov/pubmed/9824069http://www.ncbi.nlm.nih.gov/pubmed/11594901http://www.ncbi.nlm.nih.gov/pubmed/11594901http://www.ncbi.nlm.nih.gov/pubmed/11594901http://www.ncbi.nlm.nih.gov/pubmed/11594901http://www.ncbi.nlm.nih.gov/pubmed/25258352http://www.ncbi.nlm.nih.gov/pubmed/25258352http://www.ncbi.nlm.nih.gov/pubmed/25258352http://www.ncbi.nlm.nih.gov/pubmed/25258352http://www.ncbi.nlm.nih.gov/pubmed/12626967http://www.ncbi.nlm.nih.gov/pubmed/12626967http://www.ncbi.nlm.nih.gov/pubmed/12626967http://www.ncbi.nlm.nih.gov/pubmed/12626967http://www.ncbi.nlm.nih.gov/pubmed/15857560http://www.ncbi.nlm.nih.gov/pubmed/15857560http://www.ncbi.nlm.nih.gov/pubmed/15857560http://www.ncbi.nlm.nih.gov/pubmed/15857560http://www.ncbi.nlm.nih.gov/pubmed/23001437http://www.ncbi.nlm.nih.gov/pubmed/23001437http://www.ncbi.nlm.nih.gov/pubmed/23001437http://www.ncbi.nlm.nih.gov/pubmed/17698689http://www.ncbi.nlm.nih.gov/pubmed/17698689http://www.ncbi.nlm.nih.gov/pubmed/17698689http://www.ncbi.nlm.nih.gov/pubmed/17698689http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.jama.com/?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.ncbi.nlm.nih.gov/pubmed/17698689http://www.ncbi.nlm.nih.gov/pubmed/17698689http://www.ncbi.nlm.nih.gov/pubmed/23001437http://www.ncbi.nlm.nih.gov/pubmed/15857560http://www.ncbi.nlm.nih.gov/pubmed/15857560http://www.ncbi.nlm.nih.gov/pubmed/12626967http://www.ncbi.nlm.nih.gov/pubmed/12626967http://www.ncbi.nlm.nih.gov/pubmed/25258352http://www.ncbi.nlm.nih.gov/pubmed/25258352http://www.ncbi.nlm.nih.gov/pubmed/11594901http://www.ncbi.nlm.nih.gov/pubmed/11594901http://www.ncbi.nlm.nih.gov/pubmed/9824069http://www.ncbi.nlm.nih.gov/pubmed/9824069http://www.ncbi.nlm.nih.gov/pubmed/25776936http://www.ncbi.nlm.nih.gov/pubmed/25776936http://www.ncbi.nlm.nih.gov/pubmed/23579354http://www.ncbi.nlm.nih.gov/pubmed/23579354http://www.ncbi.nlm.nih.gov/pubmed/15037492http://www.ncbi.nlm.nih.gov/pubmed/15037492http://www.ncbi.nlm.nih.gov/pubmed/24717459http://www.ncbi.nlm.nih.gov/pubmed/24717459http://www.ncbi.nlm.nih.gov/pubmed/7063747http://www.ncbi.nlm.nih.gov/pubmed/24004068http://www.ncbi.nlm.nih.gov/pubmed/24004068http://www.ncbi.nlm.nih.gov/pubmed/25392034http://www.ncbi.nlm.nih.gov/pubmed/25392034http://www.ncbi.nlm.nih.gov/pubmed/23353941http://www.ncbi.nlm.nih.gov/pubmed/23353941http://www.ncbi.nlm.nih.gov/pubmed/17975201https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/7_Record_Demographics.pdfhttps://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/7_Record_Demographics.pdfhttps://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/7_Record_Demographics.pdfhttps://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/7_Record_Demographics.pdfhttp://www.ncbi.nlm.nih.gov/pubmed/16342923http://www.ncbi.nlm.nih.gov/pubmed/16342923http://www.ncbi.nlm.nih.gov/pubmed/16224307http://www.ncbi.nlm.nih.gov/pubmed/16224307http://www.ncbi.nlm.nih.gov/pubmed/13634291http://www.ncbi.nlm.nih.gov/pubmed/13634291http://www.ncbi.nlm.nih.gov/pubmed/9055718http://www.ncbi.nlm.nih.gov/pubmed/9055718http://www.ncbi.nlm.nih.gov/pubmed/17568333http://www.ncbi.nlm.nih.gov/pubmed/17568333http://www.ncbi.nlm.nih.gov/pubmed/7261642http://www.ncbi.nlm.nih.gov/pubmed/7261642http://www.ncbi.nlm.nih.gov/pubmed/20716737http://www.ncbi.nlm.nih.gov/pubmed/20716737http://www.ncbi.nlm.nih.gov/pubmed/16410346http://www.ncbi.nlm.nih.gov/pubmed/7799491http://www.ncbi.nlm.nih.gov/pubmed/25560714http://www.ncbi.nlm.nih.gov/pubmed/25560714http://www.ncbi.nlm.nih.gov/pubmed/8769590http://www.ncbi.nlm.nih.gov/pubmed/8844239http://www.ncbi.nlm.nih.gov/pubmed/8844239http://www.ncbi.nlm.nih.gov/pubmed/23934590http://www.ncbi.nlm.nih.gov/pubmed/23934590http://www.ncbi.nlm.nih.gov/pubmed/23087028http://www.ncbi.nlm.nih.gov/pubmed/23087028http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.0289&utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jama.2016.0288http://www.ncbi.nlm.nih.gov/pubmed/1303622http://www.ncbi.nlm.nih.gov/pubmed/12682500http://www.ncbi.nlm.nih.gov/pubmed/12682500http://www.ncbi.nlm.nih.gov/pubmed/11445675http://www.ncbi.nlm.nih.gov/pubmed/24838355