BMJ OpenFor peer review only STARD 2015 AIM STARD stands for “Standards for Reporting Diagnostic accuracy studies”. This list of items was developed to contribute to the completeness

For peer review only

Neutrophil Gelatinase-Associated Lipocalin in the Prediction of Acute Kidney Injury among Adult Intensive Care Unit

Patients: Determination of Optimal Thresholds in Plasma and Urine

Journal: BMJ Open

Manuscript ID bmjopen-2017-016028

Article Type: Research

Date Submitted by the Author: 23-Jan-2017

Complete List of Authors: Tecson, Kristen; Baylor Heart and Vascular Institute, ; Texas A&M College of Medicine Health Science Center, Ehrhardtsen, Elisabeth; BioPorto Diagnostics A/S Erikson, Peter; BioPorto Diagnostics A/S Gaber, A; Houston Methodist Hospital Germain, Michael; Baystate Medical Center Golestaneh, Ladan; Yeshiva University Albert Einstein College of Medicine Lavoria, Maria de los Angeles; BioPorto Diagnostics A/S Moore, Linda; Houston Methodist Hospital McCullough, Peter; Baylor Heart and Vascular Institute

<b>Primary Subject Heading</b>:

Renal medicine

Secondary Subject Heading: Diagnostics

Keywords: neutrophil gelatinase associated lipocalin, acute kidney injury, risk prediction, sensitivity, specificity

For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml

BMJ Open on July 11, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2017-016028 on 10 July 2017. D

ownloaded from

http://bmjopen.bmj.com/


1

Neutrophil Gelatinase-Associated Lipocalin in the Prediction of Acute Kidney Injury

among Adult Intensive Care Unit Patients: Determination of Optimal Thresholds in

Plasma and Urine

Kristen M. Tecson, PhD, Elisabeth Erhardtsen, DVM, Peter Mørch Eriksen, A. Osama Gaber,

MD, Michael Germain, MD, Ladan Golestaneh, MD, Maria de los Angeles Lavoria, MsC, Linda

W. Moore, MS, Peter A. McCullough, MD, MPH

Author Affiliations: Baylor Heart and Vascular Institute, Dallas, Texas (Tecson, McCullough);

Baylor Scott & White Research Institute, Dallas, Texas (Tecson); Texas A&M College of

Medicine Health Science Center, Dallas, Texas (Tecson, McCullough); BioPorto Diagnostics

A/S, Copenhagen, Denmark (Erhardtsen, Eriksen, Lavoria); Albert Einstein College of Medicine, Bronx,

NY (Golestaneh); Houston Methodist Hospital, Houston, TX (Gaber, Moore); Baystate Medical Center,

Springfield, MA (Germain); Baylor University Medical Center, Dallas, Texas (McCullough); Baylor Jack

and Jane Hamilton Heart and Vascular Hospital, Dallas, Texas (McCullough);

Short Title: Blood and Urine NGAL in Risk Prediction of AKI

Keywords: neutrophil gelatinase associated lipocalin, siderocalin-2, acute kidney injury, risk

prediction, sensitivity, specificity

Word Count: 3175

Corresponding Author:

Peter A. McCullough, MD, MPH

621 N Hall Street, Suite H-030

Dallas, TX 75226

[email protected]

Page 1 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on July 11, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2017-016028 on 10 July 2017. Dow

nloaded from



2

Abstract

Objectives: To determine the optimal threshold of blood and urine Neutrophil gelatinase-

associated lipocalin (NGAL) to predict moderate to severe AKI and persistent moderate to severe

AKI lasting at least 48 consecutive hours, as defined by an adjudication panel.

Methods: A multicenter prospective observational study enrolled intensive care unit patients and

recorded daily ethylenediaminetetraacetic acid (EDTA) plasma, heparin plasma, and urine

NGAL. We utilized natural log-transformed NGAL in a logistic regression model to predict stage

2/3 AKI (defined by Kidney Disease International Global Organization). We performed the same

analysis using the NGAL value at the start of persistent stage 2/3 AKI.

Results: Of 245 subjects, 33 (13.5%) developed stage 2/3 AKI and 25 (10.2%) developed

persistent stage 2/3 AKI. Predicting stage 2/3 AKI revealed the optimal NGAL cutoffs in EDTA

plasma (142.0 ng/ml), heparin plasma (148.3 ng/ml), and urine (78.0 ng/ml) and yielded the

following decision statistics: sensitivity (SN) of 78.8%, specificity (SP) of 73.0%, positive

predictive value (PPV) of 31.3%, negative predictive value (NPV) of 95.7%, diagnostic accuracy

(DA)=73.8% (EDTA plasma); SN=72.7%, SP=73.8%, PPV=30.4%, NPV=94.5%, DA=73.7%

(heparin plasma); SN=69.7%, SP=76.8%, PPV=32.9%, NPV=94%, DA=75.8% (urine). The

optimal NGAL cutoffs to predict persistent stage 2/3 AKI were similar: 148.3 ng/ml (EDTA

plasma), 169.6 ng/ml (heparin plasma), and 79.0 ng/ml (urine) yielding: SN=84.0%, SP=73.5%,

PPV=26.6%, NPV=97.6, DA=74.6% (EDTA plasma); SN=84%, SP=76.1%, PPV=26.8%,

NPV=96.5%, DA=76.1% (heparin plasma); SN=75%, SP=75.8%, PPV=26.1, NPV=96.4%,

DA=75.7% (urine).

Page 2 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



3

Conclusion: Blood and urine NGAL predicted stage 2/3 AKI, as well as persistent 2/3 AKI in

the ICU with acceptable decision statistics using a single cut point in each type of specimen.

Abstract word count: 261

Strengths and limitations of this study:

• This study utilized biomarker data from both plasma and urine samples.

• Acute kidney injury diagnosis was adjudicated by an expert panel.

• We used an unbiased, data-driven approach to identify a single cut point for each of plasma and

urinary NGAL samples to predict stage 2 or 3 acute kidney injury and persistent stage 2 or 3 acute

kidney injury.

• This small prospective cohort study had limited clinical follow-up.

Page 3 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



4

Introduction

Serum creatinine and urine output are used to diagnose acute kidney injury (AKI) as

defined by the 2012 Kidney Disease International Global Organization (KDIGO) guidelines

(Supplemental Table 1); however, creatinine is an imperfect marker as it peaks anywhere from 2-

7 days following an insult to the kidneys.1 Additionally, urine output may lead to an inaccurate

diagnosis of AKI in the intensive care unit (ICU), as it is commonly manipulated in the ICU

through use of intravenous fluids and diuretics. Neutrophil gelatinase-associated lipocalin

(NGAL) has been shown to peak following tubular injury earlier than changes in creatinine and

urine output, making it a desirable biomarker for predicting the development of AKI.2 Previous

studies have shown the utility of this biomarker but have suffered from heterogenous AKI

thresholds and lack of validation of parenchymal renal involvement in the injury. In those

studies, AKI may have been due to a transient hemodynamically related reduction in renal

filtration due poor forward perfusion or impaired plasma refill, which represents a distinct AKI

pathophsiology.3, 4

Furthermore, the optimal body fluid for NGAL assay performance (blood or

urine) is unknown, as is the optimal cut point for the detection of moderate to severe AKI

(KDIGO stage 2 or 3). We set out to use a commercially available NGAL assay in a multicenter,

blinded study to assess its performance in two types of plasma samples and in one urine sample

for the prediction of moderate to severe AKI, as defined by KDIGO and adjudicated by clinicians

blinded to the NGAL values.

Methods

Study Details

This study prospectively enrolled consecutive adult patients admitted to an ICU or critical

care setting after obtaining informed consent between March 5, 2014 and April 15, 2015 from 4

Page 4 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



5

participating hospitals in Boston and Springfield, Massachusetts, Bronx, New York, and

Houston, Texas. Each study site received Institutional Review Board approval prior to enrolling

patients. Those with history of nephrectomy, renal transplantation and/or renal replacement

therapy initiated before admission were not eligible to participate. In addition to the standard

clinical care and laboratory testing, one urine and two plasma samples were drawn and frozen per

day in the ICU, up to 8 days. The samples were shipped to a central laboratory where NGAL was

assayed by BioPorto Diagnostics A/S, Copenhagen, Denmark. This test has a measurable range

of 25-3000 ng/ml. 5

At mean NGAL concentrations of 97, 116, and 112 ng/ml in

ethylenediaminetetraacetic acid (EDTA) plasma, heparin plasma, and urine, respectively, the

coefficients of variation were 3.1%, 1.8%, and 2.8%, respectively. If patients were discharged

early from the ICU during the study period, serum creatinine levels were collected manually from

the hospital data system for 48 hours provided the patient was still hospitalized. Reference

standard AKI was determined daily, up to 8 days, by a three-physician adjudication panel using

the KDIGO® guidelines. AKI was confirmed by the adjudication panel and disagreements were

settled by conference calls and discussion of the cases. Adjudicators were blinded to the NGAL

values and physician notes concerning the renal diagnosis.6

Patient Involvement

Patient involvement began at the time of informed consent; patients and caregivers were not involved

in the design of, recruitment for, or conduct of this study. The development of the outcome measures was

not informed by patients’ priorities, experiences, or preferences. The participants’ contributions to this

study are deeply appreciated and the results of the study will be publicly available to the patients via this

publication.

Page 5 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



6

Analyses

NGAL values were highly skewed in all three sample types so we utilized the natural-log

transformed distributions (Figure 1). We performed logistic regression using the natural log-

transformed baseline NGAL [i.e. ln(baseline NGAL)] value as a predictor for the outcome of

moderate/severe (stage 2/3) AKI at any point during observation. Additionally, we defined

persistent moderate/severe (stage 2/3) AKI as 2 or more days of consecutive stage 2/3 AKI and

we constructed another logistic regression model for the outcome of persistent 2/3 AKI using a

wandering ln(baseline NGAL) value as the predictor variable. The wandering ln(baseline NGAL)

value was taken on the morning of the first day of a 48 hour persistent AKI period. If AKI did

not occur, the wandering ln(baseline NGAL) value was taken to be the ln(baseline NGAL). We

found the optimal NGAL value for both outcomes by using the criterion of smallest distance to

the “perfect point” on the receiver operating characteristic (ROC) curve, (0,1). Baseline NGAL

values < 5 ng/ml were adjusted using a regression model, which affected approximately 2% of

the data. We assessed differences in baseline characteristics between those who had at least 1 day

of stage 2/3 AKI at any time during observation and those who did not via Wilcoxon Rank Sum

and Chi-square tests; we did the same for those who had persistent stage 2/3 AKI and those who

did not have persistent stage 2/3 AKI at any point during observation. Continuous variables are

reported as medians [quarter 1, quarter 3]. Categorical variables are reported as frequencies and

proportions.

Results

A total of 252 patients were screened for the study, 245 of whom were enrolled and

included in analyses. There were 33 (13.5%) subjects who developed stage 2/3 AKI as

determined by the adjudicators. With this event rate, we had 95% power to detect an area under

Page 6 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



7

the curve (AUC) as low as 0.69. Age and gender did not differ significantly between those with

or without stage 2/3 AKI; however, those with stage 2/3 AKI had more comorbidities (sepsis:

18.2% v. 5.7%, p=0.01; diabetes 48.5% v. 27.8%, p=0.04; chronic kidney disease: 33.3% v.

10.4%, p<0.001), longer ICU stays (4 [2, 8] days v. 2 [1, 3] days, p=0.001), and higher baseline

NGAL levels (EDTA plasma: 276 [155, 517] ng/ml v. 98 [70, 154] ng/ml, p<0.001; heparin

plasma: 288 [150, 669] ng/ml v. 100 [68, 160] ng/ml, p < 0.001; urine: 190 [61, 1133] ng/ml v.

31 [15, 67], p < 0.001) than those without AKI (Table 1). Using the ln(baseline EDTA plasma

NGAL) as the independent variable in a logistic regression model for the outcome of stage 2/3

AKI at any time yielded an AUC = 0.76, 95% CI 0.64-0.87, p<0.0001 (Figure 2). Optimizing the

ROC curve with respect to the distance from (0,1) resulted in a cut point of 142.0 ng/ml, which

yielded a sensitivity (SN) of 78.8%, specificity (SP) of 73.0%, positive predictive value (PPV) of

31.3%, negative predictive value (NPV) of 95.7% and diagnostic accuracy (DA) of 73.8%.

Similar results are shown for heparin plasma and urine NGAL in Figures 2b and 2c.

Twenty-five (10.2%) subjects had persistent stage 2/3 AKI. Demographics did not differ

significantly between those who did and those who did not have persistent stage 2/3 AKI;

however, those with persistent stage 2/3 AKI had a higher rate of CKD (36.0% v. 10.9%, p <

0.001), longer ICU stays (6 [2,8] days v. 2 [1, 3] days, p <0.001), and higher wandering baseline

NGAL values (EDTA plasma: 287 [188, 517] ng/ml v. 98 [70, 156] ng/ml, p <0.001; heparin:

355 [178, 716] ng/ml v. 101 [68, 162] ng/ml, p<0.001; urine: 231.1[84.3, 1203] ng/ml v. 32 [15,

75] ng/ml, p<0.001) (Table 2). We utilized the wandering ln(baseline EDTA plasma NGAL) in a

logistic regression model to predict persistent stage 2/3 AKI, which yielded an AUC = 0.85, 95%

CI 0.77-0.94, p<0.001. The optimal cut point for NGAL based on the distance to (0,1) was 148.3

ng/ml (Figure 3). This value yielded SN= 84.0%, SP=73.5%, PPV= 26.6%, NPV= 97.6%,

Page 7 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



8

DA=74.6%. Similar cut points and results are shown for heparin plasma and urine NGAL in

Figures 3b and 3c. Odds ratios from the regression models are shown in Table 3.

Discussion

We found that a single measured threshold for NGAL, either in blood or urine, was

effective in risk prediction for the development of moderate to severe AKI and persistent AKI, as

defined by AKI lasting 48 consecutive hours or longer. NGAL measured on a daily basis

performed well in the prognostication of persistent AKI. Our results were internally consistent

and add to the literature as they yielded unbiased cut points from prospectively collected samples

and adjudicated AKI outcomes. We found that urine and plasma yielded different absolute

NGAL concentrations, however, within each sample type the cut point for the prediction of stage

2/3 AKI or persistent AKI was similar, making daily measurements and utilization of a single

threshold feasible, provided there is consistency in sample source and handling procedures. Our

results are concordant with a prior study that observed slightly better discriminatory power of

plasma NGAL than urinary NGAL, with an AUC as high as 0.84. 7

Thus, we believe for ease of

use, plasma NGAL is more likely to be incorporated into routine care than urinary NGAL.

Our findings build upon the work of others using a variety of NGAL assays

demonstrating the concept that 25 kilodalton protein is rapidly and reliably upregulated after

ischemia/reperfusion injury to the kidney.8,9, 4

In human clinical studies, urine NGAL measured

in the emergency department from patients with a variety of presentations was associated with

the development of end-stage renal disease requiring dialysis in the hospital.10

A 2010-2011

study in western Australia, n=102, demonstrated the effectiveness of using NGAL to predict

inpatient AKI (based on risk, injury, failure, loss of kidney function, and end-stage kidney

disease classification) in a tertiary care setting. In the Australian study, the Biosite Triage®

Page 8 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



9

NGAL in whole blood (Alere, Inverness Medical, Australia Inc) was found to have a single

optimal cutoff of 89 ng/ml had SN = 68%, SP = 70%, and AUC = 0.71, 95% CI 0.58-

0.84.11

While the threshold was similar to that found in our EDTA and heparin samples, this

study used all stages of AKI as the endpoint and did not have blinded adjudication by a

committee. Nevertheless, these data are in line with our conclusion that NGAL measured in

blood performs well as a prognostic aid for AKI. A meta-analysis of 1,478 septic patients

demonstrated that NGAL was associated with the development of AKI, need for renal

replacement therapy, and mortality.12

Numerous studies have demonstrated that NGAL rises

after cardiac surgery and is congruent with AKI.13,14

Additionally, NGAL concentrations

increase after contrast-induced AKI; however, it is to a much lesser extent than cardiac surgery

or critical illness.15

Our study is the first among these to demonstrate that NGAL can be used to

risk predict stage 2/3 AKI and its persistence in a cohort of ICU patients. We further

demonstrated a single cutoff value performs well in terms of decision statistics for the

outcomes of stage 2/3 AKI and persistent stage 2/3 AKI.

There have been large prospective studies examining novel biomarkers other than NGAL

for the prediction of stage 2/3 AKI among the critically ill. Kashani and colleagues (n=728)

measured the concentrations of urinary tissue inhibitor of metalloproteinases (TIMP)-2 and

insulin-like growth factor binding protein 7 (IGFBP7), and combined the 2 measures into 1

predictor, referred to as [TIMP-2]*[IGFBP7], NephroCheck® (Astute Medical, San Diego, CA)

to identify imminent stage 2/3 AKI (i.e. in the next 12-hours) which occurred in 14% of study

participants. Baseline urinary [TIMP-2]*[IGFBP7] yielded an AUC = 0.80, 97% CI 0.76-0.83;

however, the full decision statistics of an optimal cut point were not disclosed. 16

A second study

by Bihorac and colleagues (n=420), evaluated urinary [TIMP-2]*[IGFBP7] for the same

Page 9 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



10

endpoint. In this study, 71 (17.4%) patients had immenent stage 2/3 AKI and the model had an

AUC = 0.82, 95% CI 0.76-0.88. However, [TIMP-2]*[IGFBP7] did not appear amenable to a

single threshold value. Two cut points were presented; one yielding high SN and low SP and the

other yielding low SN and high SP. At the cutoff of 0.3 (ng/ml)2/1000 the SN = 92, 95% CI, 85–

98%, but the SP= 46, 95% CI 41–52%. At a 7-fold higher cut point of 2.0 (ng/ml)2/1000 the SN

= 37, 95% CI, 26–47% and SP= 95, 95% CI, 93–97%.17

In comparison, an NGAL EDTA plasma

level of 142.0 ng/ml in our study predicted stage 2/3 AKI and yielded SN=78.8, 95% CI 73.7-

83.9% and SP=73.0, 95% CI 67.4-78.6%. Thus, for NGAL at a single cut point, the lower

bounds of the 95% CIs for sensitivity and specificity were well above 60 and 40%, respectively.

Accordingly, NGAL appears to have considerable advantages over [TIMP-2]*[IGFBP7] in

forecasting AKI over a longer time window, at a single prognostic threshold, and avoids low

sensitivity or specificity. Additionally, we demonstrated NGAL performed well when measured

daily using similar cut points for the prediction of persistent stage 2/3 AKI. Daily use in risk

prediction has not been demonstrated in any study with [TIMP2]*[IGFBP7].

Limitations

Our study has all the limitations of small prospective cohort studies of biomarkers

including limited clinical follow-up and lack of precise follow-up information, including accurate

urine output. We did not consider stage I AKI in the definition of the primary endpoint because

prior studies suggested that many in this category could have transient hemodynamic elevations

of creatinine without parenchymal AKI. Thus, our event rate is lower than other studies which

included all KDIGO stages of AKI. Finally, we did not have information on long term outcomes

such as need for renal replacement therapy or mortality which would have been useful in

Page 10 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



11

integrating NGAL with serum creatinine and urine output in an in-depth assessment of patient

outcomes.18

Conclusion

NGAL concentrations of 142.0, 149.5, and 78.0 ng/ml in EDTA plasma, heparin plasma,

and urine, respectively, were found to be optimal in predicting stage 2/3 AKI in ICU patients.

Similarly, in daily prospective samples, NGAL concentration thresholds at 148.3, 169.6, and 79.2

ng/ml in EDTA plasma, heparin plasma, and urine, respectively, were optimal in predicting

persistent stage 2/3 AKI in the same cohort of ICU patients. These results suggest that

prospective validation of NGAL in one or more sample types in a larger study is warranted.

Page 11 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



12

Funding

The study was funded by BioPorto Diagnostics A/S.

Disclosures

Elisabeth Erhardtsen, DVM, Peter Mørch Eriksen, Maria de los Angeles Lavoria are employees of BioPorto

Diagnostics A/S.

Acknowledgements

The authors would like to thank the study participants who graciously shared their time and materials for the

purposes of this research.

Transparency

The lead author affirms that this manuscript is an honest, accurate, and transparent account of the study

being reported; that no important aspects of the study have been omitted; and that any discrepancies from

the study as planned (and, if relevant, registered) have been explained.

Copyright

“I, Peter A. McCullough, The Corresponding Author of this article contained within the original manuscript

which includes any diagrams & photographs within and any related or stand alone film submitted (the

Contribution”) has the right to grant on behalf of all authors and does grant on behalf of all authors, a licence

to the BMJ Publishing Group Ltd and its licencees, to permit this Contribution (if accepted) to be published

in the BMJ and any other BMJ Group products and to exploit all subsidiary rights, as set out in our licence

set out at: http://www.bmj.com/about-bmj/resources-authors/forms-policies-and-checklists/copyright-open-

access-and-permission-reuse.” I am one author signing on behalf of all co-owners of the Contribution.

Contributorship

PAM conceived the study. PME funded the study. AOG, MG, LG, and LWM initiated the study design and

helped with implementation. KMT conducted the statistical analysis. KMT and PAM drafted the article. All

authors contributed to the manuscript’s refinement and approved the final version.

Data: The data from this study will not be made available.

Page 12 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



13

Figure 1. Mean and standard error of natural log-transformed NGAL by day and acute kidney

injury status (a. EDTA plasma, b. heparin plasma, c. urine)

Figure 2. Receiver operating characteristic curves for natural log-transformed baseline NGAL as

a predictor of stage 2/3 acute kidney injury with the optimal NGAL threshold identified (a.

EDTA plasma, b. heparin plasma, c. urine). Decision statistics are presented for the optimal cut

point (bold) as well as at cut points approximately 20 ng/ml below and above the optimal value.

LR+ indicates positive likelihood ratio and LR- indicated negative likelihood ratio.

Figure 3. Receiver operating characteristic curves for wandering natural log-transformed

baseline NGAL as a predictor of persistent stage 2/3 acute kidney injury with the optimal NGAL

threshold identified (a. EDTA plasma, b. heparin plasma, c. urine). Decision statistics are

presented for the optimal cut point (bold) as well as at cut points approximately 20 ng/ml below

and above the optimal value. LR+ indicates positive likelihood ratio and LR- indicated negative

likelihood ratio.

Page 13 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



14

Table 1. Patient characteristics by occurrence of moderate/severe acute kidney injury (AKI)

Variable

Stage 2 or 3

AKI (n=33)

Stage 1 or No AKI

(n = 212) P-value

Age (years) 68 [56, 74] 63 [54, 73] 0.50

Gender (male) 22 (66.7%) 135 (63.7%) 0.74

Race/Ethnicity 0.86

African American 3 (9.1%) 29 (13.7%)

Asian 0 (0.0%) 2 (0.9%)

Hispanic or Latino 4 (12.1%) 28 (13.2%)

Mixed 0 (0.0%) 2 (0.9%)

White 26 (78.8%) 151 (71.2%)

Hypertension 26 (78.8%) 142 (67.0%) 0.17

Congestive Heart Failure 6 (18.2%) 38 (17.9%) 0.97

Diabetes Mellitus 0.04

Type I 0 (0.0%) 2 (0.9%)

Type II 16 (48.5%) 57 (26.9%)

None 17 (51.5%) 153 (72.2%)

Urinary Tract Infection 2 (6.1%) 6 (2.8%) 0.33

Sepsis 6 (18.2%) 12 (5.7%) 0.01

Chronic Kidney Disease History <0.001

None 22 (66.7%) 190 (89.6%)

Stage 1 1 (3.0%) 4 (1.9%)

Stage 2 1 (3.0%) 3 (1.4%)

Stage 3A 8 (24.2%) 5 (2.4%)

Stage 3B 0 (0.0%) 8 (3.8%)

Stage 4 1 (3.0%) 2 (0.9%)

Baseline EDTA plasma NGAL (ng/ml) 276 [155, 517] 98 [70, 154] <0.001

Baseline heparin plasma NGAL (ng/ml) 288 [150, 669] 100 [68, 160] <0.001

Baseline urine NGAL (ng/ml) 190 [61, 1133] 31 [15, 67] <0.001

Days in intensive care unit 4 [2, 8] 2 [1,3] 0.001

Page 14 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



15

Table 2. Sample characteristics for subjects with and without persistent stage 2/3 acute kidney

injury

Variable Persistent stage 2/3

AKI (n=25)

No persistent

stage 2/3 AKI

(n=220)

p-value

Age (years) 68 [60, 73] 63 [54, 73] 0.31

Gender (male) 16 (64.0%) 141 (64.1%) 0.99

Race/Ethnicity

0.89


Asian 0 (0.0%) 2 (0.9%)


Mixed 0 (0.0%) 2 (0.9%)

White 20 (80.0%) 157 (71.4%)

Hypertension 19 (76.0%) 149 (67.7%) 0.40


Diabetes Mellitus

0.40

Type I 0 (0.0%) 2 (0.9%)

Type II 10 (40.0%) 63 (28.6%)

None 15 (60.0%) 155 (70.5%)


Sepsis 4 (16.0%) 14 (6.4%) 0.10

Chronic Kidney Disease History

<0.001

None 16 (64.0%) 196 (89.1%)

Stage 1 1 (4.0%) 4 (1.8%)

Stage 2 1 (4.0%) 3 (1.4%)

Stage 3A 6 (24.0%) 7 (3.2%)

Stage 3B 0 (00%) 8 (3.6%)

Stage 4 1 (94.0%) 2 (0.9%)

Wandering baseline EDTA plasma NGAL (ng/ml) 287 [188,517] 98 [70,156] <0.001

Wandering baseline heparin plasma NGAL (ng/ml) 355 [178, 716] 101 [68, 162] <0.001

Wandering baseline urine NGAL (ng/ml) 231.1 [84.3, 1203] 32 [15, 75] <0.001

Days in intensive care unit 6 [2, 8] 2 [1,3] <0.001

Page 15 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



16

Table 3. Odds ratios for stage 2/3 acute kidney injury and persistent stage 2/3 acute kidney injury

using EDTA, heparin, and urine NGAL values as univariate predictors (odds are associated with

an approximate 2.73-fold increase in NGAL value)

Outcome NGAL Measure P-Value Odds Ratio (95% CI)

Stage 2/3 acute

kidney injury

EDTA plasma <0.001 2.8 (1.8, 4.4)

Heparin plasma <0.001 3.2 (2.0, 4.9)

Urine <0.001 2.2 (1.7, 2.9)

Persistent stage 2/3

acute kidney injury

EDTA plasma <0.001 3.8 (2.3, 6.3)

Heparin plasma <0.001 3.9 (2.3, 6.3)

Urine <0.001 2.2 (1.6, 3.0)

Page 16 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



17

Supplemental Table 1. Kidney Disease International Global Organization 2012 definitions of

acute kidney injury stages

Stage Serum Creatinine Urine output

1 1.5-1.9 times baseline OR ≥ 0.3 mg/dl

(≥26.5 µmol/l) increase

< 0.5 ml/hg/h for 6-12 hours

2 2.0-2.9 times baseline < 0.5 ml/kg/h ≥ 12 hours

3 3.0 times baseline OR increase in serum

creatinine to ≥ 4.0 mg/dl (≥ 353.6 µmol/l)

OR initiation of renal replacement

therapy OR in patients < 18 years,

decrease in eGFR to < 35 ml/min per

1.73 m2

< 0.3 ml/kg/h ≥ 24 hours OR Anuria for

≥ 12 hours

Page 17 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



18

References

1 KDIGO Clinical Practice Guideline for Acute Kidney Injury. Kidney Int. 2012; 2(suppl 1).

2 McCullough PA, Shaw AD, Haase M, Bouchard J, Waikar SS, Siew ED, Murray PT, Mehta

RL, Ronco C. Diagnosis of acute kidney injury using functional and injury biomarkers:

workgroup statements from the tenth Acute Dialysis Quality Initiative Consensus Conference.

Contrib Nephrol. 2013;182:13-29. doi: 10.1159/000349963. PubMed PMID: 23689653.

3 Chang CH, Yang CH, Yang HY, Chen TH, Lin CY, Chang SW, Chen YT, Hung CC, Fang

JT, Yang CW, Chen YC. Urinary Biomarkers Improve the Diagnosis of Intrinsic Acute

Kidney Injury in Coronary Care Units. Medicine (Baltimore). 2015

Oct;94(40):e1703. doi: 10.1097/MD.0000000000001703. PubMed PMID: 26448023; PubMed

Central PMCID: PMC4616771.

4 Xu K, Rosenstiel P, Paragas N, Hinze C, Gao X, Huai Shen T, Werth M, Forster

C, Deng R, Bruck E, Boles RW, Tornato A, Gopal T, Jones M, Konig J, Stauber J,

D'Agati V, Erdjument-Bromage H, Saggi S, Wagener G, Schmidt-Ott KM, Tatonetti N,

Tempst P, Oliver JA, Guarnieri P, Barasch J. Unique Transcriptional Programs

Identify Subtypes of AKI. J Am Soc Nephrol. 2016 Dec 27. pii: ASN.2016090974.

doi: 10.1681/ASN.2016090974. [Epub ahead of print] PubMed PMID: 28028135.

5 The NGAL Test

TM [package insert]. Gentofte, Denmark. BioPorto Diagnostics A/S, 2013

6 U.S. Department of Health and Human Services, Food and Drug Administration, Center for

Devices and Radiological Health. 2007 March 13. Guidance for Industry and FDA Staff:

Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests.

Washington, DC: Author.

7 Schley G, Köberle C, Manuilova E, Rutz S, Forster C, Weyand M, et al. (2015) Comparison of

Plasma and Urine Biomarker Performance in Acute Kidney Injury. PLoS ONE 10(12):

e0145042. doi:10.1371/journal.pone.0145042

8 Clerico A, Galli C, Fortunato A, Ronco C. Neutrophil gelatinase-associated lipocalin (NGAL)

as biomarker of acute kidney injury: a review of the laboratory characteristics and clinical

evidences. Clin Chem Lab Med. 2012 Feb

15;50(9):1505-17. doi: 10.1515/cclm-2011-0814. Review. PubMed PMID: 22962216.

9 Devarajan P. Neutrophil gelatinase-associated lipocalin: a promising biomarker for human

acute kidney injury. Biomarkers in medicine. 2010;4(2):265-280. doi:10.2217/bmm.10.12.

10 Nickolas TL, O'Rourke MJ, Yang J, Sise ME, Canetta PA, Barasch N, Buchen C, Khan F,

Mori K, Giglio J, Devarajan P, Barasch J. Sensitivity and specificity of a single emergency

department measurement of urinary neutrophil gelatinase-associated lipocalin for diagnosing

Page 18 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



19

acute kidney injury. Ann Intern Med. 2008 Jun 3;148(11):810-9. PubMed PMID: 18519927;

PubMed Central PMCID: PMC2909852.

11

Macdonald S, Arendts G, Nagree Y, Xu X-F. Neutrophil Gelatinase-Associated Lipocalin (NGAL)

predicts renal injury in acute decompensated cardiac failure: a prospective observational study. BMC

Cardiovascular Disorders. 2012;12:8. doi:10.1186/1471-2261-12-8.

12

Zhang A, Cai Y, Wang PF, Qu JN, Luo ZC, Chen XD, Huang B, Liu Y, Huang WQ, Wu J,

Yin YH. Diagnosis and prognosis of neutrophil gelatinase-associated lipocalin for acute kidney

injury with sepsis: a systematic review and meta-analysis. Crit Care. 2016 Feb 16;20:41. doi:

10.1186/s13054-016-1212-x. PubMed PMID: 26880194; PubMed Central PMCID:

PMC4754917.

13

Akrawinthawong K, Shaw MK, Kachner J, Apostolov EO, Basnakian AG, Shah S, Tilak J,

McCullough PA. Urine catalytic iron and neutrophil gelatinase-associated lipocalin as

companion early markers of acute kidney injury after cardiac surgery: a prospective pilot study.

Cardiorenal Med. 2013 Apr;3(1):7-16. doi:

10.1159/000346815. PubMed PMID: 23946721; PubMed Central PMCID: PMC3743453.

14 Haase-Fielitz A, Haase M, Devarajan P. Neutrophil gelatinase-associated lipocalin as a

biomarker of acute kidney injury: a critical evaluation of current status. Annals of clinical

biochemistry. 2014;51(0 3):335-351. doi:10.1177/0004563214521795.

15 Filiopoulos V, Biblaki D, Vlassopoulos D. Neutrophil gelatinase-associated lipocalin (NGAL):

a promising biomarker of contrast-induced nephropathy after computed tomography. Ren Fail.

2014 Jul;36(6):979-86. doi: 10.3109/0886022X.2014.900429. Review. PubMed PMID:

24673459.

16 Kashani K, Al-Khafaji A, Ardiles T, Artigas A, Bagshaw SM, Bell M, Bihorac A,

Birkhahn R, Cely CM, Chawla LS, Davison DL, Feldkamp T, Forni LG, Gong MN, Gunnerson

KJ, Haase M, Hackett J, Honore PM, Hoste EA, Joannes-Boyau O, Joannidis

M, Kim P, Koyner JL, Laskowitz DT, Lissauer ME, Marx G, McCullough PA, Mullaney

S, Ostermann M, Rimmelé T, Shapiro NI, Shaw AD, Shi J, Sprague AM, Vincent JL,

Vinsonneau C, Wagner L, Walker MG, Wilkerson RG, Zacharowski K, Kellum JA.

Discovery and validation of cell cycle arrest biomarkers in human acute kidney injury.

Crit Care. 2013 Feb 6;17(1):R25. doi: 10.1186/cc12503. PubMed PMID: 23388612;


17 Bihorac A, Chawla LS, Shaw AD, Al-Khafaji A, Davison DL, Demuth GE, Fitzgerald

R, Gong MN, Graham DD, Gunnerson K, Heung M, Jortani S, Kleerup E, Koyner JL,

Krell K, Letourneau J, Lissauer M, Miner J, Nguyen HB, Ortega LM, Self WH,

Sellman R, Shi J, Straseski J, Szalados JE, Wilber ST, Walker MG, Wilson J,

Wunderink R, Zimmerman J, Kellum JA. Validation of cell-cycle arrest biomarkers for

acute kidney injury using clinical adjudication. Am J Respir Crit Care Med.

Page 19 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



20

2014 Apr 15;189(8):932-9. doi: 10.1164/rccm.201401-0077OC. PubMed PMID: 24559465.

18 McCullough PA, Bouchard J, Waikar SS, Siew ED, Endre ZH, Goldstein SL, Koyner

JL, Macedo E, Doi K, Di Somma S, Lewington A, Thadhani R, Chakravarthi R, Ice C,

Okusa MD, Duranteau J, Doran P, Yang L, Jaber BL, Meehan S, Kellum JA, Haase M,

Murray PT, Cruz D, Maisel A, Bagshaw SM, Chawla LS, Mehta RL, Shaw AD, Ronco C.

Implementation of novel biomarkers in the diagnosis, prognosis, and management of

acute kidney injury: executive summary from the tenth consensus conference of the

Acute Dialysis Quality Initiative (ADQI). Contrib Nephrol. 2013;182:5-12. doi:

10.1159/000349962. PubMed PMID: 23689652; PubMed Central PMCID: PMC3856225.

Page 20 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 1a

338x190mm (300 x 300 DPI)

Page 21 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 1b

338x190mm (300 x 300 DPI)

Page 22 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 1c

338x190mm (300 x 300 DPI)

Page 23 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 2a

338x190mm (300 x 300 DPI)

Page 24 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 2b

338x190mm (300 x 300 DPI)

Page 25 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 2c

338x190mm (300 x 300 DPI)

Page 26 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 3a

338x190mm (300 x 300 DPI)

Page 27 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 3b

338x190mm (300 x 300 DPI)

Page 28 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 3c

338x190mm (300 x 300 DPI)

Page 29 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Optimal Cut Points of Plasma and Urine Neutrophil Gelatinase-Associated Lipocalin for the Prediction of Acute

Kidney Injury among Critically Ill Adults: Retrospective Determination and Clinical Validation of a Prospective

Multicenter Study

Journal: BMJ Open

Manuscript ID bmjopen-2017-016028.R1


Date Submitted by the Author: 16-Apr-2017

Complete List of Authors: Tecson, Kristen; Baylor Heart and Vascular Institute, ; Texas A&M College of Medicine Health Science Center, Ehrhardtsen, Elisabeth; BioPorto Diagnostics A/S Erikson, Peter; BioPorto Diagnostics A/S Gaber, A; Houston Methodist Hospital

Germain, Michael; Baystate Medical Center Golestaneh, Ladan; Yeshiva University Albert Einstein College of Medicine Lavoria, Maria de los Angeles; BioPorto Diagnostics A/S Moore, Linda; Houston Methodist Hospital McCullough, Peter; Baylor Heart and Vascular Institute


Renal medicine






jopen.bmj.com

/B

MJ O


ownloaded from



1

Optimal Cut Points of Plasma and Urine Neutrophil Gelatinase-Associated Lipocalin for

the Prediction of Acute Kidney Injury among Critically Ill Adults: Retrospective

Determination and Clinical Validation of a Prospective Multicenter Study


MD, Michael Germain, MD, Ladan Golestaneh, MD, Maria de los Angeles Lavoria, MsC,

Linda W. Moore, MS, Peter A. McCullough, MD, MPH




A/S, Copenhagen, Denmark (Erhardtsen, Eriksen, Lavoria); Albert Einstein College of

Medicine, Bronx, NY (Golestaneh); Houston Methodist Hospital, Houston, TX (Gaber, Moore);

Baystate Medical Center, Springfield, MA (Germain); Baylor University Medical Center,

Dallas, Texas (McCullough); Baylor Jack and Jane Hamilton Heart and Vascular Hospital,

Dallas, Texas (McCullough)




Word Count: 3661




Dallas, TX 75226

[email protected]

Page 1 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



2

Abstract






NGAL. We utilized natural log-transformed NGAL in a logistic regression model to predict

stage 2/3 AKI (defined by Kidney Disease International Global Organization). We performed the

same analysis using the NGAL value at the start of persistent stage 2/3 AKI.












DA=75.7% (urine).

Page 2 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



3







• We used an unbiased, data-driven approach to identify a single cut point for each of

plasma and urinary NGAL samples to predict stage 2 or 3 acute kidney injury and

persistent stage 2 or 3 acute kidney injury.


Page 3 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



4

Introduction









studies have shown the utility of this biomarker but have suffered from heterogeneous AKI




pathophysiology.3, 4





for the prediction of moderate to severe AKI, as defined by KDIGO and adjudicated by

clinicians blinded to the NGAL values.

Methods

Study Details



Page 4 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



5


Houston, Texas; the protocol may be found at the following:

http://www.bioporto.com/Products/Featured-areas/NGAL.aspx. Each study site received

Institutional Review Board (IRB) approval prior to enrolling patients (IRBs utilized for this study

are as follows: Baystate Health IRB #1; Houston Methodist Research Institute (HMRI) IRB 1;

Pratners Human Research Committee; Biomedical Research Alliance of New York (BRANY)

IRB). Those with history of nephrectomy, renal transplantation and/or renal replacement therapy

initiated before admission were not eligible to participate. In addition to the standard clinical care

and laboratory testing, one urine and two plasma samples were drawn and frozen per day in the

ICU, up to 8 days. The samples were shipped to a central laboratory where NGAL was assayed

by BioPorto Diagnostics A/S, Copenhagen, Denmark. This test has a measurable range of 25-

3000 ng/ml. 5

At mean NGAL concentrations of 97, 116, and 112 ng/ml in



early from the ICU during the study period, serum creatinine levels were collected manually

from the hospital data system for 48 hours provided the patient was still hospitalized. Reference





Patient Involvement

Patient involvement began at the time of informed consent; patients and caregivers were

not involved in the design of, recruitment for, or conduct of this study. The development of the

Page 5 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



6

outcome measures was not informed by patients’ priorities, experiences, or preferences. The

participants’ contributions to this study are deeply appreciated and the results of the study will

be publicly available to the patients via this publication.

Analyses

This manuscript was constructed via retrospective analyses of prospectively collected

data and was developed under STARD guidelines. NGAL values were highly skewed in all three

sample types so we utilized the natural-log transformed distributions (Figure 1). We performed

logistic regression using the natural log-transformed baseline NGAL [i.e. ln(baseline NGAL)]

value as a predictor for the outcome of moderate/severe (stage 2/3) AKI at any point during

observation. Additionally, we defined persistent moderate/severe (stage 2/3) AKI as 2 or more

days of consecutive stage 2/3 AKI and we constructed another logistic regression model for the

outcome of persistent 2/3 AKI using a wandering ln(baseline NGAL) value as the predictor

variable. The wandering ln(baseline NGAL) value was taken on the morning of the first day of a

48 hour persistent AKI period. If AKI did not occur, the wandering ln(baseline NGAL) value

was taken to be the ln(baseline NGAL). We found the optimal NGAL value for both outcomes

by using the criterion of smallest distance to the “perfect point” on the receiver operating

characteristic (ROC) curve, (0,1). Baseline NGAL values < 5 ng/ml were adjusted using a

regression model, which affected approximately 2% of the data. We assessed differences in

baseline characteristics between those who had at least 1 day of stage 2/3 AKI at any time during

observation and those who did not via Wilcoxon Rank Sum and Chi-square tests; we did the

same for those who had persistent stage 2/3 AKI and those who did not have persistent stage 2/3

AKI at any point during observation. Continuous variables are reported as medians [quarter 1,

quarter 3]. Categorical variables are reported as frequencies and proportions.

Page 6 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



7

Results

A total of 252 patients were screened for the study; 2 were screen failures, and 5 did not

have sufficient information to be included in analyses. Hence, 245 patients were enrolled and

included in analyses. Only 2 patients in this study had cardiac surgery during the observational

period. No adverse events occurred during the observation period. There were 33 (13.5%)

subjects who developed stage 2/3 AKI as determined by the adjudicators. With this event rate,

we had 95% power to detect an area under the curve (AUC) as low as 0.69. Age and gender did

not differ significantly between those with or without stage 2/3 AKI; however, those with stage

2/3 AKI had more comorbidities (sepsis: 18.2% v. 5.7%, p=0.01; diabetes 48.5% v. 27.8%,

p=0.04; chronic kidney disease: 33.3% v. 10.4%, p<0.001), longer ICU stays (4 [2, 8] days v. 2

[1, 3] days, p=0.001), and higher baseline NGAL levels (EDTA plasma: 276 [155, 517] ng/ml v.

98 [70, 154] ng/ml, p<0.001; heparin plasma: 288 [150, 669] ng/ml v. 100 [68, 160] ng/ml, p <

0.001; urine: 190 [61, 1133] ng/ml v. 31 [15, 67], p < 0.001) than those without AKI (Table 1).

Using the ln(baseline EDTA plasma NGAL) as the independent variable in a logistic regression

model for the outcome of stage 2/3 AKI at any time yielded an AUC = 0.76, 95% CI 0.64-0.87,

p<0.0001 (Figure 2). Optimizing the ROC curve with respect to the distance from (0,1) resulted

in a cut point of 142.0 ng/ml, which yielded a sensitivity (SN) of 78.8%, specificity (SP) of

73.0%, positive predictive value (PPV) of 31.3%, negative predictive value (NPV) of 95.7% and

diagnostic accuracy (DA) of 73.8%. Similar results are shown for heparin plasma and urine

NGAL in Figures 2b and 2c.



however, those with persistent stage 2/3 AKI had a higher rate of chronic kidney disease (CKD)

Page 7 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



8

(36.0% v. 10.9%, p < 0.001), longer ICU stays (6 [2,8] days v. 2 [1, 3] days, p <0.001), and

higher wandering baseline NGAL values (EDTA plasma: 287 [188, 517] ng/ml v. 98 [70, 156]

ng/ml, p <0.001; heparin: 355 [178, 716] ng/ml v. 101 [68, 162] ng/ml, p<0.001; urine:

231.1[84.3, 1203] ng/ml v. 32 [15, 75] ng/ml, p<0.001) (Table 2). We utilized the wandering

ln(baseline EDTA plasma NGAL) in a logistic regression model to predict persistent stage 2/3

AKI, which yielded an AUC = 0.85, 95% CI 0.77-0.94, p<0.001. The optimal cut point for

NGAL based on the distance to (0,1) was 148.3 ng/ml (Figure 3). This value yielded SN= 84.0%,

SP=73.5%, PPV= 26.6%, NPV= 97.6%, DA=74.6%. Similar cut points and results are shown for

heparin plasma and urine NGAL in Figures 3b and 3c. Odds ratios from the regression models

are shown in Table 3.

Discussion











plasma NGAL than urinary NGAL, with an AUC as high as 0.84. 7

Thus, we believe for ease of


Page 8 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



9




The work by Nikolas et al. revealed that NGAL

even has the ability to distinguish AKI from non-progressive CKD, a feat that creatinine has

failed to accomplish.10

Further, the urine NGAL measured in the emergency department was

associated with the development of end-stage renal disease requiring dialysis in the hospital. A

2010-2011 study in western Australia, n=102, demonstrated the effectiveness of using NGAL to

predict inpatient AKI (based on risk, injury, failure, loss of kidney function, and end-stage

kidney disease classification) in a tertiary care setting. In the Australian study, the Biosite

Triage® NGAL in whole blood (Alere, Inverness Medical, Australia Inc) was found to have a

single optimal cutoff of 89 ng/ml had SN = 68%, SP = 70%, and AUC = 0.71, 95% CI 0.58-

0.84.11

While the threshold was similar to that found in our EDTA and heparin samples, this





replacement therapy, and mortality.12

Marino et al.’s study of 101 emergency department

patients with sepsis confirmed that NGAL was related to (the severity of) AKI; however, NGAL

concentrations were also elevated in the absence of AKI in these patients.13

Numerous studies

have demonstrated that NGAL rises after cardiac surgery and is congruent with AKI.14,15

Additionally, NGAL concentrations increase after contrast-induced AKI; however, it is to a

much lesser extent than from cardiac surgery or critical illness.16

An extensive literature review

and meta-analysis by Haase et al. revealed a wide range of optimal reported NGAL cutoffs for

Page 9 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



10

AKI prediction (100 – 270 ng/ml).17

The optimal plasma cutoffs found in our study to detect

stage 2/3 AKI (EDTA: 142.0 ng/ml, heparin:149.5 ng/ml) were within the range reported in the

meta-analysis. Haase suggested cutoffs between 150-170 ng/ml for adults, which is slightly

higher than our findings, and may be attributable to the high rate of postoperative patients used

in the meta-analysis, as well as the varied definitions of AKI. Our study is the first among these

to demonstrate that NGAL can be used to risk predict stage 2/3 AKI and its persistence in a

cohort of ICU patients. We further demonstrated a single cutoff value performs well in terms of

decision statistics for the outcomes of stage 2/3 AKI and persistent stage 2/3 AKI.








however, the full decision statistics of an optimal cut point were not disclosed. 18

A second study


endpoint. In this study, 71 (17.4%) patients had imminent stage 2/3 AKI and the model had an





= 37, 95% CI, 26–47% and SP= 95, 95% CI, 93–97%.19

In comparison, an NGAL EDTA

Page 10 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



11

plasma level of 142.0 ng/ml in our study predicted stage 2/3 AKI and yielded SN=78.8, 95% CI

73.7-83.9% and SP=73.0, 95% CI 67.4-78.6%. Thus, for NGAL at a single cut point, the lower







Limitations






included all KDIGO stages of AKI. Our study sample was non-homogeneous in that it included

patients with sepsis, a diagnosis which has been shown to increase NGAL levels even for those

without AKI.13

Further, due to the small sample size, we could not analyze sepsis or CKD

subjects separately, rendering it impossible to determine whether or not different cutoffs were

needed. To examine confounders’ relations with (persistent) stage 2/3 AKI, we built multivariate

models; however, neither sepsis, nor CKD, nor DM were statistically significant when

considered with NGAL simultaneously. Consequently, the adjusted odds ratio estimates changed

only slightly, and the corresponding p-values were unchanged. Perhaps this is due, in part, to the

fact that NGAL can distinguish AKI from non-progressive CKD. Additionally, the small sample

Page 11 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



12

size did not permit the use of training and/or validation sets, which prohibited further

investigation of the models. However, all AUCs were at least 0.76, which indicates good

predictive ability. Finally, we did not have information on long term outcomes such as need for

renal replacement therapy or mortality which would have been useful in integrating NGAL with

serum creatinine and urine output in an in-depth assessment of patient outcomes.20

Conclusion


and urine, respectively, were found to be optimal in predicting stage 2/3 AKI in ICU patients

participating in a prospective observational study. Similarly, NGAL concentration thresholds of

148.3, 169.6, and 79.2 ng/ml in EDTA plasma, heparin plasma, and urine, respectively, were

optimal in predicting persistent stage 2/3 AKI in the same cohort of ICU patients. A larger

prospective study to provide external validation of these cut points is warranted.

Page 12 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



13

Funding


Disclosures

Elisabeth Erhardtsen, DVM, Peter Mørch Eriksen, Maria de los Angeles Lavoria are employees

of BioPorto Diagnostics A/S. The remaining authors have nothing to disclose.

Acknowledgements

The authors would like to thank the study participants who graciously shared their time and

materials for the purposes of this research.

Transparency

The lead author affirms that this manuscript is an honest, accurate, and transparent account of the

study being reported; that no important aspects of the study have been omitted; and that any

discrepancies from the study as planned (and, if relevant, registered) have been explained.

Copyright

“I, Peter A. McCullough, The Corresponding Author of this article contained within the original

manuscript which includes any diagrams & photographs within and any related or stand alone

film submitted (the Contribution”) has the right to grant on behalf of all authors and does grant

on behalf of all authors, a licence to the BMJ Publishing Group Ltd and its licencees, to permit

this Contribution (if accepted) to be published in the BMJ and any other BMJ Group products

and to exploit all subsidiary rights, as set out in our licence set out at:

http://www.bmj.com/about-bmj/resources-authors/forms-policies-and-checklists/copyright-open-

access-and-permission-reuse.” I am one author signing on behalf of all co-owners of the

Contribution.

Contributorship

PAM conceived the study. PME funded the study. AOG, MG, LG, and LWM initiated the study

design and helped with implementation. KMT conducted the statistical analysis. KMT and PAM

drafted the article. All authors contributed to the manuscript’s refinement and approved the final

version.


Page 13 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



14













likelihood ratio.

Page 14 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



15


Variable Stage 2 or 3 AKI

(n=33)

Stage 1 or No AKI

(n = 212) P-value

Age (years) 68 [56, 74] 63 [54, 73] 0.5

Gender (male) 22 (66.7%) 135 (63.7%) 0.74

Race/Ethnicity 0.86


Asian 0 (0.0%) 2 (0.9%)


Mixed 0 (0.0%) 2 (0.9%)

White 26 (78.8%) 151 (71.2%)

Hypertension 26 (78.8%) 142 (67.0%) 0.17



Type I 0 (0.0%) 2 (0.9%)

Type II 16 (48.5%) 57 (26.9%)

None 17 (51.5%) 153 (72.2%)


Sepsis 6 (18.2%) 12 (5.7%) 0.01


None 22 (66.7%) 190 (89.6%)

Stage 1 1 (3.0%) 4 (1.9%)

Stage 2 1 (3.0%) 3 (1.4%)

Stage 3A 8 (24.2%) 5 (2.4%)

Stage 3B 0 (0.0%) 8 (3.8%)

Stage 4 1 (3.0%) 2 (0.9%)

In-hospital renal replacement therapy 4 (12.1%) 2 (0.9%) 0.003

Nephrotoxin use at baseline 18 (54.6%) 88 (41.5%) 0.19

Baseline EDTA NGAL (ng/ml) 276 [155, 517] 98 [70, 154] <0.001





Page 15 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



16

injury

Variable Persistent AKI

(n=25)

No persistent

AKI (n=220) P-value

Age (years) 68 [60, 73] 63 [54, 73] 0.31

Gender (male) 16 (64.0%) 141 (64.1%) 0.99

Race/Ethnicity 0.89


Asian 0 (0.0%) 2 (0.9%)


Mixed 0 (0.0%) 2 (0.9%)

White 20 (80.0%) 157 (71.4%)

Hypertension 19 (76.0%) 149 (67.7%) 0.4



Type I 0 (0.0%) 2 (0.9%)

Type II 10 (40.0%) 63 (28.6%)

None 15 (60.0%) 155 (70.5%)


Sepsis 4 (16.0%) 14 (6.4%) 0.1


None 16 (64.0%) 196 (89.1%)

Stage 1 1 (4.0%) 4 (1.8%)

Stage 2 1 (4.0%) 3 (1.4%)

Stage 3A 6 (24.0%) 7 (3.2%)

Stage 3B 0 (00%) 8 (3.6%)

Stage 4 1 (94.0%) 2 (0.9%)


Use of nephrotoxins 14 (56.05) 92 (41.8%) 0.18

Wandering baseline EDTA NGAL (ng/ml) 287 [188,517] 98 [70,156] <0.001

Wandering baseline heparin plasma NGAL

(ng/ml) 355 [178, 716] 101 [68, 162] <0.001



Page 16 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



17


using EDTA, heparin, and urine NGAL values as predictors (odds are associated with an

approximate 2.73-fold increase in NGAL value)


Stage 2/3 acute

kidney injury

EDTA <0.001 2.8 (1.8, 4.4)

Heparin <0.001 3.2 (2.0, 4.9)

Urine <0.001 2.2 (1.7, 2.9)


acute kidney injury

EDTA <0.001 3.8 (2.3, 6.3)

Heparin <0.001 3.9 (2.3, 6.3)

Urine <0.001 2.2 (1.6, 3.0)

Page 17 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



18













1.73 m2


≥ 12 hours

Page 18 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



19

References

1 KDIGO Clinical Practice Guideline for Acute Kidney Injury. Kidney Int. 2012; 2(suppl 1).

2 McCullough PA, Shaw AD, Haase M, Bouchard J, Waikar SS, Siew ED, Murray PT, Mehta




3 Chang CH, Yang CH, Yang HY, Chen TH, Lin CY, Chang SW, Chen YT, Hung CC, Fang JT,

Yang CW, Chen YC. Urinary Biomarkers Improve the Diagnosis of Intrinsic Acute Kidney

Injury in Coronary Care Units. Medicine (Baltimore). 2015 Oct;94(40):e1703. doi:

10.1097/MD.0000000000001703. PubMed PMID: 26448023; PubMed Central PMCID:

PMC4616771.

4 Xu K, Rosenstiel P, Paragas N, Hinze C, Gao X, Huai Shen T, Werth M, Forster C, Deng R,

Bruck E, Boles RW, Tornato A, Gopal T, Jones M, Konig J, Stauber J, D'Agati V, Erdjument-

Bromage H, Saggi S, Wagener G, Schmidt-Ott KM, Tatonetti N, Tempst P, Oliver JA, Guarnieri

P, Barasch J. Unique Transcriptional Programs Identify Subtypes of AKI. J Am Soc Nephrol.

2016 Dec 27. pii: ASN.2016090974. doi: 10.1681/ASN.2016090974. [Epub ahead of print]

PubMed PMID: 28028135.

5 The NGAL Test

TM [package insert]. Gentofte, Denmark. BioPorto Diagnostics A/S, 2013.

















Page 19 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



20



11

Macdonald S, Arendts G, Nagree Y, Xu X-F. Neutrophil Gelatinase-Associated Lipocalin

(NGAL) predicts renal injury in acute decompensated cardiac failure: a prospective

observational study. BMC Cardiovascular Disorders. 2012;12:8. doi:10.1186/1471-2261-12-8.

12

Zhang A, Cai Y, Wang PF, Qu JN, Luo ZC, Chen XD, Huang B, Liu Y, Huang WQ, Wu J,




PMC4754917.

13

Marino R, Struck J, Hartmann O, Maisel AS, Rehfeldt M, Magrini L, Melander O, Bergmann

A, Di Somma S. Diagnostic and short-term prognostic utility of plasma pro-enkephalin (pro-

ENK) for acute kidney injury in patients admitted with sepsis in the emergency department. J

Nephrol. 2015 Dec;28(6):717-24. doi:10.1007/s40620-014-0163-z. Epub 2014 Dec 9. PubMed

PMID: 25486879.

14

Akrawinthawong K, Shaw MK, Kachner J, Apostolov EO, Basnakian AG, Shah S, Tilak J,



Cardiorenal Med. 2013 Apr;3(1):7-16. doi: 10.1159/000346815. PubMed PMID: 23946721;








24673459.

17 Haase M, Bellomo R, Devarajan P, Schlattmann P, Haase-Fielitz A; NGAL Meta-analysis

Investigator Group. Accuracy of neutrophil gelatinase-associated lipocalin (NGAL) in

diagnosis and prognosis in acute kidney injury: a systematic review and meta-analysis. Am J

Kidney Dis. 2009 Dec;54(6):1012-24. doi:10.1053/j.ajkd.2009.07.020. Epub 2009 Oct 21.

Review. PubMed PMID: 19850388.

18

Kashani K, Al-Khafaji A, Ardiles T, Artigas A, Bagshaw SM, Bell M, Bihorac A, Birkhahn

R, Cely CM, Chawla LS, Davison DL, Feldkamp T, Forni LG, Gong MN, Gunnerson KJ, Haase

M, Hackett J, Honore PM, Hoste EA, Joannes-Boyau O, Joannidis M, Kim P, Koyner JL,

Laskowitz DT, Lissauer ME, Marx G, McCullough PA, Mullaney S, Ostermann M, Rimmelé T,

Page 20 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



21

Shapiro NI, Shaw AD, Shi J, Sprague AM, Vincent JL, Vinsonneau C, Wagner L, Walker MG,

Wilkerson RG, Zacharowski K, Kellum JA. Discovery and validation of cell cycle arrest

biomarkers in human acute kidney injury. Crit Care. 2013 Feb 6;17(1):R25. doi:

10.1186/cc12503. PubMed PMID: 23388612; PubMed Central PMCID: PMC4057242.

19

Bihorac A, Chawla LS, Shaw AD, Al-Khafaji A, Davison DL, Demuth GE, Fitzgerald R,

Gong MN, Graham DD, Gunnerson K, Heung M, Jortani S, Kleerup E, Koyner JL, Krell K,

Letourneau J, Lissauer M, Miner J, Nguyen HB, Ortega LM, Self WH, Sellman R, Shi J,

Straseski J, Szalados JE, Wilber ST, Walker MG, Wilson J, Wunderink R, Zimmerman J,

Kellum JA. Validation of cell-cycle arrest biomarkers for acute kidney injury using clinical

adjudication. Am J Respir Crit Care Med. 2014 Apr 15;189(8):932-9. doi: 10.1164/rccm.201401-

0077OC. PubMed PMID: 24559465.

20

McCullough PA, Bouchard J, Waikar SS, Siew ED, Endre ZH, Goldstein SL, Koyner JL,

Macedo E, Doi K, Di Somma S, Lewington A, Thadhani R, Chakravarthi R, Ice C, Okusa MD,

Duranteau J, Doran P, Yang L, Jaber BL, Meehan S, Kellum JA, Haase M, Murray PT, Cruz D,

Maisel A, Bagshaw SM, Chawla LS, Mehta RL, Shaw AD, Ronco C. Implementation of novel

biomarkers in the diagnosis, prognosis, and management of acute kidney injury: executive

summary from the tenth consensus conference of the Acute Dialysis Quality Initiative (ADQI).

Contrib Nephrol. 2013;182:5-12. doi:10.1159/000349962. PubMed PMID: 23689652; PubMed


Page 21 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 1a

338x190mm (300 x 300 DPI)

Page 22 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 1b

338x190mm (300 x 300 DPI)

Page 23 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 1c

338x190mm (300 x 300 DPI)

Page 24 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 2a

338x190mm (300 x 300 DPI)

Page 25 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 2b

338x190mm (300 x 300 DPI)

Page 26 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 2c

338x190mm (300 x 300 DPI)

Page 27 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 3a

338x190mm (300 x 300 DPI)

Page 28 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 3b

338x190mm (300 x 300 DPI)

Page 29 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 3c

338x190mm (300 x 300 DPI)

Page 30 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Section & Topic No Item Reported on page #

TITLE OR ABSTRACT

1 Identification as a study of diagnostic accuracy using at least one measure of accuracy

(such as sensitivity, specificity, predictive values, or AUC)

1, 2

ABSTRACT

2 Structured summary of study design, methods, results, and conclusions

(for specific guidance, see STARD for Abstracts)

2-3

INTRODUCTION

3 Scientific and clinical background, including the intended use and clinical role of the index test 4

4 Study objectives and hypotheses 4

METHODS

Study design 5 Whether data collection was planned before the index test and reference standard

were performed (prospective study) or after (retrospective study)

4, 6

Participants 6 Eligibility criteria 4-5

7 On what basis potentially eligible participants were identified

(such as symptoms, results from previous tests, inclusion in registry)

4-5

8 Where and when potentially eligible participants were identified (setting, location and dates) 4-5

9 Whether participants formed a consecutive, random or convenience series 4-5

Test methods 10a Index test, in sufficient detail to allow replication 5

10b Reference standard, in sufficient detail to allow replication 4, 5, Sup. Table 1

11 Rationale for choosing the reference standard (if alternatives exist) 4, 5, Sup. Table 1

12a Definition of and rationale for test positivity cut-offs or result categories

of the index test, distinguishing pre-specified from exploratory

6

12b Definition of and rationale for test positivity cut-offs or result categories

of the reference standard, distinguishing pre-specified from exploratory

4, 5, 6

13a Whether clinical information and reference standard results were available

to the performers/readers of the index test

4, 5

13b Whether clinical information and index test results were available

to the assessors of the reference standard

4, 5

Analysis 14 Methods for estimating or comparing measures of diagnostic accuracy 7-8, Figures

15 How indeterminate index test or reference standard results were handled 7

16 How missing data on the index test and reference standard were handled 7

17 Any analyses of variability in diagnostic accuracy, distinguishing pre-specified from exploratory

18 Intended sample size and how it was determined 7

RESULTS

Participants 19 Flow of participants, using a diagram 7 – in text

20 Baseline demographic and clinical characteristics of participants 7, Table 1

21a Distribution of severity of disease in those with the target condition Table 1

21b Distribution of alternative diagnoses in those without the target condition Table 1

22 Time interval and any clinical interventions between index test and reference standard 5

Test results 23 Cross tabulation of the index test results (or their distribution)

by the results of the reference standard

Table 1

24 Estimates of diagnostic accuracy and their precision (such as 95% confidence intervals) 7-8, Figures

25 Any adverse events from performing the index test or the reference standard 7

DISCUSSION

26 Study limitations, including sources of potential bias, statistical uncertainty, and generalisability 11

27 Implications for practice, including the intended use and clinical role of the index test 8, 11-12

OTHER

28 Registration number and name of registry 13

29 Where the full study protocol can be accessed 5

30 Sources of funding and other support; role of funders 13

Page 31 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



STARD 2015

AIM

STARD stands for “Standards for Reporting Diagnostic accuracy studies”. This list of items was developed to contribute to the

completeness and transparency of reporting of diagnostic accuracy studies. Authors can use the list to write informative

study reports. Editors and peer-reviewers can use it to evaluate whether the information has been included in manuscripts

submitted for publication.

EXPLANATION

A diagnostic accuracy study evaluates the ability of one or more medical tests to correctly classify study participants as

having a target condition. This can be a disease, a disease stage, response or benefit from therapy, or an event or condition

in the future. A medical test can be an imaging procedure, a laboratory test, elements from history and physical examination,

a combination of these, or any other method for collecting information about the current health status of a patient.

The test whose accuracy is evaluated is called index test. A study can evaluate the accuracy of one or more index tests.

Evaluating the ability of a medical test to correctly classify patients is typically done by comparing the distribution of the

index test results with those of the reference standard. The reference standard is the best available method for establishing

the presence or absence of the target condition. An accuracy study can rely on one or more reference standards.

If test results are categorized as either positive or negative, the cross tabulation of the index test results against those of the

reference standard can be used to estimate the sensitivity of the index test (the proportion of participants with the target

condition who have a positive index test), and its specificity (the proportion without the target condition who have a negative

index test). From this cross tabulation (sometimes referred to as the contingency or “2x2” table), several other accuracy

statistics can be estimated, such as the positive and negative predictive values of the test. Confidence intervals around

estimates of accuracy can then be calculated to quantify the statistical precision of the measurements.

If the index test results can take more than two values, categorization of test results as positive or negative requires a test

positivity cut-off. When multiple such cut-offs can be defined, authors can report a receiver operating characteristic (ROC)

curve which graphically represents the combination of sensitivity and specificity for each possible test positivity cut-off. The

area under the ROC curve informs in a single numerical value about the overall diagnostic accuracy of the index test.

The intended use of a medical test can be diagnosis, screening, staging, monitoring, surveillance, prediction or prognosis. The

clinical role of a test explains its position relative to existing tests in the clinical pathway. A replacement test, for example,

replaces an existing test. A triage test is used before an existing test; an add-on test is used after an existing test.

Besides diagnostic accuracy, several other outcomes and statistics may be relevant in the evaluation of medical tests. Medical

tests can also be used to classify patients for purposes other than diagnosis, such as staging or prognosis. The STARD list was

not explicitly developed for these other outcomes, statistics, and study types, although most STARD items would still apply.

DEVELOPMENT

This STARD list was released in 2015. The 30 items were identified by an international expert group of methodologists,

researchers, and editors. The guiding principle in the development of STARD was to select items that, when reported, would

help readers to judge the potential for bias in the study, to appraise the applicability of the study findings and the validity of

conclusions and recommendations. The list represents an update of the first version, which was published in 2003.

More information can be found on http://www.equator-network.org/reporting-guidelines/stard.

Page 32 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Optimal Cut Points of Plasma and Urine Neutrophil Gelatinase-Associated Lipocalin for the Prediction of Acute

Kidney Injury among Critically Ill Adults: Retrospective Determination and Clinical Validation of a Prospective

Multicenter Study

Journal: BMJ Open

Manuscript ID bmjopen-2017-016028.R2


Date Submitted by the Author: 09-May-2017

Complete List of Authors: Tecson, Kristen; Baylor Heart and Vascular Institute, ; Texas A&M College of Medicine Health Science Center, Ehrhardtsen, Elisabeth; BioPorto Diagnostics A/S Erikson, Peter; BioPorto Diagnostics A/S Gaber, A; Houston Methodist Hospital

Germain, Michael; Baystate Medical Center Golestaneh, Ladan; Yeshiva University Albert Einstein College of Medicine Lavoria, Maria de los Angeles; BioPorto Diagnostics A/S Moore, Linda; Houston Methodist Hospital McCullough, Peter; Baylor Heart and Vascular Institute


Renal medicine






jopen.bmj.com

/B

MJ O


ownloaded from



1

Optimal Cut Points of Plasma and Urine Neutrophil Gelatinase-Associated Lipocalin for

the Prediction of Acute Kidney Injury among Critically Ill Adults: Retrospective

Determination and Clinical Validation of a Prospective Multicenter Study


MD, Michael Germain, MD, Ladan Golestaneh, MD, Maria de los Angeles Lavoria, MsC, Linda

W. Moore, MS, Peter A. McCullough, MD, MPH




A/S, Copenhagen, Denmark (Erhardtsen, Eriksen, Lavoria); Albert Einstein College of

Medicine, Bronx, NY (Golestaneh); Houston Methodist Hospital, Houston, TX (Gaber, Moore);

Baystate Medical Center, Springfield, MA (Germain); Baylor University Medical Center,

Dallas, Texas (McCullough); Baylor Jack and Jane Hamilton Heart and Vascular Hospital,

Dallas, Texas (McCullough)




Word Count: 3671




Dallas, TX 75226

[email protected]

Page 1 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



2

Abstract






NGAL. We utilized natural log-transformed NGAL in a logistic regression model to predict

stage 2/3 AKI (defined by Kidney Disease International Global Organization). We performed the

same analysis using the NGAL value at the start of persistent stage 2/3 AKI.












DA=75.7% (urine).

Page 2 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



3







• We used an unbiased, data-driven approach to identify a single cut point for each of

plasma and urinary NGAL samples to predict stage 2 or 3 acute kidney injury and

persistent stage 2 or 3 acute kidney injury.


Page 3 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



4

Introduction









studies have shown the utility of this biomarker but have suffered from heterogeneous AKI




pathophysiology.3, 4





for the prediction of moderate to severe AKI, as defined by KDIGO and adjudicated by

clinicians blinded to the NGAL values.

Methods

Study Details



Page 4 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



5


Houston, Texas; the protocol may be found at the following:

http://www.bioporto.com/Products/Featured-areas/NGAL.aspx. Each study site received

Institutional Review Board (IRB) approval prior to enrolling patients (IRBs utilized for this study

are as follows: Baystate Health IRB #1; Houston Methodist Research Institute (HMRI) IRB 1;

Pratners Human Research Committee; Biomedical Research Alliance of New York (BRANY)

IRB). Those with history of nephrectomy, renal transplantation and/or renal replacement therapy

initiated before admission were not eligible to participate. In addition to the standard clinical care

and laboratory testing, one urine and two plasma samples were drawn and frozen per day in the

ICU, up to 8 days. The samples were shipped to a central laboratory where NGAL was assayed

by BioPorto Diagnostics A/S, Copenhagen, Denmark. This test has a measurable range of 25-

3000 ng/ml. 5 At mean NGAL concentrations of 97, 116, and 112 ng/ml in



early from the ICU during the study period, serum creatinine levels were collected manually

from the hospital data system for 48 hours provided the patient was still hospitalized. Reference





Patient Involvement

Patient involvement began at the time of informed consent; patients and caregivers were

not involved in the design of, recruitment for, or conduct of this study. The development of the

Page 5 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



6

outcome measures was not informed by patients’ priorities, experiences, or preferences. The

participants’ contributions to this study are deeply appreciated and the results of the study will

be publicly available to the patients via this publication.

Analyses

This manuscript was constructed via retrospective analyses of prospectively collected

data and was developed under STARD guidelines; ethics committee approval was obtained prior

to conducting these analyses. NGAL values were highly skewed in all three sample types so we

utilized the natural-log transformed distributions (Figure 1). We performed logistic regression

using the natural log-transformed baseline NGAL [i.e. ln(baseline NGAL)] value as a predictor

for the outcome of moderate/severe (stage 2/3) AKI at any point during observation.

Additionally, we defined persistent moderate/severe (stage 2/3) AKI as 2 or more days of

consecutive stage 2/3 AKI and we constructed another logistic regression model for the outcome

of persistent 2/3 AKI using a wandering ln(baseline NGAL) value as the predictor variable. The

wandering ln(baseline NGAL) value was taken on the morning of the first day of a 48 hour

persistent AKI period. If AKI did not occur, the wandering ln(baseline NGAL) value was taken

to be the ln(baseline NGAL). We found the optimal NGAL value for both outcomes by using the

criterion of smallest distance to the “perfect point” on the receiver operating characteristic (ROC)

curve, (0,1). Baseline NGAL values < 5 ng/ml were adjusted using a regression model, which

affected approximately 2% of the data. We assessed differences in baseline characteristics

between those who had at least 1 day of stage 2/3 AKI at any time during observation and those

who did not via Wilcoxon Rank Sum and Chi-square tests; we did the same for those who had

persistent stage 2/3 AKI and those who did not have persistent stage 2/3 AKI at any point during

Page 6 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



7

observation. Continuous variables are reported as medians [quarter 1, quarter 3]. Categorical

variables are reported as frequencies and proportions.

Results

A total of 252 patients were screened for the study; 2 were screen failures, and 5 did not

have sufficient information to be included in analyses. Hence, 245 patients were enrolled and

included in analyses. Only 2 patients in this study had cardiac surgery during the observational

period. No adverse events occurred during the observation period. There were 33 (13.5%)

subjects who developed stage 2/3 AKI as determined by the adjudicators. With this event rate,

we had 95% power to detect an area under the curve (AUC) as low as 0.69. Age and gender did

not differ significantly between those with or without stage 2/3 AKI; however, those with stage

2/3 AKI had more comorbidities (sepsis: 18.2% v. 5.7%, p=0.01; diabetes 48.5% v. 27.8%,

p=0.04; chronic kidney disease: 33.3% v. 10.4%, p<0.001), longer ICU stays (4 [2, 8] days v. 2

[1, 3] days, p=0.001), and higher baseline NGAL levels (EDTA plasma: 276 [155, 517] ng/ml v.

98 [70, 154] ng/ml, p<0.001; heparin plasma: 288 [150, 669] ng/ml v. 100 [68, 160] ng/ml, p <

0.001; urine: 190 [61, 1133] ng/ml v. 31 [15, 67], p < 0.001) than those without AKI (Table 1).

Using the ln(baseline EDTA plasma NGAL) as the independent variable in a logistic regression

model for the outcome of stage 2/3 AKI at any time yielded an AUC = 0.76, 95% CI 0.64-0.87,

p<0.0001 (Figure 2). Optimizing the ROC curve with respect to the distance from (0,1) resulted

in a cut point of 142.0 ng/ml, which yielded a sensitivity (SN) of 78.8%, specificity (SP) of

73.0%, positive predictive value (PPV) of 31.3%, negative predictive value (NPV) of 95.7% and

diagnostic accuracy (DA) of 73.8%. Similar results are shown for heparin plasma and urine

NGAL in Figures 2b and 2c.

Page 7 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



8



however, those with persistent stage 2/3 AKI had a higher rate of chronic kidney disease (CKD)

(36.0% v. 10.9%, p < 0.001), longer ICU stays (6 [2,8] days v. 2 [1, 3] days, p <0.001), and

higher wandering baseline NGAL values (EDTA plasma: 287 [188, 517] ng/ml v. 98 [70, 156]

ng/ml, p <0.001; heparin: 355 [178, 716] ng/ml v. 101 [68, 162] ng/ml, p<0.001; urine:

231.1[84.3, 1203] ng/ml v. 32 [15, 75] ng/ml, p<0.001) (Table 2). We utilized the wandering

ln(baseline EDTA plasma NGAL) in a logistic regression model to predict persistent stage 2/3

AKI, which yielded an AUC = 0.85, 95% CI 0.77-0.94, p<0.001. The optimal cut point for

NGAL based on the distance to (0,1) was 148.3 ng/ml (Figure 3). This value yielded SN= 84.0%,

SP=73.5%, PPV= 26.6%, NPV= 97.6%, DA=74.6%. Similar cut points and results are shown for

heparin plasma and urine NGAL in Figures 3b and 3c. Odds ratios from the regression models

are shown in Table 3.

Discussion










Page 8 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



9


plasma NGAL than urinary NGAL, with an AUC as high as 0.84. 7 Thus, we believe for ease of





The work by Nikolas et al. revealed that NGAL

even has the ability to distinguish AKI from non-progressive CKD, a feat that creatinine has

failed to accomplish.10 Further, the urine NGAL measured in the emergency department was

associated with the development of end-stage renal disease requiring dialysis in the hospital. A

2010-2011 study in western Australia, n=102, demonstrated the effectiveness of using NGAL to

predict inpatient AKI (based on risk, injury, failure, loss of kidney function, and end-stage

kidney disease classification) in a tertiary care setting. In the Australian study, the Biosite

Triage® NGAL in whole blood (Alere, Inverness Medical, Australia Inc) was found to have a

single optimal cutoff of 89 ng/ml had SN = 68%, SP = 70%, and AUC = 0.71, 95% CI 0.58-

0.84.11While the threshold was similar to that found in our EDTA and heparin samples, this





replacement therapy, and mortality.12 Marino et al.’s study of 101 emergency department

patients with sepsis confirmed that NGAL was related to (the severity of) AKI; however, NGAL

concentrations were also elevated in the absence of AKI in these patients.13 Numerous studies

have demonstrated that NGAL rises after cardiac surgery and is congruent with AKI.14,15

Page 9 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



10

Additionally, NGAL concentrations increase after contrast-induced AKI; however, it is to a

much lesser extent than from cardiac surgery or critical illness.16 An extensive literature review

and meta-analysis by Haase et al. revealed a wide range of optimal reported NGAL cutoffs for

AKI prediction (100 – 270 ng/ml).17 The optimal plasma cutoffs found in our study to detect

stage 2/3 AKI (EDTA: 142.0 ng/ml, heparin:149.5 ng/ml) were within the range reported in the

meta-analysis. Haase suggested cutoffs between 150-170 ng/ml for adults, which is slightly

higher than our findings, and may be attributable to the high rate of postoperative patients used

in the meta-analysis, as well as the varied definitions of AKI. Our study is the first among these

to demonstrate that NGAL can be used to risk predict stage 2/3 AKI and its persistence in a

cohort of ICU patients. We further demonstrated a single cutoff value performs well in terms of

decision statistics for the outcomes of stage 2/3 AKI and persistent stage 2/3 AKI.








however, the full decision statistics of an optimal cut point were not disclosed. 18 A second study


endpoint. In this study, 71 (17.4%) patients had imminent stage 2/3 AKI and the model had an



Page 10 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



11



= 37, 95% CI, 26–47% and SP= 95, 95% CI, 93–97%.19 In comparison, an NGAL EDTA

plasma level of 142.0 ng/ml in our study predicted stage 2/3 AKI and yielded SN=78.8, 95% CI

73.7-83.9% and SP=73.0, 95% CI 67.4-78.6%. Thus, for NGAL at a single cut point, the lower







Limitations






included all KDIGO stages of AKI. Our study sample was non-homogeneous in that it included

patients with sepsis, a diagnosis which has been shown to increase NGAL levels even for those

without AKI.13 Further, due to the small sample size, we could not analyze sepsis or CKD

subjects separately, rendering it impossible to determine whether or not different cutoffs were

needed. To examine confounders’ relations with (persistent) stage 2/3 AKI, we built multivariate

models; however, neither sepsis, nor CKD, nor DM were statistically significant when

Page 11 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



12

considered with NGAL simultaneously. Consequently, the adjusted odds ratio estimates changed

only slightly, and the corresponding p-values were unchanged. Perhaps this is due, in part, to the

fact that NGAL can distinguish AKI from non-progressive CKD. Additionally, the small sample

size did not permit the use of training and/or validation sets, which prohibited further

investigation of the models. However, all AUCs were at least 0.76, which indicates good

predictive ability. Finally, we did not have information on long term outcomes such as need for

renal replacement therapy or mortality which would have been useful in integrating NGAL with

serum creatinine and urine output in an in-depth assessment of patient outcomes.20

Conclusion


and urine, respectively, were found to be optimal in predicting stage 2/3 AKI in ICU patients

participating in a prospective observational study. Similarly, NGAL concentration thresholds of

148.3, 169.6, and 79.2 ng/ml in EDTA plasma, heparin plasma, and urine, respectively, were

optimal in predicting persistent stage 2/3 AKI in the same cohort of ICU patients. A larger

prospective study to provide external validation of these cut points is warranted.

Page 12 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



13

Funding


Disclosures

Elisabeth Erhardtsen, DVM, Peter Mørch Eriksen, Maria de los Angeles Lavoria are employees

of BioPorto Diagnostics A/S. The remaining authors have nothing to disclose.

Acknowledgements

The authors would like to thank the study participants who graciously shared their time and

materials for the purposes of this research.

Transparency

The lead author affirms that this manuscript is an honest, accurate, and transparent account of the

study being reported; that no important aspects of the study have been omitted; and that any

discrepancies from the study as planned (and, if relevant, registered) have been explained.

Copyright

“I, Peter A. McCullough, The Corresponding Author of this article contained within the original

manuscript which includes any diagrams & photographs within and any related or stand alone

film submitted (the Contribution”) has the right to grant on behalf of all authors and does grant

on behalf of all authors, a licence to the BMJ Publishing Group Ltd and its licencees, to permit

this Contribution (if accepted) to be published in the BMJ and any other BMJ Group products

and to exploit all subsidiary rights, as set out in our licence set out at:

http://www.bmj.com/about-bmj/resources-authors/forms-policies-and-checklists/copyright-open-

access-and-permission-reuse.” I am one author signing on behalf of all co-owners of the

Contribution.

Contributorship

PAM conceived the study. PME funded the study. AOG, MG, LG, and LWM initiated the study

design and helped with implementation. KMT conducted the statistical analysis. KMT and PAM

drafted the article. All authors contributed to the manuscript’s refinement and approved the final

version.


Page 13 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



14













likelihood ratio.

Page 14 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



15


Variable Stage 2 or 3 AKI

(n=33)

Stage 1 or No AKI

(n = 212) P-value

Age (years) 68 [56, 74] 63 [54, 73] 0.5

Gender (male) 22 (66.7%) 135 (63.7%) 0.74

Race/Ethnicity 0.86


Asian 0 (0.0%) 2 (0.9%)


Mixed 0 (0.0%) 2 (0.9%)

White 26 (78.8%) 151 (71.2%)

Hypertension 26 (78.8%) 142 (67.0%) 0.17



Type I 0 (0.0%) 2 (0.9%)

Type II 16 (48.5%) 57 (26.9%)

None 17 (51.5%) 153 (72.2%)


Sepsis 6 (18.2%) 12 (5.7%) 0.01


None 22 (66.7%) 190 (89.6%)

Stage 1 1 (3.0%) 4 (1.9%)

Stage 2 1 (3.0%) 3 (1.4%)

Stage 3A 8 (24.2%) 5 (2.4%)

Stage 3B 0 (0.0%) 8 (3.8%)

Stage 4 1 (3.0%) 2 (0.9%)


Nephrotoxin use at baseline 18 (54.6%) 88 (41.5%) 0.19

Baseline EDTA NGAL (ng/ml) 276 [155, 517] 98 [70, 154] <0.001





Page 15 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



16

injury

Variable Persistent AKI

(n=25)

No persistent

AKI (n=220) P-value

Age (years) 68 [60, 73] 63 [54, 73] 0.31

Gender (male) 16 (64.0%) 141 (64.1%) 0.99

Race/Ethnicity 0.89


Asian 0 (0.0%) 2 (0.9%)


Mixed 0 (0.0%) 2 (0.9%)

White 20 (80.0%) 157 (71.4%)

Hypertension 19 (76.0%) 149 (67.7%) 0.4



Type I 0 (0.0%) 2 (0.9%)

Type II 10 (40.0%) 63 (28.6%)

None 15 (60.0%) 155 (70.5%)


Sepsis 4 (16.0%) 14 (6.4%) 0.1


None 16 (64.0%) 196 (89.1%)

Stage 1 1 (4.0%) 4 (1.8%)

Stage 2 1 (4.0%) 3 (1.4%)

Stage 3A 6 (24.0%) 7 (3.2%)

Stage 3B 0 (00%) 8 (3.6%)

Stage 4 1 (94.0%) 2 (0.9%)


Use of nephrotoxins 14 (56.05) 92 (41.8%) 0.18

Wandering baseline EDTA NGAL (ng/ml) 287 [188,517] 98 [70,156] <0.001

Wandering baseline heparin plasma NGAL

(ng/ml) 355 [178, 716] 101 [68, 162] <0.001



Page 16 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



17


using EDTA, heparin, and urine NGAL values as predictors (odds are associated with an

approximate 2.73-fold increase in NGAL value)


Stage 2/3 acute

kidney injury

EDTA <0.001 2.8 (1.8, 4.4)

Heparin <0.001 3.2 (2.0, 4.9)

Urine <0.001 2.2 (1.7, 2.9)


acute kidney injury

EDTA <0.001 3.8 (2.3, 6.3)

Heparin <0.001 3.9 (2.3, 6.3)

Urine <0.001 2.2 (1.6, 3.0)

Page 17 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



18

References

1 KDIGO Clinical Practice Guideline for Acute Kidney Injury. Kidney Int. 2012; 2(suppl 1). 2 McCullough PA, Shaw AD, Haase M, Bouchard J, Waikar SS, Siew ED, Murray PT, Mehta




3 Chang CH, Yang CH, Yang HY, Chen TH, Lin CY, Chang SW, Chen YT, Hung CC, Fang JT,

Yang CW, Chen YC. Urinary Biomarkers Improve the Diagnosis of Intrinsic Acute Kidney

Injury in Coronary Care Units. Medicine (Baltimore). 2015 Oct;94(40):e1703. doi:

10.1097/MD.0000000000001703. PubMed PMID: 26448023; PubMed Central PMCID:

PMC4616771.

4 Xu K, Rosenstiel P, Paragas N, Hinze C, Gao X, Huai Shen T, Werth M, Forster C, Deng R,

Bruck E, Boles RW, Tornato A, Gopal T, Jones M, Konig J, Stauber J, D'Agati V, Erdjument-

Bromage H, Saggi S, Wagener G, Schmidt-Ott KM, Tatonetti N, Tempst P, Oliver JA, Guarnieri

P, Barasch J. Unique Transcriptional Programs Identify Subtypes of AKI. J Am Soc Nephrol.

2016 Dec 27. pii: ASN.2016090974. doi: 10.1681/ASN.2016090974. [Epub ahead of print]

PubMed PMID: 28028135.

5 The NGAL Test

TM [package insert]. Gentofte, Denmark. BioPorto Diagnostics A/S, 2013.

















Page 18 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



19



11 Macdonald S, Arendts G, Nagree Y, Xu X-F. Neutrophil Gelatinase-Associated Lipocalin

(NGAL) predicts renal injury in acute decompensated cardiac failure: a prospective

observational study. BMC Cardiovascular Disorders. 2012;12:8. doi:10.1186/1471-2261-12-8.

12 Zhang A, Cai Y, Wang PF, Qu JN, Luo ZC, Chen XD, Huang B, Liu Y, Huang WQ, Wu J,




PMC4754917.

13 Marino R, Struck J, Hartmann O, Maisel AS, Rehfeldt M, Magrini L, Melander O, Bergmann

A, Di Somma S. Diagnostic and short-term prognostic utility of plasma pro-enkephalin (pro-

ENK) for acute kidney injury in patients admitted with sepsis in the emergency department. J

Nephrol. 2015 Dec;28(6):717-24. doi:10.1007/s40620-014-0163-z. Epub 2014 Dec 9. PubMed

PMID: 25486879.

14 Akrawinthawong K, Shaw MK, Kachner J, Apostolov EO, Basnakian AG, Shah S, Tilak J,



Cardiorenal Med. 2013 Apr;3(1):7-16. doi: 10.1159/000346815. PubMed PMID: 23946721;








24673459.

17 Haase M, Bellomo R, Devarajan P, Schlattmann P, Haase-Fielitz A; NGAL Meta-analysis

Investigator Group. Accuracy of neutrophil gelatinase-associated lipocalin (NGAL) in

diagnosis and prognosis in acute kidney injury: a systematic review and meta-analysis. Am J

Kidney Dis. 2009 Dec;54(6):1012-24. doi:10.1053/j.ajkd.2009.07.020. Epub 2009 Oct 21.

Review. PubMed PMID: 19850388.

18 Kashani K, Al-Khafaji A, Ardiles T, Artigas A, Bagshaw SM, Bell M, Bihorac A, Birkhahn

R, Cely CM, Chawla LS, Davison DL, Feldkamp T, Forni LG, Gong MN, Gunnerson KJ, Haase

M, Hackett J, Honore PM, Hoste EA, Joannes-Boyau O, Joannidis M, Kim P, Koyner JL,

Laskowitz DT, Lissauer ME, Marx G, McCullough PA, Mullaney S, Ostermann M, Rimmelé T,

Page 19 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



20

Shapiro NI, Shaw AD, Shi J, Sprague AM, Vincent JL, Vinsonneau C, Wagner L, Walker MG,

Wilkerson RG, Zacharowski K, Kellum JA. Discovery and validation of cell cycle arrest

biomarkers in human acute kidney injury. Crit Care. 2013 Feb 6;17(1):R25. doi:

10.1186/cc12503. PubMed PMID: 23388612; PubMed Central PMCID: PMC4057242.

19 Bihorac A, Chawla LS, Shaw AD, Al-Khafaji A, Davison DL, Demuth GE, Fitzgerald R,

Gong MN, Graham DD, Gunnerson K, Heung M, Jortani S, Kleerup E, Koyner JL, Krell K,

Letourneau J, Lissauer M, Miner J, Nguyen HB, Ortega LM, Self WH, Sellman R, Shi J,

Straseski J, Szalados JE, Wilber ST, Walker MG, Wilson J, Wunderink R, Zimmerman J,

Kellum JA. Validation of cell-cycle arrest biomarkers for acute kidney injury using clinical

adjudication. Am J Respir Crit Care Med. 2014 Apr 15;189(8):932-9. doi: 10.1164/rccm.201401-

0077OC. PubMed PMID: 24559465.

20 McCullough PA, Bouchard J, Waikar SS, Siew ED, Endre ZH, Goldstein SL, Koyner JL,

Macedo E, Doi K, Di Somma S, Lewington A, Thadhani R, Chakravarthi R, Ice C, Okusa MD,

Duranteau J, Doran P, Yang L, Jaber BL, Meehan S, Kellum JA, Haase M, Murray PT, Cruz D,

Maisel A, Bagshaw SM, Chawla LS, Mehta RL, Shaw AD, Ronco C. Implementation of novel

biomarkers in the diagnosis, prognosis, and management of acute kidney injury: executive

summary from the tenth consensus conference of the Acute Dialysis Quality Initiative (ADQI).

Contrib Nephrol. 2013;182:5-12. doi:10.1159/000349962. PubMed PMID: 23689652; PubMed


Page 20 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 1a

338x190mm (300 x 300 DPI)

Page 21 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 1b

338x190mm (300 x 300 DPI)

Page 22 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 1c

338x190mm (300 x 300 DPI)

Page 23 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 2a

338x190mm (300 x 300 DPI)

Page 24 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 2b

338x190mm (300 x 300 DPI)

Page 25 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 2c

338x190mm (300 x 300 DPI)

Page 26 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 3a

338x190mm (300 x 300 DPI)

Page 27 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 3b

338x190mm (300 x 300 DPI)

Page 28 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Figure 3c

338x190mm (300 x 300 DPI)

Page 29 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from















1.73 m2


≥ 12 hours

Page 30 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



Section & Topic No Item Reported on page #

TITLE OR ABSTRACT

1 Identification as a study of diagnostic accuracy using at least one measure of accuracy

(such as sensitivity, specificity, predictive values, or AUC)

1, 2

ABSTRACT

2 Structured summary of study design, methods, results, and conclusions

(for specific guidance, see STARD for Abstracts)

2-3

INTRODUCTION

3 Scientific and clinical background, including the intended use and clinical role of the index test 4

4 Study objectives and hypotheses 4

METHODS

Study design 5 Whether data collection was planned before the index test and reference standard

were performed (prospective study) or after (retrospective study)

4, 6

Participants 6 Eligibility criteria 4-5

7 On what basis potentially eligible participants were identified

(such as symptoms, results from previous tests, inclusion in registry)

4-5

8 Where and when potentially eligible participants were identified (setting, location and dates) 4-5

9 Whether participants formed a consecutive, random or convenience series 4-5

Test methods 10a Index test, in sufficient detail to allow replication 5

10b Reference standard, in sufficient detail to allow replication 4, 5, Sup. Table 1

11 Rationale for choosing the reference standard (if alternatives exist) 4, 5, Sup. Table 1

12a Definition of and rationale for test positivity cut-offs or result categories

of the index test, distinguishing pre-specified from exploratory

6

12b Definition of and rationale for test positivity cut-offs or result categories

of the reference standard, distinguishing pre-specified from exploratory

4, 5, 6

13a Whether clinical information and reference standard results were available

to the performers/readers of the index test

4, 5

13b Whether clinical information and index test results were available

to the assessors of the reference standard

4, 5

Analysis 14 Methods for estimating or comparing measures of diagnostic accuracy 7-8, Figures

15 How indeterminate index test or reference standard results were handled 7

16 How missing data on the index test and reference standard were handled 7

17 Any analyses of variability in diagnostic accuracy, distinguishing pre-specified from exploratory

18 Intended sample size and how it was determined 7

RESULTS

Participants 19 Flow of participants, using a diagram 7 – in text

20 Baseline demographic and clinical characteristics of participants 7, Table 1

21a Distribution of severity of disease in those with the target condition Table 1

21b Distribution of alternative diagnoses in those without the target condition Table 1

22 Time interval and any clinical interventions between index test and reference standard 5

Test results 23 Cross tabulation of the index test results (or their distribution)

by the results of the reference standard

Table 1

24 Estimates of diagnostic accuracy and their precision (such as 95% confidence intervals) 7-8, Figures

25 Any adverse events from performing the index test or the reference standard 7

DISCUSSION

26 Study limitations, including sources of potential bias, statistical uncertainty, and generalisability 11

27 Implications for practice, including the intended use and clinical role of the index test 8, 11-12

OTHER

28 Registration number and name of registry 13

29 Where the full study protocol can be accessed 5

30 Sources of funding and other support; role of funders 13

Page 31 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from



STARD 2015

AIM

STARD stands for “Standards for Reporting Diagnostic accuracy studies”. This list of items was developed to contribute to the

completeness and transparency of reporting of diagnostic accuracy studies. Authors can use the list to write informative

study reports. Editors and peer-reviewers can use it to evaluate whether the information has been included in manuscripts

submitted for publication.

EXPLANATION

A diagnostic accuracy study evaluates the ability of one or more medical tests to correctly classify study participants as

having a target condition. This can be a disease, a disease stage, response or benefit from therapy, or an event or condition

in the future. A medical test can be an imaging procedure, a laboratory test, elements from history and physical examination,

a combination of these, or any other method for collecting information about the current health status of a patient.

The test whose accuracy is evaluated is called index test. A study can evaluate the accuracy of one or more index tests.

Evaluating the ability of a medical test to correctly classify patients is typically done by comparing the distribution of the

index test results with those of the reference standard. The reference standard is the best available method for establishing

the presence or absence of the target condition. An accuracy study can rely on one or more reference standards.

If test results are categorized as either positive or negative, the cross tabulation of the index test results against those of the

reference standard can be used to estimate the sensitivity of the index test (the proportion of participants with the target

condition who have a positive index test), and its specificity (the proportion without the target condition who have a negative

index test). From this cross tabulation (sometimes referred to as the contingency or “2x2” table), several other accuracy

statistics can be estimated, such as the positive and negative predictive values of the test. Confidence intervals around

estimates of accuracy can then be calculated to quantify the statistical precision of the measurements.

If the index test results can take more than two values, categorization of test results as positive or negative requires a test

positivity cut-off. When multiple such cut-offs can be defined, authors can report a receiver operating characteristic (ROC)

curve which graphically represents the combination of sensitivity and specificity for each possible test positivity cut-off. The

area under the ROC curve informs in a single numerical value about the overall diagnostic accuracy of the index test.

The intended use of a medical test can be diagnosis, screening, staging, monitoring, surveillance, prediction or prognosis. The

clinical role of a test explains its position relative to existing tests in the clinical pathway. A replacement test, for example,

replaces an existing test. A triage test is used before an existing test; an add-on test is used after an existing test.

Besides diagnostic accuracy, several other outcomes and statistics may be relevant in the evaluation of medical tests. Medical

tests can also be used to classify patients for purposes other than diagnosis, such as staging or prognosis. The STARD list was

not explicitly developed for these other outcomes, statistics, and study types, although most STARD items would still apply.

DEVELOPMENT

This STARD list was released in 2015. The 30 items were identified by an international expert group of methodologists,

researchers, and editors. The guiding principle in the development of STARD was to select items that, when reported, would

help readers to judge the potential for bias in the study, to appraise the applicability of the study findings and the validity of

conclusions and recommendations. The list represents an update of the first version, which was published in 2003.

More information can be found on http://www.equator-network.org/reporting-guidelines/stard.

Page 32 of 32


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2017-016028 on 10 July 2017. Dow

nloaded from


Documents

BMJ OpenFor peer review only STARD 2015 AIM STARD stands for “Standards for Reporting Diagnostic accuracy studies”. This list of items was developed to contribute to the completeness