Event-Based Assessments of Sensor Accuracy during

Exercise-Induced Reductions in Glucose in Children with

Type 1 Diabetes

Craig Kollman,1 Darrell Wilson,2 Roy Beck1 and William Tamborlane3 for the Diabetes Research in

Children Network (DirecNet) Study Group 1. Tampa, FL; 2. Stanford, CA; 3. New Haven, CT;

Event-Based Assessments of Sensor Accuracy during Exercise-Induced Reductions in Glucose in Children with Type 1 Diabetes Craig Kollman,1 Darrell Wilson,2 Roy Beck1 and William Tamborlane3 for the Diabetes Research in Children Network (DirecNet) Study Group 1Tampa, FL; 2Stanford, CA; 3New Haven, CT;.

Currently, the accuracy of near-continuous glucose sensors is typically assessed using point-by-point comparisons. While useful in evaluating meters, these methods can be skewed by important temporal aspects of hypo- or hyperglycemic excursions.

To compare event-based vs. point-by-point analyses of accuracy during exercise-induced falls in plasma glucose, we studied 22 children with T1D 8-17 y who wore a Freestyle Navigator Continuous Glucose Monitoring System (Abbott Diabetes Care, Alameda, CA) during a 24h CRC admission that included an exercise session. Blood samples for reference laboratory glucose measurements were taken q15 min during exercise. Three subjects were excluded from analysis because the blood samples were insufficient for laboratory measurements (N=1) or the glucose did not fall during exercise (N=2).

Point-by-point analysis resulted in a median relative absolute difference (RAD) of 17% during exercise compared with 12% during non-exercise. Event analysis looking at the entire glycemic excursion (baseline to nadir) shows this discrepancy was primarily accounted for by a slower rate of fall in Navigator vs reference values (median 0.8 vs 1.4 mg/dL/min; p<0.001) and by a 5-30 min lag between the nadir Navigator and reference glucose levels (median 100 vs 75 min, respectively p<0.001). The sensor-measured fall was similar to that of the reference in each patient (figure). Point-by-point estimates of error can be unduly inflated by sensor lag during periods of rapid change. Event-based methods offer a simple way to incorporate the time dimension into analyses without a complicated or arbitrary adjustment for lag.

Abstract

Introduction• Many techniques used to assess the

accuracy of near-continuous glucose sensors are borrowed from methods developed for meters.

• Rely on point-by-point comparisons.

• Do not assess trends over time.

• Event-based methods intended to incorporate the time element into analyses in a clinically meaningful way.

Point-by-Point Analysis

• Each glucose reading is paired to a reference measurement (within a few minutes).

• Sensor readings often adjusted for a fixed time lag.

• Difference measures calculated for each pair used to describe accuracy.

Point-by-Point AnalysisExample with Hourly Reference Sampling

100

125

150

175

200

4:00 4:10 4:20 4:30 4:40 4:50 5:00 5:10 5:20 5:30 5:40 5:50 6:00

Time

Glu

cose

(m

g/d

L)

SensorReference

Pair #1

Pair #2 Pair #3

Point-by-Point Analysis Example Measures of Accuracy

• Difference: Sensor minus reference.• Absolute Difference: Absolute value of

Difference (always positive).• Relative Absolute Difference (RAD): Absolute

Difference divided by reference (expressed as a percentage).

• ISO Criteria: If reference ≤75 mg/dL sensor within ±15mg/dL; if reference>75 mg/dL sensor within ±20%.

• Error Grid: (Clarke or Consensus): Zone A or B.

Point-by-Point AnalysisPrevious Example Revisited

Pair #1 Pair #2 Pair #3

Sensor Glucose 156 170 151

Reference Glucose 139 140 165

Difference +17 +30 –14

Absolute Difference 17 30 14

RAD 12% 21% 8%

ISO Criteria Met Yes No Yes

Zone A or B Yes Yes Yes

Point-by-Point AnalysisConsensus Error Grid

Event Analysis• Combine multiple points into clinically

meaningful events.• Define events to capture more of the time

element.• Example events would be hypo- or

hyperglycemic episodes or hypo- or hyperglycemic excursions.

• Define difference measures based on the entire event rather than individual points.

• Requires more frequent reference sampling.

Event AnalysisDefining Excursions

• Define the hypoglycemic excursion as the drop in glucose from baseline to nadir.

• Calculate separately for sensor vs. reference.

• Calculate rate of change from baseline until the nadir.

• Note the sensor and reference nadirs will often occur at different times. Rates of change therefore calculated over slightly different time periods.

Event AnalysisExample Comparison of Hypoglycemic Excursions

100

125

150

175

200

4:00 4:10 4:20 4:30 4:40 4:50 5:00 5:10 5:20 5:30 5:40 5:50 6:00

Time

Glu

co

se

(m

g/d

L)

SensorReference

Reference Nadir

Sensor Baseline

Sensor Nadir

Reference Baseline

Event AnalysisCalculate Excursion Summaries from Previous Graph

Sensor Reference

Baseline Glucose 184 170

Nadir Glucose 126 109

Drop in Glucose 184-126=58 170-109=61

Minutes to Nadir 80 50

Rate of Change* 58÷80=0.7 61÷50=1.2

* mg/dL/minute.

Event AnalysisHypoglycemic Events

• Multiple glucose readings below the hypoglycemic threshold (e.g., 70 mg/dL) considered part of the same event.

• Calculation of false positive and sensitivity rates based on events rather than individual points.

• Allow ±10 mg/dL margin of error (e.g., sensor=68/reference=72 not counted as a false positive).

• Allow ± 30 minute range from beginning of event to confirm a sensor low or detect a low reference.

• Analogous rules for hyperglycemic events.

Illustration Using Actual Data

• DirecNet Pilot Study of the FreeStyle Navigator™ Continuous Glucose Monitoring System, (Abbott Diabetes Care, Alameda, CA).

• One of the objectives was to assess accuracy.

• Type 1 diabetes for at least 1 year.• Age 3 to <18 years.• Using a downloadable insulin infusion

pump for a least 6 months.

Study Procedures• Admitted to Clinical Research Center

– Approximately 24 hours

• Exercise session for those older than 7 yrs– Four 15-min sessions walking on a treadmill at a

target heart rate 140 bpm interspersed with three 5-min rest breaks

• Venous blood sampling– Just prior to start of exercise (Baseline)– Each rest period– End of exercise

• Reference glucoses measured at DirecNet Central Laboratory.

Point-by-Point Analysis

N=19 subjects Median (25th, 75th percentiles)

During Other

Exercise Times

N=97 pairs N=1,203 pairs

Difference +17 (+3, +27) 0 (-16, +16)

Absolute Difference 22 (12, 33) 16 (8, 28)

RAD 17% (9%, 26%) 12% (6%, 20%)

ISO Criteria 58% 75%

Event AnalysisHypoglycemic Excursions during Exercise

-25

0

25

50

75

100

125

150

175

200

-25 0 25 50 75 100 125 150 175 200

Reference

Navigator

N=19 SubjectsDrop in Glucose from Baseline to Nadir (mg/dL)

Event AnalysisHypoglycemic Excursions during Exercise

-0.5

0.0

0.5

1.0

1.5

2.0

2.5

-0.5 0.0 0.5 1.0 1.5 2.0 2.5

Reference

Navigator

N=19 SubjectsRate of Change from Baseline to Nadir (mg/dL/minute)

Event AnalysisHypoglycemic Excursions during Exercise

N=19 subjects

median (25th-75th percentiles)

Navigator Lab Difference Abs. Diff.

Baseline 161 172 -17 (-26, +18) 23 (17, 36)

Glucose Dropa 82 86 -11 (-27, +4) 17 (8, 28)

Minutes to Nadir 100 75 +3 (-7, +14) 22 (13, 42)

Rate of Changeb 0.8 1.4 -0.4 (-0.8, -0.2) 0.4 (0.2, 0.8)

a-baseline minus nadir; b-mg/dL/minute (timed from baseline to nadir)

Event AnalysisDetection of Hypo- and Hyperglycemic events

Sensitivity False Positive

Inpatient

Hypoglycemia* 49% 37%

Hyperglycemia** 83% 18%

Outpatient

Hypoglycemia* 62% 28%

Hyperglycemia** 87% 12%

* Multiple readings ≤70 mg/dL within 30 min treated as same event.** Multiple readings ≥200 mg/dL within 30 min treated as same event.

Remarks• Point-by-point analysis may give the

impression that the Navigator did not perform well during exercise.

• However, the Navigator generally tracked the exercise-induced drops in glucose fairly well.

• Rate of change was less when estimated by the Navigator which may have reflected the longer time necessary for the compartment it measures to reach the nadir.

Summary• Event-based analyses can be used to

supplement point-by-point comparisons to further describe the clinical utility of near-continuous glucose sensors.

• Requires more frequent reference sampling than is necessary for point-by-point analysis.

• Comparison of glucose excursions does not require adjustment for a fixed lag time.

• Amount of lag can vary by subject and within the same subject over time.

Contact Information

• Craig Kollman

• Address: Jaeb Center for Health Research

15310 Amberly Dr. – Suite 350

Tampa, FL 33647

• E-mail: ckollman@jaeb.org

• Phone: 813-975-8690

• Barbara Davis Center– H. Peter Chase– Rosanna Fiallo-Scharer– Laurel Messer– Barbara Tallant

• University of Iowa– Eva Tsalikian– Michael Tansey– Linda Larson– Julie Coffey– Joanne Cabbage

• Nemours Children’s Clinic– Tim Wysocki– Nelly Mauras– Larry Fox– Keisha Bird– Kim Englert

• Stanford University– Bruce Buckingham– Darrell Wilson– Jennifer Block– Paula Clinton

• Yale University– William Tamborlane– Stuart Weinzimer– Elizabeth Doyle– Melody Martin– Amy Steffen

• Jaeb Center for Health Research– Roy Beck– Katrina Ruedy– Craig Kollman– Dongyuan Xing– Cynthia Stockdale