Healthcare Across Borders - September 2003 Head-To-Head Comparison Of The Two Currently Available Continuous Monitors North County Endocrine 700 West El

Healthcare Across Borders - September 2003

Head-To-Head Comparison Of The Two Head-To-Head Comparison Of The Two Currently Available Continuous MonitorsCurrently Available Continuous Monitors

North County EndocrineNorth County Endocrine

700 West El Norte Pkwy700 West El Norte Pkwy

Escondido, CA 92126Escondido, CA 92126

(760) 743-1431 (760) 743-1431

The Diabetes Mall

www diabetesnet.com

(619) 497-0900 (619) 497-0900 [email protected]@diabetesnet.com

John Walsh, P.A., C.D.E., and Ruth Roberts, M.A.John Walsh, P.A., C.D.E., and Ruth Roberts, M.A. January, 2007January, 2007

http://www.diabetesnet.com/





Dexcom STS Monitor

FDA release 3/27/06 Approved for 18 and olderReadings every 5 min. 3-14 days of readings per sensorOne high and two low alerts

$800 (often less) + $35 per “3” day sensorTransmitter replacement: ~$250 every 6 mos

Transmitter ~0.8 x 1.5”


Medtronic Paradigm RT

NOT a closed loop

FDA release: 4/13/06 Paradigm 522/722 pump Sensor connected to transmitter by wireApproved for 18 and olderOne high and one low alert

Trend arrowsReadings every 5 min. $999 + $35 per “3 day” sensor Transmitter replacement: ~$900 every 6 mos


Continuous Monitors Compared

The Dexcom STS & Paradigm RT continuous monitors are currently available in the U.S.This study was designed as a head to head comparison while being worn by one person with Type 1 diabetes.An Ultra meter was used as the standard against which each monitor’s accuracy was tested.

Each monitor’s screen is shown above over the same 3 hour time span over a glucose range of 0 to 400 mg/dl. The Ultra meter reading at the time read 73 mg/dl compared to 93 and 122 mg/dl.


Research Design

1. The low alert in each monitor was set at 80 mg/dl and the high alert at 160 mg/dl

2. 262 readings from an Ultra meter were performed over 33 days (7.94 tests per day)

3. Simultaneous readings from each monitor were compared to the Ultra’s readings

4. Testing with the Ultra meter was performed:• As soon as either monitor’s low or high alert sounded• When values between the monitors disagreed• And at routine intervals, including calibrations


Calibrations

Each monitor required 2 calibrations with an Ultra glucose meter each day to reduce “drift” or loss of accuracy over time.

Calibrations were generally done when the glucose level was stable.

Calibrations were usually done on both monitors when requested by one of them.

In all, 3 to 4 calibrations were done per day on both monitors.

Care was taken not to do more than 4 calibrations per day on either monitor due to the recommendation by one of the manufactures that no more than 4 be done per day. (This recommendation makes no sense to us, though we have discussed it with various engineering and sales personnel from the company.)

Accuracy of the other monitor appears to improve when more calibrations are done from observations before and after the study.


Readings Sorted By Ultra BG Values

Red = first to detect a low Yellow = first to detect a high Purple = over 50 mg/dl off

Sorted By Consecutive Readings

Minor note: Sensor B always gives results in even numbers for reasons we do not understand. However, an odd number would sometimes result when a second reading was displayed before the Ultra test was complete and the two readings were averaged.

Date Time Ultra Sensor A Sensor BNew sensor A

New sensor B13-Dec 18:36 140 135 10213-Dec 20:54 181 164 12813-Dec 22:01 177 194 16014-Dec 9:10 109 220 4614-Dec 19:25 67 80 8014-Dec 20:07 57 60 7614-Dec 22:19 151 138 16414-Dec 22:43 164 164 25815-Dec 2:34 99 86 11015-Dec 2:40 97 80 10815-Dec 8:03 65 67 8615-Dec 9:23 61 51 7215-Dec 11:24 47 67 9015-Dec 11:28 43 57 7815-Dec 20:22 98 113 10415-Dec 20:57 60 80 7815-Dec 23:06 53 59 6216-Dec 4:40 164 160 16016-Dec 7:28 91 80 10616-Dec 8:13 65 58 7816-Dec 8:22 56 51 7016-Dec 9:19 95 88 8816-Dec 11:11 69 77 9016-Dec 11:19 58 75 7816-Dec 12:18 81 78 6416-Dec 13:00 77 72 7116-Dec 13:36 198 168 118


Readings Sorted By Ultra BG Values


New sensor B28-Dec 15:38 42 45 6815-Dec 11:28 43 57 7829-Dec 6:52 46 63 7815-Dec 11:24 47 67 9012-Dec 12:53 50 74 12411-Dec 23:32 50 70 88

3-Oct 21:24 51 43 7810-Dec 15:45 52 50 6615-Dec 23:06 53 59 6231-Dec 17:27 54 62 8031-Dec 14:34 55 65 9216-Dec 8:22 56 51 7031-Dec 14:48 56 52 7829-Sep 19:34 56 54 9514-Dec 20:07 57 60 76

1-Oct 20:29 57 68 4411-Dec 23:40 57 65 7616-Dec 11:19 58 75 7816-Oct 16:15 58 40 7829-Sep 19:52 58 50 65

7-Oct 11:39 59 55 7630-Sep 16:37 60 67 9015-Dec 20:57 60 80 78

3-Oct 21:18 60 54 8415-Dec 9:23 61 51 72


New sensor B9-Oct 22:17 341 344 358

11-Oct 3:23 297 230 18814-Oct 17:03 286 276 16614-Oct 16:08 282 280 144

2-Oct 7:25 278 294 2729-Oct 20:42 273 249 196

29-Dec 11:38 271 201 16414-Oct 15:40 252 234 14229-Dec 12:51 237 230 27031-Dec 22:18 236 209 22229-Dec 13:58 233 159 23830-Sep 0:55 231 163 15830-Sep 11:27 229 144 22417-Oct 9:55 227 217 16414-Oct 11:50 218 231 130

8-Oct 13:42 217 210 1648-Oct 11:15 212 218 202

10-Dec 16:20 211 207 13810-Dec 19:11 211 231 220

9-Oct 20:17 211 250 16428-Dec 13:34 210 168 11612-Dec 20:09 210 212 16816-Oct 19:20 209 221 16214-Oct 17:58 208 206 14010-Oct 17:17 205 219 172

Red = first to detect a low Yellow = first to detect a high Purple = over 50 mg/dl off

Low to High High to Low


Plotted Readings With Trend LinesUltra

0

100

200

300

400

0 50 100 150 200 250 300 350 400Sensor A

0

100

200

300

400

0 100 200 300 400

Ultra

Sensor B

Monitor A shows linearity from 50 mg/dl up to at least 240 mg/dl.

Monitor B shows more scattered values and “flattens out” with higher low readings and lower high readings than the Ultra.

Ultra

Ultra

A

B


GlycensitTM Analysis

Blue dotted lines = ISO standardsYellow area = where 95% of data points will fallRed lines = minimum and maximum deviation by star pointsIdeally, all readings would fall between the blue dotted lines -- this

is the standard for today’s glucose meters

A B

http://tomcatbackup.esat.kuleuven.be/GLYCENSIT/


Difference From Ultra Meter Readings For Each SensorSensor A Sensor B

0-9 mg/dl off 116 5210-19 mg/dl off 83 6220-29 mg/dl off 32 4630-39 mg/dl off 11 3640-49 mg/dl off 9 2750-59 mg/dl off 4 1460-69 mg/dl off 2 7over 70 mg/dl off 5 18

Monitor Accuracy

Table shows how much each monitor’s glucose values differed from a simultaneous reading on the Ultra meter.

44.3%31.7%12.2%

4.2%3.4%1.5%0.8%1.9%

19.8%23.7%17.6%13.7%10.3%

5.3%2.7%6.9%

76.0%

19.8%

4.2%

43.5%

41.6%

14.9%


Avg. Diff. in mg/dl and % between Sensor and Ultra Meter at different BG rangesUltra Sensor A Sensor B

BG Range # of Readings Avg. BG Diff in mg/dl Diff in % Diff in mg/dl Diff in %0-80 mg/dl 66 64.74 3.23 5.00% 21.94 33.89%81-160 mg/dl 127 117.07 -0.12 0.00% -3.02 -2.58%161-240 mg/dl 61 188.7 -5.97 -3.16% -32.25 -17.09%241 or over mg/dl 8 285 -21.5 -7.54% -81.25 -28.51%

Monitor Accuracy

Monitor A read slightly high (5%) for readings below 80, while B’s readings were 34% high for this range.

Between 81 and 160, both sensors were within 3% of the Ulta.

Between 161 and 240, A averaged 3% lower than the Ultra, while B averaged 17% lower.

Above 240, there are too few readings to judge accuracy, but both monitors appear to read lower, although the difference was more pronounced for Monitor B.


A B

BGs < 80

BGs 80-160

BGs > 160

BGs < 80

BGs 80-160

BGs > 160

Monitor Accuracy

This GlycensitTM graphic summarizes data shown on the previous slide. The average reading for Monitor A, shown by the middle of the spool-shaped objects for low, middle, and high, is much closer to that of the Ultra meter (the green line in the middle) with less variation (the height of the spool the standard deviation) than Monitor B. The shorter the spool and the closer it’s middle is to the line the more accurate the Monitor.

http://tomcatbackup.esat.kuleuven.be/GLYCENSIT/

Lower than the Ultra

Higher than the Ultra


Which Monitor Alerted First?

One of the most important functions of a continuous monitor is to accurately warn of low and high readings. This table shows which monitor alerted at least 5 min earlier for lows and highs.

Monitor A was first to alert for a reading below 80 mg/dl 76% of the time, Monitor B was first 3% of the time, and 21% were ties.

For detection of highs above 160, Monitor A was first 68% of the time, Monitor B was first 5% of the time, and 27% were ties.

Monitor A Monitor B Tie

For BGs less than 80 28 1 8

For BGs over 160 25 2 10

Which Monitor Alerted First?


Time Advantge For First Alerts

Time Advantage To First Alert For Each MonitorReadings Less Than 80 mg/dl

Advantage Details

A & B 8 ties

A 19* firsts – time advantage averaged 30.3 min with median of 18 min

A 9 additional firsts – no alert given by B

B 1 first – no alert given by A

Readings Above 160 mg/dl

A & B 10 ties

A 17** firsts – time advantage averaged 32.1 min with median of 27 min

A 8 additional firsts – no alert given by B

B 2 firsts with time advantage of 11 and 14 minutes

* Time not recorded for 2 of these events ** Time not recorded for 4 of these


Time Delays For Low And High Alerts

In situations where Monitor A was first to alert*, the time delays for Monitor B to give an alert are shown.

* confirmed with the Ultra meter

** Readings were maintained below 80 mg/dl for long periods to test these delays.

Minutes Of Advance Warning

Less than 80 mg/dl Over 160 mg/dl

+ 15 min + 33 min

+ 6 + 49

+ 14 + 83

+ 80 ** + 24

+ 90 ** + 42

+ 8 + 36

+ 18 + 37

+ 17 + 7

+ 20 + 27

+ 45 ** + 24

+ 8 + 19

+ 8 + 20

+ 19 + 16

+ 16 Avg = 32.1 min

+ 80 ** Median = 27 min

+ 14

+ 57 ** Avg = 30.3 m Med 17


Problems Needing Answers To Close The Loop

Sensor inaccuracyNot reliableSlow insulin

Lifestyle variabilityNo control algorithm

Sensor lag timeInsulin overdose

Glucose

Engineering / MaterialsInternal Checking

Viaject / Oral insulinRapid Prediction

Software / TestingEngineering

Dual delivery / Glucagon

Insulin


Findings

Neither continuous monitor has any hope for use in a closed loop system at this time, especially Monitor B.

In our single case study, Monitor A when compared with Monitor B was noticeably:

• More accurate• Quicker to detect changes in glucose• Quicker to warn of low and high glucose readings

Monitor B was slower to respond to changes in glucose and tended to have “flat” readings that were more likely to read high for lows and low for highs.

This study involved only one individual, so each monitors may perform differently in others.

Additional head-to-head comparisons of different continuous monitors are recommended.


Addendum

To eliminate a possible equipment problem as the cause for poor accuracy with Monitor B, two different transmitters and receivers were used over two different periods. The first combo was used for 19 days in late September and early October of 2006, and the second for 14 days in December, 2006. This change of equipment did not improve accuracy.

Sensors for both monitors have been shown to maintain accuracy when used longer than 3 days. Over the 33 days of the study, 6 different sensors were used for Monitor A and 9 sensors were used for Monitor B. This had no impact on accuracy.

Two sensors (one for each monitor) were replaced within a few hours of each other in late September after 4 bad readings (over 50 mg/dl off) were received from each. These readings were also included with no real effect on overall accuracy.

Documents

Healthcare Across Borders - September 2003 Head-To-Head Comparison Of The Two Currently Available Continuous Monitors North County Endocrine 700 West El