PASSING & BABLOK AND DEMING REGRESSION · 2019-09-20 · LOA . Bland-Altman plot Mean difference Mean of methods ference 0 +1.96 s LOA First method or Reference method (true value)

PASSING & BABLOK AND DEMING REGRESSION

Vanja Radišić Biljak

Department of medical laboratory diagnostics, University Hospital „Sveti Duh”, Zagreb, Croatia

PASSING & BABLOK AND DEMING REGRESSION

BLAND & ALTMAN PLOT



METHOD COMPARISON



Method verification

• Confirming that declared test specifications can be

achieved in the clinical laboratory

• Precision

• Accuracy

• Linearity

• Interferences

• LOD, LOQ...

• Method comparison

When and why?

Introducing new method or analyzer

Multiple analytical systems in laboratory

Using services of another laboratory

Method change is not going to influence

laboratory result for the patient

How?

• Experimental procedures following protocols

• CLSI EP09-A3: Measurement procedure comparison and

bias estimation using patient samples

1. Number of samples

2. Measurement range

3. Time of analysis

4. Data analysis

5. Data interpretation

Number of samples

• Min: 40 samples

• Optimal: 100 samples • To identify unexpected errors from sample matrix or

interferences

• Measurements in duplicate: use mean value

• Measurements in triplicate: use median value

Measurement range

• Cover 90% of the method measurement range

Method A

Method B

Measurement range

Good

agreement

between

methods Difference in

higher

concentration

range

Measurement range

• Overlaping measurement range for both methods

Glucose

concentration

Method A

Method B

Method A using

dilution protocol

Method B

reported as

LOQ

Data analysis

Several statistical aproaches:

•Correlation

•Paired test for difference

•Bland-Altman analysis

•Linear regression

• Deming regression

• Passing-Bablok regresion

Data analysis


•Correlation






Analyzing results

• Comparison of two methods for testosterone

concentration measurement

Summary data

nmol/L Method 1

N=122

Method 2

N=122

Analyzer E411

(Roche)

Architect 1000i

(Abbott)

Min-Max 0.297 – 3.34 0.78 – 2.83

Mean ± SD 1.38 ± 0.50 1.43 ± 0.38

Median (IQR) 1.33 (0.99 – 1.69) 1.38 (1.15 – 1.63)

P (normality) <0.001

REJECTED

<0.001

REJECTED

Scatter plot

Analyzing results



Summary data

nmol/L

Method 1

N=122

Method 2

N=122

Analyzer E411

(Roche)

Architect 1000i

(Abbott)

Min-Max 0.297 – 3.34 0.78 – 2.83

Mean ± SD 1.38 ± 0.50 1.43 ± 0.38

Median (IQR) 1.33 (0.99 – 1.69) 1.38 (1.15 – 1.63)


REJECTED

<0.001

REJECTED

Types of correlation

• Pearson (rp) and Spearman correlation (rs)

122 samples < 30 Spearman

correlation

Numerical data At least one

ordinal data

Spearman

correlation

Reject normality

Both variables

do not follow

normal

distribution

Spearman

correlation

Pearson correlation

Correlation

• Spearman coefficient of correlation

Coefficient of correlation (r)

Coefficient of

correlation (r) Interpretation

0-0.24 No association

0.25-0.49 Poor association

0.50-0.74 Moderate to good

association

0.75-1.00 Very good to excellent

association

r can be interpreted only if P < level of significance (0.05)

What is the meaning of this result?

• Methods are significantly associated

• Linear relation between methods

• ↑ of Method A associated with ↑ of Method B

• Nothing about interchangeability of

methods!

WHY?

Example 1

r (95% CI) =

1.00 (1.00-1.00)

Method 2 =

Method 1 + 10

Method 2 = Method 1

Example 2

r (95% CI) =

1.00 (1.00-1.00)

Method 3 =

1,5 x Method 1

Method 2 = Method 1

Example 3

r (95% CI) =

1.00 (1.00-1.00)

Method 4 =

1,5 x Method 1 + 20

Method 2 = Method 1

Data analysis


•Correlation






Data analysis


•Correlation






Analyzing results



Summary data

nmol/L

Method 1

N=122

Method 2

N=122

Analyzer E411

(Roche)

Architect 1000i

(Abbott)

Min-Max 0.297 – 3.34 0.78 – 2.83

Mean ± SD 1.38 ± 0.50 1.43 ± 0.38

Median (IQR) 1.33 (0.99 – 1.69) 1.38 (1.15 – 1.63)


REJECTED

<0.001

REJECTED

Significance of difference

• Wilcoxon test (normality failed)

P=0.02

Significance of difference

• Wilcoxon test (normality failed)

P=0.02

Significant

difference between

methods

What is the meaning of this result?

• Calculating differences for each pair of measurement

• Comparing number of negative and positive differences

• If there is no difference between methods, number of

differences is equal

More measurements were higher using Method 2

Data analysis


•Correlation






Data analysis


•Correlation






Bland-Altman analysis

• Graphical method to compare two measurements

technique

• Analyzing differences between measurement pairs

• Identification of constant and proportional bias

Giavarina D. Understanding Bland Altman analysis. Biochem Med (Zagreb) 2015;25(2):141-51.

Calculating differences

Average = - 0.05

Ideal situation: all differences = 0

Real situation (analytical

variability): all differences

close to 0 (some +; some -)

The average of all calculated

differences between measurement

pairs close to 0

Bland-Altman plot

Mean difference

Mean of methods

Diffe

rence

0

+1.96 s

LOA

First method or

Reference method

(true value)

Laboratory methods

(no true value)

95% CI

95% CI

-1.96 s

LOA

Bland-Altman plot

Mean difference

Mean of methods

Diffe

rence

0

+1.96 s

LOA

First method or

Reference method

(true value)

Laboratory methods

(no true value)

95% CI

95% CI

-1.96 s

LOA

Evaluating difference

Mean difference

Mean of methods

Diffe

rence

0

+1.96 s

LOA

95% CI

95% CI

-1.96 s

LOA

95% CI Including 0

No statistically significant

difference

Evaluating difference

Mean difference

Mean of methods

Diffe

rence

0

+1.96 s

LOA

95% CI

95% CI

-1.96 s

LOA

95% CI Excluding 0

Statistically significant

difference

Type of difference

Mean difference

Mean of methods

Diffe

rence in m

easure

ment

units

Meth

od 2

– M

eth

od 1

0

+1.96 s

LOA

95% CI

95% CI

-1.96 s

LOA

Evaluating

constant bias

Type of difference

Mean difference

Mean of methods

Diffe

rence in p

erc

enta

ges

(Me

thod 2

– M

eth

od 1

)/M

eth

od 1

x100%

0

+1.96 s

LOA

95% CI

95% CI

-1.96 s

LOA

Evaluating

proportional

bias

Limits of agreement

Normal distribution → 95% cases within ± 1.96 s

Limits of agreement

Mean difference

Mean of methods

Diffe

rence

0

+1.96 s

LOA

95% CI

95% CI

-1.96 s

LOA

95% of the

differences within

LOA

Limits of agreement

Mean difference

Mean of methods

Diffe

rence

0

+1.96 s

LOA

95% CI

95% CI

-1.96 s

LOA

Narrow LOA

Good agreement

Wide LOA

Poor agreement

Precision of limits of

agreement

Mean difference

Mean of methods

Diffe

rence

0

+1.96 s

LOA

95% CI

95% CI

-1.96 s

LOA

95% CI of limits of

agreement

Distribution of differences – Mountain plot


P = 0.097

Normality accepted


P = 0.097

Normality accepted

How are differences

distributed

according to

concentration level?


P = 0.097

Normality accepted

How are differences

distributed

according to

concentration level?

Bland-Altman

plot

Testosterone comparison (1)

1) Mean difference -0.05 mmol/L

2) Difference statistically

significant

3) Constant bias

4) Shape of the cloud

indicates aditional

proportional bias


1) Mean difference -0.05 mmol/L


significant

3) Constant bias

4) Shape of the cloud

indicates aditional

proportional bias


1) Mean difference -6.9%


significant

3) Proportional bias


1) Mean difference -6.9%


significant

3) Proportional bias Is this difference clinically significant?

Data interpretation

Statistical significance Clinical significance

Comparing values

with predefined

acceptance

criteria

https://www.westgard.com/biodatabase1.htm


13,6%

allowed

TE

https://biologicalvariation.eu

BA = 0.25 (CVI2 + CVG

2)1/2


Summary data

nmol/L

Method 1

N=122

Method 2

N=122

Reference

method

Analyzer E411 (Roche) Architect 1000i

(Abbott)

LCMS

Min-Max 0.30 – 3.34 0.78 – 2.83 0.54 – 2.43

Mean ± SD 1.38 ± 0.50 1.43 ± 0.38 1.22 ± 0.39

Median (IQR) 1.33

(0.99 – 1.69)

1.38

(1.15 – 1.63)

1.15

(0.94 – 1.42)


REJECTED

<0.001

REJECTED

<0.001

REJECTED

Bland-Altman plot

Mean difference

Mean of methods

Diffe

rence

0

+1.96 s

LOA

First method or

Reference method

(true value)

Laboratory methods

(no true value)

95% CI

95% CI

-1.96 s

LOA


Data analysis


•Correlation






Data analysis


•Correlation






Linear regression

High correlation

Linear relationship

Equation to describe relationship

between methods

Determine proportional and constant error

Deming regression

Passing and Bablok regression

Bilić-Zulle L. Comparison of methods: Passing and Bablok regression. Biochem Med (Zagreb) 2011;21:49-52.

Linear regression

Regression equation

y = a + bx

Intercept = a

tg (α) = b

x

y

Method B

α

Method A

95% confidence

intervals y = x

Regression equation

y = a (95% CI) +

b (95% CI) x

Constant and proportional error

Intercept = a

tg (α) = b

x

y

Method B

α

Method A

Regression equation

y = a (95% CI) + b (95% CI) x

Excluding 0

Constant

error

y = x Excluding 1

Proportional

error

Deming regression

• Includes analytical variability of both methods (CV)

• Both methods prone to errors (clinical laboratory)

There are limitations to using Deming regression!!!

Assumes that errors are independent and normally distributed!

Sensitive to outliers!




y = 0.44 (0.36 to 0.52) + 0.72 (0.65 to 0.78) x

Constant bias Proportional difference

Why don’t we recalculate results?

Testo (Architect) = 0.44+ 0.72 x Testo (e411)

Testo (e411) = (Testo (Architect) – 0.44) / 0.72

Residual analysis

• Calculation of the results using the regression equation

• Calculation of the difference between calculated and

measured value

Residual analysis

y

x

Y – F(x)

0

> 0

< 0

All measuremets near the regression line Difference between actual Y and

calculated Y close to 0

Good fit; equation can be used

Residual analysis

x

y

x0

> 0

< 0

Many measuremets far from the regression line Difference between actual Y and

calculated Y different than 0

Poor fit; equation can’t be used

Residual analysis

Differences between measured and calculated values

Passing & Bablok regression

• Non-parametric method

• No assumptions about distributions of samples

• No assumptions about distributions of errors

• Not sensitive to outliers

Confirming linear relation between methods

If linearity test failed, Passing Bablok regression should not be used!


Deming regression

y = 0.44 (0.36 to 0.52) +

0.72 (0.65 to 0.78) x

Passing & Bablok

regression

y = 0.42 (0.35 to 0.50) +

0.73 (0.66 to 0.80) x

Data analysis


•Correlation






Some examples…

Summary data

RBC

1012/L

Analyzer 1

N=300

Analyzer 2

N=300

Min-Max 1.82 – 7.15 1.82 – 7.24

Mean ± SD 4.49 ± 0.84 4.56 ± 0.85

Median (IQR) 4.50 (4.07 – 5.01) 4.58 (4.17 – 5.05)

P (normality) 0.01

REJECTED

<0.001

REJECTED

Some examples…

Some examples…

Some examples…

Some examples…

No constant nor

proportional differences

Some examples…

Summary data

HGB

g/L

Analyzer 1

N=300

Analyzer 2

N=300

Min-Max 66 – 243 64 – 245

Mean ± SD 135 ± 27 134.7 ± 26

Median (IQR) 135 (121 – 149) 135 (121.5 – 148)


REJECTED

<0.001

REJECTED

Some examples…

Some examples…

Some examples…

Some examples…

Some examples…

Some examples…

If linearity test failed, Passing Bablok regression should not be used!

What to do?

CLSI EP09-A3

CLSI EP09-A3

Some examples…

Summary data

PLT

109/L

Analyzer 1

N=300

Analyzer 2

N=300

Min-Max 8 – 1693 8 – 1566

Mean ± SD 248 ± 192 231 ± 178

Median (IQR) 215 (113 – 325) 201 (106 – 302)


REJECTED

<0.001

REJECTED

Some examples…

Some examples…

Proportional error

0 500 1000 1500 2000

0

200

400

600

800

1000

1200

1400

1600

1800

TRC-S

TR

C-N

Method comparison

• Important laboratory procedure for verification

• Included into validation protocols for new reagents

• Comparison with the reference method

• Comparison with different manufacturers

• Comparison with same manufacturer

• Results are presented in manufacturers declarations

P values missing

95%CI missing

Small number of

samples

P values missing

95%CI missing

P values missing

95%CI missing

P values missing

95%CI missing

P values missing

95%CI missing

THE END

Documents

PASSING & BABLOK AND DEMING REGRESSION · 2019-09-20 · LOA . Bland-Altman plot Mean difference Mean of methods ference 0 +1.96 s LOA First method or Reference method (true value)