Combining Confidence Distributions

Rare Event Meta-Analysis

Brinley Zabriskie and Chris Corcoran Utah State University, Logan, UT, USA

for

Pralay Senchaudhuri Cytel Software Corporation, Cambridge, MA, USA

What Is a Meta-Analysis?

What Is a Confidence

Distribution?

Confidence Distributions

in Meta-Analysis

Innovation

What Is a Meta-Analysis?

. . . Study 1 n=326�

𝑂𝑅 ↓1 =7.63�pval=0.004

Study 2 n=44�𝑂𝑅 ↓2 =2.43�pval=0.49

Study 3 n=570�𝑂𝑅 ↓3 =2.24�pval=0.14

Study 9 n=333�𝑂𝑅 ↓6 =4.74�pval=0.06

Combined Info

𝑂𝑅 ↓𝐶 =1.80�pval=0.009

Cerebral Microbleeds (CMBs) Example

Average Treatment Event Rate: 9.3% Average Control Event Rate: 3.6%

What Is a Meta-Analysis?

Random sample of 500 systematic reviews in 2009 demonstrated that over half included outcomes with event rates of < 5%

Cerebral Microbleeds (CMBs) Example

What Is a Meta-Analysis? Cerebral Microbleeds (CMBs) Example

θ  95% CI p-value

Peto Odds Ratio

1.83 (1.17, 2.87) 0.008 (significant)

Mantel-Haenszel Odds Ratio 1.71 (1.14, 2.57) 0.010

(significant)

Inverse Variance (0.5 continuity correction)

Odds Ratio 1.80 (1.16, 2.80) 0.009

(significant)

DerSimonian and Laird (0.5 continuity correction)

Odds Ratio 2.46 (1.22, 4.97) 0.012

(significant)

Conclude the presence of CMBs may increase the risk of sICH

Simulation with Rare Events (baseline probability: 0.7%)

Rates Differ Between TRT and CTRL

Can we lower the Type I Error rate by using methods better suited for

meta-analyses with rare events?

What Is a Meta-Analysis?

Point estimate

Interval estimate

Distribution estimate

𝝁 Population Parameter

What Is a Confidence Distribution?

. . . + + +

Singh et al. (2005), Tian et al. (2009), and Liu et. al (2014)

Confidence Distributions in Meta-Analysis Cerebral Microbleeds (CMBs) Example

p↓c (θ)=F(c)[w↓1 h(p↓1 (θ))+…+w↓6 h(p↓6 (θ))]

Combining Confidence Intervals

Tian et al. (2009)

Θ risk difference p↓i (⋅) an exact method h(⋅) logit w↓i  based on sample size

of study

p↓c (θ)=F(c)[w↓1 h(p↓1 (θ))+…+w↓6 h(p↓6 (θ))]

Combining p-value Functions

Liu et al. (2014)

Θ log odds ratio p↓i (⋅) Fisher’s exact test

(mid-p) h(⋅) inverse Normal CDF w↓i  based on event rates

Combining p-values Fisher (1932)

Θ - p↓i (⋅) Fisher’s exact test (mid-p) h(⋅) log w↓i  none

Confidence Distributions in Meta-Analysis

Confidence Distributions in Meta-Analysis Cerebral Microbleeds (CMBs) Example

θ  95% CI p-value

Combining Confidence Intervals

Tian et al. (2009)

Risk Difference 0.03 (-0.0002, 0.058) 0.054

(non-significant)

Combining p-value Functions

Liu et al. (2014)

Odds Ratio 1.73 (1.12, 2.62) 0.014

(significant)

Combining p-values Fisher (1932)

- - 0.016 (significant)

Conflicting conclusions as to whether CMBs increase risk

Confidence Distributions in Meta-Analysis

Relatively Fast, ↑ Error Very Slow, ↓ Error Very Fast, ↑ Error

Rates Differ Between TRT and CTRL

Innovation

Modification 1 Liu et al. (2014)

with logit and weights

Θ log odds ratio p↓i (⋅) Fisher’s exact test (mid-p) h(⋅) logit w↓i  based on event rates

p↓c (θ)=F(c)[w↓1 h(p↓1 (θ))+…+w↓6 h(p↓6 (θ))]

Modification 2 Liu et al. (2014)

with logit and no weights

Θ log odds ratio p↓i (⋅) Fisher’s exact test

(mid-p) h(⋅) logit w↓i  none

Modification 3 Tian et al. (2009)

with Liu et al. (2014) weights

Θ risk difference p↓i (⋅) an exact method h(⋅) logit w↓i  based on event rates

Innovation

θ  95% CI p-value

Modification 1 Liu et al. (2014)

with logit and weights

Odds Ratio 1.69 (1.16, 2.33) 0.010

(significant)

Modification 2 Liu et al. (2014)

with logit and no weights

Odds Ratio 1.22 (1.37, 3.07) 0.001

(significant)

Modification 3 Tian et al. (2009)

with Liu et al. (2014) weights

Risk Difference 0.01 (-0.008, 0.054) (non-significant)

Cerebral Microbleeds (CMBs) Example

Conclude CMBs may not increase the risk of sICH

Rates Differ Between TRT and CTRL

Can we lower the Type I Error rate by

using methods better suited for

meta-analyses with rare events?

YES!

Innovation

Fairly Fast,

↓ Error

Thank you!

K. Singh, M. Xie, and W. E. Strawderman. Combining Information from Independent Sources through Confidence Distributions. The Annals of Statistics, 33(1):159-183, 2005.

D. Liu, R. Y. Liu, and M. Xie. Exact Meta-Analysis Approach for Discrete Data and its Application to 2 x 2 Tables With Rare Events. Journal of the American Statistical Association, 109(508):1450-1465, 2014.

L. Tian, T. Cai, M. A. Pfeer, N. Piankov, P. Cremieux, and L. J. Wei. Exact and efficient inference procedure for meta-analysis and its application to the analysis of independent 2 x 2 tables with all available data but without artificial continuity correction. Biostatistics, 10(2):275-281, 2009.

R. A. Fisher. Statistical Methods for Research Workers. 4 edition, 1932.