On Sensitivity Value of Pair-MatchedObservational Studies
Qingyuan Zhao
Department of Statistics, University of Pennsylvania
August 2nd, JSM 2017
Manuscript and slides are available athttp://www-stat.wharton.upenn.edu/~qyzhao/.
1/12
Sensitivity Analysis
I Observational studies = Treatment is not randomized.
I The core but unverifiable assumption: treatment ignorability,aka no unmeasured confounding. (Fisher’s criticism of“smoking causes lung cancer”.)
I Sensitivity analysis: what if this assumption is violated (in acontrolled way captured by one or a few sensitivityparameters)?
I There is a long list of approaches of sensitivity analysis. I willconsider Rosenbaum’s sensitivity analysis for a pair-matchedstudy. (Cornfield’s response to Fisher.)
2/12
What does a sensitivity analysis look like?
I Sensitivity parameter Γ ≥ 1 in Rosenbaum’s model: withineach matched pair,
1/Γ ≤ odds ratio(1st unit treated, 2nd unit treated) ≤ Γ.
I Γ = 1 corresponds to ignorable treatment. Can test the sharpnull hypothesis by e.g. Wilcoxon’s signed rank test.
I When Γ > 1: Rosenbaum obtained lower and upper bounds ofthe p-value of any signed score test.
I An example:
probe set sensitivity analysis sensitivity value
37583 atΓ 1 2 3 5 7 10 1.84 2.44pΓ 0.00 0.02 0.13 0.60 1.00 1.00 0.01 0.05
3/12
Sensitivity value
Definition (“Truncated” sensitivity value)
Γ∗∗α = inf
{Γ ≥ 1 | pΓ > α
}.
Interpretation: If the unmasured confounder changes thewithin-pair odds ratio of treatment by more than Γ∗∗
α , then thesharp null hypothesis could be not significant.
I Γ∗∗α > 1 iff p1 = p1 ≤ α. What if the null hypothesis is not
significant even when Γ = 1?
I Mathematically, pΓ can be defined for 0 < Γ < 1 as well. It ismore convenient to work with Γ∗
α that takes the infimum overΓ > 0.
4/12
Some motivations I
I Sensitivity value is a concise summary of the study’s“sensitivity to measured confounding”.
I A value vs. A table.
I Analogy: p-value for a randomized experiment vs. sensitivityvalue for an observational study.
5/12
Some motivations II
I Asymptotics of sensitivity value ↔ Power of sensitivityanalysis:
P(Γ∗α ≥ Γ) = P(pΓ ≤ α).
I The “favorable situation”: no unmeasured confounding andnonzero causal effect.
I Fixed Γ asymptotics: Rosenbaum [2015] considered theBahadur efficiency of a sensitivity analysis by studying howfast pΓ → 0.
I Fixed α asymptotics: examine the distribution of Γ∗α.
6/12
Background
I A general and common strategy is to use the signed score test(Yi is the within-pair difference)
T =
∑Ii=1 sgn(Yi )qi∑I
i=1 qi
, qi = ψ
(rank(|Yi |)
I + 1
).
I Rosenbaum found bounding variable T̄Γ in the sense that
P(T ≥ t|F) ≤ P(T Γ ≥ t|F) = pΓ,
I CLT for T̄Γ:
√I · T Γ − Γ/(1 + Γ)√
Γ/(1 + Γ)2σ2q,I
d→ N(0, 1), σ2q,I =
I−1∑I
i=1 q2i(
I−1∑I
i=1 qi
)2.
7/12
CLT for sensitivity value
Theorem (Z, 2017)Suppose Yi
i.i.d.∼ F and√
I · (T − µF )/σFd→ N(0, 1), the transformed
sensitivity value κ∗α = Γ∗α/(1 + Γ∗
α) for fixed 0 < α < 1 satisfies
√I ·[κ∗α − µF
]d→ N
(− σqΦ̄−1(α)
√µF (1− µF ), σ2
F
).
κ*
µFc(µF, σq) I
σF I
8/12
Design sensitivity
κ*
µFc(µF, σq) I
σF I
I Rosenbaum [2004] noticed a phase transition at µF :I If κ > µF , sensitivity analysis has no asymptotic power.I If κ < µF , power → 1 as I →∞.I He calls the value µF/(1− µF ) “design sensitivity”.
I The CLT for sensitivity value separates the contribution bydesign and by obtaining more sample.
8/12
Design sensitivity
κ*
µF
I Rosenbaum [2004] noticed a phase transition at µF :I If κ > µF , sensitivity analysis has no asymptotic power.I If κ < µF , power → 1 as I →∞.I He calls the value µF/(1− µF ) “design sensitivity”.
I The CLT for sensitivity value separates the contribution bydesign and by obtaining more sample.
8/12
Design sensitivity
κ*
µF
I Rosenbaum [2004] noticed a phase transition at µF :I If κ > µF , sensitivity analysis has no asymptotic power.I If κ < µF , power → 1 as I →∞.I He calls the value µF/(1− µF ) “design sensitivity”.
I The CLT for sensitivity value separates the contribution bydesign and by obtaining more sample.
9/12
Implication: select statistics
I The goal: maximize Γ∗α stochastically.
√I ·[κ∗α − µF
]d→ N
(− σqΦ̄−1(α)
√µF (1− µF ), σ2
F
).
I The expecation of Γ∗α is determined by two quantities:
1. µF =< ψ, gF >;2. σq = ‖ψ‖2
2/‖ψ‖21.
I Tradeoff: want to maximize µF without making σq too large.
I Manuscript has detailed comparison of different choices of ψunder different F .
I Practically, can estimate µF by sample splitting and thendecide.
10/12
Implication: select subgroupsI Hsu, Small, and Rosenbaum [2013] noticed a dilemma.
Suppose there are two subgroups with unequal µF .Heuristically,
I If I →∞, should use the subgroup with larger µF .I If I is small, should combine the samples.
I Can use the asymptotics to compute the critical sample size.I For example, for Noether’s statistics ψ(u) ≡ 1 (ratio of
sample sizes below: 1:1 and 3:1).
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●●●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2
prefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer both
10
30
100
300
1000
3000
10000
−0.2 −0.1 0.0 0.1 0.2µF1 − µF2
criti
cal s
ampl
e si
ze
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1
prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2
prefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer both
10
30
100
300
1000
3000
10000
−0.2 −0.1 0.0 0.1 0.2µF1 − µF2
criti
cal s
ampl
e si
ze
10/12
Implication: select subgroupsI Hsu, Small, and Rosenbaum [2013] noticed a dilemma.
Suppose there are two subgroups with unequal µF .Heuristically,
I If I →∞, should use the subgroup with larger µF .I If I is small, should combine the samples.
I Can use the asymptotics to compute the critical sample size.I For example, for Noether’s statistics ψ(u) ≡ 1 (ratio of
sample sizes below: 1:1 and 3:1).
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●●●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2
prefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer both
10
30
100
300
1000
3000
10000
−0.2 −0.1 0.0 0.1 0.2µF1 − µF2
criti
cal s
ampl
e si
ze
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1
prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2
prefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer both
10
30
100
300
1000
3000
10000
−0.2 −0.1 0.0 0.1 0.2µF1 − µF2
criti
cal s
ampl
e si
ze
11/12
Implication: select hypotheses
I One treatment and hundreds of outcomes that are susceptibleto unmeasured confounding.
I Can use sensitivity value to screen causal hypotheses.
I The manuscript has an application to genomics screening.Wilcoxon (8,7,8) (8,6,7)
−4 −2 0 2 4 −4 −2 0 2 4 −4 −2 0 2 4
0.00
0.25
0.50
0.75
1.00
Quantile of N(0, 1)
Sam
ple
quan
tile
of κ
*
histogram
0 5 10 15
0.00
0.25
0.50
0.75
1.00
density
Wilcoxon
(8,7,8)
(8,6,7)
I Zhao, Small, and Rosenbaum [2017] proposed a relatedmethod called “Cross Screening”.
12/12
References
I Manuscript:I Q. Zhao. On sensitivity value of pair-matched observational
studies. arXiv:1702.03442.I Additional references:
J. Y. Hsu, D. S. Small, and P. R. Rosenbaum. Effect modification and designsensitivity in observational studies. Journal of the American StatisticalAssociation, 108(501):135–148, 2013.
P. R. Rosenbaum. Design sensitivity in observational studies. Biometrika, 91(1):153–164, 2004.
P. R. Rosenbaum. Bahadur efficiency of sensitivity analyses in observationalstudies. Journal of the American Statistical Association, 110(509):205–217,2015.
Q. Zhao, D. S. Small, and P. P. Rosenbaum. Cross-screening in observationalstudies that test many hypotheses. arXiv:1703.02078, 2017.