Invariance Principles

in Theoretical Computer Science

Ryan ’Donnell

Carnegie Mellon University

O

1. Describe some TCS results requiring

variants of the Central Limit Theorem.

Talk Outline

2. Show a flexible proof of the CLT

with error bounds.

3. Open problems and an advertisement.

1. Describe some TCS results requiring

variants of the Central Limit Theorem.

Talk Outline

2. Show a flexible proof of the CLT

with error bounds.

3. Open problems and an advertisement.

Linear Threshold Functions

Linear Threshold Functions

Learning Theory [O-Servedio’08]

Thm: Can learn LTFs f in poly(n) time,

just from correlations E[f(x)xi].

Key:

when all |ci| ≤ ϵ.

Property Testing [Matulef-O-Rubinfeld-Servedio’09]

Thm: Can test if is

ϵ-close to an LTF with poly(1/ϵ) queries.

Key:

when all |ci| ≤ ϵ.

Derandomization [Meka-Zuckerman’10]

Thm: PRG for LTFs with seed

length O(log(n) log(1/ϵ)).

Key:

even when xi’s not fully independent.

Multidimensional CLT?

when all small compared to

For

Derandomization+ [Gopalan-O-Wu-Zuckerman’10]

Thm: PRG for “functions of O(1) LTFs”

with seed length O(log(n) log(1/ϵ)).

Key: Derandomized multidimensional CLT.

Property Testing+ [Blais-O’10]

Thm: Testing if is a

Majority of k bits needs kΩ(1) queries.

Key:

assuming E[Xi] = E[Yi], Var[Xi] = Var[Yi],

and some other conditions.

(actually, a multidimensional version)

Social Choice,Inapproximability [Mossel-O-Oleszkiewicz’05]

Thm: a) Among voting schemes where no

voter has unduly large influence,

Majority is most robust to noise.

b) Max-Cut is UG-hard to .878-approx.

Key: If P is a low-deg. multilin. polynomial,

assuming P has “small coeffs. on each coord.”

1. Describe some TCS results requiring

variants of the Central Limit Theorem.

Talk Outline

2. Show a flexible proof of the CLT

with error bounds.

3. Open problems and an advertisement.

Gaussians

Standard Gaussian: G ~ N(0,1). Mean 0, Var 1.

a + bG also a “Gaussian”: N(a,b2)

Sum of independent Gaussians is Gaussian:

If G ~ N(a,b2), H ~ N(c,d2) are independent,

then G + H ~ N(a+c,b2+d2).

Anti-concentration: Pr[ G ∈ [u−ϵ, u+ϵ] ] ≤ O(ϵ).

X1, X2, X3, … independent, ident. distrib.,

mean 0, variance σ2,

Central Limit Theorem (CLT)

CLT with error bounds

X1 + · · · + Xnis “close to” N(0,1),

assuming Xi is not too wacky.

X1, X2, …, Xn independent, ident. distrib.,

mean 0, variance 1/n,

wacky:

Niceness of random variables

Say E[X] = 0, stddev[X] = σ.

eg: ±1. N(0,1). Unif on [-a,a].

not nice:

def: (≥ σ).

“def”: X is “nice” if

Niceness of random variables

Say E[X] = 0, stddev[X] = σ.

eg: ±1. N(0,1). Unif on [-a,a].

not nice:

def: (≥ σ).

def: X is “C-nice” if

Y “ϵ-close” to Z:

Berry-Esseen Theorem

X1, X2, …, Xn independent, ident. distrib.,

mean 0, variance 1/n,

X1 + · · · + Xnis ϵ-close to N(0,1),

assuming Xi is C-nice, where

[Shevtsova’07]: .7056

General Case

X1, X2, …, Xn independent, ident. distrib.,

mean 0,

X1 + · · · + Xnis ϵ-close to N(0,1),

assuming Xi is C-nice,

Berry-Esseen: How to prove?

1. “Characteristic functions”

2. “Stein’s method”

3. “Replacement” = think like a cryptographer

X1, X2, …, Xn indep., mean 0,

S = X1 + · · · + XnG ~ N(0,1).ϵ-close to

Indistinguishability of random variables

S “ϵ-close” to G:

Indistinguishability of random variables

S “ϵ-close” to G:

u

Indistinguishability of random variables

S “ϵ-close” to G:

ut

Indistinguishability of random variables

S “ϵ-close” to G:

Replacement method

S “ϵ-close” to G:

uδ

Replacement method

X1, X2, …, Xn indep., mean 0,

S = X1 + · · · + Xn

G ~ N(0,1)

For smooth

Replacement method

X1, X2, …, Xn indep., mean 0,

G = G1 + · · · + Gn

For smooth

S = X1 + · · · + Xn

Hybrid argument

X1, X2, …, Xn indep., mean 0,

SY = Y1 + · · · + Yn

For smooth

SX = X1 + · · · + Xn

Invariance principle

Y1, Y2, …, Yn Var[Xi] = Var[Yi] =

Hybrid argument

Def: Zi = Y1 + · · · + Yi + Xi+1 + · · · + Xn

SX = Z0, SY = Zn

X1, X2, …, Xn, Y1, Y2, …, Yn, independent,

matching means and variances.

SX = X1 + · · · + Xn SY = Y1 + · · · + Ynvs.

Hybrid argument

Zi = Y1 + · · · + Yi + Xi+1 + · · · + Xn

Goal:

X1, X2, …, Xn, Y1, Y2, …, Yn, independent,

matching means and variances.

Zi = Y1 + · · · + Yi + Xi+1 + · · · + Xn

Zi = Y1 + · · · + Yi + Xi+1 + · · · + Xn

where U = Y1 + · · · + Yi−1 + Xi+1 + · · · + Xn.

Note: U, Xi, Yi independent.

Goal:

−

=

by indep. and matching means/variances!

∴

Variant Berry-Esseen: Say

If X1, X2, …, Xn & Y1, Y2, …, Yn indep.

and have matching means/variances, then

Usual Berry-Esseen:

If X1, X2, …, Xn indep., mean 0,

uδ

Hack

Usual Berry-Esseen:

If X1, X2, …, Xn indep., mean 0,

Variant Berry-Esseen

+ Hack

Usual Berry-Esseen

except with error O(ϵ1/4)

Extensions are easy!

Vector-valued version:

Use multidimensional Taylor theorem.

Derandomized version:

If X1, …, Xm C-nice, 3-wise indep., then

X1+···+ Xm is O(C)-nice.

Higher-degree version:

X1, …, Xm C-nice, indep., Q is a deg.-d poly.,

then Q(X1, …, Xm) is O(C)d-nice.

1. Describe some TCS results requiring

variants of the Central Limit Theorem.

Talk Outline

2. Show a flexible proof of the CLT

with error bounds.

3. Open problems, advertisement, anecdote?

Open problems

1. Recover usual Berry-Esseen via the

Replacement method.

2. Vector-valued: Get correct dependence

on test sets K. (Gaussian surface area?)

3. Higher-degree: improve (?) the

exponential dependence on degree d.

4. Find more applications in TCS.

Do you like LTFs and PTFs?

Do you like probability and geometry?

Oct. 21-22 (“just before FOCS”) workshopat the Princeton Intractability Center:

Analysis and Geometry of Boolean Threshold Functions

Diakonikolas! Kane! Meka! Rubinfeld! Servedio! Shpilka! Vempala! And more!

http://intractability.princeton.edu/blog/2010/08/workshop-ltfptf/