Pseudorandomness from Shrinkage David Zuckerman University of Texas at Austin Joint with Russell...

Pseudorandomness from Shrinkage

David ZuckermanUniversity of Texas at Austin

Joint with Russell Impagliazzo and Raghu Meka

Randomness and Computing

• Randomness extremely useful in computing.– Randomized algorithms– Monte Carlo simulations– Cryptography– Distributed computing

• Problem: high-quality randomness expensive.

What is minimal randomness requirement?

• Can we eliminate randomness completely?• If not:–Can we minimize quantity of randomness?–Can we minimize quality of randomness?• What does this mean?

What is minimal randomness requirement?

• Can we eliminate randomness completely?• If not:–Can we minimize quantity of randomness?• Pseudorandom generator

–Can we minimize quality of randomness?• Randomness extractor

Pseudorandom Numbers

• Computers rely on pseudorandom generators:

PRG71294 141592653589793238

short random string

long “random-enough”string

What does “random enough” mean?

Modern Approach to PRGs[Blum-Micali 1982, Yao 1982]

Alg

Alg

random

pseudorandom

≈ samebehavior

Require PRG to “fool” all efficient algorithms.

Which efficient algorithms?

• Most functions fool all polynomial-time circuits.– Construct explicitly?

• Poly-time PRG fooling all polynomial-time circuits implies NP≠P.

• So either:– Make unproven assumption.– Try to fool interesting subclasses of algorithms.

Two Major Challenges

1. Prove circuit lower bounds.– EXP does not have poly-size circuits.

2. Derandomize algorithms.

• Hardness vs. Randomness paradigm– (1) implies (2) [Nisan-Wigderson, BFNW,…]– Almost equivalent [Kabanets-Impagliazzo …]

Pseudorandom Generators

• PRG fools class F of functions if|Pr[f(Un)=1] - Pr[f(PRG(Ud))=1]| ≤ ε.

• Cryptography: e.g., F=BPTIME(nlog n).– Equivalent to one-way functions [HILL].

• Derandomizing BPP: F=nc-size circuits.– Need unproven lower bound assumptions.

• What F, d without unproven assumptions?

PRGpseudorandomrandom seed

nd

Pseudorandom Generators

• PRG fools class F of functions if|Pr[f(Un)=1] - Pr[f(PRG(Ud))=1]| ≤ ε.

• PRG fooling {f | sizeM(f)≤s} with seed length s1/c implies g in NP with sizeM(g)≥≈nc.

• Can we achieve converse: does g in P with sizeM(g)≥nc imply PRG with seed of length ≈ s1/c?

• Previous work gives nothing in this case.

PRGpseudorandomrandom seed

nd

New Results

• Construct such near optimal PRGs if lower bound is proved via “shrinkage.”

• Obtain following seed lengths to fool size s, error = 1/poly.– Formulas over {∨,∧,NOT}: s1/3+o(1)

– Formulas over arbitrary basis: s1/2+o(1)

– Read-once formulas over {∨,∧,NOT}: s.234…

– Branching programs: s1/2+o(1)

Previous Work

• Seed length (1-α)n fooling read-once formulas and read-once branching programs of width 2αn, α>0 small enough constant.

[Bogdanov, Papakonstantinou, Wan].• For ROBPs reading bits in known order, seed

length O(log2 n) [Nisan,…].

Random Restrictions

• Choose random restriction ρ, fraction p unset.• E[size(f|ρ)] ≤ p size(f), size(formula)= # leaves.• Whp size(f|ρ) ≤ 2p size(f).• Holds even if ρ chosen k-wise independently.

Shrinkage Exponent• Random ρ, fraction p unset. Shrinkage Γ:

E[size(f|ρ)] = O(pΓ s).• Example: Formulas.– Formulas over arbitrary basis: Γ = 1.– Formulas over DM={∨,∧,NOT}: Γ = 2

[Subbotovskaya ‘61, …., Hastad ‘93]– Read-once formulas over DM: Γ = 3.27…

[Paterson-Zwick ‘91, Hastad-Razborov-Yao ‘95]• General circuits: Γ = 0.

Branching Programs

• Layered, ordered, read-once BPs needed for PRG for Space• Size = # edges ≤ 2wn.• Γ = 1: size of shrunken BP proportionally to |{unfixed var’s}|.• |{layered, ordered ROBPs}| ≤ w2wn.• We consider arbitrary BPs, reading bits in arbitrary order.

n+1 layers

width w

0

01

1

x1

x2

acc

rej

PRGs from Shrinkage• Random ρ, fraction p unset. Shrinkage Γ:

E[size(f|ρ)] = O(pΓ s).• Shrinkage Γ nΓ+1/polylog(n) lower bounds

[Andreev].• Main theorem: High probability shrinkage Γ

wrt pseudorandom restrictions gives PRG with seed length s1/(Γ+1) + o(1).

• Showing shrinkage wrt pseudorandom restrictions is nontrivial when Γ ≠ 1.

Outline

• Background on Randomness Extractors• New Theorem about Old PRG• New PRG• Correctness Proof• Pseudorandom Restrictions• Conclusions

Weak Random Source […CG ‘85 Z ‘90]

• Random variable X on {0,1}r.• General model: min-entropy

• Flat source:– Uniform on A,

|A| ≥ 2k.|A| ³ 2k

{0,1}r

How Arise in PRGs

• Condition on information– E.g., TM configuration

• Uniform X in {0,1}r, f:{0,1}r {0,1}b.• f regular: H∞(X|f(X) = a) = r - b.• Any f:

Pra=f(X’)[H∞(X|f(X) = a) ≥ r – b – Δ] ≥ 1-2-Δ.

Goal: Extract Randomness

Ext r bits m bits

statistical error

Problem: Impossible, even for k=r-1, m=1, ε<1/2.

Impossibility Proof

• Suppose f:{0,1}r {0,1} satisfies sources X ∀with H∞(X) ≥ r-1, f(X) ≈ U.

f-1(0)f-1(1)

Take X=f-1(0)

Randomness Extractor: short seed[Nisan-Z ‘93,…, Guruswami-Umans-Vadhan ‘07]

Ext r bits m =.99k bits

statistical error

d=O(log (r/ε)) random bit seed Y

Extractor-Based PRG for Read-Once Branching Programs [Nisan-Z ‘93]

• Basic PRG: G(x, y1,…, yt)=Ext(x,y1)…Ext(x,yt)• Parameters: r = |x| = 2√n

d = |yi| = O(log n)

t = m = |Ext(x,yi)| = √n

PRG for Ordered Read-Once BPs

• G(x, y1,…, yt)=Ext(x,y1)…Ext(x,yt)

• Condition on v reached after reading up to Ext(X,Yi-1).

• Whp H∞(X|reach v) ≥ |x| – log w - Δ.

• Hence (Ext(X,Yi)|reach v) ≈ uniform.

n+1 layers

width w

0

01

1

z1

z2

acc

rej

v

New: Same PRG works if bits read in any order

• z1,z2,…,zm can appear anywhere.

• Still, after fixing all zi, i>m, restricted function is a ROBP on z1,z2,…,zm read in the same order as original ROBP.

n+1 layers

width w

0

01

1

z41

z26

acc

rej

New: Same PRG works if bits read in any order

• Still, after fixing all zi, i>m, restricted function is a ROBP on z1,z2,…,zm read in the same order as original ROBP.

• Information = lg(# restricted functions) = lg(w2wm)

n+1 layers

width w

0

01

1

z41

z26

acc

rej

New: Works if bits read in any order

• PRG: G(x, y1,…, yt)=Ext(x,y1)…Ext(x,yt)=z1…zn

• BP could read in order z12z7z8…• D=distribution of PRG output, U=Unif({0,1}n).• Suppose |Pr[f(D)=1] – Pr[f(U)=1]| > δ.• Let Zi=Ext(X,Yi), Ui =Unif({0,1}m)– Z1=z1z2…zm,Z2=zm+1…z2m,…

• Bits in Zi can appear anywhere.

New: Works if bits read in any order

• PRG: G(x, y1,…, yt)=Ext(x,y1)…Ext(x,yt).• D=distribution of PRG output, U=Unif({0,1}n).• Suppose |Pr[f(D)=1] – Pr[f(U)=1]| > δ.• Let Zi=Ext(X,Yi), Ui =Unif({0,1}m).• Hybrid argument.• Let Di = (U1,…,Ui,Zi+1,…,Zt). D0=D, Dt=U.

• Exists i: |Pr[f(Di)=1] – Pr[f(Di-1=1)]| > δ/t.

• Changing Zi=Ext(X,Yi) to Ui changes Pr[accept].

New: Works if bits read in any order

• Exists i: |Pr[f(Di)=1] – Pr[f(Di-1=1)]| > δ/t.

• Changing Zi=Ext(X,Yi) to Ui changes Pr[accept].

• Consider ρ = (Z1,…,Zi-1,**…*,Ui+1,…,Ut)

• Then g = f|ρ is a ROBP on m bits.• f(Di)=g(Zi), f(Di-1)=g(Ui). Goal: whp g(Zi) ≈ g(Ui). • Only w2wm possibilities for g.• Whp, H∞(X|G=g) ≥ r – 2mw log w - Δ.

• Whp, conditioned G=g, Ext(X,Yi) ≈ Ui.

General Branching Programs

• Even PRG for unordered ROBPs is new– Our seed length is O(√(wn) log n)– Previous was (1-α)n [Bogdanov, Papakonstantinou, Wan]– Known order: O(log2 n) [Nisan,…].

• What if not read once?– Some variables could be read many times.– Pseudorandomly permute variables before construction.– Gives seed length size(f)½+o(1).

• What about formulas? General reduction?

General PRG Construction

• Assume have pseudorandom restrictions which give shrinkage Γ whp.

ρ1 = 0 1 * 1 1 0 1 1 * 0 0 1 0 * 0 1 0 0 1 1 1

ρ2 = 0 0 1 0 1 0 * 0 1 1 0 1 * 0 1 1 0 * * 1 0

…ρt = * 0 1 0 1 1 * 1 * 0 0 1 0 0 0 1 * 0 1 1 1

• Set t=c(log n)/p so whp all columns have *.

General PRG Construction

ρ1 = 0 1 * 1 1 0 1 1 * 0 0 1 0 * 0 1 0 0 1 1 1

ρ2 = 0 0 1 0 1 0 * 0 1 1 0 1 * 0 1 1 0 * * 1 0

…ρt = * 0 1 0 1 1 * 1 * 0 0 1 0 0 0 1 * 0 1 1 1

• Choose X, Y1,…,Yt randomly.

• Replace *’s in ith row with Ext(X,Yi).• PRG output = XOR of resulting strings.

Correctness Proof

• D=distribution of PRG output, U=uniform.• Suppose |Pr[f(D)=1] – Pr[f(U=1)]| > δ.• Let Zi=Ext(X,Yi). Hybrid argument.

• Change Z1,…,Zi to U1,…,Ui to get Di.

• Dt ≈ U: Whp *’s cover all columns.

• Exists i: |Pr[f(Di)=1] – Pr[f(Di-1=1)]| > δ/t.

• Changing Zi to Ui changes Pr[f accepts].

Correctness Proof

• Exists i: changing Zi=Ext(X,Yi) to Ui changes Pr[f accepts].

• Fix everything but ρ=ρi, Zi, Ui. Let v = ith row.

• Let fi(v) = f(v+w), w = XOR of rows except ith.

• Let g = fi|ρ, so g(v|A) = fi (v) , A = *’s of ρ.

• f(Di)=g(Zi), f(Di-1)=g(Ui). Goal: whp g(Zi) ≈ g(Ui).

• E=event that size(g) ≤ s=cpΓ size(fi). Pr[E] ≥ 1-ε.

• Conditioned on E, g describable by b ≈ s log s bits.

• Whp, H∞(X|E,G=g) ≥ r – b - Δ.

• Whp conditioned on E and G=g, Ext(X,Yi) ≈ Ui.

Improving the PRG

• To get nearly optimal output length for Γ > 1, replace *’s with Gk-wise(Ext(X,Yi)).

Pseudorandom Restrictions

• Need pseudorandom restrictions that yield shrinkage.

• BPs and formulas over arbitrary basis:– clog n wise independence suffices.– Deal with heavy variables separately.

• Formulas over {∧,∨,NOT}, incl. read-once:– More work.– Hastad and Hastad-Razborov-Yao as black boxes.– They only guarantee shrinkage in expectation for truly

random restrictions.

Proof Idea

Decompose formula:O(n/k) subformulas of size ≤k=no(1).Use k2-wise independence.Goal: p ≈ n-1/(Γ+1). Too small here.Instead, shrink by q ≈ k-.1 and iterate.

Unrestrictable inputs

• Many subformulas have inputs that must = *.• Does shrinkage for random restrictions imply

shrinkage when some inputs must = *?• Further decomposition: each subformula has

≤ 2 such inputs.• h such inputs increase size by ≤ 2h.– For each setting of variables have subformula.– Combine with selector formula.

Read-Once Formulas

• Need different trick for read-once formula.

• g small but unlikely to shrink to nothing.

* *g g

Dependencies

• Read-once case: k-wise independence.• Read-t case: Consider independent sets in

dependency graph on subformulas.• General case: tricky dependencies.

Conclusions

• New, extractor-based PRG based on shrinkage.• Without improving lower bounds, essentially

best possible PRGs for:– Formulas over {∨,∧,NOT}: s1/3+o(1) seed length.– Formulas over arbitrary basis: s1/2+o(1)

– Read-once formulas over {∨,∧,NOT}: s.234…

– Branching programs: s1/2+o(1)

Open Questions

• Better PRGs for unordered ROBPs?– Can we recurse somehow?– Subsequent work: Reingold-Steinke-Vadhan give O(log2 n)

seed for unordered permutation ROBPs.• PRGs from other lower bound techniques?– Subsequent work: Trevisan-Xue on PRGs for AC0.

• Improve lower bounds?– Our PRG gives alternate function f:formula-size(f) ≥ n3-o(1), matching Hastad/Andreev.– Subsequent: average-case lower bound of n3-o(1)

[Komargodski-Raz-Tal] (improving [Komargodski-Raz])

Thank you!