Average-case Complexityluca/average/slides.pdf · Lipton (1990): If there is algorithm that solves Perm() efficiently on most random matrices, Then there is an algorithm that solves

Average-case Complexity

Luca Trevisan

UC Berkeley

Distributional Problem

<P,D>

P computational problem

– e.g. SAT– e.g. SAT

D distribution over inputs

– e.g. n vars 10n clauses

Positive Results:

• Algorithm that solves P efficiently on most

inputs

– Interesting when P useful problem, D

distribution arising “in practice”

Negative Results:Negative Results:

• If <assumption>, then no such algorithm

– P useful, D natural

• guide algorithm design

– Manufactured P,D,

• still interesting for crypto, derandomization

Positive Results:

• Algorithm that solves P efficiently on most

inputs

– Interesting when P useful problem, D

distribution arising “in practice”

Negative Results:Negative Results:

• If <assumption>, then no such algorithm

– P useful, D natural

• guide algorithm design

– Manufactured P,D,

• still interesting for crypto, derandomization

Holy Grail

If there is algorithm A that solves P efficiently on

most inputs from D

Then there is an efficient worst-case algorithm Then there is an efficient worst-case algorithm

for [the complexity class] P [belongs to]

Part (1)

In which the Holy Grail proves elusive

The Permanent

Perm (M) := Σσ Πi M(i,σ(i))

Perm() is #P-complete

Lipton (1990):

If there is algorithm that solves Perm() efficiently

on most random matrices,

Then there is an algorithm that solves it efficiently

on all matrices (and BPP=#P)

Lipton’s Reduction

Suppose operations are over finite field of size >n

A is good-on-average algorithm

(wrong on < 1/(10(n+1)) fraction of matrices)(wrong on < 1/(10(n+1)) fraction of matrices)

Given M, pick random X, compute

A(M+X), A(M+2X),…,A(M+(n+1)X)

Whp the same as

Perm(M+X),Perm(M+2X),…,Perm(M+(n+1)X)

Lipton’s Reduction

Given

Perm(M+X),Perm(M+2X),…,Perm(M+(n+1)X)

Find univariate degree-n polynomial p such thatFind univariate degree-n polynomial p such that

p(t) = Perm(M+tX) for all t

Output p(0)

Improvements / Generalizations

• Can handle constant fraction of errors

[Gemmel-Sudan]

• Works for PSPACE-complete, EXP-complete,…• Works for PSPACE-complete, EXP-complete,…[Feigenbaum-Fortnow, Babai-Fortnow-Nisan-Wigderson]

Encode the problem as a polynomial

Strong Average-Case Hardness

• [Impagliazzo, Impagliazzo-Wigderson] Manufacture problems in E, EXP, such that

– Size-t circuit correct on ½ + 1/t inputs

implies

– Size poly(t) circuit correct on all inputs– Size poly(t) circuit correct on all inputs

Motivation:

[Nisan-Wigderson] P=BPP if there is problem in E of exponential average-case complexity


• [Impagliazzo, Impagliazzo-Wigderson]

Manufacture problems in E, EXP, such that

– Size-t circuit correct on ½ + 1/t inputs implies


Motivation:

[Impagliazzo-Wigderson]

P=BPP if there is problem in E of exponential

average worst-case complexity

Open Question 1

• Suppose there are worst-case intractable

problems in NP

• Are there average-case intractable problems?• Are there average-case intractable problems?


• [Impagliazzo, Impagliazzo-Wigderson] Manufacture

problems in E, EXP, such that

– Size-t circuit correct on ½ + 1/t inputs

implies


• [Sudan-T-Vadhan]

– IW result can be seen as coding-theoretic

– Simpler proof by explicitly coding-theoretic ideas

Encoding Approach

• Viola proves that an error-correcting code

cannot be computed in AC0

• The exponential-size error-correcting code • The exponential-size error-correcting code

computation not possible in PH

Problem-specific Approaches?

[Ajtai]

• Proves that there is a lattice problem such that:

– If there is efficient average-case algorithm

– There is efficient worst-case approximation

algorithm

Ajtai’s Reduction

• Lattice Problem


– There is efficient worst-case approximation

algorithmalgorithm

The approximation problem is in NPIcoNP

Not NP-hard

Holy Grail

• Distributional Problem:


– P=NP

(or NP in BPP, or NP has poly-size circuits,…)(or NP in BPP, or NP has poly-size circuits,…)

Already seen: no “encoding” approach works

Can extensions of Ajtai’s approach work?

A Class of Approaches

• L problem in NP, D distribution of inputs

• R reduction of SAT to <L,D>:

• Given instance f of SAT,– R produces instances x1,…,xk of L, each distributed according to D

– R produces instances x1,…,xk of L, each distributed according to D

– Given L(x1),…,L(x1), R is able to decide f

If there is good-on-average algorithn for <L,D>,

we solve SAT in polynomial time

[cf. Lipton’s work on Permanent]


• L,W problems in NP, D (samplable) distribution of inputs

• R reduction of W to <L,D>

• Given instance w of W,– R produces instances x1,…,xk of L, each distributed – R produces instances x1,…,xk of L, each distributed according to D

– Given L(x1),…,L(x1), R is able to decide w

If there is good-on-average algorithm for <L,D>, we solve W in polynomial time;

Can W be NP-complete?


• Given instance w of W,

– R produces instances x1,…,xk of L, each distributed according to D

– Given L(x1),…,L(x1), R is able to decide w

Given good-on-average algorithm for <L,D>, we Given good-on-average algorithm for <L,D>, we solve W in polynomial time;

If we have such reduction, and W is NP-complete, we have Holy Grail!

Feigenbaum-Fortnow: W is in “coNP”

Feigenbaum-Fortnow


– R produces instances x1,…,xk of L, each

distributed according to D

– Given L(x1),…,L(x1), R is able to decide wGiven L(x1),…,L(x1), R is able to decide w

• Using R, Feigenbaum-Fortnow design a 2-round

interactive proof with advice for coW

• Given w, Prover convinces Verifier that

R rejects w after seeing L(x1),…,L(x1)

Feigenbaum-Fortnow


– R produces instances x of L distributed as in D

– w in L iff x in L

Suppose we know PrD[ x in L]= ½

VP

w

R(w) = x1R(w) = x2. . .

R(w) = xm

x1, x2,. . . , xm

(Yes,w1),No,. . . , (Yes, wm)

Accept iff all simulations of R reject

and m/2 +/- sqrt(m) answers are certified Yes

Feigenbaum-Fortnow

• Given instance w of W, p:= Pr[ xi in L]

– R produces instances x1,…,xk of L, each distrib. according to D

– Given L(x1),…,L(xk), R is able to decide w

VP

wR(w) -> x1

1,…,xk1

. . .

R(w) -> x1m,…,xk

m

x11,…,xk

m

(Yes,w11),…,NO

Accept iff

-pkm +/- sqrt(pkm) YES with certificates

-R rejects in each case

Generalizations

• Bogdanov-Trevisan: arbitrary non-adaptive

reductions

• Main Open Question:• Main Open Question:

What happens with adaptive reductions?

Open Question 1

Prove the following:

Suppose:

W,L are in NP, D is samplable distribution, W,L are in NP, D is samplable distribution,

R is poly-time reduction such that

– If A solves <L,D> on 1-1/poly(n) frac of inputs

– Then R with oracle A solves W on all inputs

Then W is in “coNP”

By the Way

• Probably impossible by current techniques:

If NP not contained in BPP

There is a samplable distribution D and an NP problem L

Such that <L,D> is hard on average

By the Way




Such that for every efficient A

A makes many mistakes solving L on DA makes many mistakes solving L on D

By the Way




Such that for every efficient A

A makes many mistakes solving L on DA makes many mistakes solving L on D

• [Guttfreund-Shaltiel-TaShma] Prove:


For every efficient A

There is a samplable distribution D

Such that A makes many mistakes solving SAT on D

Part (2)

In which we amplify average-case complexity and In which we amplify average-case complexity and

we discuss a short paper

Revised Goal

• Proving“If NP contains worst-case intractable problems, then NP contains average-case intractable problems”

Might be impossible

• Average-case intractability comes in different quantitative degrees

• Equivalence?

Average-Case Hardness

What does it mean for <L,D>

to be hard-on-average?

Suppose A is efficient algorithm Suppose A is efficient algorithm

Sample x ~ D

Then A(x) is noticeably likely to be wrong

How noticeably?

Average-Case Hardness

Amplification

Ideally:

• If there is <L,Uniform>, L in NP,

such that every poly-time algorithm (poly-size such that every poly-time algorithm (poly-size

circuit) makes > 1/poly(n) mistakes

• Then there is <L’,Uniform>, L’ in NP,

such that every poly-time algorithm (poly-size

circuit) makes > ½ - 1/poly(n) mistakes

Amplification

“Classical” approach: Yao’s XOR Lemma

Suppose: for every efficient A

PrD [ A(x) = L(x) ] < 1- δPrD [ A(x) = L(x) ] < 1- δ

Then: for every efficient A’

PrD [ A’(x1,…,xk) = L(x1) xor … xor L(xk) ]

< ½ + (1 - 2δ)k + negligible

Yao’s XOR Lemma

Suppose: for every efficient A

PrD [ A(x) = L(x) ] < 1- δ

Then: for every efficient A’Then: for every efficient A’

PrD [ A’(x1,…,xk) = L(x1) xor … xor L(xk) ]

< ½ + (1 - 2δ)k + negligible

Note: computing L(x1) xor … xor L(xk) need not be

in NP, even if L is in NP

O’Donnell Approach

Suppose: for every efficient APrD [ A(x) = L(x) ] < 1- δ

Then: for every efficient A’

Pr [ A’(x ,…,x ) = g(L(x ), …, L(x )) ] PrD [ A’(x1,…,xk) = g(L(x1), …, L(xk)) ]

< ½ + small(k, δ)

For carefully chosen monotone function g

Now computing g(L(x1),…, L(xk)) is in NP, if L is in NP

Amplification (Circuits)

Ideally:

• If there is <L,Uniform>, L in NP, such that every poly-time

algorithm (poly-size circuit) makes > 1/poly(n) mistakes

• Then there is <L’,Uniform>, L’ in NP, such that every poly-time

algorithm (poly-size circuit) makes > ½ - 1/poly(n) mistakes

Achieved by [O’Donnell, Healy-Vadhan-Viola] for poly-size

circuits

Amplification (Algorithms)

• If there is <L,Uniform>, L in NP, such that every poly-

time algorithm makes > 1/poly(n) mistakes

• Then there is <L’,Uniform>, L’ in NP, such that every

poly-time algorithm makes > ½ - 1/polylog(n) mistakes

[T]

[Impagliazzo-Jaiswal-Kabanets-Wigderson]

½ - 1/poly(n) but for PNP||

Open Question 2

Prove:

• If there is <L,Uniform>, L in NP, such that every

poly-time algorithm makes > 1/poly(n) mistakespoly-time algorithm makes > 1/poly(n) mistakes

• Then there is <L’,Uniform>, L’ in NP, such that

every poly-time algorithm makes

> ½ - 1/poly(n) mistakes

Completeness

• Suppose we believe there is L in NP, D

distribution, such that <L,D> is hard

• Can we point to a specific problem C such that • Can we point to a specific problem C such that

<C,Uniform> is also hard?

Completeness


distribution, such that <L,D> is hard



Must put restriction on D, otherwise assumption is

the same as P != NP

Side Note

Let K be distribution such that x has probability

proportional to 2-K(x)

Suppose A solves <L,K> on 1-1/poly(n) fraction of Suppose A solves <L,K> on 1-1/poly(n) fraction of

inputs of length n

Then A solves L on all but finitely many inputs

Exercise: prove it

Completeness


samplable distribution, such that <L,D> is hard



Completeness


samplable distribution, such that <L,D> is hard



Yes we can!

[Levin, Impagliazzo-Levin]

Levin’s Completeness Result

• There is an NP problem C, such that

• If there is L in NP, D computable distribution,

such that <L,D> is hardsuch that <L,D> is hard

• Then <C,Uniform> is also hard

Reduction

Need to define reduction that preserves

efficiency on average

(Note: we haven’t yet defined efficiency on average)

R is a (Karp) average-case reduction

from <A,DA> to <B,DB> if

1. x in A iff R(x) in B

2. R(DA) is “dominated” by DB:

Pr[ R(DA)=y] < poly(n) * Pr [DB = y]

Reduction

R is an average-case reduction from <A, DA> to <B, DB> if

• x in A iff R(x) in B

• R(DA) is “dominated” by DB:

Pr[ R(DA)=y] < poly(n) * Pr [DB = y]

Suppose we have good algorithm for <B, DB>

Then algorithm also good for <B,R(DA)>

Solving <A, DA> reduces to solving <B,R(DA)>

Reduction

If Pr[ Y=y] < poly(n) * Pr [DB = y]

and we have good algorithm for <B, DB >

Then algorithm also good for <B,Y>

Reduction works for any notion of average-case

tractability for which above is true.


Follow presentation of [Goldreich]

• If <BH,Uniform> is easy on average

• Then for every L in NP, • Then for every L in NP, every D computable distribution, <L,D> is easy on average

BH is non-deterministic Bounded Halting: given <M,x,1t>,does M(x) accept with t steps?


BH, non-deterministic Bounded Halting:

given <M,x,1t>,

does M(x) accept with t steps?

Suppose we have good-on-average alg ASuppose we have good-on-average alg A

Want to solve <L,D>, where L solvable by NDTM M

First try: x -> <M,x, 1poly(n)>


First try: x -> <M,x, 1poly(n)>

Doesn’t work: x may have arbitrary distribution,

we need target string to be nearly uniform we need target string to be nearly uniform

(high entropy)

Second try: x -> <M’,C(x), 1poly(n)>

Where C() is near-optimal compression alg, M’

recover x from C(x), then runs M


Second try: x -> <M’,C(x), 1poly(n)>

Where C() is near-optimal compression alg, M’

recover x from C(x), then runs M

Works! Provided C(x) has length at most

O(log n) + log 1/PrD[x]

Possible if cumulative distribution function of D is

computable.

Impagliazzo-Levin

Do the same but for all samplable distribution

Samplable distribution not necessarily efficiently

compressible in coding theory sense. compressible in coding theory sense.

(E.g. output of PRG)

Hashing provides “non-constructive” compression

Complete Problems

BH with Uniform distribution

Tiling problem with Uniform distribution [Levin]

Generalized edge-coloring [Venkatesan-Levin]

Matrix representability [Venkatesan-Rajagopalan]

Matrix transformation [Gurevich]

. . .

Open Question 3

L in NP, M NDTM for L is specified by k bits

Levin’s reduction incurs 2k bits in fraction of

“problematic” inputs“problematic” inputs

(comparable to having 2k slowdown)

Limited to problems having non-deterministic

algorithm of 5 bytes

Inherent?

More Reductions?

Still relatively few complete problems

Similar to study of inapproximability before

Papadimitriou-Yannakakis and PCPPapadimitriou-Yannakakis and PCP

Would be good, as in Papadimitriou-Yannakakis,

to find reductions between problems that are

not known to be complete but are plausibly

hard

Open Question 4

(Heard from Russell Impagliazzo)

Prove that

If 3SAT is hard on instances with n variables and

10n clauses,

Then it is also hard on instances with 12n clauses

See

• http://www.cs.berkeley.edu/~luca/average

[slides, references, addendum to Bogdanov-T, coming soon]

• http://www.cs.uml.edu/~wang/acc-forum/

[average-case complexity forum]

• Impagliazzo A personal view of average-case complexity

Structures’95

• Goldreich Notes on Levin’s theory of average-case complexity

ECCC TR-97-56

• Bogdanov-T. Average case complexity

F&TTCS 2(1): (2006)

Documents

Average-case Complexityluca/average/slides.pdf · Lipton (1990): If there is algorithm that solves Perm() efficiently on most random matrices, Then there is an algorithm that solves