(S)ETH and A Survey of Consequences€¦ · Exact Algorithms and ComplexityExponential Time HypothesisExplanatory Value of ETH and SETH Probabilistic Polynomial Time AlgorithmsOpen

Exact Algorithms and Complexity Exponential Time Hypothesis Explanatory Value of ETH and SETH Probabilistic Polynomial Time Algorithms Open Problems

(S)ETH and A Survey of Consequences

Mohan Paturi

Simons Institute, August 2015

Paturi (S)ETH and A Survey of Consequences


Outline

1 Exact Algorithms and Complexity

2 Exponential-Time Hypothesis

3 Explanatory Value of ETH and SETH

4 Probabilistic Polynomial Time Algorithms

5 Open Problems



Exact Algorithms and Complexity

All NP-complete problems are equivalent as far as polynomialtime solvability is concerned.

However, some NP-complete problems have better exactalgorithms than others

Exact algorithms — deterministic or randomized algorithmsthat produce exact solutions

Exact complexity — worst-case complexity of exact algorithms

What improvements can we expect over exhaustive search orstandard algorithms?

What are the obstructions that limit the improvements?

What principles explain the exact complexities ofNP-complete problems?































































Exact Complexity — Parametrization of NP Problems

Two (or more) parameters with each instance: m, the size ofthe input and n, a complexity parameter

Natural and robust complexity parameters1 k-sat: formula F −→ (formula size m, number of variables n)2 Also, k-sat: formula F −→ (formula size m, number of

clauses)3 Hamiltonian Path: graph G = (V ,E ) −→ (size of the

graph m, number of vertices n)4 Also, Hamiltonian Path: graph G = (V ,E ) −→ (size of

the graph m, log n!)





Natural and robust complexity parameters1 k-sat: formula F −→ (formula size m, number of variables n)

2 Also, k-sat: formula F −→ (formula size m, number ofclauses)

3 Hamiltonian Path: graph G = (V ,E ) −→ (size of thegraph m, number of vertices n)

4 Also, Hamiltonian Path: graph G = (V ,E ) −→ (size ofthe graph m, log n!)






clauses)

3 Hamiltonian Path: graph G = (V ,E ) −→ (size of thegraph m, number of vertices n)








graph m, number of vertices n)








graph m, number of vertices n)4 Also, Hamiltonian Path: graph G = (V ,E ) −→ (size of

the graph m, log n!)



NP Parametrization

Circuit Sat: circuit F −→ (circuit size m, number ofvariables n)

NP : Existentially quantified circuit ∃yC (x , y) → (circuit sizem, number of existentially quantified Boolean variables |y |)Circuit Sat is the “mother” of all NP-complete problemsunder this natural parametrization.

Any NP-complete problem can be reduced to Circuit Satpreserving the complexity parameter exactly.



NP Parametrization


NP : Existentially quantified circuit ∃yC (x , y) → (circuit sizem, number of existentially quantified Boolean variables |y |)

Circuit Sat is the “mother” of all NP-complete problemsunder this natural parametrization.




NP Parametrization






NP Parametrization






Improved Exact Algorithms

Given an NP problem instance with size parameter m andcomplexity parameter n,

Standard (deterministic or random) algorithms are thoseachieve worst-case time complexity poly(m)2n

Improved exact algorithms are those that achieve worst-casetime complexity poly(m)2µn for µ < 1.

Also known as moderately exponential-time or nontrivialexponential-time algorithms
























Improved Exponential Time Algorithms for k-sat andk-Colorability

k-sat, number of variables as the complexity parameter —backtracking and local searchHertli (2012), PPSZ, Schoning, PPZ, Rolf, Iwama, · · · ,Monien and Speckenmeyer (1985)

3-sat — 20.386n

4-sat — 20.554n

k-sat — 2(1−µk/(k−1))n where µk ≈ 1.6 for large k.

k-Colorability, number of vertices as the complexityparameter — backtrackingBeigel and Eppstein (2005), Byskov (2004), · · ·

3-Colorability — 20.41n






3-sat — 20.386n

4-sat — 20.554n









3-sat — 20.386n

4-sat — 20.554n









3-sat — 20.386n

4-sat — 20.554n









3-sat — 20.386n

4-sat — 20.554n









3-sat — 20.386n

4-sat — 20.554n









3-sat — 20.386n

4-sat — 20.554n







Improved Algorithms for Hamiltonian Path

Hamiltonian Path, number of vertices as the complexityparameter — dynamic programming, inclusion-exclusion,determinant summation formulas, algebraic sievingBjorklund (2010), Bax (1993), Karp (1982), Kohn, Gottlieb,and Kohn (1977), Held and Karp (1962), Bellman (1962)

Hamiltonian Path — 2n

undirected Hamiltonian Path — 20.67n















Improved Algorithms for Colorability

Colorability, number of vertices as the complexityparameter — dynamic programming, Mobius inversionBjorklund, Husfeldt, Kaski, Koivisto (2006-2008), · · · , Byskov(2004), Eppstein (2003), Lawler (1976)

Colorability — 2n in exponential space



Improved Algorithms for Max Independent Set

Max Independent Set, number of vertices as thecomplexity parameter — backtracking, measure and conquerFomin, Grandoni, and Kratsch (2005), Robson (1986), Jian(1986), Tarjan and Trojanowski (1977)

20.287n in polynomial space20.276n in exponential space

Circuit Sat — split and list, random restrictions, dynamicprogramming, algebraization, matrix multiplicationImpagliazzo, P, William (2012), Williams (2011), Santhanam(2011), Tamaki and Seto (2012), Impagliazzo, P, andSchneider (2013)

AC0 Satisfiability for circuits of size cn and depth d —

2n(1−1/Θ(cd ))

ACC Satisfiability — 2n−nε

Formula Satisfiability for formulas of size cn — 2n(1−1/c2)

Formula Satisfiability for formulas of size cn over the full

binary basis — 2n(1−1/2−c2)

Depth-2 Threshold Circuit Satisfiability for circuits with cn

wires — 2n(1−1/cc2

)





20.287n in polynomial space

20.276n in exponential space



2n(1−1/Θ(cd ))







)








2n(1−1/Θ(cd ))







)








2n(1−1/Θ(cd ))







)








2n(1−1/Θ(cd ))







)








2n(1−1/Θ(cd ))







)








2n(1−1/Θ(cd ))







)








2n(1−1/Θ(cd ))







)








2n(1−1/Θ(cd ))







)Paturi (S)ETH and A Survey of Consequences


Exact Complexity — Motivating Questions

Which problems have improved algorithms?Is there a 2µn algorithm for Colorability or cnfsat orCircuit Sat for µ < 1?

Can the improvements extend to arbitrarily small exponents?Is 3-sat solvable in subexponential-time? How about3-Colorability?

Can we prove improvements beyond a certain point are notpossible (at least under some complexity assumption)?Lower bounding the exponent for 3-sat under suitablecomplexity assumptions?

Is progress on different problems connected?Do improved algorithms for 3-sat imply improved algorithmsfor 3-Colorability or vice versa?If Colorability has a 2cn algorithm, can we prove cnfsathas a 2dn algorithm for some c, d < 1?
























A Zoo of Algorithms, Techniques, and Analyses

A lot of effort has gone into improving the exponents.

A disparate variety of algorithmic techniques and analyseshave been used.backtracking, local search, split and list, random restrictions,dynamic programming, algebraization, matrix multiplication,Mobius inversion, measure and conquer, inclusion-exclusion,determinant summation formulas, algebraic sieving

A priori, it is not clear that we can expect a common principleto govern the exact complexities.















Connections between Problems

Is there a connection between the exponential complexities ofproblems?

If 3-Colorability has a subexponential time (2εn forarbitrarily small ε) algorithm, does it imply a subexponentialtime algorithms for 3-sat?

Problem: In the standard reduction from 3-sat of n variablesand m clauses to 3-Colorability, we get a graph onO(n + m) vertices and O(n + m) edges.

Complexity parameter increases polynomially, thus preventingany useful conclusion about 3-sat.
























Sparsification Lemma

Lemma (Sparsification Lemma, IPZ 1997)

∃ algorithm A ∀k ≥ 2, ε ∈ (0, 1], φ ∈ k-CNF with n variables,Ak,ε(φ) outputs φ1, . . . , φs ∈ k-CNF in 2εn time such that

1 s ≤ 2εn; Sol(φ) =⋃

i Sol(φi ), where Sol(φ) is the set ofsatisfying assignments of φ

2 ∀i ∈ [s] each literal occurs ≤ O(kε )3k times in φi .

To complete the reduction from 3-sat to 3-Colorabilityand preserve linearity in the parameter value,Apply Sparsification Lemma to the given 3-cnf φ.Consider each 3-cnf φi with linearly many clauses and reduceit to a graph with linearly many vertices.Now, a subexponential time algorithm for 3-Colorabilityimplies a subexponential time algorithm for 3-sat.Key ideas: subexponential time reductions, sparsification






1 s ≤ 2εn; Sol(φ) =⋃



To complete the reduction from 3-sat to 3-Colorabilityand preserve linearity in the parameter value,

Apply Sparsification Lemma to the given 3-cnf φ.Consider each 3-cnf φi with linearly many clauses and reduceit to a graph with linearly many vertices.Now, a subexponential time algorithm for 3-Colorabilityimplies a subexponential time algorithm for 3-sat.Key ideas: subexponential time reductions, sparsification






1 s ≤ 2εn; Sol(φ) =⋃



To complete the reduction from 3-sat to 3-Colorabilityand preserve linearity in the parameter value,Apply Sparsification Lemma to the given 3-cnf φ.

Consider each 3-cnf φi with linearly many clauses and reduceit to a graph with linearly many vertices.Now, a subexponential time algorithm for 3-Colorabilityimplies a subexponential time algorithm for 3-sat.Key ideas: subexponential time reductions, sparsification






1 s ≤ 2εn; Sol(φ) =⋃



To complete the reduction from 3-sat to 3-Colorabilityand preserve linearity in the parameter value,Apply Sparsification Lemma to the given 3-cnf φ.Consider each 3-cnf φi with linearly many clauses and reduceit to a graph with linearly many vertices.

Now, a subexponential time algorithm for 3-Colorabilityimplies a subexponential time algorithm for 3-sat.Key ideas: subexponential time reductions, sparsification






1 s ≤ 2εn; Sol(φ) =⋃



To complete the reduction from 3-sat to 3-Colorabilityand preserve linearity in the parameter value,Apply Sparsification Lemma to the given 3-cnf φ.Consider each 3-cnf φi with linearly many clauses and reduceit to a graph with linearly many vertices.Now, a subexponential time algorithm for 3-Colorabilityimplies a subexponential time algorithm for 3-sat.

Key ideas: subexponential time reductions, sparsification






1 s ≤ 2εn; Sol(φ) =⋃



To complete the reduction from 3-sat to 3-Colorabilityand preserve linearity in the parameter value,Apply Sparsification Lemma to the given 3-cnf φ.Consider each 3-cnf φi with linearly many clauses and reduceit to a graph with linearly many vertices.Now, a subexponential time algorithm for 3-Colorabilityimplies a subexponential time algorithm for 3-sat.Key ideas: subexponential time reductions, sparsification



SNP

SNP — class of properties expressible by a series of secondorder existential quantifiers, followed by a series of first orderuniversal quantifiers, followed by a basic formula—Papadimitriou and Yannakakis 1991

SNP includes k-sat and k-Colorability for k ≥ 3.∃S∀(y1, . . . , yk)∀(s1, . . . , sk)[R(s1,...,sk )(y1, . . . , yk) =⇒∧1≤i≤kSsi (yi ), where si ∈ {+,−} and S is a subset of [n].

Vertex Cover,Clique, Independent Set andk-Set Cover are in size-constrained SNP.

Hamiltonian Path is SNP-hard.



SNP


SNP includes k-sat and k-Colorability for k ≥ 3.

∃S∀(y1, . . . , yk)∀(s1, . . . , sk)[R(s1,...,sk )(y1, . . . , yk) =⇒∧1≤i≤kSsi (yi ), where si ∈ {+,−} and S is a subset of [n].





SNP







SNP







SNP







Completeness of 3-sat in SNP

Theorem (Impagliazzo, P, and Zane (1977))

3-sat admits a subexponential-time algorithm if and only if everyproblem in (size-constrained) SNP admits one.

Proof Sketch: Show that every problem in SNP is stronglymany-one reducible to k-sat for some k . Complexityparameter is the number of existential quantifiers.

Reduce k-sat to the union of subexponentially manylinear-size k-sat using Sparsification Lemma.

Reduce each linear-size k-sat to 3-sat with linearly manyvariables.



























Exponential-time Hypothesis (ETH)

The previous theorem gives evidence that 3-sat does nothave a subexponential-time algorithm as it is unlikely that thewhole class SNP has such algorithms.

While it seems beyond our scope to prove this, our plan is toexplore the state of affairs given the likelihood.

Let sk = inf{δ|∃ 2δn algorithm for k-sat};ETH — Exponential Time Hypothesis: s3 > 0

Assuming ETH, we conclude none of the problems in(size-constrained) SNP have a subexponential time algorithms

Furthermore, SNP-hard problems such asHamiltonian Path cannot have a subexponential timealgorithm.














Let sk = inf{δ|∃ 2δn algorithm for k-sat};

ETH — Exponential Time Hypothesis: s3 > 0





























Explanatory Value of ETH

We have a very little understanding of exponential timealgorithms.

For ETH to be useful,

it must be able to provide an explanation for the exactcomplexities of various other problems,ideally, by providing lower bounds that match the upperbounds from the best known algorithms.

ETH will be useful if it helps factor out the essential difficultyof dealing with exponential time algorithms for NP-completeproblems.













it must be able to provide an explanation for the exactcomplexities of various other problems,

ideally, by providing lower bounds that match the upperbounds from the best known algorithms.


















Lower Bounds based on ETH — I

We follow the nice summary provided by Lokshtanov, Marxand Saurabh (2011).

All the following results assume ETH.Subexponential time lower bounds: There is no 2o(

√n)

algorithm for Vertex Cover, 3-Colorability, andHamiltonian Path for planar graphs.Lower bounds for FPT problems: There is no 2o(k)nO(1)

algorithm to decide whether the graph has a vertex cover ofsize at most k .Similar results hold for the problemsFeedback Vertex Set and Longest Path. Cai andJuedes (2003)Lower bounds for W [1]-complete problems: There is nof (k)no(k) algorithm for Clique or Independent Set.Chen, Chor, Fellows, Huang, Juedes, Kanj, and Xia (2005,2006)




We follow the nice summary provided by Lokshtanov, Marxand Saurabh (2011).All the following results assume ETH.

Subexponential time lower bounds: There is no 2o(√n)






We follow the nice summary provided by Lokshtanov, Marxand Saurabh (2011).All the following results assume ETH.Subexponential time lower bounds: There is no 2o(

√n)

algorithm for Vertex Cover, 3-Colorability, andHamiltonian Path for planar graphs.

Lower bounds for FPT problems: There is no 2o(k)nO(1)






√n)


algorithm to decide whether the graph has a vertex cover ofsize at most k .Similar results hold for the problemsFeedback Vertex Set and Longest Path. Cai andJuedes (2003)

Lower bounds for W [1]-complete problems: There is nof (k)no(k) algorithm for Clique or Independent Set.Chen, Chor, Fellows, Huang, Juedes, Kanj, and Xia (2005,2006)





√n)





Lower Bounds based on ETH — II

Lower bounds for W [2]-complete problems: There is nof (k)no(k) algorithm for Dominating Set. Fellows (2011),Lokshtanov (2009)

Lower bounds for problems parameterized by treewidthChromatic Number parameterized by treewidth t does notadmit an algorithm that runs in time 2o(t lg t)nO(1).Lokshtanov, Marx, and Saurabh (2011), Cygan, Nederlof,Pilipczuk, van Rooij, Wojtaszczyk (2011)

List Coloring parameterized by treewidth does not admitalgorithms that run in f (t)no(t). Fellows, Fomin, Lokshtanov,Rosamond, Saurabh, Szeider, and Thomassen (2011)

Workflow Satisfiability Problem parameterized by the numberof steps k cannot have a 2o(k lg k)nO(1) algorithm. Crampton,Cohen, Gutin, and Jones (2013)

Many others · · ·
































Many others · · ·Paturi (S)ETH and A Survey of Consequences


Complexity of k-sat with Increasing k

Theorem (Impagliazzo and P, 1999)

If ETH is true, sk increases infinitely often

Let s∞ = limk→∞ sk .

More specifically, we prove s∞ − sk ≥ d/k for some absoluteconstant d > 0.

Provides evidence to the observation that heuristics for k-satperform worse as k increases.

Proof Sketch: Trade clause width up to reduce the number ofvariables: reduce k-sat to k ′-CNF for k ′ � k such that theresultant formula has fewer variables.







































Complexity of CSP Problems

ETH implies that (d , 2)-CSP requires dcn time where c is anabsolute constant. The constant c depends on s3. Traxler2008

(d , 2)-CSP is the class of constraint satisfaction problemswhere variables take d values and each clause has twovariables.

Proof involves reducing a (d , 2)-CSP instance to a(d ′, 2)-CSP instance for d ′ � d , but with fewer variables.

A special case of (k , 2)-CSP, k-Colorability, has a 2n

algorithm (exponent is independent of k).

Greater expressiveness of k ′-CNF and (d ′, 2)-CSP has beenexploited.






























SETH — Strong Exponential Time Hypothesis

Earlier result regarding the increasing complexity of k-sattempts one to hypothesizeSETH — Strong Exponential Time Hypothesis: s∞ = 1



SETH and Its Equivalent Statements

Theorem

The following statements are equivalent:

∀ε < 1, ∃k, k-sat, the satisfiability problems for n-variablek-cnf formuals, cannot be computed in time O(2εn) time.

∀ε < 1, ∃k, k-Hitting Set, the Hitting Set problem forset systems over [n] with sets of size at most k , cannot becomputed in time O(2εn) time.

∀ε < 1, ∃k, k-Set Splitting, the Set Splitting problemfor set systems over [n] with sets of size at most k, cannot becomputed in time O(2εn) time.

— Cygan, Dell, Lokshtanov, Marx, Nederlof, Okamoto, P,Saurabh, Wahlstrom, 2012




Theorem









Theorem








Lower Bounds based on SETH - I

If SETH holds, k-dominating set does not have af (k)nk−ε time algorithm. — Patrascu and Williams, 2009

SETH implies that Independent Set parameterized bytreewidth cannot be solved faster than 2twnO(1) —Lokshtanov, Marx, and Saurabh 2010

SETH implies that Dominating Set parameterized bytreewidth cannot be solved faster than 3twnO(1) —Lokshtanov, Marx, and Saurabh 2010


















Lower Bounds based on SETH - II

Theorem

SETH determines the exact complexities of the following problemsin P.

∀ε > o, the Orthogonal Vectors problem for n binaryvectors of dimension ω(log n) cannot be solved in timeO(n2−ε). — Williams - 2004

∀ε > o, the Vector Domination problem for n vectors ofdimension ωlog n cannot be solved in time O(n2−ε). —Williams - 2004, Impagliazzo, Paturi, Schneider - 2013

∀ε > o, the Frechet Distance problem for two piece-wiselinear curves with n pieces n cannot be solved in timeO(n2−ε). — Bringmann - 2014

Many others · · · — Borassi, Crescenzi, Habib - 2014,Abboud, Vassilevska Williams, 2014




Theorem









Theorem








Probabilistic Polynomial Time Algorithms

Consider natural, though restricted, models of computationfor exponential time algorithms.

OP(T (n,m)): one-sided error probabilistic algorithms thatrun in time T (n,m)

OPP: OP(T (n,m)) where T (n,m) is polynomially bounded.

Includes several Davis-Putnam style backtracking algorithms,local search algorithms

OPP: space efficiency, parallelization, speed-up by quantumcomputation

What is the best success probability achievable in OPP orOP(T (n,m))?


























































Circuit Sat problem can be solved with probability2−n+O(lg T (n,m)) using OP(T (n,m)) algorithms.

Best-known deterministic algorithm takes time 2npoly(m).

Hamiltonian path problem can be solved with probability 1/n!in OPP.

The best known deterministic exponential time algorithmtakes time 2O(n)poly(m).
























Time and Success Probability Trade-off

Let X (n) = (lg t + lg 1/p)/n where p is the best successprobability for time t.

Let X = limn→∞ X (n).

How does X behave as a function of t?

For the Circuit Sat problem, based on the best knownalgorithms, X = 1 whether t is polynomial in n or exponentialin n.

On the other hand, for Hamiltonian Path, based on thebest known algorithms, X =∞ when t is polynomial in n andX ≤ 1 when t is exponential is n.

For what problems, does this quantity decrease/stay the sameover a certain range of time?

We present (weak) evidence that for Circuit Sat,(log t + log 1/p)/n may not significantly decrease withincreasing time.































































Circuit Family for deciding Circuit Sat

Fn,m

Fn,m(y, z)

6 6 6 66 6

y = desc(D), m = |y|

66666666666

random bits (z)

D

6666666

Circuit D with n variables

Probabilistic Circuit for CircuitSatPaturi (S)ETH and A Survey of Consequences


Success Probability for Circuit Sat with OPPalgorithms

Theorem (P, Pudlak 2010)

If Circuit Sat can be decided with probabilistic circuits of sizemk for some k with success probability 2−δn for δ < 1, then thereexists a µ < 1 depending on k and δ such that Circuit Sat(n,m)

can be decided by deterministic circuits of size 2O(nµ lg1−µ m).



Results: Quasilinear Size Circuits

Theorem

If Circuit Sat can be decided with probabilistic circuits of sizeO(m) with success probability 2−δn for δ < 1, thenCircuit Sat(n,m) can be decided by deterministic circuits ofsize O(poly(m)nO(lg lg m)).

The consequence is very close to the statementNP ⊆ P/poly.



Results: Quasilinear Size Circuits

Theorem

If Circuit Sat can be decided with probabilistic circuits of sizeO(m) with success probability 2−δn for δ < 1, thenCircuit Sat(n,m) can be decided by deterministic circuits ofsize O(poly(m)nO(lg lg m)).

The consequence is very close to the statementNP ⊆ P/poly.



Success Probability for Circuit Sat with SubexponentialSize Circuits

Theorem (P, Pudlak 2010)

If Circuit Sat can be decided with probabilistic circuits of size2o(n)O(m) with success probability 2−δn for δ < 1, thenCircuit Sat(n,m) can be decided by deterministic circuits ofsize 2o(n)poly(m).



Exponential Amplification Lemma

Lemma (P and Pudlak 2010)

Exponential Amplification Lemma: Let F be an f -boundedfamily for some f : N× N→ R that decides Circuit Sat withsuccess probability 2−δn for 0 < δ < 1. Then there exists ag -bounded circuit family G that decides Circuit Sat withsuccess probability at least 2−δ

2n whereg(n,m) = O(f (dδne+ 5, O(f (n,m)))).

F : (f (n,m), δn)→ G : (g(n,m), δ2n)



Exponential Amplification Lemma

Lemma (P and Pudlak 2010)

Exponential Amplification Lemma: Let F be an f -boundedfamily for some f : N× N→ R that decides Circuit Sat withsuccess probability 2−δn for 0 < δ < 1. Then there exists ag -bounded circuit family G that decides Circuit Sat withsuccess probability at least 2−δ

2n whereg(n,m) = O(f (dδne+ 5, O(f (n,m)))).

F : (f (n,m), δn)→ G : (g(n,m), δ2n)



Drucker’s Recent Result

Theorem (Drucker, 2013)

For any µ < 1, if there is an OPP algorithm which takes thedescription of a 3-sat formula of length m as input and decides itssatisfiability with success probability at least 2−m

µ, then

NP ⊆ coNP/poly



Other Connections

Hardest instances

Satisfiability and circuit lower bounds

· · ·



Open Problems

Assuming ETH or other suitable assumption, prove

a specific lower bound on s3

s∞ = 1 (SETH)

Assuming SETH, can we prove a 2n lower bound onColorability?

Are there better non-OPP algorithms for k-sat orCircuit Sat?

Does there exist a c−n success probability OPP algorithm forHamiltonian Path?



Open Problems



s∞ = 1 (SETH)






Open Problems



s∞ = 1 (SETH)






Open Problems



s∞ = 1 (SETH)






Open Problems



s∞ = 1 (SETH)






Open Problems



s∞ = 1 (SETH)






Thank You


Documents

(S)ETH and A Survey of Consequences€¦ · Exact Algorithms and ComplexityExponential Time HypothesisExplanatory Value of ETH and SETH Probabilistic Polynomial Time AlgorithmsOpen