CRYPTOGRAPHY AND NP-HARDNESS

CRYPTOGRAPHY AND NP-HARDNESS

Andrej BogdanovChinese University of Hong Kong

Bertinoro Summer School | July 2014

f(p, q) = p∙qeasy

hard?

P ≠ NP

One-way functions and NP

“Invert a one-way function” is an NP problem

One-way functions are necessary for crypto:

So if P = NP, there is no crypto

If P ≠ NP is there crypto?(Can crypto be based on NP-hardness?)

As hard as possible…

One-way functions are NP-hard to invert.

“Theorem”

“Proof:”f(f, a) = (f, f(a))

If algorithm I inverts f, then I(f, true) solves SAT

Worst-case versus average-case

Solve SAT:

Invert a one-way function:

For every satisfiable f, A(f) outputs a satisfying assignment

For a (noticeable) fraction of inputs x, I(f(x)) is a preimage for f(x)

x

f(x)

One-way permutations

One-way functions are NP-hard to invert.

Fake Theorem

Real TheoremOne-way permutations are not NP-hard to invert, unless NP = coNP.

NP and coNP

Verifier Prover

(x1 ∨ x2) ∧ (x1 ∨ x3) ∧ (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x3)

T, T, FOK!

NP:Honest prover can certify true instances.No cheating prover can certify false instances.

has a satisfying assignment.

NP and coNP

Verifier RefuterT, T, FOK!

coNP:Honest refuter can refute false instances.No cheating one can refute true instances.

(x1 ∨ x2) ∧ (x1 ∨ x3) ∧ (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x3)has no satisfying assignment.

NP and coNP

Is NP = coNP?

“f is satisfiable” has efficient proofs,but no reason to believe it has efficient refutations.

SAT ∈ NP, but most likely SAT ∉ coNP

... and so NP ≠ coNP

What does it mean?

Suppose some permutation f is NP-hard to invert. What does this mean?

SAT Ireduction

f invert f at 762193

One-way permutations are not NP-hard to invert, unless NP = coNP.

168321

invert f at 111111111111f ∈/∉ L

L

x

x

Simulating reduction by a proof system

L Ireduction

x invert f at y1

x1. . .

Verifier Prover

Verifier: On input f, simulate reduction.Prover: Provide query/answer pairs y1, x1, ..., yt, xt

Verifier: If f(x1) = y1 and ... and f(xt) = yt , and consistentoutput answer of reduction,

otherwise declare “prover is cheating!”

From a proof system to a proof

invert f at y1

x1

. . .

Verifier Prover

invert f at yt

xt

y1, x1, ..., yt, xt

x ∈/∉ L

This is an NP proof system for Land also an NP proof system for Lso L is both in NP and in coNP.

Conclusion

If one-way permutations are “L-hard” to invert, then L is both in NP and in coNP.

In particular, if L is SAT, we conclude SAT is in coNP, so NP = coNP.

Next: Variations of this theme, but alwaysVerifier emulates reduction

Prover emulates inversion oracle

A trick question

Where did we use that f is a permutation?

How realistic is our model of the reduction?

The “right” question to ask:

L Ireduction

x invert f at y1

x1. . .

The reduction may be randomized

An example

Fix a cyclic group with generator g and look at f(x) = gx

Worst-case to average-case reduction:If y = f(x) can be inverted for 1% of x, then it can be inverted for all x

Proof:

Repeat until y = f(x).^Choose random r and compute x s.t. x + r = I(y∙gr)

^^On input y,

Emulating randomized reductions

y1, x1, ..., yt, xtVerifier Prover(reduction R?) (inverter I)x ∈/∉ L

Who should provide the randomness r? Answer: The verifier

Correctness of randomized reductions:Given an inversion oracle I for f

For most r, RI(x; r) = L(x)

Emulating randomized reductions

y1, x1, ..., yt, xt

Verifier Prover(reduction R?) (inverter I)x ∈/∉ L

randomness r

Verifier is randomized, proof system is interactive

If one-way permutations are L-hard to invert wrt randomized reductions, then L ∈ AM ∩ coAM.

Permutations vs. functions

Where did we use that f is a permutation?Correctness of randomized

reductions:Given an inversion oracle I for fFor most r, RI(x; r) = L(x)

Verifier and prover should agree on inversion oracle before r is chosenWhen f is a permutation, I is unique

Two-to-one functions

f: {0, 1}n → {0, 1}n-1 is 2-to-1 if for every y, |f-1(y)| = 2

choi

ce o

f r

choice of I

RI(x; r) = L(x)

y1, x1, ..., yt, xtVerifier Proverr

If for every i, f(yi) = xi

output answer of reduction,otherwise “prover is cheating!”

Such an f has 22 possible inversion oraclesn-1

Proof system for 2-1 functions

Prover and verifier agree to emulate inversion oracle that answers with lexicographically smaller inverse

y1, x1, x1’ ..., yt, xt, xt’Verifier Proverr

output answer of reduction

otherwise “prover is cheating!”

If for every i, f(xi) = f(xi’) = yi and xi < xi’

with transcript y1, x1, ..., yt, xt ch

oice

of r

choice of I

RI(x; r) = L(x)

K-to-one functions

Say f is K-to-1 if for every y, |f-1(y)| = K

Complexity of proof system grows linearly in K When say K = 2n/2 this is exponential in n

Can we do better?

INTERACTIVE PROOFS

Graph isomorphism

is isomorphic to

Claim:

Proof:

Graph non-isomorphism

is not isomorphic to

Claim:

Interactive proof:G0

G1

Verifier:Choose random bit b, permutation pSend graph G = p(Gb)

Prover: Answer with b’Verifier:If b’ = b, declare “probably not isomorphic”

Graph non-isomorphism

Analysis:If G0, G1 not isomorphic, then prover knows for surethat G came from Gb, so he can answer b If G0, G1 isomorphic, then G is equally likely to have come from G0 /G1, so he can guess b with prob 1/2

Is there a classical proof system for graph non-isomorphism?

Interactive proofs

Given a promise problem L = (LYES, LNO)

Verifier Proverx ∈ LYES ∪ LNO randomized

efficientunboundedq1

a2

qR-1aR

. . .

YES/NO

Completeness:If x ∈ LYES, Pr[VP(x) = YES] ≥ 0.6Soundness: If x ∈ LNO, Pr[VP*(x) = YES] < 0.4

for every P*

Normal form for interactive proofs

The class AM consists of those decision problems that have constant round interactive proofs Such proofs have a normal form

a(x, r)Verifier Proverpublic randomness r

There is a compiler for converting protocols into this form; we’ll do an example instead.

The set size lower bound protocol

Input:

A set C ⊆ {0, 1}n (specified by a circuit) A size estimate 0 < S < 2n

LYES: |C| ≥ S

LNO: |C| < S/8

Verifier:

Interactive proof:Send a random 2-universal hash functionh: {0, 1}n → {0, 1}r where 2S ≤ 2r < 4S

Prover: Send x (and a proof that x ∈ C) Verifier:

Accept if x ∈ C and h(x) = 0.

The set size lower bound protocol

Input:

A set C ⊆ {0, 1}n

A size estimate 0 < S < 2n

LYES: |C| ≥ S

LNO: |C| < (1 – e)S

An error parameter e > 0

Running time of verifier is linear in |C|/e

Proof:

Run original protocol on (Ck, Sk), k = 3/e

Graph non-isomorphism via set size

Given G0, G1 we want a proof of non-isomorphism For simplicity we’ll assume G0, G1 have no automorphisms

C = {p(Gb): p is a permutation, b is a bit}

G0, G1 are isomorphic |C| = n!

G0, G1 are not isomorphic |C| = 2∙n!

Reduction to set size lower bound:

AM ≈ NP

a(x, r)Verifier Proverpublic randomness r

If we replace r by the output of a suitable pseudo-random generator, proof can be derandomizedUnder a plausible assumption in complexity theory, AM = NP.

BACK TO CRYPTOGRAPHY

Hardness of regular one-way functions

Say f: {0, 1}n → {0, 1}n - k is 2k-to-1Suppose we have a reduction R? that, given an inverter I for f, solves L

Verifier will emulate reduction

Prover will emulate random inverter IGiven a query y, return each x s.t. f(x) = y with probability 2-k independently of previous queries and answers

Hardness of regular one-way functions

y1

x1 = I(y1)

. . .

Verifier Prover

yt

xt = I(yt)x ∈/∉ L

x ∈ L Prr, I[RI (x; r) accepts] ≥ 2/3

x ∉ L Prr, I[RI (x; r) accepts] < 1/3

|{(r, x1, …, xt) valid and accepting}| ≥ (2/3) 2|r| + kt

|{(r, x1, …, xt) valid and accepting}| < (1/3) 2|r| + kt

Hardness of regular one-way functions

y1

x1 = I(y1)

. . .

Verifier Prover

yt

xt = I(yt)x ∈/∉ L

x ∈ L Prr, I[RI (x; r) rejects] ≥ 2/3

x ∉ L Prr, I[RI (x; r) rejects] < 1/3

|{(r, x1, …, xt) valid and rejecting}| ≥ (2/3) 2|r| + kt

|{(r, x1, …, xt) valid and rejecting}| < (1/3) 2|r| + kt

General one-way functions

If one-way functions are L-hard to invert wrt non-adaptive reductions, then L ∈ AM ∩ coAM.A non-adaptive reduction can query the inverter many times, but all queries made at the same timeExampleWorst-case to average-case for matrix multiply

Injective one-way functions

As a model example we’ll look at injective one-way functions f: {0, 1}n → {0, 1}m where m = n + 1

QuestionWhat goes wrong if we apply the proof system for permutations?

Injective one-way functions

To begin, we’ll assume that the marginal distribution of every query is uniform over {0, 1}m That is, each of the queries y1, …, yt is uniformly distributed, but they can be arbitrarily correlatedProof system will emulate inverter I that returns unique inverse when one exists, ⟘when not

Statistics of coin tosses

Toss k coins, each heads with probability p

Pr[more than (p + e)k heads] < exp(-e2k/2)

E[number of heads] = pk

Pr[fewer than (p - e)k heads] < exp(-e2k/2)

Proof system for uniform queries

Idea: Use query statistics to hold prover in check

y11 y12 … y1t y21 y22 … y2t

yk1 yk2 … ykt

r1

r2

rk

. . .

. . .

In every column, we expect to see k/2 queries that have an inverse.Moreover, this number will be within (½ ± e)k with probability 1 - 2exp(-e2k/2)

Run the reduction independently many times

Proof system for uniform queries

Union bound over columns:Every column will contain (½ ± e)k invertible entries with probability 1 – 2t exp(-e2k/2)

y11 y12 … y1t y21 y22 … y2t

yk1 yk2 … ykt

. . .

For e = 1/(10t), k = O(t2

log t), w/p 90% whole table has tk/2 ± k/9 invertible entriesThis is fewer than one “error”

per row

Proof system for uniform queries

Verifier:

Send k random seeds for reductionProver: For every query yij, send inverse xij

or ⟘Verifier:

If

Interactive proof:

1. f(xij) = yij for every xij

≠ ⟘2. At least tk/2 - k/9 of the xij’s

≠ ⟘For every i, emulate reduction assuming transcript is ri; yi1, xi1, ..., yik, xik Output majority of answers

Analysis of proof system

1. f(xij) = yij for every xij

≠ ⟘2. At least tk/2 - k/9 of the xij’s

≠ ⟘By statistics, honest prover can meet his quota of inverses with high probability

Honest prover analysis:

Analysis of proof system

1. f(xij) = yij for every xij

≠ ⟘2. At least tk/2 - k/9 of the xij’s

≠ ⟘Cheating prover is required to provide ptk - k/9 inversesBy statistics, table has at most tk/2 + k/9 invertible entries whpSo prover cannot cheat on more than 2/9 of rowsThis is unlikely to affect majority of reduction runs

Conclusion

If one-way functions are L-hard to invert wrt non-adaptive reductions with uniform marginal query distribution, then L ∈ AM ∩ coAM.QuestionWhere did we use uniformity of marginals?

Answers1. All queries are identically distributed2. To calculate the expected fraction of inverses

easy✓

Smooth distributions

Smooth distributionsDistribution Y is K-smooth if its probability density function is uniformly bounded by K

Namely, for every y in {0, 1}m, PrY ~ Y[Y = y] ≤ K 2-m

Idea: Use the prover to learn it

In general, the verifier may not know the probability that a reduction query is invertible

Abstracting the problem

Verifier does not know Y (beyond it being K-smooth), but can sample from Y Verifier wants to learn pY = PrY ~ Y [y has inverse] Verifier: Generate many independent samples from Y and ask which are invertibleIf prover is honest, verifier can estimate pY as the fraction of invertible samples

Inverting smoothly distributed queries

Given a sample Y ~ Y generated by the verifier, determine (with confidence) if Y has an inverse Idea: Leverage similarity between Y and the Uniform distributionExample:

U = uniform over {0, 1, 2, ..., 999}Y = uniform over {0, 3, 6, ..., 999}

11 samples from U, one from Y. Which one?778 882 63 78 752 158 685 718 657 222 874780

Inverting smoothly distributed queries

Verifier: Generate one sample y from Yand q samples u1, ..., uq from UShuffle them randomly and send to prover.Prover: Send an inverse or for every ⟘sample.Verifier: If at least (½ – e)q of the inverses are correct, accept the prover’s answer for y.

Inverting smoothly distributed queries

Intuition of analysisBy statistics, honest prover can pass requirementCheating prover can lie on at most 2eq U-queriesCheating prover cannot tell a U-query from a Y-query with advantage more than K so the chances it slips an undetected lie on

the Y-query is at most 2eq/K Complexity is polynomial in K and 1/confidence

HISTOGRAMS

Histograms

A histogram of distribution Y maps probabilities p in [0, 1] to number of outcomes with probability p

Example: 3 tosses of a 1/3-biased coin

1/272/27

4/27

8/27

For distributions over {0, 1}m, a histogram may require description length exponential in m

Approximate histograms

An e-(approximate) histogram assigns to every bin the number of outcomes in that probability range

Partition the interval [0, 1] into bins of the form[(1 - e)i, (1 - e)i-1)

For distributions described by a sampler S, size of such histogram is polynomial in |S| and 1/e

Learning histograms

Input:

A distribution Y (specified by a sampler) A histogram h

HISTYES: h is the e-histogram of Y HISTN

O: h is far from the e-histogram of Y

An accuracy parameter e > 0

HIST ∈ AM [Haitner, Mahmoody, and Xiao]

Learning histograms

Verifier: Check ∑hi = 2m

Generate q samples from Y

≳ h0l-1q are l-1-heavy ≳ (h0l-1 + h1l-2)q are l-2-heavy

etc.

Ask prover to show:

and ∑hi l-i ≥ 1

l-5

l-4

l-3

l-2

l-1

1

h0

h1

l = 1 – eh2

h4

h∞

0

Why cheating provers fail

Verifier: Check ∑hi = 2m

Generate q samples from Y

≳ h0l-1q are l-1-heavy ≳ (h0l-1 + h1l-2)q are l-2-heavy

etc.

Ask prover to show:

and ∑hi l-i ≥ 1

l-5

l-4

l-3

l-2

l-1

1

h0h1h4

h∞

0

h2

Why cheating provers fail

Verifier: Check ∑hi = 2m

Generate q samples from Y

≳ h0l-1q are l-1-heavy ≳ (h0l-1 + h1l-2)q are l-2-heavy

etc.

Ask prover to show:

and ∑hi l-i ≥ 1

l-5

l-4

l-3

l-2

l-1

1

h0h1h4

h∞

0

h2

Learning pY via histograms

Prover: For each bin i, make aclaim for number ni of invertible ys in binVerifier: Check ∑ni 2-m ≥ ½Generate q samples from Y; ask for proofs ≳ n0l-1q are l-1-heavy with inverse

≳ (n0l-1 + n1l-2)q are l-2-heavy with inverse etc. Output pY = n0 + n1l-1 + ... .^

[Holenstein and Künzler]

l-5

l-4

l-3

l-2

l-1

10

n0n4

n2n∞

ARBITRARY QUERIES

Heavy queries

Call a sample y ~ Y K-heavy if PrY ~ Y[Y = y] > K 2-m

f(U)Y

K

y ∈ {0, 1}n

Pry ~ U[ y is K-heavy ] ≤ 1/KPry ~ f(U) [ y is K-heavy ] ≤ 2/K = 1% for K =

200

Irrelevance of heavy queries

Since only 1% of f’s outputs are heavy, it doesn’t matter what the inverter does on them

Prover and verifier agree to emulate inverter that

If y is in image of f, outputs unique inverse xIf not, outputs (no inverse)⟘

If y is heavy, outputs HOtherwise,

The proof system

r1 y11 y12 … y1t r2 y21 y22 … y2t

rk yk1 yk2 … ykt

. . .

. . .

Verifier Prover

Verifier can hold prover in check if she knowspH = Pry ~ Y [ y is K-heavy ]

pX = Pry ~ Y [ y is invertible and not K-heavy ]

x11 ⟘ … x1t x21 x22 … H

xk1 H … ⟘

. . .

Probabilities from the histogram

l-2

l-1

1K2-m2-m

pH = Pry ~ Y [ y is K-heavy ]

pX = Pry ~ Y [ y is invertible and not K-heavy ]

The proof system

Verifier:

Verify historgams. Calculate pX and pH

Prover:Send queries for k reduction runs

Prover: Send histogram claims for Y andinvertible samples

Answer each query with H, xij, or ⟘Verifier:

Check ≳ pXkt answers are xij and correct Check ≳ pHkt answers are H

Prover: Prove queries answered by H are K-heavy

Summary

We saw how to rule out NP-hardness* of inverting

Regular one-way functions(more generally, range-computable functions) General one-way functions, but only under non-adaptive reductions

One-way permutations