Upload
yen
View
50
Download
4
Tags:
Embed Size (px)
DESCRIPTION
Privacy by Learning the Database. Moritz Hardt DIMACS, October 24, 2012. Isn’t privacy the opposite of learning the database?. Curator. query set Q. Analyst. privacy-preserving structure S accurate on Q. data set D = multi-set over universe U. - PowerPoint PPT Presentation
Citation preview
Privacy by Learningthe Database
Moritz Hardt DIMACS, October 24, 2012
Isn’t privacy the opposite of learningthe database?
Curator
Analyst
data set D= multi-set
over universe U
query set Q
privacy-preservingstructure S
accurate on Q
. . .1 2 3 N4 5
Data set D as N-dimensional histogram where N=|U|
D[i] = # elements in Dof type i
Normalized histogram = distribution over universe
1
0
Statistical query q (aka linear/counting):
. . .1 2 3 N4 5
Vector q in [0,1]N
q(D) := <q,D>
q(D) in [0,1]
Why statistical queries?
• Perceptron, ID3 decision trees, PCA/SVM, k-means clustering [BlumDworkMcSherryNissim’05]
• Any SQ-learning algorithm [Kearns’98]– includes “most” known PAC-learning algorithms
Lots of data analysis reduces to multiple statistical queries
Curator’s wildest dream:
This seems hard!
Curator’s 2nd attempt:
Intuition:Entropy implies privacy
Two pleasant surprises
Approximately solved by multiplicative weights update [Littlestone89,...]
Can easily be made differentially private
Why did learning theorists care to solve privacy problems 20 years ago?
Answer:Entropy implies generalization
Learnerexample set Q
hypothesis haccurate on all
examples
Maximizing entropy implieshypothesis generalizes
Unknown concept
Sensitive databaseQueries labeled by
answer on DB
Synopsisapproximates DB on
query set
Must Preserve Privacy
Unknown conceptExamples labeled by concept
Hypothesisapproximates target concept on examples
Must Generalize
Privacy Learning
How can we solve this?
Concave maximizations.t. linear constraints
EllipsoidWe’ll take adifferent route.
Start with uniform D0
“What’s wrong with it?”Query q violates constraint!
Minimize entropy losss.t. correction
Closed form expression for Dt+1? Well...
Closed form expression for Dt+1? YES!
Relax
Approximate Think
Multiplicative Weights Update
. . .1 2 3 N4 5
0
1
Dt
D
At step t
. . .1 2 3 N4 5
0
1
Dt
Dq
At step tSuppose q(Dt) < q(D)
. . .1 2 3 N4 5
0
1
Dt
Dq
After step t
Multiplicative Weights Update
Algorithm:D0 uniformFor t = 1...T Find bad query q Dt+1 = Update(Dt,q)
How quickly do we run out ofbad queries?
Progress Lemma:
if q bad
Put
Facts:
Progress Lemma:
if q bad
At most steps
Error bound
Algorithm:D0 uniformFor t = 1...T Find bad query q Dt+1 = Update(Dt,q)
What about privacy?
Only step that interacts with D
Differential Privacy [Dwork-McSherry-Nissim-Smith-06]
Two data sets D,D’ are called neighboring if they differ in one element.
Definition (Differential Privacy):A randomized algorithm M(D) is called (ε,δ)-differentially privateif for any two neighboring data sets D,D’ and all events S:
Laplacian Mechanism [DMNS’06]
Given query q:1. Compute q(D)2. Output q(D) + Lap(1/ε0n)
Fact: Satisfies ε0-differential privacy
Note: Sensitivity of q is 1/n
Query selection
… q1 q2 q3 qk
|q(D)-q(Dt)|
Query selection
… q1 q2 q3 qk
|q(D)-q(Dt)|
Add Lap(1/ε0n)
Pick maximal violation
Query selection
… q1 q2 q3 qk
|q(D)-q(Dt)|
Pick maximal violation
Query selection
… q1 q2 q3 qk
|q(D)-q(Dt)|
Lemma [McSherry-Talwar’07]:Selected index satisfies ε0-differential privacyand w.h.p Violation >
Algorithm:D0 uniformFor t = 1...T Noisy selection of q Dt+1 = Update(Dt,q)
Now: Each step satisfies ε0-differential privacy!
What is the total privacy guarantee?
Also use noisy answer in update rule
New error bound:
T-fold composition ofε0-differential privacy satisfies:
Answer 1 [DMNS’06]:
ε0T-differential privacy
Answer 2 [DRV’10]:
(ε,δ)-differential privacy
Note: for small enough ε
Composition Theorem
s Erro
r bou
nd
Optim
ize T,
ε 0
ε,δ
Theorem 1. On databases of size nMW achieves ε-differential privacywith
Theorem 2. MW achieves (ε, δ)-differential privacy with
Optimal dependence on |Q| and n
Offline (non-interactive)
S
Q…
Online (interactive)
q1
q2
a2
a1
?✔H-Ligett-McSherry12,Gupta-H-Roth-Ullman11
See also: Roth-Roughgarden10, Dwork-Rothblum-Vadhan10,Dwork-Naor-Reingold-Rothblum-Vadhan09, Blum-Ligett-Roth08
H-Rothblum10
Algorithm:Given query qt:
• If |qt (Dt)- qt (D) | < α/2 + Lap(1/ε0n)– Output qt (Dt)
• Otherwise– Output qt (D) + Lap(1/ε0n)– Dt+1 = Update(Dt, qt )
Private MW Online [H-Rothblum’10]
Achieves same error bounds!
Overview: Privacy Analysis
• Offline setting: T << n steps– Simple analysis using Composition Theorems
• Online setting: k >> n invocations of Laplace– Composition Thms don’t suggest small error!
• Idea: Analyze privacy loss like lazy random walk (goes back to Dinur-Dwork-Nissim’03)
Privacy Loss as a lazy random walk
Number of Steps
Privacy Loss as a lazy random walk
Number of Steps
Privacy loss
Privacy Loss as a lazy random walk
Number of Steps
lazy lazy lazy lazy lazy
Privacy loss
busy busy busy busy busy
busy round = noisy answer close to forcing update
Privacy Loss as a lazy random walk
Number of Steps
lazy lazy lazy lazy lazy
Privacy loss
1
busy busy busy busy busy
busy round = noisy answer close to forcing update
Privacy Loss as a lazy random walk
Number of Steps
lazy lazy lazy lazy lazy
Privacy loss
busy busy busy busy busy
1 1
busy round = noisy answer close to forcing update
Privacy Loss as a lazy random walk
Number of Steps
lazy lazy lazy lazy lazy
Privacy loss
busy busy busy busy busy
1 1 1
busy round = noisy answer close to forcing update
Privacy Loss as a lazy random walk
Number of Steps
lazy lazy lazy lazy lazy
Privacy loss
busy busy busy busy busy
1 1 1 1
busy round = noisy answer close to forcing update
Privacy Loss as a lazy random walk
Number of Steps
lazy lazy lazy lazy lazy
Privacy loss
busy busy busy busy busy
1 1 1 1 1 1 1 1
busy round = noisy answer close to forcing update
Privacy Loss as a lazy random walk
Number of Steps
lazy lazy lazy lazy lazy
Privacy loss
busy busy busy busy busy
1 1 1 1 1 1 1 1
busy round = noisy answer close to forcing update
W.h.p. boundedby O(sqrt(#busy))
Formalizing the random walk
Imagine output of PMW is 0/1 indicator vector
where vt=1 if round t update, 0 otherwise
Recall: Very few updates! Vector is sparse.
Theorem: Vector v is (ε,δ)-diffpriv.
Let D,D’ be neighboring DBs
Let P,Q be corresponding output distributions
Lemma: (3) implies (ε,δ)-diffpriv.
Approach:1.Sample v from P2.Consider X = log(P(v)/Q(v))3.Argue Pr{ |X| > ε } ≤ δ
Intution:X = privacy
loss
Privacy loss in round t
We’ll show:1. Xt = 0 if t not busy2.|Xt| ≤ ε0 if t busy 3. Number of busy rounds O(#updates)
Total privacy loss
DRV’10E[X1+...+Xk] ≤ O(ε0
2#updates)
AzumaStrong concentrationaround expectation
Defining “busy” eventUpdate condition:
Busy event
…
Offline (non-interactive) Online (interactive)
q1
S q2
a2
Q a1
✔ ✔
What we can do
• Offline/batch setting: every set of linear queries• Online/interactive setting: every sequence of
adaptive and adversarial linear queries• Theoretical performance: Nearly optimal in the
worst case– For instance-by-instance guarantee see H-Talwar10,
Nikolov-Talwar (upcoming!), different techniques• Practical performance: Compares favorably to
previous work! See Katrina’s talk.
Are we done?
What we would like to do
Running time: Linear dependence on |U||U| exponential in #attributes of data
Can we get poly(n)?No, in the worst-case for synthetic data [DNRRV09]even for simple query classes [Ullman-Vadhan10]
No, in interactive setting without restricting query class [Ullman12]
What can we do about it?
Look beyond the worst-case!Find meaningful assumptionson data, queries, models etc
Design better heuristics!
In this talk:Get more mileage out of learning theory!
Sensitive databaseQueries labeled by
answer on DB
Synopsisapproximates DB on
query set
Unknown conceptExamples labeled by concept
Hypothesisapproximates target concept on examples
Privacy Learning
Can we turn this into an efficient reduction?
Yes. [H-Rothblum-Servedio’12]
Informal Theorem: There is an efficient differentially private release mechanism for a query class Q provided that there is an efficient PAC-learning algorithm for related concept class Q’• Interfaces nicely with existing learning
algorithms:– Learning based on polynomial threshold
functions [Klivans-Servedio]– Harmonic Sieve [Jackson] and extension [Jackson,
Klivans, Servedio]
Database as a function
Observation:Enough to learn Ft for t=α,2α,...,(1-α)in order to approximate F
Query q q(D)
High-level idea
Learning algorithm
labeled examples
Observation: If all labels are privacy-preserving,then so will be hypothesis h
Hypothesis h such that
Main hurdles
• Privacy requires noise, noise might defeat learning algorithm
• Can only generate |D| examples efficiently before running out of privacy
Learning algorithm
Threshold Oracle
Compute a=F(x)+N If |a-t| tiny: output “fail”Else if a>t: output 1 Else if a<t: output 0
Ensures:1. Privacy2. “Removes” noise3. Complexity independent of |D|
Generate samples:
1. Pick x1,x2,..,.xm
2. Receive b1,b2,...,bm from TO3. Remove all “failed” examples4. Pass on remaining labeled examples to learner
(y1,l1),....,(yr,lr)
“F(x)>t”?
b in {0,1,fail}
Application: Boolean Conjunctions
Important class of queries in differential privacy [BCDKMT07,KRSU10,GHRU11,HMT12,...]
Salary > $50k Syphilis Height > 6’1 Weight < 180 Male
True False True False True
True True True True True
False False False True False
True False False True True
False False False False False
Example Conjunction: “(Salary > $50k) AND (Male)”Evaluates to on this database
Universe U = {0,1}d
Informal Corollary (Subexponential algorithm for conjunctions).There is a differentially private release algorithm with running time poly(|D|) such that for any distribution over Boolean conjunctions the algorithm is w.h.p. α-accurate provided that:
Informal Corollary (Small width).There is a differentially private release algorithm with running time poly(|D|) such that for any distribution over width-k Boolean conjunctions the algorithm is w.h.p. α-accurate provided that:
Previous:2O(d)
Previous:dO(k)
Follow-up work
• Thaler-Ullman-Vadhan12: Can remove distributional relaxation and get exp(O(d1/2)) complexity for all Boolean conjunctions
Idea: Use polynomial encodings from learning algorithm directly
Summary
• Derived simple and powerful private data release algorithm from first principles
• Privacy/learning analogy as a guiding principle– Can be turned into efficient reduction
• Can we use these ideas outside theory and in new settings?
Thank you
Open problems
• Is PMW close to instance optimal?• Is there a converse to privacy-to-learning
reduction?• No barriers for cut/spectral analysis of
graphs/matrices (universe small)• Releasing k-way conjunctions in time poly(n),
error poly(d,k)