Differentially Private Aggregation of Distributed Time-series ...rlu1/slide/20131026lechen.pdfIntroduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion Motivation

Introduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion

Differentially Private Aggregation of DistributedTime-series with Transformation and Encryption

Part 4 — Differentially Private Aggregation 2

presenter: Le Chen

Nanyang Technological University

[email protected]

October 27, 2013

presenter: Le Chen Nanyang Technological University [email protected]

Differentially Private Aggregation of Distributed Time-series with Transformation and Encryption


Introduction

Outline

Introduction

L2 Sensitivity

Discrete Fourier Transform

Conclusion & Discussion




Introduction

Reference

Cynthia Dwork.Differential Privacy.Invited talk at ICALP, Venice, Italy, July 10-14, 2006.Automata, Languages and Programming, Lecture Notes inComputer Science Volume 4052, 2006, pp 1-12.

Wikipedia.<http:

//en.wikipedia.org/wiki/Differential_privacy>

Elaine Shi, T-H. Hubert Chan, Eleanor Rieffel, Richard Chowand Dawn Song.Privacy-Preserving Aggregation of Time-Series Data.In Network and Distributed System Security Symposium(NDSS), 2011.



<http://en.wikipedia.org/wiki/Differential_privacy>

<http://en.wikipedia.org/wiki/Differential_privacy>


Introduction

Reference

T-H. Hubert Chan, Elaine Shi, and Dawn SongPrivacy-Preserving Stream Aggregation with Fault Tolerance.16th International Conference, FC 2012, Kralendijk, Bonaire,Februray 27-March 2, 2012. Financial Cryptography and DataSecurity, Lecture Notes in Computer Science Volume 7397,2012, pp 200-214.

Vibhor Rastogi, Suman NathDifferentially Private Aggregation of Distributed Time-serieswith Transformation and Encryption.ACM SIGMOD 2010, New York, NY. Proceedings of the 2010ACM SIGMOD International Conference on Management ofData, pages 735-746.




Motivation

Motivation of Aggregation

Figure: System Model




Motivation

Notations

I I = I1 ∪ I2 · · · ∪ IU is a set of vectors.

I Ii = [Ii1, Ii2, · · · , Iin] is a n × 1 vector, where Iij ∈ {0, 1}represents whether the average weight of user is over 200 lb inmonth j .

I nbrs(I ): the data set obtained from adding/removing oneuser’s data from I .

I Qj(I ) outputs ”the number of users over 200 lb” in month j ,

i.e.∑U

n=1 Iikj .

I Q(I ) = {Q1(I ),Q2(I ), · · ·Qn(I )}.




Motivation

LPA (Laplace Perturbation Algorithm)

I By adding LAP(λ) noise to the final results, LPA(Q, λ) isε-differentially private for λ = ∆(Q)/ε.

I Q̃ = Q(I ) + LAPn(λ).

I DLPA: Each user u adds its noise nu to xu.

I Q̃ =∑

u(xu + nu) = Q(I ) +∑

u nu.




Motivation

Basic Distributed Protocol




Motivation

Distributed Decryption

I Distributed keys: The private key λ is shared by U users asλ =

∑u λu.

I Each user u computes his decryption share cu = cλu .

I The aggregator combines c ′ =∏U

u=1 cu.

I The final result t = L(c ′ mod N2)L(gλ mod N2)

mod N.




Motivation

Protocol for Computing Exact Sum

I Encrypt-Sum(xu, ru): the encryption of∑Uu=1(xu + ru) = Q +

∑Uu=1 ru.

I Decrypt-Sum(c , ru): each user u computes a decryption sharec ′u = cλug−ruλ.

I Final result: the aggregator aggregates c ′ =∏U

u=1 c ′u and gets

Q = L(c ′ mod N2)L(gλ mod N2)

mod N.




Motivation

Protocol for Computing Noisy Sum

I Let Yi ∼ N(0, λ) for i ∈ {1, 2, 3, 4} be four Gaussian randomvariables. Then Z = Y 2

1 + Y 22 − Y 2

3 − Y 24 is a Lap(2λ2)

random variable.




Motivation

Protocol for Computing Noisy Sum

where a =∑

u au, Enc(a2) is public, and∑

u bu = 0.




L2 Sensitivity

L2 Sensitivity

I If Q is a query sequence, Q(I ) and Q(I ′) are each vectors.

I L1 distance metric, denoted as |Q(I )− Q(I ′)|1, that measuresthe Manhattan distance

∑j |Qj(I )− Qj(I ′)| between these

vectors.

I L2 distance metric, denoted as |Q(I )− Q(I ′)|2, that measures

the Euclidean distance√∑

j(Qj(I )− Qj(I ′))2.




L2 Sensitivity

Sensitivity




L2 Sensitivity

Sensitivity Example





DFT (Discrete Fourier Transform)

I The DFT of an n-dimensional sequence X is a lineartransform giving another n-dimensional sequence.

I DFT (X ): the j th element DFT (X )j =∑n

i=1 e2π√−1

njiXi .

I DFT (X ): the j th element of the Inverse DFT is

IDFT (X )j = 1n

∑ni=1 e−

2π√−1

njiXi .

I Furthermore, IDFT (DFT (X )) = X .





DFT k(X )

I Denote DFT k(X ) as the first k elements of DFT (X ).

I The elements of DFT k(X ) are called the Fourier coefficientsof the k lowest frequencies and they compactly represent thehigh-level trends in X .

I An approximation X ′ to X can be obtained from DFT k(X ) asfollows:

I PADn(DFT k(X )): appends n − k 0 to DFT k(X ),I compute X ′ = IDFT (PADn(DFT k(X ))).





DFT k(X )

I Obviously X ′ may be different from X as ignoring the lastn − k Fourier coefficients may introduce some error.

I Denote RE kj (X ), short for reconstruction error at the j th

position, to be the value |X ′j − Xj |.





Reconstruction Error





Reconstruction




Fourier Perturbation Algorithm

FPA (Fourier Perturbation Algorithm)

I The parameter λ in FPAk needs to be adjusted in order to getε-differential privacy.

I Since FPAk perturbs the sequence F k , λ has to be calibratedaccording to the L1 sensitivity, ∆1(F k), of F k .





Proof

I (i) holds since ∆2(F k) ≤ ∆2(Q), as the n Fourier coefficientshave the same L2 norm as Q, while F k ignores the last n − kFourier coefficients,

I and ∆1(F k) ≤√

k∆2(F k), due to a standard inequalitybetween the L1 and L2 norms of a sequence.

I Let xj = |Qj(I )− Qj(I ′)|, then ∆1(Q) =∑

j |Qj(I )− Qj(I ′)|=

∑j xj , ∆2(Q) =

√∑j(Qj(I )− Qj(I ′))2 =

√∑j x2

j . Since

the inequality ∑j xj

n≤

√∑j x2

j

n

holds, we can see that ∆1(F k) ≤√

k∆2(F k).





Proof

I (ii) follows since for λ =√

k∆2(Q)/ε ≥ ∆1(F k)/ε,

I F̃ k = LPA(F k , λ) computed in step 2 is ε-differential private,and Q̃ in step 3 is obtained using F̃ k only.





Accuracy

I The theorem shows that the error by FPAk for each query isk/ε+ RE k

i (Q(I )), but the LPA yields an error of n/ε.

I Since the reconstruction error, RE ki (Q(I )), is often small even

for k << n, we expect the error in FPAk to be much smallerthan in LPA.

I This hypothesis is confirmed in our experiments that showthat FPAk gives orders of magnitude improvement over LPAin terms of error.





Choosing the Right k

I So far we have assumed that k is known to us.

I Since errori (FPAk) is k/ε+ RE ki (Q(I )), a good value of k is

important in obtaining a good trade-off between theperturbation error, k/ε, and the reconstruction error,RE k

i (Q(I )).

I If k is too big, the perturbation error becomes too big, while ifk is too small the reconstruction error becomes too high.





Choosing the Right k

I We can often choose k based on prior assumptions aboutQ(I ).

I For instance, if Q(I ) is such that the Fourier coefficientscorresponding to Q(I ) decrease exponentially fast, then only aconstant number (say k = 10) of Fourier coefficients need tobe retained during perturbation.

I Our experiments show that this naive method is applicable inmany practical scenarios as Fourier coefficients of manyreal-word sequences decrease very rapidly.






I Last week, we introduced an aggregation protocol supportsdistributed differential privacy and distributed decryption.

I This week we talked about using (DFT) Discrete FourierTransform in LPA (Laplace Perturbation Algorithm) to getFPA (Fourier Perturbation Algorithm), and discussed the errorof DFT and chosen of k.

I FPA can achieve the same differential privacy level with lesserror, the noise actually reduces by a factor of n/k .



Documents

Differentially Private Aggregation of Distributed Time-series ...rlu1/slide/20131026lechen.pdfIntroduction L2 Sensitivity Discrete Fourier Transform Conclusion & Discussion Motivation