University of British Columbia Dancing in the Dark: Private Multi-Party Machine Learning in an Untrusted Setting Clement Fung, Jamie Koerner, Stewart Grant, Ivan Beschastnikh Challenges ● Machine learning (ML) at scale is popular: ○ But not privacy focused, vulnerable ● Differentially private ML exists: ○ But theoretical and not well applied ● Tor allows anonymity in P2P networks ○ But how do we use it for ML? Partners Our contributions 1. Brokered Learning: A design paradigm and API for ML in untrusted setting 2. TorMentor: a system for anonymous, differentially-private multi-party ML 3. We translate known ML attacks (Information Leakage, Poisoning) and defenses (RONI, Differential Privacy) to a private, untrusted setting Evaluation results 1. Effects of client batch size and privacy parameters in multi-party ML 2. Defending against inversion: Including users, DP mitigates attacks 3. Defending against poisoning at scale: Tuning RONI threshold References 1. McMahan et al. “Communication-Efficient Learning of Deep Networks from Decentralized Data ”, AISTATS ‘17re Aggr 2. Song et al. “Stochastic Gradient Descent With Differentially Private Updates, GlobalSIP’ 13 3. Huang et al. "Adversarial Machine Learning", AISec ‘11 4. Hitaj et al. "Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning" CCS'17 TorMentor design Brokered Learning API 1. Curator: define model, validation set 2. Client: Join model, privacy constraints We trust cloud providers with our data for Machine Learning ( ML). Future Research Directions 1. Investigate client incentives for anonymous ML 2. Improve understandability of privacy parameters 3. Mitigation of attacks with fewer assumptions 4. Remove trust model in broker Defense ingredients Differentially private model ML with ε noise 3. Tor: Anonymize network traffic source 2. RONI: Verify correct ML; use validation score RONI Model Verify ᵂ Model update Information Leakage: Copy private training data through observing ML model while training ML attacks we consider Global ML Server Random Model Poisoning: Manipulate performance of global model with bad updates Global ML Server Provide trusted guarantees to facilitate anonymous ML Privately use training data, (Privacy ε, batch size bystanders) 3. Train model via differentially private SGD 4. Anonymously return model to curator Wants to train ML model, need training data Can we support ML in private, anonymous, untrusted settings? 1. Differential Privacy: Privacy in ML from ε noise

Dancing in the Dark: Private Multi-Party Machine Learning ... · Dancing in the Dark: Private Multi-Party Machine Learning in an Untrusted Setting Clement Fung, Jamie Koerner, Stewart

Download PDF Report

Upload
others
View
0
Download
0

Embed Size (px)

Citation preview

Page 1: Dancing in the Dark: Private Multi-Party Machine Learning ... · Dancing in the Dark: Private Multi-Party Machine Learning in an Untrusted Setting Clement Fung, Jamie Koerner, Stewart

University of British Columbia

Dancing in the Dark: Private Multi-Party Machine Learning in an Untrusted SettingClement Fung, Jamie Koerner, Stewart Grant, Ivan Beschastnikh

Challenges● Machine learning (ML) at scale is popular:

○ But not privacy focused, vulnerable● Differentially private ML exists:

○ But theoretical and not well applied● Tor allows anonymity in P2P networks

○ But how do we use it for ML?

Partners

Our contributions1. Brokered Learning: A design paradigm and API for ML in untrusted setting2. TorMentor: a system for anonymous, differentially-private multi-party ML3. We translate known ML attacks (Information Leakage, Poisoning) and defenses (RONI,

Differential Privacy) to a private, untrusted setting

Evaluation results1. Effects of client batch size and privacy parameters in multi-party ML2. Defending against inversion: Including users, DP mitigates attacks3. Defending against poisoning at scale: Tuning RONI threshold

References1. McMahan et al. “Communication-Efficient Learning of Deep Networks from Decentralized Data ”, AISTATS ‘17re Aggr

2. Song et al. “Stochastic Gradient Descent With Differentially Private Updates, GlobalSIP’ 133. Huang et al. "Adversarial Machine Learning", AISec ‘114. Hitaj et al. "Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning" CCS'17

TorMentor design

Brokered Learning API1. Curator: define model, validation set2. Client: Join model, privacy constraints

We trust cloud providers with our data for Machine Learning (ML).

Future Research Directions1. Investigate client incentives for anonymous ML2. Improve understandability of privacy parameters3. Mitigation of attacks with fewer assumptions4. Remove trust model in broker

Defense ingredients

Differentially private modelML with ε noise

3. Tor: Anonymize network traffic source

2. RONI: Verify correct ML; use validation score

RONI Model

VerifyModel update

Information Leakage: Copy private training data through observing ML model while training

ML attacks we consider

Global ML Server

Random Model Poisoning: Manipulate performance of global model with bad updates

Global ML Server

Provide trusted guarantees to facilitate anonymous ML