View
216
Download
1
Category
Tags:
Preview:
Citation preview
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 1ETH Zurich – Distributed Computing Group
PhD Thesis Raphael Eidenbenz Prof. Roger Wattenhofer, examinerProf. Dov Monderer, co-examiner
Prof. Karl Aberer, co-examiner
Coping with Selfishness in Distributed SystemsMechanism Design in Multi-Core and
Peer-to-Peer Systems
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 2
Selfishness in Computer Science
„The Internet is unique among all computer systems in that it is built, operated, and used by a multitude of diverse economic interests, in varying relationships of collaboration and competition with each other.“ C. Papadimitriou, STOC 2001
Game theory attempts to mathematically capture behavior in strategic situations (games), in which an individual's success in making choices depends on the choices of others.
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 3
Game Theory & Mechanism Design
• Game Theory explains/predicts behavior
• Mechanism Design Selfishness-aware optimization
ETH Zurich – Distributed Computing Group Raphael Eidenbenz 4ETH Zurich – Distributed Computing Group
Raphael Eidenbenz
Mechanism Design with Payments
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 5
Mechanism Design with Payments
silent testify
silent3 0
3 4
testify4 1
0 1
1 2 1 00 0
2 0
+ =
4 2 4 44 1
2 1
cost = 2net gain = 2
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 6
Optimal Implementations
20 11 10 10
0 9 10 10
11 20 10 10
9 0 10 1019 10 9 0 10 19 11 20
10 19 0 9
19 10 20 11
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 7
20 11 10 10
0+ 9+ 10 10
11 20 10 10
9+ 0+ 10 1019+1 10+1 9+∞ 0+∞
10+1 19+1 11 20
10+1 19+1 0+∞ 9+∞
19+1 10+1 20 11
Optimal Implementations
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 8
Complexity
5 0 0 0 0 1 0
0 5 0 0 0 1 0
0 0 5 0 0 1 0
0 0 0 5 0 1 0
0 0 0 0 5 1 0
5 0 0 5 0 0 0
0 5 0 5 0 0 0
0 5 5 0 5 0 0
5 5 5 0 0 0 0
,
, ,
,
, ,
𝒔𝟏
𝒔𝟐
𝒔𝟑
𝒔𝟒
1 2
Optimal exact implementation NP-hard
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 9
Worst-Case Leverage
• «maximum worst-case net gain»
𝑋∗max
𝐺
𝑚𝑖𝑛(𝑈−𝑉 ¿)¿
𝑂
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 10
Uniform Leverage
• «maximum average net gain»
𝑋∗𝐺
𝑂
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 11
Result Overview
Uniform Worst-case
Exact implementation cost NP-hard NP-hard (conjecture)
Implementation cost NP-hard NP-hard (conjecture)
Singleton implementation cost
Polynomial approximation ratio -
(Malicious) exact leverage NP-hard -
(Malicious) leverage NP-hard As hard as cost
Singleton leverage
Polynomial approximation ratio -
ETH Zurich – Distributed Computing Group Raphael Eidenbenz 12ETH Zurich – Distributed Computing Group
Raphael Eidenbenz
Multi-Core Architecture
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 13
The Multicore Revolution
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 14
Transactional Memory
explicit locks
Which transaction shall I abort??
transactions
Timestamp, Polite, Karma,Polka, Randomized, ...
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 15
Transactional Memory is a Game
– Players = programmers– Strategy space = placing of transactions– Their goal: fast execution– Social goal: maximize system throughput
„My thread is the fastest!“
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 16
Good Programming
incRingCounters(Node start){ var cur = start; transaction{ repeat{ cur.doSomething(); cur = cur.next; } until(cur==start) }}
incRingCountersGP(Node start){ var cur = start; repeat{ transaction{cur.doSomething();} cur = cur.next; until(cur==start) }}
long transactions vs short transactions
R1
R3
t
R2
Rs
R1
R3
t
R2
Rs
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 17
Good Programming Incentives
• A CM is GPI compatible iff it – punishes unnecessary locking – and rewards partitioning.
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 18
Quasi Priority Accumulating CM
R1
R3
t
R2
Rs
R1
R3
t
R2
Rs
Thm: Timestamp is not GPI compatible.
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 19
• Not priority based– „Choose random winner“
• Proof Intuition– Unnecessary Locks: stupid because only risk conflict (no priority gain)– Partitioning:
Ti2
Randomized CM
Ti
Ti1
Lemma: Randomized CM is GPI compatible.
Ti2
Ti
Ti1
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 20
• Not priority based– „Choose random winner“
• Proof Intuition– Unnecessary Locks: stupid because only risk conflict (no priority gain)– Partitioning:
Randomized CM
Lemma: Randomized CM is GPI compatible.
Ti
Ti1 Ti2
Ti
Ti2
throughput collaborators (updates/s)
Karma Polka
Timestampthroughput collaborators (updates/s)
throughput collaborators (updates/s)
Randomized
throughput collaborators (updates/s)
thro
ughp
ut s
elfis
h (u
pdat
es/s
)
thro
ughp
ut s
elfis
h (u
pdat
es/s
)
thro
ughp
ut s
elfis
h (u
pdat
es/s
)
thro
ughp
ut s
elfis
h (u
pdat
es/s
)
ETH Zurich – Distributed Computing Group Raphael Eidenbenz 22ETH Zurich – Distributed Computing Group
Raphael Eidenbenz
Peer-to-Peer Computing
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 23
Peer-to-Peer File Sharing
• History
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 24
𝑆2
𝑆1
𝑝1
𝑆3
𝑝3 𝑝2
𝑝4
𝑝5𝑝6
Cyclic Tit-for-Tat Trading
𝑝5𝑆2
𝑝1
𝑝6𝑆3
𝑝4
𝑝5
𝑆1
𝑝3 𝑝2
𝑝4𝑝1
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 25
Cyclic Tit-for-Tat Trading
𝑆2
𝑆1
𝑝1
𝑆3
𝑝3 𝑝2
𝑝4
𝑝5𝑝6
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 26
BitTorrent: Downloads per Peer
1 2 3 4 5 6-10 11-20 >20
52.2%
18.0%
9.2%5.4%
3.5%6.9%
3.0% 1.8%
# downloads
# pe
ers
in %
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 27
Cross-Swarm Cycles in BitTorrent
2-cycles 3-cycles 4-cycles1E+1
1E+3
1E+5
1E+7
1E+9
1E+11
KATBTJTPB
aver
age
# cy
cles
per
pee
r
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 28
Evaluation
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 320
20000
40000
60000
80000
100000
Intra-swarmCycle(2)Cycle(3)Cycle(4)
time (h)
tota
l thr
ough
put
(KB/
s)
10
100
1000Intra-swarm
Cycle(2)
Cycle(3)
Cycle(4)
aver
age
spee
d (K
B/s)
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 29
Evaluation
• Results
IS ISas* C2 C2as*
C3 C3as*
C4 C4as*
0
50
100
150
200a
ve
rag
e s
pe
ed
(K
B/s
)
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 30
𝑝
Cyclic Trading Protocol CYCT4T
source via dist
…
inTableid
…
outTable
Update:SELECT DISTINCT source, distFROM inTableWHERE distance < k − 1 AND via q;
Out-neighbors on cycles from :SELECT idFROM outTable INNER JOIN inTableON outTable.id = inTable.sourceWHERE via = r;
𝑟 𝑞
𝑘− 2
𝑥
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 31
𝑝
Cyclic Trading Protocol CYCT4T
𝑟 𝑞
𝑥
…
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 32
Established Equilibrium
utility
n
?
T4T
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 33
BitThief: Smooth Transition
utility
n
?
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 34
BitThief: Smooth Transition
utility
n
?
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 35
Steganographic Handshake in BitThief
• BitThief is a BitTorrent client that – Free rides with BitTorrent clients [1], and– Trades tit-for-tat (T4T) with other BitThiefs [2]
• Block request sequence• Hybrid approach using PEX
– Order of peer addresses– Forged peer address
BitTorrent
T 4T
[1] Locher et al., Free Riding in Bittorrent is Cheap, HotNets 2006[2] Locher et al., Rescuing Tit-for-Tat with Source Coding, P2P 2007
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 36
?
Steganographic Handshake
Π
Π
Π
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 37
?
Steganographic Handshake
Π
Π
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 38
Steganographic Channels
• P2P File sharing– Block request sequence– Block subset selection
• Timing
• Bandwidth
• Ports
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 39
Encoding Bits Into a Permutation
• Encode a message M in a permutation– Represent M as a factorial number– because – M is encoded into as the Lehmer Code of is 2210.
• Lehmer Code– Counts the # swaps to get to – (1,2,3,4) ..2 swaps.. – (3,1,2,4) ..2 swaps..– (3,4,1,2) ..1 swap..– (3,4,2,1) ..0 swaps..– (3,4,2,1)
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 40
Hidden Broadcast
• Send a secret message to all conspirers ncn-c
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 41
Steganographic Broadcast
• Send a message to all conspirers
• Bittorrent-like p2p file sharing system
k
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 42
Efficient Broadcast
8𝑛𝑐
ln (𝑛𝑐 )
If each conspirer randomly connects to peers, then the subnetwork induced by the conspirers is connected w.h.p.𝑐
Lemma
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 43
Efficient Broadcast
• Space complexity • Communication complexity w.h.p.
Get peer addressesAcquire blocksReveal types of connected peersBroadcast message in conspirer subnetwork
If each conspirer randomly connects to peers, then the subnetwork induced by the conspirers is connected w.h.p.𝑐
Lemma
Algorithm
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 44
Proof of Theorem 10.1
• If each conspirer randomly connects to peers, then the subnetwork induced by the conspirers is connected w.h.p.
• Proof:• For each conspirer , it holds that .• (Chernoff)• • If each edge of a graph G with c nodes is present with probability then G
is connected with probability (Corollary from [Hofstad 2007])
• In such a graph G, all nodes have less than neighbors w.h.p.• Each conspirer implicitly chooses random neighbors in the conspirer
subgraph.
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 45
Stronger Authority Models
• Individual Monitoring– Authority monitors individual communication links, no correlation– where m is the # of blocks
• Complete Monitoring– Authority monitors complete network
• Stochastic Monitoring– Trade-off: Hidden communication vs. False positives
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 46
Broadcast under Stochastic Monitoring
• Regular peers choose their request order permutation according to a distribution C
• Authority classifies a peer as a conspirer if it uses a request order permutation with
• Trade-off in the choice of threshold
– Amount of hidden communication vs. False positives
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 47
Broadcast under Stochastic Monitoring
• Regular peers choose their request order permutation according to a distribution C
• Authority classifies a peer as a conspirer if it uses a request order permutation with
• is a deterministic PRG
ETH Zurich – Distributed Computing Group Raphael Eidenbenz, PhD Thesis 48
Conclusion
t t
Thank You!Questions & Comments?
Recommended