View
38
Download
0
Category
Tags:
Preview:
DESCRIPTION
Distributed Publish/Subscribe. Nalini Venkatasubramanian (with slides from Roberto Baldoni, Pascal Felber, Hojjat Jafarpour etc.). Publish/Subscribe (pub/sub) systems. Asynchronous communication Selective dissemination Push model Decoupling publishers and subscribers. - PowerPoint PPT Presentation
Citation preview
Distributed Publish/Subscribe
Nalini Venkatasubramanian
(with slides from Roberto Baldoni, Pascal Felber, Hojjat Jafarpour etc.)
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 2
Publish/Subscribe (pub/sub) systems
Pub/Sub Service
Stock ( Name=‘IBM’; Price < 100 ; Volume>10000 )
Stock ( Name=‘IBM’; Price < 110 ; Volume>10000 )
Stock ( Name=‘HP’; Price < 50 ; Volume >1000 )
Football( Team=‘USC’; Event=‘Touch Down’)
Stock ( Name=‘IBM’; Price =95 ; Volume=50000 )
Stock ( Name=‘IBM’; Price =95 ; Volume=50000 )
Stock ( Name=‘IBM’; Price =95 ; Volume=50000 )
What is Publish/Subscribe (pub/sub)?• Asynchronous communication • Selective dissemination• Push model• Decoupling publishers and subscribers
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 3
Publish/Subscribe (pub/sub) systems Applications:
News alerts Online stock quotes Internet games Sensor networks Location-based
services Network management Internet auctions …
Subscription Model: Topic based VS Content based Topic based
Generally also known as topic based, group based or channel based event filtering.
Each event is published to one of these channels by its publisher.
Subscribers subscribes to a particular channel and will receive ALL events published to the subscribed channel.
Topic-based subscription
Simple process for matching an event to subscriptions. However, limited expressiveness.
Event filtering is easy, event routing is difficult (Heavy load on the network). The challenge is to multicast event effectively to subscribers.
Subscription Model: Content based Pub/Sub Content based
More flexibility and power to subscribers, by allowing more expression in arbitrary/customized query over the contents of the event.
Event publication by a key/value attribute pair, and subscriptions specify filters using a explicit subscription language.
E.g. Notify me of all stock quotes of IBM from New York stock exchange if the price is greater than 150
Content-based Subscription
Added complexity in matching an event to subscriptions. (Implementation: Subscription arranged in a matching tree, where each node is a partial condition.
However, more precision is provided and event routing is easier
Scalable Publish/Subscribe Architectures & Algorithms — P.
Felber 8
Publish/subscribe architectures Centralized
Single matching engine Limited scalability CORBA Event Services, JMS
Broker overlay Multiple P/S brokers Participants connected to some
broker Events routed through overlay SIENA, Gryphon
Peer-to-peer Publishers & subscribers
connected in P2P network Participants collectively filter/route
events, can be both producer & consumer
Hybrid
Distributed pub/sub systems Broker – based pub/sub
A set of brokers forming an overlay Clients use system through brokers Benefits
Scalability, Fault tolerance, Cost efficiency
Dissemination Tree
Dissemination Tree
10
Challenges in distributed pub/sub systems
Broker overlay architecture• How to form the broker network• How to route subscriptions and publications
Broker internal operations • Subscription management
• How to store subscriptions in brokers
• Content matching in brokers
• How to match a publication against subscriptions
Broker ResponsibilitySubscription Management Matching: Determining the recipients for an eventRouting: Delivering a notification to all the recipients
MINEMA Summer School - Klagenfurt (Austria) July 11-15,
2005 11
EVENT vs SUBSCRIPTION ROUTING
Extreme solutions Sol 1 (event flooding)
flooding of events in the notification event box each subscription stored only in one place within the
notification event box Matching operations equal to the number of brokers
Sol 2 (subscription flooding) each subscription stored at any place within the
notification event box each event matched directly at the broker where the
event enters the notification event box
Major distributed pub/sub approaches Tree-based
Brokers form a tree overlay [SIENA, PADRES, GRYPHON]
DHT-based: Brokers form a structured P2P overlay [Meghdoot, Baldoni et al.]
Channel-based: Multiple multicast groups [Phillip Yu et al.]
Probabilistic: Unstructured overlay [Picco et al.]
12
13
Tree-based
Brokers form an acyclic graph
Subscriptions are broadcast to all brokers
Publications are disseminated along the tree with applying subscriptions as filters
14
Tree-based
Subscription dissemination load reduction Subscription Covering Subscription Subsumption
Publication matching Index selection
MINEMA Summer School - Klagenfurt (Austria) July 11-15,
2005 15
Pub/Sub Sysems: Tib/RV [Oki et al 03] Topic Based Two level hierarchical architecture of brokers
(deamons) on TCP/IP Event routing is realized through one
diffusion tree per subject Each broker knows the entire network
topology and current subscription configuration
MINEMA Summer School - Klagenfurt (Austria) July 11-15,
2005 16
Pub/Sub systems: Gryphon [IBM 00] Content based Hierarchical tree from publishers to
subscribers Filtering-based routing Mapping content-based to network level
multicast
MINEMA Summer School - Klagenfurt (Austria) July 11-15,
2005 17
DHT Based Pub/Sub: SCRIBE [Castro et al. 02]
Topic Based Based on DHT (Pastry) Rendez-vous event routing A random identifier is assigned to each topic The pastry node with the identifier closest to
the one of the topic becomes responsible for that topic
MINEMA Summer School - Klagenfurt (Austria) July 11-15,
2005 18
DHT-based pub/sub MEGHDOOT Content Based Based on Structured Overlay CAN Mapping the subscription language and the
event space to CAN space Subscription and event Routing exploit CAN
routing algorithms
19
Fault-tolerance Pub/Sub architecture
Brokers are clustered Each broker knows all brokers in
its own cluster and at least one broker from every other clusters
Subscriptions are broadcast just in clusters
Every brokers just have the subscriptions from brokers in the same cluster
Subscription aggregation is done based on brokers
20
Fault-tolerance Pub/Sub architecture Broker overlay
Join Leave Failure
Detection Masking Recovery
Load Balancing Ring publish load Cluster publish load Cluster subscription load
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 21
CCD: Customized Content Delivery with Pub/Sub Leveraging pub/sub framework for
dissemination of rich content formats, e.g., multimedia content.
Same content format may not be consumable by all
subscribers!!!
CCD: Customized content delivery with pub/sub
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 22
Español Español!!!Español Español!!!
Customize content to the required formats before
delivery!
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 23
Subscriptions in CCD
How to specify required formats?
Receiving context: Receiving device
capabilities Display screen, available
software,… Communication capabilities
Available bandwidth User profile
Location, language,…
Subscription:• Team: USC• Video: Touch Down
Subscription:• Team: USC• Video: Touch Down
Subscription:• Team: USC• Video: Touch Down
Context: PC, DSL, AVI
Context: Phone, 3G, FLV
Context: Laptop, 3G, AVI, Spanish subtitle
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 24
Content customization
How content customization is done? Adaptation operators
Original contentSize: 28MB
Low resolution and smallcontent suitable for mobile clientsSize: 8MB
TranscoderOperator
Q? - How to perform customization in distributed pub/sub?
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 25
Challenges Option 1: Perform all the required customizations
in the sender broker
28MB
28MB 28MB15MB12MB8MB
8MB
8MB 8MB
15MB
28+12+8 = 48MB 28+12+8 = 48MB
12MB8MB
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 26
Challenges Option 2: Perform all the required customization
in the proxy brokers (leaves)
28MB
28MB 28MB15MB12MB8MB
8MB
8MB 8MB
15MB
28MB 28MB
28MB
Repeated Operator
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 27
Challenges Option 3: Perform all the required customization
in the broker overlay network
28MB
28MB 28MB15MB12MB8MB
8MB
8MB 8MB
15MB
28
CCD: DHT-based pub/sub DHT-based routing schema using
Tapestry [ZHS04]
RendezvousPoint
29
Example using DHT based pub-sub Tapestry (DHT-based) pub/sub and routing
framework Event space is partitioned among peers
Single content matching Each partition is assigned to a peer (RP) Publications and subscriptions are matched in RP
All receivers and preferences are detected after matching Content dissemination among matched subscribers
are done through a dissemination tree rooted at RP where leaves are subscribers.
30
Background
Tapestry DHT-based overlay Each node has a unique L-digit ID
in base B Each node has a neighbor map
table (LxB) Routing from one node to another
node is done by resolving one digit in each step
Sample routing map table for 2120
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 31
Dissemination tree For a published content we can estimate the
dissemination tree in broker overlay network Using DHT-based routing properties The dissemination tree is rooted at the
corresponding rendezvous brokerRendezvous
Point
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 32
Content Adaptation Graph (CAG) All possible content formats in the system All available adaptation operators in the system
Size: 28MBFrame size: 1280x720Frame rate: 30
Size: 8MBFrame size: 128x96Frame rate: 30
Size: 15MBFrame size: 704x576Frame rate: 30
Size: 10MBFrame size: 352x288Frame rate: 30
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 33
Content Adaptation Graph (CAG) A transmission (communication) cost is associated with each format Sending content in format Fi from a broker to another
one has the transmission cost of A computation cost is associated with each
operator Performing operator O(i,j) on content has the
computation cost of F1/28
F3/12F2/15 F4/8
60 60 60
25
25
25
V={F1,F2,F3,F4}E={O(1,2),O(1,3),O(1,4),O(2,3),O(2,4),O(3,4)}
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 34
CCD plan A CCD plan for a content is the dissemination
tree: Each node (broker) is annotated with the
operator(s) that are performed on it Each link is annotated with the format(s) that are
transmitted over it{O(1,2),O(2,4)}
{O(2,3)}{}
{}
{}
{}{}
{F2} {F2} {F4}
{F2} {F3} {F4}
F1/28
F3/12F2/15 F4/8
60 60 60
25
25
25
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 35
CCD algorithm
Input: A dissemination tree A CAG The initial format Requested formats by each broker
Output: The minimum cost CCD plan
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 36
CCD Problem is NP-hard
Directed Steiner tree problem can be reduced to CCD
Given a directed weighted graph G(V,E,w) , a specified root r and a subset of its vertices S, find a tree rooted at r of minimal weight which includes all vertices in S.
CCD algorithm Based on dynamic programming Annotates the dissemination tree in a bottom-up
fashion For each broker:
Assume all the optimal sub plans are available for each child
Find the optimal plan for the broker accordingly
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 37
Ni
NjNk….
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 38
CCD algorithm
F1
F1 F1F2F3
F4
F4
F4
F2
F1/28
F3/12F2/15 F4/8
60 6060
25
25
25
39
Fast and scalable notification using Pub/Sub A general purpose notification system
On line deals, news, traffic, weather,… Supporting heterogeneous receivers
Pub/SubServer
Client
User Profile
User Subscriptions
Notifications
Web
40
User profile
Personal information Name Location Language
Receiving modality PC, PDA
Email Live notification IM (Yahoo Messenger, Google Talk, AIM, MSN)
Cell phone SMS Call
41
Subscription
Subscription language in the system SQL
Subscriptions language for clients Attribute value
E.g., Website = www.dealsea.com Keywords = Laptop, Notebook Price <= $1000 Brand = Dell, HP, Toshiba, SONY
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 42
Experimental evaluation System setup
1024 brokers Matching ratio: percentage of brokers with
matching subscription for a published content Zipf and uniform distributions
Communication and computation costs are assigned based on profiling
42
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 43
Experimental evaluation
Dissemination scenarios Annotated map Customized video dissemination Synthetic scenarios
43
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 44
Cost reduction in CCD algorithm
Matching Ratio
Cost
red
uct
ion
perc
en
tag
e (
%)
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 45
Cost reduction in Heuristic CCD
Matching Ratio
Cost
red
uct
ion
perc
en
tag
e (
%)
Hojjat Jafarpour
CCD: Efficient Customized Content Dissemination in
Distributed Pub/Sub 46
References
[AT06] Ioannis Aekaterinidis, Peter Triantafillou: PastryStrings: A Comprehensive Content-Based Publish/Subscribe DHT Network. IEEE ICDCS 2006.
[CRW04] A. Carzaniga, M.J. Rutherford, and A.L. Wolf: A Routing Scheme for Content-Based Networking. IEEE INFOCOM 2004.
A.Carzaniga, D.Rosenblum, A.Wolf . Design and Evaluation of a Wide-Area Event Notification Service. ACM Transactions on Computer Systems, Vol. 19, No. 3, August 2001.
[DRF04] Yanlei Diao, Shariq Rizvi, Michael J. Franklin: Towards an Internet-Scale XML Dissemination Service. VLDB 2004.
[GSAE04] Abhishek Gupta, Ozgur D. Sahin, Divyakant Agrawal, Amr El Abbadi: Meghdoot: Content-Based Publish/Subscribe over P2P Networks. ACM Middleware 2004
[JHMV08] Hojjat Jafarpour, Bijit Hore, Sharad Mehrotra and Nalini Venkatasubramanian. Subscription Subsumption Evaluation for Content-based Publish/Subscribe Systems, ACM/IFIP/USENIX Middleware 2008.
[JHMV09] Hojjat Jafarpour, Bijit Hore, Sharad Mehrotra and Nalini Venkatasubramanian.CCD: Efficient Customized Content Dissemination in Distributed Publish/Subscribe. ACM/IFIP/USENIX Middleware 2009.
[JMV08] Hojjat Jafarpour, Sharad Mehrotra and Nalini Venkatasubramanian. A Fast and Robust Content-based Publish/Subscribe Architecture, IEEE NCA 2008.
[JMVM09] Hojjat Jafarpour, Sharad Mehrotra, Nalini Venkatasubramanian and Mirko Montanari, MICS: An Efficient Content Space Representation Model for Publish/Subscribe Systems, ACM DEBS 2009.
[OAABSS00] Lukasz Opyrchal, Mark Astley, Joshua S. Auerbach, Guruduth Banavar, Robert E. Strom, Daniel C. Sturman: Exploiting IP Multicast in Content-Based Publish-Subscribe Systems. Middleware 2000.
[ZHS04] Ben Y. Zhao, Ling Huang, Jeremy Stribling, Sean C. Rhea, Anthony D. Joseph, John Kubiatowicz: Tapestry: a resilient global-scale overlay for service deployment. IEEE Journal on Selected Areas in Communications 22(1).
P.Eugster, P.Felber,RGuerraoui and A.Kermarrec. The Many Faces of Publish/Subscribe. In ACM Computing Surveys, Vol. 35, No.2, June 2003.
EXTRA SLIDES AND EXAMPLES
47
48
Super Peer Network
2230
1330
2130
0130
1130
2330
2330
1230
1030
3130
0330
1130
2130
1130
Publisher of C
RP Peer for C
[(Shelter Information, Irvine,
School), (English,Text)]
[(Shelter Information,
Irvine, School),
(English,Text)]
[(Shelter Info, Santa Ana,
School),(Spanish,Voice)]
Speech to text
Speech to text
Translation
49
Super Peer Network
2230
1330
2130
0130
1130
2330
2330
1230
1030
3130
0330
1130
2130
1130
Publisher of C
RP Peer for C
[(Shelter Information, Irvine,
School), (English,Text)]
[(Shelter Information,
Irvine, School),
(English,Text)]
[(Shelter Info, Santa Ana,
School),(Spanish,Voice)]
Speech to text
Translation
50
Super Peer Network
2230
1330
2130
0130
1130
2330
2330
1230
1030
3130
0330
1130
2130
1130
Publisher of C
RP Peer for C
[(Shelter Information, Irvine,
School), (English,Text)]
[(Shelter Information,
Irvine, School),
(English,Text)]
[(Shelter Info, Santa Ana,
School),(Spanish,Voice)]
Speech to text
Translation
51
CCD: System model
Set of supported formats and communication cost for transmitting content in each format
Set of operators with cost of performing each operator
Operators are available is all brokers
52
CCD: System model
Content Adaptation Graph Represents available formats and operators and their
relation G = (V , E) where V = F and E = O FxF
Optimal content adaptation is NP-Hard Steiner tree problem
For a given CAG and dissemination tree, , find CCD plan with minimum total cost.
53
CCD: System model
Subscription model: [SC,SF ] where SC is the content subscription and SF
corresponds to the format in which the matching publication is to be delivered. S=[{SC:Type = ’image’, Location = ’Southern California’,
Category = ’Wild Fire’},{Format = ’PDA-Format’}] Publication model:
A publication P = [PC,PF ] also consists of two parts. PC contains meta data about the content and the content itself. The second part represents the format of the content. [{Location = ’Los Angeles County’ , Category
=’Fire,Wildfire, Burning’, image},{Format = ’PC-Format’}]
54
CCD: Customized dissemination in homogeneous overlay Optimal operator placement
Results in minimum dissemination cost Needs to know the dissemination tree for the published content Assumes small adaptation graphs (Needs enumeration of different
subsets of formats) Observation:
If B is a leaf in dissemination tree
Otherwise
55
CCD: Customized dissemination in homogeneous overlay The minimum cost for customized dissemination tree in node B is
computed as follow. If B is a leaf in the dissemination tree then
Otherwise
56
CCD: Operator placement in homogeneous overlay Optimal operator placement
57
Experimental evaluation
Implemented scenarios Homogeneous overlay
Optimal Only root TRECC All in root All in leaves
Heterogeneous Optimal All in root All in leaves
58
Experimental evaluation
Publish/Subscribe System
Decoupling in time, space and synchronization Provides decoupling in time, space and
synchronization.
Classification of Pub/Sub Architectures Centralized Broker model
Consists of multiple publishers and multiple subscribers and centralized broker/brokers (an overlay network of brokers interacting with each other).
Subscribers/Publishers will contact 1 broker, and does not need to have knowledge about others.
E.g. CORBA event services, JMS, JEDI etc…
Classification of Pub/Sub Architectures Peer-to-Peer model
Each node can be publisher, subscriber or broker. Subscribers subscribe to publishers directly and
publishers notify subscribers directly. Therefore they must maintain knowledge of each other.
Complex in nature, mechanisms such as DHT and CHORD are employed to locate nodes in the network.
E.g. Java distributed event service
Key functions implemented by P/Smiddleware service Event filtering (event selection)
The process which selects the set of subscribers that have shown interest in a given event. Subscriptions are stored in memory and searched when a publisher publishes a new event.
Event routing (event delivery) The process of routing the published events from the
publisher to all interested subscribers
Event Filtering (Subscription Model)Topic based VS Content based Topic based
Generally also known as topic based, group based or channel based event filtering.
Each event is published to one of these channels by its publisher.
Subscribers subscribes to a particular channel and will receive ALL events published to the subscribed channel.
Topic-based subscription
Simple process for matching an event to subscriptions. However, limited expressiveness.
Event filtering is easy, event routing is difficult (Heavy load on the network). The challenge is to multicast event effectively to subscribers.
Event Filtering- Subscription ModelTopic based VS Content based Content based
More flexibility and power to subscribers, by allowing more expression in arbitrary/customized query over the contents of the event.
Event publication by a key/value attribute pair, and subscriptions specify filters using a explicit subscription language.
E.g. Notify me of all stock quotes of IBM from New York stock exchange if the price is greater than 150
Content-based Subscription
Added complexity in matching an event to subscriptions. (Implementation: Subscription arranged in a matching tree, where each node is a partial condition.
However, more precision is provided and event routing is easier
Event Routing
After filtering the events, the broker/brokers must route the events to the corresponding subscribers.
Can be done in the following ways: Unicast Multicast Server push/ client pull
Event Routing
The broker makes the decision: how to route the message to the subscriber.
Several optimization schemes are available. Profile forwarding scheme – brokers only forward the
event to their neighbor broker which fulfill their subscription
Filtering the total covering of the subscription of the system – accept publisher events only if a subscriber has subscribed this event.
Example: SIENA
SIENA is a wide area notification service that uses covering-based routing. Consists of Nodes and servers (access points), Event
notifications & filters, Publish/subscribe protocol + advertisements, Identities and handlers, Filtering
Siena system can be configured in three types of inter-connection topologies: Hierarchical client/server architecture Acyclic P2P architecture General P2P architecture
SIENA: Hierarchical Architecture
•Servers interact with each other in an asymmetric client-server fashion. •Server is not distinguished from objects of interest or interested parties•Potential overloading of server stationed at higher level of hierarchy•Failure of one node in hierarchy causes all the nodes below that node to fail
Acyclic P2P architectureand General P2P architecture The acyclic P2P architecture and General P2P architecture are very similar.
Both represented by an undirected graph and allows bidirectional communication.
Scaling an issue for both.
Acyclic P2P Restriction on the configuration of connections between servers to forming
acyclic graph representation Therefore no redundant connections/ multiple paths are not allowed.
(Enforcement by a cycle avoiding algorithm) Can be difficult to maintain and not as robust as general P2P architecture.
General P2P architecture Requires less coordination among servers. Redundancy enforces robustness of Siena system with respect to failure of
single servers. Drawback: Special algorithms must be run to choose the best path.
Siena: Routing
Simplest strategy is to maintain the subscriptions at their access point and broadcast the notification throughout the network Least efficient Consumes lots of bandwidth
Send the notification towards the event servers that have clients that are interested in that notification (possibly using shortest path)
SIENA : Routing
Downstream Replication Events are kept as one copy as long as possible and only
replicated when it is as close as possible to the subscribing servers/clients.
SIENA : Routing
Upstream Evaluation Applying filters upstream, that is as close to the event publisher
as possible
Advantages of Pub/Sub
Highly suited for mobile applications, ubiquitous computing and distributed embedded systems
Robust – Failure of publishers or subscribers does not bring down the entire system
Scalability- Suited to build distributed applications consisting a large number of entities
Adaptability- can be varied to suit different environments (mobile, internet game, embedded systems etc…)
Disadvantages of Pub/Sub
Reliability – no strong guarantee on broker to deliver content to subscriber. After a publisher publishes the event, it assumes that all corresponding subscribers would receive it.
Potential bottleneck in brokers when subscribers and publishers overload them. (Solve by load balancing techniques)
Security an issue: Encryption hard to implement when the brokers has to filter out
the events according to context. Brokers might be fooled into sending notifications to the wrong
client, amplifying denial of service requests against the client.
Recommended