Design and Evaluation Design and Evaluation of a Wide-Area Event of a Wide-Area Event Notification ServiceNotification Service

Antonio CarzanigaAntonio Carzaniga

David S. RosenblumDavid S. Rosenblum

Alexander L. WolfAlexander L. Wolf

OverviewOverview IntroductionIntroduction Critical design issuesCritical design issues TopologiesTopologies Routing StrategiesRouting Strategies EvaluationEvaluation Simulation studiesSimulation studies Future ChallengesFuture Challenges

Introduction (1)Introduction (1)

Distributed event notification service Distributed event notification service

Introduction (2)Introduction (2)

Key activities of an event notification Key activities of an event notification serviceservice– Notification selectionNotification selection– Notification deliveryNotification delivery

Challenges in a wide-area settingChallenges in a wide-area setting– Maximize expressiveness in the Maximize expressiveness in the

selection mechanismselection mechanism– Maintain scalability in the delivery Maintain scalability in the delivery

mechanismmechanism

Critical Design IssuesCritical Design Issues

Interconnection topologyInterconnection topology– Hierarchy, adopted by JEDIHierarchy, adopted by JEDI– General graph, adopted by SIENAGeneral graph, adopted by SIENA

Routing algorithmRouting algorithm– Broadcast notificationsBroadcast notifications– Broadcast subscriptionsBroadcast subscriptions– Broadcast advertisementsBroadcast advertisements

Processing strategyProcessing strategy– Pruning – Pruning – A server propagates a subscription only if it A server propagates a subscription only if it

defines new notifications that are not covered by previously defines new notifications that are not covered by previously propagated subscriptions.propagated subscriptions.

API and Semantics (1)API and Semantics (1) Interfaces of SIENAInterfaces of SIENA

– publishpublish (notification n) (notification n)– subscribesubscribe (string identity, pattern expression) (string identity, pattern expression)– unsubscribeunsubscribe (string identity, pattern expression) (string identity, pattern expression)– advertiseadvertise (string identity, filter expression) (string identity, filter expression)– unadvertiseunadvertise (string identity, filter expression) (string identity, filter expression)

NotificationNotification – a set of typed attributes – a set of typed attributes– Each attribute has a type, a name, and a value.Each attribute has a type, a name, and a value.

API and Semantics (2)API and Semantics (2) FiltersFilters -- -- specify a set of attributes and specify a set of attributes and

constraints on the values of those attributesconstraints on the values of those attributes

PatternsPatterns – – a sequence of filters matched a sequence of filters matched by a temporally ordered sequence of by a temporally ordered sequence of notifications, each one matching the notifications, each one matching the corresponding filtercorresponding filter

API and Semantics (3)API and Semantics (3)

Two variants of the semantics of Two variants of the semantics of SIENASIENA– Subscription-based semanticsSubscription-based semantics

A notification A notification nn is delivered to an interested party is delivered to an interested party XX XX submitted at least one subscription submitted at least one subscription ss such that such that nn matches matches s.s.

– Advertisement-based semanticsAdvertisement-based semantics A notification A notification nn published by object published by object YY is delivered to is delivered to

interested party interested party XX YY advertised a filter advertised a filter aa that that covers covers nn and and XX registered a subscription registered a subscription ss that covers that covers nn. .

Architectures (1)Architectures (1) Hierarchical client/server architectureHierarchical client/server architecture

Acyclic peer-to-peer architectureAcyclic peer-to-peer architecture

ArchitecturesArchitectures (2)(2) General peer-to-peer architectureGeneral peer-to-peer architecture

Hybrid architecturesHybrid architectures

Routing Strategies used in SIENARouting Strategies used in SIENA

To ensure delivery of notifications to To ensure delivery of notifications to interested parties that subscribe for them.interested parties that subscribe for them.

Employs principles found in multicast Employs principles found in multicast routing protocols.routing protocols.

Main Idea – Send Notification only towards Main Idea – Send Notification only towards event servers that have interested clients , event servers that have interested clients , possibly using shortest path.possibly using shortest path.

Routing Strategies : Generic principlesRouting Strategies : Generic principles

1.1. Downstream Replication Downstream Replication

Routing Strategies: Generic Principles Routing Strategies: Generic Principles (2)(2)

2. Upstream Evaluation2. Upstream Evaluation

Two Classes of Routing AlgorithmsTwo Classes of Routing Algorithms

Subscription forwarding :Subscription forwarding :

• Subscriptions set the path for notificationsSubscriptions set the path for notifications

• Subscriptions are broadcast throughout the Subscriptions are broadcast throughout the networknetwork

• Forms a spanning tree that connects Forms a spanning tree that connects subscribers to all servers in the networksubscribers to all servers in the network

• No advertisements usedNo advertisements used

Two classes of Routing Algorithms Two classes of Routing Algorithms (2)(2)

Advertisement forwarding : Advertisement forwarding : • Advertisements set the path for SubscriptionsAdvertisements set the path for Subscriptions

• And Subscriptions set the path for NotificationsAnd Subscriptions set the path for Notifications

• Advertisements are broadcast throughout the Advertisements are broadcast throughout the network thereby forming treenetwork thereby forming tree

• Subscriptions are sent only to advertisers that Subscriptions are sent only to advertisers that submitted relevant advertisementssubmitted relevant advertisements

Optimizations in SIENAOptimizations in SIENA

Exploit commonalitiesExploit commonalities among subscriptions and among subscriptions and advertisementsadvertisements

Prune propagation treesPrune propagation trees by propagating only by propagating only along paths not covered by previous requestsalong paths not covered by previous requests

Unsubscriptions and unadvertisements are Unsubscriptions and unadvertisements are handled similarlyhandled similarly

The Filters PosetThe Filters Poset

This data structure represents a partially ordered This data structure represents a partially ordered set of filters.set of filters.

Partial order is defined by covering relations for Partial order is defined by covering relations for subscription filters & advertisement filters.subscription filters & advertisement filters.

Event server maintains this data structure that Event server maintains this data structure that keeps track of :keeps track of : previous requestsprevious requests their relationshipstheir relationships their sourcetheir source where they are forwardedwhere they are forwarded

The Filters Poset (2)The Filters Poset (2)

Poset extends vertically -> subscriptions very much inter-Poset extends vertically -> subscriptions very much inter-dependent -> few subscriptions summarize all others.dependent -> few subscriptions summarize all others.

Poset extends horizontally -> few similarities among Poset extends horizontally -> few similarities among subscriptions -> few opportunities to reduce trafficsubscriptions -> few opportunities to reduce traffic

Algorithms for Hierarchical ArchitectureAlgorithms for Hierarchical Architecture

Hierarchical server maintains :Hierarchical server maintains :

– Subscriptions in a poset s.Subscriptions in a poset s.

– Subscribers(s) – set contains identities of Subscribers(s) – set contains identities of subscribers of subscription.subscribers of subscription.

– Variable Variable mastermaster – contains identity of – contains identity of mastermaster server.server.

Algorithms for Hierarchical Architecture Algorithms for Hierarchical Architecture (2) : Subscriptions(2) : Subscriptions

Upon receiving subscribe Upon receiving subscribe (X,f),(X,f), – if server E finds subscription if server E finds subscription f’f’ in poset in poset PsPs such that such that f’f’ covers covers ff

and contains and contains XX in it’s subscriber’s set , no action taken. in it’s subscriber’s set , no action taken.

– If If ff already exists in already exists in PsPs, insert , insert XX in it’s subscriber’s set. in it’s subscriber’s set.

– Otherwise, if Otherwise, if f’f’ or or ff not found, not found, ff is inserted with empty is inserted with empty immediate successor and predecessor. immediate successor and predecessor.

– ONLY if ONLY if f f is inserted as root subscription, does the server is inserted as root subscription, does the server forward subscribe forward subscribe (E,f)(E,f) to to mastermaster..

– If immediate predecessor is NOT null, server removes X from If immediate predecessor is NOT null, server removes X from subscriber’s set of all subscriptions covered by subscriber’s set of all subscriptions covered by f.f.

Algorithms for Hierarchical Architecture Algorithms for Hierarchical Architecture

(3) : Notifications(3) : Notifications Upon receiving a Notification n: Upon receiving a Notification n:

– The server determines all the subscriptions The server determines all the subscriptions that cover n.that cover n.

– The server sends a copy of n to all the The server sends a copy of n to all the subscribers of the subscriptions.subscribers of the subscriptions.

– The server sends copy of n to master server, if The server sends copy of n to master server, if master was not sender of n.master was not sender of n.

Algorithms for Hierarchical Architecture Algorithms for Hierarchical Architecture (4) : Unsubscriptions(4) : Unsubscriptions

Single unsubscription might cancel one or more previous Single unsubscription might cancel one or more previous subscriptionssubscriptions

If root subscription is canceled , the server must forward If root subscription is canceled , the server must forward unsubscription to master AND must forward new root unsubscription to master AND must forward new root subscriptions.subscriptions.

Algorithms for Hierarchical ArchitectureAlgorithms for Hierarchical Architecture(5) : Advertisement forwarding(5) : Advertisement forwarding

Advertisement forwarding technique does Advertisement forwarding technique does not apply to hierarchical architecture. not apply to hierarchical architecture. Why?Why?

Advertisements and Unadvertisements are Advertisements and Unadvertisements are silently dropped.silently dropped.

Algorithms for peer-to-peer ArchitectureAlgorithms for peer-to-peer Architecture

In peer-to-peer architectures, a server maintainsIn peer-to-peer architectures, a server maintains

– A set of neighborsA set of neighbors– Subscriptions in poset Ps Subscriptions in poset Ps – Set of subscribers with each subscriptionSet of subscribers with each subscription– Set of forwards with subset of neighbors to Set of forwards with subset of neighbors to

which s is forwarded. which s is forwarded.

• A subscription or notification is propagated A subscription or notification is propagated from origin to it’s destination following a from origin to it’s destination following a minimum spanning tree.minimum spanning tree.

Algorithms in peer-to-peer ArchitectureAlgorithms in peer-to-peer Architecture(2) : Subscriptions(2) : Subscriptions

Extension of algorithm of the hierarchical serverExtension of algorithm of the hierarchical server

Algorithms differ in how peer-to-peer server Algorithms differ in how peer-to-peer server forwards subscriptions to its neighbors.forwards subscriptions to its neighbors.

ff is forwarded to all neighbors except those is forwarded to all neighbors except those – Upstream from the server along spanning tree rooted at Upstream from the server along spanning tree rooted at

subscriber of subscriber of ff..

– To those to which To those to which f’f’ covering covering ff has already been has already been forwarded.forwarded.

Algorithms in peer-to-peer ArchitectureAlgorithms in peer-to-peer Architecture(3) : Notifications (3) : Notifications

Processed the same way as in Processed the same way as in hierarchical architectures.hierarchical architectures.

Algorithms in peer-to-peer ArchitectureAlgorithms in peer-to-peer Architecture(4) : Advertisement Forwarding(4) : Advertisement Forwarding

Main difference : Two interacting Main difference : Two interacting computationscomputations

Advertisement forwarding constraints Advertisement forwarding constraints subscription forwardingsubscription forwarding. .

EvaluationEvaluation

How can we demonstrate ability to How can we demonstrate ability to scale?scale?

Qualitative reasoning with regard to Qualitative reasoning with regard to expressiveness of the notification expressiveness of the notification selection mechanismselection mechanism

Performing simulation studiesPerforming simulation studies Constructing a prototype Constructing a prototype

implementationimplementation

Simulation StudiesSimulation Studies

Reasoning for Level of Reasoning for Level of ExpressivenessExpressiveness

Some scalability factors such as network Some scalability factors such as network latency cannot be controlled by the latency cannot be controlled by the design design

Expressiveness level approximates SQLExpressiveness level approximates SQL Conclusion: covering relations have Conclusion: covering relations have

reasonable complexities for a scalable reasonable complexities for a scalable event notification system.event notification system.

Simulation StudiesSimulation Studies

IssuesIssues Scalability w.r.t. Scalability w.r.t.

architectures/algorithmsarchitectures/algorithms Architecture/algorithms vs increased Architecture/algorithms vs increased

application demands - how is application demands - how is communication cost affected?communication cost affected?

Simulation StudiesSimulation Studies

Network configuration:Network configuration: Assumptions Assumptions costs are linear functions of loadcosts are linear functions of load latencies of links are constantlatencies of links are constant sites and links have infinite capacity sites and links have infinite capacity

(congestion?)(congestion?) only homogenous architectures are chosenonly homogenous architectures are chosen centralized architectures are considered centralized architectures are considered

as a basis for comparison (simplicity)as a basis for comparison (simplicity)

Simulation StudiesSimulation Studies

Application BehaviorApplication Behavior involves collective behavior of an involves collective behavior of an

applications' objects of interestapplications' objects of interest Done using scenario generationDone using scenario generation

Simulation StudiesSimulation Studies

Total CostTotal Cost as a basic metric for event notification services as a basic metric for event notification services

captures important aspects of scalabilitycaptures important aspects of scalability total cost remains almost constant when there total cost remains almost constant when there

are more than 100 interested parties are more than 100 interested parties (saturation)(saturation)

very high numbers of objects of interest results in very high numbers of objects of interest results in client/server architecture performing worse than client/server architecture performing worse than acyclic peer to peer with subscription architectureacyclic peer to peer with subscription architecture

acyclic peer to peer with advertisement acyclic peer to peer with advertisement architecture shows unstable cost profile at low architecture shows unstable cost profile at low densitiesdensities

Simulation StudiesSimulation Studies

Cost per Service requestCost per Service request total cost / number of client requeststotal cost / number of client requests low value -> low overheadslow value -> low overheads it can be seen that centralized it can be seen that centralized

architectures are unreasonablearchitectures are unreasonable

Cost per SubscriptionCost per Subscription total cost of all subscription messages/ total cost of all subscription messages/

number of subscriptions processednumber of subscriptions processed

Simulation StudiesSimulation Studies

ConclusionsConclusions Hierarchical client/server architecture has lower Hierarchical client/server architecture has lower

per-subscription costs than acyclic peer to peer per-subscription costs than acyclic peer to peer therefore it is better suited for low densities of therefore it is better suited for low densities of clients clients

For both architectures high densities of interested For both architectures high densities of interested parties results in higher subscription costsparties results in higher subscription costs

Cost of delivering a notification to interested Cost of delivering a notification to interested parties is more or less the same for the 2 parties is more or less the same for the 2 architecturesarchitectures

Peer to peer is better when total cost of Peer to peer is better when total cost of communication is dominated by notificationscommunication is dominated by notifications

Future ChallengesFuture Challenges

Enhancing the design of the interface and Enhancing the design of the interface and algorithms to support mobility of clientsalgorithms to support mobility of clients

Extensions that support quality of service Extensions that support quality of service parameters suited for integration of parameters suited for integration of software componentssoftware components

Explore other important issues of WANs Explore other important issues of WANs such as mechanisms for reliability and such as mechanisms for reliability and fault tolerance of event notification fault tolerance of event notification systemssystems

Realization of content based routingRealization of content based routing