Looking into the Past: EnhancingLooking into the Past: EnhancingMobile Publish/Subscribe MiddlewareMobile Publish/Subscribe Middleware

Exponent:Exponent:Pablo E. GuerreroPablo E. Guerrero

Thesis Supervisor:Thesis Supervisor:Mariano A. CiliaMariano A. Cilia

Universidad Nacional del Centro de la Provincia de Buenos Universidad Nacional del Centro de la Provincia de Buenos AiresAires

Facultad de Ciencias ExactasFacultad de Ciencias Exactas

Tandil, Argentina, Tandil, Argentina, OctoberOctober, 2004, 2004

22

AgendaAgenda

MotivationMotivation Problem StatementProblem Statement Proposed ApproachProposed Approach Design & ImplementationDesign & Implementation Experimental ResultsExperimental Results ConclusionsConclusions

33

MotivationMotivation

Convergence of TechnologiesConvergence of Technologies Usage of Large-Scale Distributed SystemsUsage of Large-Scale Distributed Systems Mobile Devices & Wireless TechnologiesMobile Devices & Wireless Technologies Client/Server (Request/Reply) PredominanceClient/Server (Request/Reply) Predominance

New Paradigm for Information-Driven ApplicationsNew Paradigm for Information-Driven Applications WWW, Sensor Networks, Enterprise StrategiesWWW, Sensor Networks, Enterprise Strategies Pull-based (Request/Reply) Does Not FitPull-based (Request/Reply) Does Not Fit

Polling implies resource wastePolling implies resource waste Leads to Network Saturation or Server Breakdown!Leads to Network Saturation or Server Breakdown! Not Suitable for Mobile DevicesNot Suitable for Mobile Devices

Push-basedPush-based Event-based DisseminationEvent-based Dissemination

Initiator of InteractionInitiator of Interaction

ConsumerConsumer ProducerProducer

Knowledge about Knowledge about CounterpartCounterpart

FullFull Request/Request/ReplyReply

One-to-One One-to-One MessagingMessaging

NoNo Anonymous Anonymous Request/ReplyRequest/Reply

Event-Based Event-Based DisseminationDissemination

44

Motivation (II)Motivation (II)

Publish/Subscribe Middleware providesPublish/Subscribe Middleware provides Asynchronous communicationAsynchronous communication Loose couplingLoose coupling Producers don’t know who is subscribedProducers don’t know who is subscribed Clients can “dynamically” join and leaveClients can “dynamically” join and leave Doesn’t require publishers and subscribersDoesn’t require publishers and subscribers

at the same timeat the same time Reflects intrinsic behavior of Information-Reflects intrinsic behavior of Information-

driven Applicationdriven Application Support for roaming clientsSupport for roaming clients Notion of LocationNotion of Location Location TransparencyLocation Transparency

Pub/Sub needs an Intermediator Pub/Sub needs an Intermediator

55

Publish/Subscribe Systems StructurePublish/Subscribe Systems Structure

Independent Process Data Flow StyleIndependent Process Data Flow Style Message Manager or Notification ServiceMessage Manager or Notification Service Clients (producers/consumers/both)Clients (producers/consumers/both) Communication linksCommunication links

Service SemanticsService Semantics ImplementationImplementation

Decentralized brokersDecentralized brokers Connected graphConnected graph Client’s proxiesClient’s proxies

:aConsumer1 :aProducer1 :aConsumerNProducer

:aProducer3:aConsumer2 :aProducer2

notification service boundary

:aBroker1

:aBroker2

:aBroker3:aBroker4

home stub home stubhome stub

home stub home stubhome stub

:aPublishSubscribeSystem

66

Problem StatementProblem Statement

Problem Found Problem Found @@ Applications Startup Applications Startup Initial Bootstrapping Sequence phaseInitial Bootstrapping Sequence phase Getting a Consistent State Getting a Consistent State

Request/Reply (combination)Request/Reply (combination) Observation phase (pure Pub/Sub)Observation phase (pure Pub/Sub)

Cannot assume how long will it takeCannot assume how long will it take

Meaningful Event

EvntEvntEvntEvntEvnt:anEventBasedApp Sub Evnt:aNotificationService

77

Problem Statement (II)Problem Statement (II)

Problem Aggravated in Mobile Problem Aggravated in Mobile EnvironmentsEnvironments DisconnectionsDisconnections Context Changes (new location)Context Changes (new location) Pervasive environment’s dynamic behaviorPervasive environment’s dynamic behavior

A new bootstrapping phase must be initiated!A new bootstrapping phase must be initiated!

System responsiveness degradedSystem responsiveness degraded

88

Proposed ApproachProposed Approach

Goal:Goal: Reduce bootstrapping latencyReduce bootstrapping latency Use already published (past) notificationsUse already published (past) notifications

Extend an existing pub/sub system, RExtend an existing pub/sub system, REBECAEBECA Store notifications in its InfrastructureStore notifications in its Infrastructure Caches Distributed in the NetworkCaches Distributed in the Network Minimize underlying system changesMinimize underlying system changes

Meaningful Event

Sub EvntEvnt Evnt Evnt Evnt Evnt:aNotificationService:anEventBasedApp

99

The RThe REBECAEBECA Notification Service Notification Service:aConsumer

:anAppObjec t1 :anAppObjec t2

:anAppObjec t3

:aBroker1

:aProducervirtual l ink

notification service boundary

sock

et c

onn

ect

ion

:anEventBroker :aRoutingEngine

so

cket co

nn

ectio

n

home stub layer

applicat ion layer

subscribe(F)

process(E)

:a nEve ntR ou ter1 :aR ou ti ng Eng in e

:aBroker2

:a nEve ntR ou ter2 :aR ou ti ng Eng in e

:aBroker3

:a nEve ntR ou ter3 :aR ou ti ng Eng in e

socket connection

socket connection

1010

RREBECAEBECA Brokers Network Brokers Network

clients

local event brokersX1

border brokers(event routers)

broker’s network

inner broker

routing tables

----

LB1X2----LB2

X4

LB4

X5

LB5

X3---- LB3

X6----LB6

X7----LB7

X8----LB8

ER1

----

----

----

ER5

----

ER4

----

ER3

----

ER2

----

1111

RREBECAEBECA Routing Algorithms Routing Algorithms

FloodingFlooding Filter ForwardingFilter Forwarding

Simple RoutingSimple Routing Identity-based RoutingIdentity-based Routing Covering-based RoutingCovering-based Routing Merging-based RoutingMerging-based Routing

Routing with AdvertisementsRouting with Advertisements

1212

ProposedProposed ApproachApproach

Approach AnalysisApproach Analysis Caching RequirementsCaching Requirements Caching in Event-Based SystemsCaching in Event-Based Systems Notifications StorageNotifications Storage Cache Querying StrategiesCache Querying Strategies GuidelinesGuidelines

1313

Proposed Approach Analysis Proposed Approach Analysis

Store Notifications in Distributed CachesStore Notifications in Distributed Caches Extending External BehaviorExtending External Behavior

Past BoundPast Bound Interface ExtensionInterface Extension

+ void subscribe(Subscription s, PastBound past, EventP rocessor proc)new!

EventBroker

...

+ void advertise(Advertisement a, EventProcessor proc)+ void unadvertise(Advert isement a)+ void subscribe(Subscript ion s, EventProcessor proc)+ void unsubscribe(Subscript ion s)+ void publish(Event e)

1414

Proposed Approach Analysis (II) Proposed Approach Analysis (II)

:aConsumer :aBroker:aCachingStrategy

1.1: doSomething(..)

1.2.1: notify(aCachedEvent1)

1.3: notify(aNewlyPublishedEvent1)

1.4: notify(aNewlyPublishedEvent2)

1: subscribe(aSubscription, aBound, this)

1.2.2: notify(aCachedEvent2)

1.2.3: notify(aCachedEvent3)

Extend Internal BehaviorExtend Internal Behavior Extend System BehaviorExtend System Behavior

Support for Different StrategiesSupport for Different Strategies

1515

Caching Strategies RequirementsCaching Strategies Requirements

Response TimeResponse Time Time OverheadTime Overhead Space UsageSpace Usage IntegrabilityIntegrability

CorrectnessCorrectness TransparenceTransparence DynamismDynamism

Separation of ConcernsSeparation of Concerns Specification of the RequirementsSpecification of the Requirements

from the Infrastructurefrom the Infrastructure

1616

Caching in Event-Based SystemsCaching in Event-Based Systems

Caching applied in a different fashion Caching applied in a different fashion than traditional systemsthan traditional systems

Main differences:Main differences: Cache Coherency and ConsistencyCache Coherency and Consistency Cache SizesCache Sizes Information LifetimeInformation Lifetime Caching vs. BufferingCaching vs. Buffering

1717

Notifications StorageNotifications Storage

When? Locality Principle not Applicable!When? Locality Principle not Applicable! Distributed decision based on:Distributed decision based on:

Subscriptions and UnsubscriptionsSubscriptions and Unsubscriptions Advertisements and UnadvertisementsAdvertisements and Unadvertisements NotificationsNotifications

How? Behaving as FIFO queuesHow? Behaving as FIFO queues Circular Ring BufferCircular Ring Buffer

1818

Notifications Storage (II)Notifications Storage (II)

Parallel Circular Ring BuffersParallel Circular Ring Buffers

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

F1

F2

F5

F4

F3

Take advantage of Content-based NSTake advantage of Content-based NS Filter Identity, Covering and MergingFilter Identity, Covering and Merging

1919

Notifications Storage (III)Notifications Storage (III)

A Combination, whichA Combination, which avoids Replicating Notificationsavoids Replicating Notifications allows Ordering Notificationsallows Ordering Notifications

F1

F2

- -

F3

01

00

01

02

0001 000000000000000101020101020101010101

MaxGlobalBuffers

MaxFilterBuffers

Ma

xInd

exed

Filt

ers

F1

F2

- -

F3

01

00

01

02

0001 000000000000000101020101020101010101

MaxGlobalBuffers

MaxFilterBuffers

Ma

xInd

exed

Filt

ers

MaxGlobalBuffers

MaxFilterBuffers

Ma

xInd

exed

Filt

ers

MaxGlobalBuffers

MaxFilterBuffers

Ma

xInd

exed

Filt

ers

MaxGlobalBuffers

MaxFilterBuffers

Ma

xInd

exed

Filt

ers

MaxGlobalBuffers

MaxFilterBuffers

Ma

xInd

exed

Filt

ers

MaxGlobalBuffers

MaxFilterBuffers

Ma

xInd

exed

Filt

ers

MaxGlobalBuffers

MaxFilterBuffers

Ma

xInd

exed

Filt

ers

MaxGlobalBuffers

MaxFilterBuffers

Ma

xInd

exed

Filt

ers

MaxGlobalBuffers

MaxFilterBuffers

Ma

xInd

exed

Filt

ers

2020

Cache QueryingCache Querying

NS is an Undirected Acyclic GraphNS is an Undirected Acyclic Graph How deep to search? (i.e. How many levels How deep to search? (i.e. How many levels kk?)?) Link traversals involve costs similar to RPC’s:Link traversals involve costs similar to RPC’s:

De / Serialization, Un / MarshallingDe / Serialization, Un / Marshalling RPC Runtime Costs, Physical Network TransferRPC Runtime Costs, Physical Network Transfer

RPC requires 10RPC requires 10..000~15000~15..000 machine instructions000 machine instructions LPC 100 times faster than RPCLPC 100 times faster than RPC

Strategies Classification:Strategies Classification: k k = 1 = 1 Restricted to Local Event Brokers Restricted to Local Event Brokers 1 < k 1 < k m m Restricted to maximum depth ‘m’ Restricted to maximum depth ‘m’ k k = = Allowed to perform exhaustive search Allowed to perform exhaustive search

2121

Cache Querying: LocalBrokerCachingCache Querying: LocalBrokerCaching

X1

broker’s network

----

LB1X2---- LB2

X4

LB4

X5

LB5

X3---- LB3

X6----LB6

X7----LB7

X8----LB8

ER1

----

----

----

ER5

----

ER4

----

ER3

----

ER2

----

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ngLo ca l

C ac h i ngLo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ngLo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ngLo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ngLo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ngLo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ngLo ca l

C ac h i ng

Lo ca l

C ac h i ng

Lo ca l

C ac h i ngLo ca l

C ac h i ng

Lo ca l

C ac h i ngLo ca l

C ac h i ng

Lo ca l

C ac h i ng

2222

Cache Querying: BorderBrokerCachingCache Querying: BorderBrokerCaching

X1

broker’s network

----

LB1X2---- LB2

X4

LB4

X5

LB5

X3---- LB3

X6----LB6

X7----LB7

X8----LB8

ER1

----

----

----

ER5

----

ER4

----

ER3

----

ER2

----

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

Bo rd er

C ac h i n g

2323

Cache Querying: MergingCachingCache Querying: MergingCaching

X1

broker’s network

----

LB1X2---- LB2

X4

LB4

X5

LB5

X3---- LB3

X6----LB6

X7----LB7

X8----LB8

ER1

----

----

----

ER5

----

ER4

----

ER3

----

ER2

----

M e rg i n g

C ac h i n g

M ergingCach in g

M ergingCach in g

M e rg i n g

C ac h i n g

M e rg i n g

C ac h i n g

M e rg i n g

C ac h i n g

M ergingCach in g

M ergingCach in g

M e rg i n g

C ac h i n g

M e rg i n g

C ac h i n g

M ergingCach in g

M ergingCach in g

M e rg i n g

C ac h i n g

M e rg i n g

C ac h i n g

M ergingCach in g

M e rg i n g

C ac h i n g

M e rg i n g

C ac h i n g

M e rg i n g

C ac h i n g

M e rg i n g

C ac h i n g

M e rg i n g

C ac h i n g

2424

Cache QueryingCache Querying

Caching w/More Complex Routing AlgorithmsCaching w/More Complex Routing Algorithms Routing Algorithms selects which neighbors to Routing Algorithms selects which neighbors to

forward a subscriptionforward a subscription Solution: Force Forwarding an explicit Solution: Force Forwarding an explicit FetchFetch

messagemessage Caching w/AdvertisementsCaching w/Advertisements

Avoid asking every peer by usingAvoid asking every peer by usingadvertisements informationadvertisements information

Considering the Time DimensionConsidering the Time Dimension Border Brokers’ clocks synchronizationBorder Brokers’ clocks synchronization Attach a Timestamp to notificationsAttach a Timestamp to notifications

2525

Cache Querying GuidelinesCache Querying Guidelines

Floo

din

g

Mer

gin

gR

ou

ting

Co

veri

ngR

ou

ting

Iden

tity

Ro

uti

ng

Sim

ple

Rou

tin

g

Merging Caching

Border Broker Caching

Local Broker Caching

[Cach

ing

Str

ate

gy]

[Routing Algorithm]

[with Advs]

[without Advs]

2626

Design & ImplementationDesign & Implementation

OOPOOPAproachesAproaches ClassClass

InheritanceInheritance DecoratorDecorator

DesignDesignPatternPattern

Flooding SimpleRouting

IdentityRouting

CoveringRouting

MergingRouting

RoutingEngine

CachingEngine

+ decorated)CachingEngine(Engine+ void addSubscript ion( )+ void removeSubscription( )+ void addAdvert isement( )+ void removeAdvert isement( )+ void processEvent( )...

SubscriptionSubscription

AdvertisementAdvertisement

Event

...

MergingCachingEngine ...LocalBrokerCachingEngine

Engine

engine

this.addSubscription(s,ep);

...

+ void addSubscription(Subscription ), EventProcessor

+ void removeSubscript ion(Subscript ion)+ void addSubscription( s, p, ep)Subscription PastBound EventProcessor

+ void addAdvertisement(Advertisement)+ void removeAdvert isement(Advert isement)+ void processEvent(Event)...

new!

2727

AOP Architectural ApproachAOP Architectural Approach

Usage ofUsage ofAOP toolsAOP tools

RoutingEngine

+ void addSubscript ion( )+ void removeSubscript ion( )+ void addAdvertisement( )+ void removeAdvert isement( )+ void processEvent( )# void handleAdminEvent( )...

SubscriptionSubscription

AdvertisementAdvertisement

EventEvent

...

EventBroker

EventRouter

RoutingTable

EventProcessor

CachingStrategy

+ (..)+ void setRout ingEngine( )+ void setSubscriptionEntries( )+ void setAdvertisementEntries( )+ void setProcessors( procs)

CachingStrategyRoutingEngine

RoutingTableRoutingTable

Collection+ void beforeProcessEvent( )Event+ void afterProcessEvent( )# void processInfoEvent( )# void processAdminEvent( )

+ void beforeUnsubscribe( )

EventEvent

AdminEvent

Subscription

+ void beforeSubscribe( , )+ void afterSubscribe( )

+ void beforeForwardSubsAndUnsubs(..)+ getDestinations( , )

Subscription EventProcSubscription

Set Event EventProcessor

- cache- engine...

EventCacheRoutingEngine

en

gin

e

BorderBrokerCaching ...LocalBrokerCaching

cacheEventCache

...

...

2828

AOP Architectural Approach (II)AOP Architectural Approach (II)

Method InterceptionsMethod Interceptions AdvicesAdvices Inter-Type DeclarationsInter-Type Declarations

AspectsAspects

aRoutingEngine

dispatchnew()

dispatch

dispatch

subscribe (Subscript ion,EventPro cessor)

aLocalEventBroker

aRoutingEngine

addSubscription(Subscription,EventProcessor)

&&

pointcut initRoutingEngine():pointcut initRoutingEngine():initialization(RoutingEngine.new());initialization(RoutingEngine.new());

pointcut subscription():pointcut subscription():withincode(void LocalEventBroker.subscribe(..)) &&withincode(void LocalEventBroker.subscribe(..)) &&call(void RoutingEngine.addSubscription(..));call(void RoutingEngine.addSubscription(..));

before (RoutingEngine routingEngine,before (RoutingEngine routingEngine, Subscription sub,Subscription sub, EventProcessor processor):EventProcessor processor): subscription(routingEngine, sub, processor)subscription(routingEngine, sub, processor){{ if (sub.isBounded())if (sub.isBounded()) routingEngine.getCachingStrategy().beforeSubscribe(sub, processor);routingEngine.getCachingStrategy().beforeSubscribe(sub, processor);}}

public void EventBroker.subscribe(Subscription sub,public void EventBroker.subscribe(Subscription sub, EventProcessor source,EventProcessor source, PastBound bound);PastBound bound);

private PastBound Subscription.bound;private PastBound Subscription.bound;

2929

AOP Architectural Approach (III)AOP Architectural Approach (III)

CachingCachingClassClassHierarchyHierarchy

+ (..)+ void afterProcessEvent( )# void processInfoEvent( )# void processAdminEvent( )+ void beforeSubscribe( )+ void afterSubscribe( )+ void beforeUnsubscribe( )+ void beforeForwardSubsAndUnsubs(. .)+ getDest inations( , )

LocalBrokerCachingEvent

EventAdminEvent

SubscriptionSubscription

Subscription

Set Event EventProcessor

LocalBrokerCaching

cache

engine

...

RoutingEngine

EventCache

CachingStrategy

+ (..)+ void afterProcessEvent( )# void processInfoEvent( )# void processAdminEvent( )+ void beforeSubscribe( )+ void afterSubscribe( )+ void beforeUnsubscribe( )+ void beforeForwardSubsAndUnsubs(. .)+ getDest inations( , )# void checkDeliver( , )# void prepareReply( , )# void handleReply( )# getRoutingEntryFor( )

BorderBrokerCachingEvent

EventAdminEvent

SubscriptionSubscription

Subscription

Set Event EventProcessorSubscription EventProcessorSubscription EventProcessor

ReplyRoutingEntry Subscription

BorderBrokerCaching

...

... ...

+ (. .)# void prepareReply( , )# void checkDeliver( , )# void processAdminEvent( )+ void afterProcessEvent( )# void handleReply( )

MergingCachingSubscription EventProcessorSubscription EventProcessor

AdminEventEvent

Reply

MergingCaching

HashTable fetchedReplies

3030

Experimental ResultsExperimental Results

Provide details on Caching BehaviorProvide details on Caching Behavior Evaluations SetupEvaluations Setup

Hierarchical NetworkHierarchical Network Consumers issue RandomConsumers issue Random

SubscriptionsSubscriptions Producers publishProducers publish

Notifications of Random TypesNotifications of Random Types Caching Efficacy MetricCaching Efficacy Metric

Measures Best-EffortMeasures Best-Effort

3131

LocalBrokerCaching EvaluationLocalBrokerCaching Evaluation

3232

BorderBrokerCaching EvaluationBorderBrokerCaching Evaluation

3333

MergingCaching EvaluationMergingCaching Evaluation

3434

Caching Strategies ComparisonCaching Strategies Comparison

3535

ConclusionsConclusions

Notification Service was ExtendedNotification Service was Extended Reduced Bootstrap LatencyReduced Bootstrap Latency Used a pure pub/sub ApproachUsed a pure pub/sub Approach Minimal Changes on the pub/sub Minimal Changes on the pub/sub

Subscription InterfaceSubscription Interface Incorporated Distributed CachingIncorporated Distributed Caching

Efficient, Content-based Notification Efficient, Content-based Notification StorageStorage

Several Strategies Analyzed & Several Strategies Analyzed & ImplementedImplemented

Aspect-OrientationAspect-Orientation Achieved good modularity for a Achieved good modularity for a

crosscutting concerncrosscutting concern Allowed a plug-and-play functionalityAllowed a plug-and-play functionality

EvaluationEvaluation Measured & Compared Caching EfficacyMeasured & Compared Caching Efficacy Gave Insights of Caching BehaviorGave Insights of Caching Behavior

NotificationService

Caching

3636

Future WorkFuture Work

Caching Functionality IntegrationCaching Functionality Integration On-line IntegrationOn-line Integration

through Dynamic AOPthrough Dynamic AOP Experiment with otherExperiment with other

Notification ServicesNotification Services

Extend EvaluationsExtend Evaluations Develop Analytical Response Time Develop Analytical Response Time

ModelsModels Perform Large Scale SimulationsPerform Large Scale Simulations

Consider Other EnhancementsConsider Other Enhancements Priorization and Interactions with Priorization and Interactions with

other Aspects of the Notification other Aspects of the Notification Service Service

NotificationService

Caching