Upload
claude-horatio-owen
View
220
Download
0
Embed Size (px)
Citation preview
1
Distributed Monitoring of
Peer-to-Peer SystemsBy
Serge Abiteboul, Bogdan Marinoiu
Docflow meeting, Bordeaux
2
2
Outline The Monitoring Problem & Approach
A language for specifying monitoring tasks: P2PML
P2PMonitor System ActiveXML Stream Algebra Architecture of P2PMonitor Monitoring Plan Generation & Query Rewriting
Focus on Filtering
Reusing running tasks
Work in progress
3
3
The Monitoring Problem P2P systems are:
a popular support for content sharing communities, distributed applications
highly dynamic (intense communications, content changing rapidly, peers come/leave)
and
difficult to observe
Observation is important ! error management & diagnosis
statistics gathering & optimization issues : the « busiest » peer in a network
business applications : billing & quality of service
Web surveillance
4
4
Is it possible to observe & analyse a P2P system ?
Difficult (if not impossible) in a centralized way
Yes, in a distributed manner
5
5
XM
L
strea
m
Approach
Detect events at the (monitored) peer level
Data changes, Web service calls -> alerters
Each event is represented as an XML document
XML Stream
(distributed) XML stream processing system
XML Streams are published
6
6
Outline The Monitoring Problem & Approach
A language for specifying monitoring tasks: P2PML
P2PMonitor System ActiveXML Stream Algebra Architecture of P2PMonitor Monitoring Plan Generation & Query Rewriting
Focus on Filtering
Reusing running tasks
Work in progress
7
7
P2PML statement structureXQuery FLWR flavourFor – maps streams to XML variables
Let – assigns new XML variables
Where – imposes conditions on events (filtering and join criteria)
Return – generates reports / restructures XML
By – specifies publication means : in channels for inside system publication e-mails, Web pages, RSS feeds for outside system publication
8
8
P2PML statement examplefor $c on local: outCOM
let $timeCall := $c.call.time
and $duration := $c.response.time - $timeCall
where
$c.call.method = “GetTemp” and $duration > 10
and $c.call.site = "http://meteofrance.fr"
return <longGetTemp>
<callTime>{$timeCall}</callTime>
<duration>{$duration}</duration>
</longGetTemp>
by channel “QoS:Alerts”
9
9
Outline The Monitoring Problem & Approach
A language for specifying monitoring tasks: P2PML
P2PMonitor System ActiveXML Stream Algebra Monitoring Plans Architecture of P2PMonitor
Focus on Filtering
Reusing running tasks
Work in progress
10
10
ActiveXML Stream Algebra
is the support for monitoring plan representation and the basis for the its optimization :
Distribute the work among the peers
Try to place computation close to data if possible
Try to reduce redundancy
11
11
Scenario
P2PMLQueries
XML streams
12
12
Monitoring Plans
13
13
Architecture of P2PMonitor(1)
Subscription Manager
Alerters (WS Alerter, Database Alerter, RSS Alerter)
Stream Processors Without « storage »: Filter, Restructure, Union With « storage »: Join, Group-By, Duplicate Removal
Publishers E-mail, WebPage, RSS Channel Publisher : a user or another peer may subscribe to it
14
14
Architecture of P2P Monitor(2)
15
15
Outline The Monitoring Problem & Approach
A language for specifying monitoring tasks: P2PML
P2PMonitor System ActiveXML Stream Algebra Monitoring Plans Architecture of P2PMonitor
Focus on Filtering
Reusing running tasks
Work in progress
16
16
Focus on Filtering
Filtering is a crucial operator in stream processing !
E.g., Many users might be interested in events coming from the same source / alerter : bottleneck hazard
Our approach to the problem : two-stage filtering
Reasons:
Attributes of XML document’s root reflect the most important properties of an event
The event’s details can be given intentionnally (ActiveXML style)
17
17
Two-stage filteringA subscription is viewed as a conjunction of simple conditions (e.g., « attribute » = « value ») and « more difficult » XPath queries
1st data structure regroups the (ordered) simple conditions of all the subscriptions by commonalities (Atomic Event Set structure)
2nd data structure regroups XPath queries of all the subscriptions (path –based indexing YFilter style – using NFA)
On a XML document:
1st stage: read the root, evaluate AES, detect the « difficult » XPath queries that remain to be evaluated
2nd stage (if needed): adapt the second structure and evaluate the set of XPath queries on the body of the XML document (if necessary execute Web service calls).
The output is the set of the subscriptions « hit » by the XML document
18
18
Outline The Monitoring Problem & Approach
A language for specifying monitoring tasks: P2PML
P2PMonitor System ActiveXML Stream Algebra Monitoring Plans Architecture of P2PMonitor
Focus on Filtering
Reusing streams / running tasks
Work in progress
19
19
Reusing running tasks
Optimization by trying to avoid redundancy
Before building new operators (and streams), try to discover useful ones
Stream representation in XML:
Stream Definition Database – description of available streamsDistributed, not centralized (avoid bottlenecks)Implemented using KadoP – index and repository system over
a DHT
20
20
Stream replication and equivalence(1)Streams can be replicated between peers
With two similar operators on two replicas of the same stream, we obtain two equivalent streams
Replication can be represented in the Stream Definition Database
Stream Equivalence is difficult to detect
21
21
Algorithm for discovering useful streams It uses XPath queries on the Stream Definition Database:
E.g. for identifying the output stream of alerter inCOM :
/Stream[@PeerId=$p1][Operator/inCOM] ->(S1, P1)
It goes bottom-up
on the query tree
E.g.,
JoinP(σF(inCOM@P1),
outCOM@P2)
(S5, P1)
22
22
Outline The Monitoring Problem & Approach
A language for specifying monitoring tasks: P2PML
P2PMonitor System ActiveXML Stream Algebra Monitoring Plans Architecture of P2PMonitor
Focus on Filtering
Reusing streams / running tasks
Work in progress
23
23
Work in progress (1)Link with Incremental View Maintenance
Defining a monitoring task by a tree-pattern query on an active document with streams - powerful way of expressing complex monitoring tasks (difficult to express directly in P2PML)
24
24
Work in progress (2)Introducing explictly the « time » in P2PML -possible impact on P2PMonitor performance (reactivity) and resource consumption (needed storage)
E.g.
for $e1 on P1:inCOM, $e2 on P2:outCOMwhere $e1.timeEvent > $e2.timeEvent +25
…
Queries on traces obtained by P2PMonitor – diagnosis, detecting patterns of evolution for the monitored system
E.g. Trace = I1,I2…In - instances of a document
For each new order detected in instance Ik, there is a payment present in one of the following instances
25
25
Thank you very much!