An Active Events Model for Systems Monitoring

Philip GrossColumbia University

Programming Systems Lab

Director: Gail Kaiser

2

Motivation: Loose Coupling

Want to maximize ease of distributed programming

Not only should middleware deliver events… It should completely shield participants from

knowledge of each other Information goes into the cloud, information

comes out of the cloud

3

Problem: No Event Context

Consumers get data that they’re “supposed” to get

Confusion and ambiguity can be major problems

4

Example: Stock Trading Program

Sub (MSFT > 80) // sell and cash in Event published with MSFT = 60: ignored Event published with MSFT = 1515: received,

all shares sold Are we rich yet? Unfortunately no: was appointment with MS

sales rep at 3:15pm

5

We Conclude That…

We need to associate semantics with event data

Somehow saying “this is what I mean,” or “here’s how to interpret me”

Events that explicitly indicate semantics, we call Active Events, or ActEvents

They’re not just passive data containers

6

Two Flavors of Information Bus

Light model: information is carried by a notification service– High speed, low latency, simple behavior

Heavy model: information is carried by self-transmitting software agents– Lower speed, higher latency, more sophisticated

behavior

7

And Two Flavors of ActEvents

By reference or by value By reference: SmartEvents

– Data includes structured tags– External processor maps tags to semantics

By value: Gaugents– Agent ferries not just data…– But also mobile code that has semantic knowledge

8

SmartEvent Implementation

XML embedded in event Certain tags “exposed” for routing XML annotated to provide syntactic and

semantic information FleXML (Flexible XML) enables dynamic

runtime schema and semantics discovery and composition

9

Initial SmartEvents

Language recognition problem Validate against Schema When unknown element encountered, ask

“Semantic Server” for – New schema fragment– Processing modules (Tag Processors)

Tag Processors give higher-level events Supports dynamic schema composition

10

Newer SmartEvents

XML is described with multiple namespaces Namespaces provide direct syntactic

identification Which allows lookup of Tag Processors Semantics still looked up dynamically Still supports dynamic schema composition Tag Processors can maintain state over event

streams

11

Two Flavors of Information Bus

Light model: information is carried by a notification service– High speed, low latency, simple behavior

Heavy model: information is carried by self-transmitting software agents– Lower speed, higher latency, more sophisticated

behavior

12

Gaugent Implementation

Specialization of Worklets mobile agents Worklet Virtual Machine (WVM) installed at

each participating node Java bytecode based Routing can be altered on the fly Carries data and processing code

13

Gaugents Continued

Worklets normally carry one junction for each WVM they plan to visit– Encapsulates computation to be performed at that

location– Contains semantic information

May carry jackets– Provide a mini-workflow for a particular junction, e.g.

repetition, start/exit constraints, etc.– Contain manipulable routing information

14

In Use: KX SmartEvents

Legacy System

Legacy System

Legacy System

Packager

DB

Metaparser

tagprocessor

SemanticServerDB

Distiller

worklet worklet

workflakes

15

In Use: KX Gaugents

Legacy System

Legacy System

Legacy System

Packager

DB

Metaparser

tagprocessor

SemanticServerDB

Distiller

worklet

worklet

workflakes

16

Future Directions

New event system Extended FleXML Web services

17

Future Event System

MEEP? CHIEF? Native XML Scalability, survivability, performance VPENs: Virtual Private Event Networks

– Event diodes for Multi-Level Security Content-based routing, but with significant

optimizations for topic-based advertisements GPL

18

Conclusions

Semantic annotation of events is essential for loosely coupled architectures

Different solutions are appropriate for different types of information exchange

We have built two implementations More experimentation needed

19

Questions?

20

Extended FleXML

Currently one interpretation per incoming event per Metaparser

Multiple consumers may need multiple interpretations

Thus sets of Tag Processors may be applied to produce multiple high-level “interpretations”

21

Web Services

Juggernaut: Sun, IBM, Microsoft all pushing it hard

Has significant implications for service interoperability projects

In particular: SOAP and WSDL

22

SOAP

Very nice encapsulation Future probe-gauge standard?<env:Envelope xmlns:env="http://www.w3.org/2001/09/soap-envelope"><env:Header> <n:alertcontrol xmlns:n="http://example.org/alertcontrol"> <n:priority>1</n:priority> <n:expires>2001-06-22T14:00:00-05:00</n:expires> </n:alertcontrol></env:Header><env:Body> <m:alert xmlns:m="http://example.org/alert"> <m:msg>Pick up Mary at school at 2pm</m:msg> </m:alert></env:Body></env:Envelope>

23

WSDL

Web Services Description Language Useful for where Pub/Sub meets RPC

– Types/Messages– Operation: organize into inputs and outputs– PortType: collection of Operations– Binding: mapping of PortType to e.g. SOAP– Service: set of bound PortTypes at an actual

location