A General Extension System for Event Processing Languages

<Insert Picture Here>

A General Extension System for Event Processing LanguagesAlexandre Alves - Oracle CEP

Monday, July 11, 2011

DEBS2011

Text

Agenda

• Scenario 1• String manipulation (programming-in-the-small)

• Blending CQL and Java• Architecture• Scenario 2

• Geo-fence (extensibility)• Blending CQL and Spatial• Architecture• Q/A


DEBS2011

Text

Scenario 1:Programming-in-the-small

• Correlate security price changes to real-time event news.


DEBS2011

Text


SELECT * FROM news [RANGE 1 HOUR], stock_tick [RANGE 1 HOUR] WHERE /* join-criteria */

Oracle CQLProcessor


DEBS2011

Text


• Problem: the NEWS feed is a unstructured (String), hence not trivial to identify its event properties• For example, a naive join criteria is to look for a stock’s

symbol• The problem of parsing a String can be solved using

Regular Expressions. • Regular Expressions and other programming-in-the-

small tasks have long been solved by general purpose programming languages

• Can we leverage this within an event processing language (e.g. CQL)?


DEBS2011

Text

Java Regular Expressions

• Let’s look at a simple solution to the problem using Java:

Matcher matcher = Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message);

if (matcher.find()) { symbol = message.substring(matcher.start() + 1,

matcher.end() - 1));}

Matches three letter symbols, separated by leading and trailing white-spaces

Text


DEBS2011

Text

Blending CQL with Java

CREATE VIEW filtered_news(message, matcher) AS SELECT message, Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) as matcher FROM news [RANGE 1 HOUR]

SELECT location, item_description, message FROM filtered_news, stock_tick[RANGE 1 HOUR] WHERE matcher.find() = true AND filtered_news.message.substring(matcher.start() + 1, matcher.end() - 1) = stock_tick.symbol


DEBS2011

Text




Static method invocation

Method invocation

CQL char conversion to Java String


DEBS2011

Text




Overload of CQL equality to handle Java String

Overload of CQL ‘+’ operator to handle Java Integer

‘matcher’ is a Java Object, and treated as a complex type by CQL


DEBS2011

Text

Architecture

Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)

• Is this symbol: • A stream attribute (e.g. message)?• A stream alias (e.g. FROM S1 as A)• A CQL function (e.g. SELECT myfunc())• The l-value of complex type (e.g. myObj.myMethod())• A constructor• A (static) method• A class name



DEBS2011

Text

Architecture


• To be able to semantically compile symbols, we need to have type information.

• However, CQL is not aware of Java, nor should it be.• There is a need of a generic type-system, and an

extension model for providers to plug-in their own languages



DEBS2011

Text

Architecture

CQL Parser

Semantic Analyzer

Code Generator

Code Executor

Type Registry

Java Type-System


DEBS2011

Text

Architecture

Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)

“Pattern” Type Registry

Java Type-System

methods

fieldsComplexType


DEBS2011

Text

Architecture

“Pattern” Type Registry

Java Type-System

methods

fieldsComplexType

CQL deals with a ‘complex type’ abstraction, and does not know of its ‘Java Language’ binding.


DEBS2011

Text

Architecture

Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)

Find a “compile” method and its return typemethods

fieldsComplexType

Pattern


DEBS2011

Text

Architecture


Store ‘matcher’ as a stream/view attributemethods

fieldsComplexType

Pattern

methods

fieldsComplexType

Matcher


DEBS2011

Text

Architecture

CREATE VIEW filtered_news(message, matcher)

stream attribute*1 Type

ComplexType

<<is-of>>

Extensible Language

Java Class

<<metadata binding>>

Extension Java Language

‘matcher’ is an attribute of the view of type ‘java.util.regex.Matcher’


DEBS2011

Text

Extensibility

stream attribute*1 Type

ComplexType

<<is-of>>

Extensible Language

Java Class

<<metadata binding>>

Extension Java LanguageHow do we know which ‘language extension’ to use?How to provide new extensions?


DEBS2011

Text

Scenario 2:Spatial Integration

• Targeted marketing for a mobile subscriber• CEP application checks if the location of the subscriber is

within the distance of a registered shop

TextText

Oracle Spatial

id: CHARgeometry: SDO_GEOMETRY

Shop


DEBS2011

Text

Blending CQL with Spatial

CREATE VIEW CustomerLocation-Stream(point, custId) AS SELECT createPoint@spatial(lng, lat) as point, custId FROM Location-Stream

SELECT loc.custId, shop.idFROM CustomerLocation-Stream[NOW] AS loc, Shop as shopWHERE contain@spatial(shop.geometry, loc.point, 2.0d) ‘spatial’ link points to

Oracle Spatial ‘extension’, where ‘contain’ function resides

point is a spatial type


DEBS2011

Text

Architecture

contain@spatial(shop.geometry, loc.point, 2.0d)

CQL link specifies language extension, which are plugged into the system as a CQL cartridge

CQL Cartridge

CQL

Cartridge

<<link>>


DEBS2011

Text

Conclusion

• Blending of CQL with other languages allow for the creation of feature-rich CEP applications while still being highly descriptive• Example:

• String manipulation using Java• Geo-fencing using Oracle Spatial

• Generic frameworks allows for the dynamic plugin of extensions (cartridges)


Technology

A General Extension System for Event Processing Languages