Monitoring Your Business with WSO2 BAM

Anjana FernandoSenior Technical Lead

WSO2 Inc.

Chathura EkanayakeSenior Technical Lead

WSO2 Inc.

Outline• Introduction to Business Activity Monitoring

• WSO2 Business Activity Monitor

• WSO2 BAM Components

• WSO2 BAM Toolboxes

• WSO2 BAM High Availability Distributed Deployment

• Demonstrations

Business Activity Monitoring

“The aggregation, analysis, and presentation of real-time information about activities inside organizations and involving customers and partners” - Gartner

Aggregation

● Capturing data

● Data storage

● What data to capture?

Analysis● Data operations

● Building KPIs

● Operate on large amounts of historic data or new data

● Building BI

Presentation

● Visualizing KPIs/BI

● Custom Dashboards

● Visualization tools

● Not just dashboards!

WSO2 Business Activity Monitor

• Fully-open source solution for aggregating and analyzing data and presenting information about business activities.

• Cloud-enabled, lightweight, developer-friendly and easy-to-deploy

• High performance data capture framework

• Elastically-scalable data analysis powered by Apache Hadoop

• Pre-built Data Agents for WSO2 products

Architecture

Data Agents

Data Agents● Compatible with CEP/BAM

● Get data across to BAM

− Service monitoring feature – WSO2 AS, DSS, ESB, API Manager

− Mediation monitoring feature – BAM Mediator for WSO2 ESB

− Custom data-agents

● Asynchronous & non-blocking

● Thrift for high performance message throughput

Custom Agents

• Data agents are meant to be installed/plugged into the server to be monitored

• A Java SDK is provided

• Through Thrift, different languages can be supported

Data Receiver

Data Receiver● Receives data and stores it in Cassandra

○ Scalable, big data repository

● Asynchronous & non-blocking

○ Combination of Cassandra, Thrift and the non-blocking nature results in extremely fast writes

● Shared with WSO2 CEP for real time analysis

● Supports Thrift & REST API

● Supports plugging in of different receiver types

Data Model• Data is sent over using strongly typed Data Streams

{ 'name':'phone.retail.shop', 'version':'1.0.0', 'nickName': 'Phone_Retail_Shop', 'description': 'Phone Sales', 'metaData':[ {'name':'clientType','type':'STRING'} ], 'payloadData':[ {'name':'brand','type':'STRING'}, {'name':'quantity','type':'INT'}, {'name':'total','type':'INT'}, {'name':'user','type':'STRING'} ]}

● Data Streams are versioned○ Allows for easier analysis

The Analyzer Engine

The Analyzer Engine● Powered by Apache Hadoop with querying/managing

through Apache Hive

● Data transfer to and from data sources is handled through custom storage handlers

● Parallel, distributed processing through the MapReduce programming model

• Runs on local Hadoop node or delegates to Hadoop cluster

• Scalable analytics

• Cluster can range from a couple of nodes to 1000s

● Analysis is carried out based on analytics scripts

● Scripts are based on an easy-to-learn, SQL-like query language

INSERT OVERWRITE TABLE UserTable SELECT userName, COUNT(DISTINCT orderID),SUM(quantity) FROM PhoneSalesTable WHERE version= "1.0.0" GROUP BY userName;

The Analyzer Engine

The Analyzer Engine

● Scripts can be scheduled○ Ex: once a minute, every Wednesday at 4:15 p.m., every

30th at 12 midnight

● Polyglot data architectures are supported○ Write summarised information to any kind of datastore

○ Custom written Hive JDBC handler used by default for output to relational DBs

The Presentation Layer

The Presentation Layer

• Gadget Portal for the dashboard

• Gadget Generation Wizard – Tool for generating custom

gadgets for the dashboard

• Activity Dashboard for correlating activities

• Message Console

• Plug in your own report server/ dashboard server

The Presentation LayerBAM Dashboards

The Presentation LayerGadget Portal

The Presentation Layer

Activity Dashboard

Activity Dashboard (cont'd)

The Presentation LayerMessage Console (HL7)

Message Console (cont'd)

BAM Toolboxes● BAM Toolboxes are installable and hot deployable artifacts used for

deploying functionalities to a BAM server

○ Stream definitions○ Analytics scripts○ Dashboards

● Supports plugging in of one or more of the above features

● Toolboxes for monitoring and auditing most WSO2 products are available OOTB

● Toolboxes for custom scenarios can be created easily

HA Distributed BAM deployment● WSO2 BAM can be clustered and deployed in a distributed manner to enable

high-availability, fail-over scenarios

○ Partially distributed deployment

■ Cassandra (storage) nodes are clustered

○ Fully distributed deployment

■ All components of BAM are clustered (data receiver, storage, analyzer and presentation)

● Hazelcast in-memory data grids are used for clustering implementation

Partially Distributed BAM deployment

Fully Distributed BAM deployment

Deployment – Shared events with CEP

Performance of WSO2 BAM

3-Node Cassandra Cluster used

Demo: From Publishing to Visualization

Monitoring a business process - Sales order handling

Verify order

Select supplier

Produce items

Ship products

In_production = true

Monitoring a business process - Business events

Verify order

Select supplier

Produce items

Ship products

In_production = true

Case ID = case_1Activity = Verify orderAssignee = SmithDuration = 20 mins

Case ID = case_1Activity = Verify orderAssignee = SmithDuration = 20 mins

Business events...

Case ID Activity Assignee Duration

Case_1 Verify order Smith 10

Case_2 Verify order John 15

Case_1 Produce items Nick 30

Case_1 Ship products Peter 150

Case_2 Produce items John 200

Case_3 Verify order Smith 20

...

WSO2 BAM - Deployment

Workflowsystem

BAM data publisher

Events store(e.g. Cassendra)

WSO2 BAM

Analyticsscripts

Results store(e.g. MySQL)

Gadget server

Gadgets

Visualizations /Reports

Demo: Activity Monitoring

Questions?

Thank You