Get the Big Picture! End-to-End Monitoring of Heterogeneous Middleware and Apps

Tom Lubinski, CTO, SL Corporation

8/9 October, 2013

RTView – Get the Big Picture !

• What is end-to-end monitoring• Why should you care about it ?

• Traditional system management tools are not enough• Agent-based transaction monitoring tools not enough

• What RTView does differently to address this need• Customer Use Case

What is End-To-End Monitoring ?

• Most people think this is end-to-end monitoring:

A B C D E

A Linear Data Flow …

Where A, B, C are JSPs, Servlets, Topics, Queues, etc.

• Sometimes it can be more complex:

A B C D E

With Loops and Error Paths …but it’s still “one-dimensional”

• It’s even more complex than this

• Two more dimensions must be added to the picture …

• A second dimension to capture the nested levels systems are implemented in – component layering

Host LayerPhysical Servers, Network, Disk, OS

App Server CachingMessaging

Servlet JSP

Topic Queue

Cache Service

WebLogicServer

CoherenceCache

Load-Balanced Servers backed by Distributed Cache

A third dimension that most people don’t even think of … process distribution

Why Should You Care ?

• These three dimensions create complexity – all going on at same time

• Taken together they generate huge volumes of monitoring data and complex relationships between the application and the underlying components.

• Add in a fourth dimension, time, and the challenge becomes even greater

Why Should You Care ?

• System Mgmt tools give you lots of data, but typically one component at time … only the 2nd dimension

• Transaction monitoring tools only 1st dimension.

• Both are essentially “after the fact” - like alerts … you can tell when a transaction or component has failed but it needs to be correlated with state of entire system

How RTView Addresses This Problem

• RTView collects data from all these dimensions and is aware of the relationships.

• If an app is dependent on a specific msg queue, RTView can map this to the JMS server that contains the queue, as well as the system that hosts the server and correlate the metrics

• RTView is aware of Server load-balancing and the distributed nature of Coherence caches

How RTView Addresses This Problem

www.sl.com

RTView Use Case

Application: On-Line Store

Large on-line store providing product search and ordering services for consumer software products

WebLogic for Application Server

Coherence for Database Caching

Multiple other technologies, including TIBCO EMS for communications services

VMWare Virtual Infrastructure

RTView Use Case

Application Team

Small Group (< 10) responsible for 100+ WebLogic Servers + 100 Coherence nodes replicated in DEV,

TEST, and PROD

Some peripheral monitoring, e.g. Splunk for log files, Omniture for web tracking, OEM for WLS and OC

RTView Use Case

Support Challenges

Difficult to have confidence that store is “OK”Only know when something goes wrong

When it does, difficult to determine the cause

Each subsystem implemented as a WL ClusterWebLogic = OEM only allows them to see one server at a

time, but the store works off of “clusters”

Coherence = complete black box

RTView Use Case

RTView Solution

Hi-Level Overview Diagram showing “health state” – provides confidence that all subsystems are OK

Present WebLogic information in “clusters” – makes it possible to see aggregate metrics and load

balancing for each app

Correlate Coherence metrics with WL to provide confidence that Coherence black box is OK

On-Line Store Overview Diagram

RTView used to create system overview diagram

WebLogic Cluster/Server Summary

All Servers Organized by Cluster, with Health State

WebLogic Cluster App Summary

Each Cluster shown as a unit, with server metrics aggregated

Load Balance Analysis

Load Balance Comparison of multiple metrics across WebLogic and Coherence

Aggregating Other Middleware Information

Health State of each service aggregated from multiple components

Including Aggregate Service Alert History over Time

Including Detailed History of Coherence Cache Service

The Emerging Standard Oracle Stack

VMware

Oracle Coherence/DatabasesMessaging Middleware

Fusion / TIBCO / MQ

Oracle WebLogic

Conclusion

• 3-Dimensional End-to-end monitoring required for the emerging standard Oracle application stack

• High-level overviews showing “health state” give confidence that all subsystems are OK

• Ability to present WebLogic information in “clusters” makes it possible to see aggregate metrics and load balancing for each app

• Ability to correlate Coherence metrics with WebLogic helps to ensure that Coherence black box is OK

• Include Virtual Infrastructure State and Components from Other Vendors

Thank you!For more information, please visit

www.sl.com

Get the Big Picture! End-to-End Monitoring of Heterogeneous Middleware and Apps

Technology

Award Number - DTIC · systems. The middleware application must be compatible with multiple interface specifications for the heterogeneous computerized information systems. The middleware

Dynamic Reconfiguration of IP Domain Middleware Stacks to ... · 7/18/2005 · Our end result is a Java middleware for multimedia streaming to heterogeneous mobile clients, utilising

Applying Adaptive Middleware to Manage End-to-End QoS …schmidt/PDF/CC.pdf · Applying Adaptive Middleware to Manage End-to-End QoS for Next-generation Distributed Applications Christopher

ASKAP Computing - ASKAIC · 2009. 7. 14. · • Apache Tuscany (SCA/SOA) • ActiveMQ / JMS (Message Oriented Middleware ... Python, Ruby, and PHP • Ice supports heterogeneous

DADO: Enhancing Middleware to Support …devanbu/icse2003-dado.pdfDADO: Enhancing Middleware to Support Crosscutting Features in Distributed, Heterogeneous Systems Eric Wohlstadter,

End-to-End QoS Provision over Heterogeneous IP and … · University of the Aegean Karlovasi, Samos End-to-End QoS Provision over Heterogeneous IP and non IP Broadband Wired and Wireless

Chapter 7 Message-Oriented Middleware...n servers receive requests from queues (“dequeue”) n replies are sent/received to/from queues as well Middleware for Heterogeneous and Distributed

Simulation Model for End-to-end QoS across Heterogeneous Networks

Building a SOA Solution with Oracle Fusion Middleware · PDF fileBuilding a SOA Solution with Oracle Fusion Middleware for Oracle E-Business Suite ... Application Integration End-to-end

Rational Application Developer End-2-End Highlights Tools, Servers and middleware Bill.Hahn@us.ibm.com

AmbiStream: A Middleware for Multimedia Streaming on ...dl.ifip.org/db/conf/middleware/middleware2011/AndriescuCI11.pdf · AmbiStream: A Middleware for Multimedia Streaming on Heterogeneous

E METRICS - ISBSG · Separation of presentation (Front-End) and processing (Back-End) functions Multi-channel and multi-supplier Front-End Back-End environement with heterogeneous

Contact: Hamid Mukhtar Middleware for Ad hoc User Task Composition in Heterogeneous Environments Considering User Preferences Hamid Mukhtar PhD Defense

Building Portable Middleware Services for Heterogeneous ...osl.cs.illinois.edu/media/papers/mechitov-2012-building_portable... · Building Portable Middleware Services for Heterogeneous

Evaluation of Cluster Middleware in a Heterogeneous ... · This chapter describes the state of the art in cluster computing and discusses the tools mostly used for building a heterogeneous

An Integrated Management Supervisor for End-to-End ...multimedia framework supporting the end-to-end management of heterogeneous contents, networks, and terminals while enabling Quality

Date of presentation 1 PROJECT IDEA Topic: PRIME: “Proactive Inter-Middleware for Self- Configurable Heterogeneous Cloud EcoSystems” –Objective Cloud Computing,

Climb Out of Your Monitoring Silo – Enable Real End-to-End Visibility for Your Complex, Heterogeneous Applications

Data Sharing Middleware Prototype (DSMP) for Information Dissemination Among Heterogeneous Sources

End to End Quality of Service over Heterogeneous Networks ...2F978-0-387...90 End to End Quality of Service over Heterogeneous Networks (EuQoS) 3 End-to-end QoS path - EQ-path 3.1