© 2004 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice

Availability Monitoring Using ASAP

MA-10-HP

Bob Rojewski and Collin O’BrienSoftware Designers, HP

bob.rojewski@hp.comcollin.obrien@hp.com

http://www.NonStopASAP.com

NonStopAsap.com 2

Agenda

• ASAP Overview

• ASAP Status Update

• Futures: ASAP 2.5

NonStopAsap.com 3

ASAP Overview

NonStopAsap.com 4

What Is ASAP?

• Availability Monitor

• State Monitor

• Performance Monitor

• Fault-Tolerant Infrastructure To Support And Enhance Monitoring Functions

• Not An Enterprise Management Framework, ASAP interfaces with various open EMF’s

NonStopAsap.com 5

ASAP Performance & State Monitoring• Monitors performance & state of system

components, including CPUs, disks, processes, etc.

• Includes all state and performance-related attributes of each object (usage percentages, rates, status, …)

• Allows thresholds to be specified for attributes; if an object/attribute exceeds a threshold, an alert will be generated

• Can be extended to applications via ASAPX API

NonStopAsap.com 6

ASAP Availability Monitoring• Performance and state monitoring make a

quantitative statement about objects

• Availability monitoring makes a user-defined, qualitative statement about objects

• Can define a rich set of service level objectives for monitored objects and attributes

• Alerting infrastructure driven off of objectives

• Applications can utilize this capability via ASAPX API; application service level objectives are defined and handled in the same way as system objectives

NonStopAsap.com 7

ASAP - Extension API ASAPX

Data_item - Defined by EDL

Value - Defined by application

Math = 0 - Add Value to Data_itemMath = 1 - Assign Value to Data_itemMath = 2 - Assign Text to Data_item

ASAPX

SharedMemory

Provider

ASAP

Consumer

ASAP_REGISTER_( domain_name : length

, segment_offset );

.

.ENDTRANSACTION;

ASAP_UPDATE_( segment_offset, error_detail

, Data_item, Value, Math );

ASAPDatabase

NonStopAsap.com 8

ASAP Infrastructure• Distributed monitoring environment• Centralized historical database• Fault-tolerant, massively parallel architecture for

scalability and low overhead• Wide variety of published and open interfaces:

ASAP Client NonStop EMS Events SNMP Agent Traps

Published Database Batch Query Interface

Conversational Interface

EMF Interfaces ASAPX API Consumer APIs

HP OpenView WebViewpoint Plug-in 3rd Party Entities

NonStopAsap.com 9

ASAP Status Update

NonStopAsap.com 10

ASAP 2.4 – Status Update• ASAP core software monitors the following

subsystems:

• ASAP Extension (ASAPX) allows applications to utilize ASAP monitoring infrastructure

• Third-party plug-ins provide monitoring for additional subsystems:

CPU Disk Expand File

Process Process Busy RDF Spooler

Tape TMF System

Base24 EMS OSS Pathway

ServerClass ServerNet SNAX SQL

TCP/IP X.25

NonStopAsap.com 11

ASAP 2.4 – New Features• ASAP 2.4 focuses on improved scalability:

− Process ZLE grouping and aggregation of abstract process domains

− Support for 10,000 abstract domains per processor

− New in-memory detail and aggregation functions

NonStopAsap.com 12

ASAP 2.4 – Scalability

• That’s what’s coming! Super-fast, super-scalar systems

• Requirements:− Must analyze each individual process and compare against

predefined service levels, alerting when necessary− Must be able to group processes and view at the group level− Must be able to aggregate/propagate data and alerts to the group

level− Must be able to set service level objectives at aggregate levels− Must be able to store summary information about the group for

historical purposes− Must have access to detail information when needed

Can you monitor?

processes in a system?

processes in an Expand network?

10,000160,000

40,800,000

processes in a processor?

NonStopAsap.com 13

ASAP 2.4 – Process Monitoring

• ASAP:− Examines each individual

monitored process at each interval− Computes and analyzes over two

dozen attributes for each process− Compares each state-pair attribute

against pre-defined objectives− Generates EMS and/or ASAP alerts

when objectives aren’t met− Stores historical information about

each process in the ASAP database

− Provides alert, status and/or performance data to a variety of optional user interfaces.

MONITOR PROCESS $A

$F

$E$D

$C$B

$A

$A $B $C$D $E $F

NonStopAsap.com 14

ASAP 2.4 – Process ZLE

Sales

Sales

$F

$E$D

$C$B

$A

Sales

• Hierarchical Process Grouping− Allows logical groups of processes− For example group all servers for

the sales application − Or group all spooler processes

• Aggregation/Propagation to Group Levels− Provides aggregate summaries

and alerts− Real-time historical data

reduction

• Configuration Options− 5 Levels of hierarchical naming− Aggregate domains at any level− Control of aggregate/propagate

function

MONITOR PROCESS SALES\$A

NonStopAsap.com 15

ASAP 2.4 – Hierarchical Process Grouping• The MONITOR Command Defines Process Domains

− MONITOR PROCESS $ABC− MONITOR PROCESS SALES\ORDERS\SERVERS\$ABC− MONITOR PROCESS SPOOLER\SUPERVISORS\$SPLS

• Benefits of Process Groups− Hierarchical views and state propagation− Aggregate summaries at each level− More manageable navigation and process monitoring− Monitor the service and the individual components

• Process Domain Name Rules− Up to 64 bytes in length− Up to 5 hierarchical levels, separated by a backslash

(“\”)− Last level denotes process name in abstract group

NonStopAsap.com 16

ASAP 2.4 – Aggregate Domains

• Use the MONITOR command to create aggregate domains− MONITOR PROCESS SALES\ORDERS\SERVERS\#− MONITOR PROCESS SALES\ORDERS\#− MONITOR PROCESS SALES\#

• Each MONITOR PROCESS <name>\# command instructs ASAP to create an aggregate record of that name

• The aggregate record contains attribute values that are computed across all group members at and below that level in the name hierarchy

• Attribute values in the summary records are determined by Data Aggregation or Attribute Propagation

• Attribute states in the summary records are determined using State Propagation or an objective comparison

NonStopAsap.com 17

ASAP 2.4 – Aggregate Domains• Data Aggregation

− ASAP combines attribute values using a mathematical comparison or computation across all members of the group

− ASAP uses SUM, AVG, MIN, MAX or CNT functions to determine the attribute value in the aggregate record

• Attribute Propagation− If no Data Aggregation is specified for an attribute ASAP propagates

an attribute value to the aggregate record− The attribute value with the worst (highest) ASAP state gets

propagated to the aggregate record

• State Propagation− ASAP propagates the worst (highest) ASAP state to the aggregate

record unless an objective is set on the aggregate domain attribute

NonStopAsap.com 18

ASAP 2.4 – Aggregate-Only Domains• The MONITOR Command Defines Aggregate-Only

Domains− MONITOR PROCESS SALES\ORDERS\##− MONITOR PROCESS SALES\##

• Aggregate Only− Process each detail record− Combine data into an aggregate record− Write ONLY the aggregate record to the ASAP database− Store the data from hundreds or thousands of processes in a

single historical record

• Detail Records Can Still be Seen !− PROCESS, MEMORY option retrieves detailed process information

directly from ASAP memory

NonStopAsap.com 19

ASAPX 2.4 – Scalability

• That’s what’s here today in ASAPX 2.4!• Requirements:

− Must analyze each individual domain and compare against predefined service levels, alerting when necessary

− Must be able to group domains and view at the group level− Must be able to aggregate/propagate data and alerts to the group

level− Must be able to set service level objectives on aggregate data− Must be able to store summary information about the group for

historical purposes− Must have access to detail information when needed

Can you monitor?

domains in a system?

domains in an Expand network?

10,000160,000

40,800,000

application domains in a processor?

NonStopAsap.com 20

ASAPX 2.4 – ASAPX ZLE

Sales

Sales

Sales\

Sales\Sales\

Sales\Sales\

Sales\Inv

Sales

• Hierarchical Process Grouping

• Aggregation/Propagation to Group Levels

• 10,000 domains per CPU• 10,000 aggregates per CPU• Control of aggregate

function− DataItem and S operands now

have optional aggregate codes• #<n> [SUM | AVG | MIN | MAX | CNT]• S [SUM | AVG]

• APP, MEMORY option retrieves detailed process information directly from ASAP memory

ASAP_REGISTER_

NonStopAsap.com 21

ASAPX 2.4 – Other Enhancements

• New ASAPX Dynamic Link Library− ASAPXDLL− Compiled from same code base as all other libraries− Use with PIC applications

• New DataItem Type− DATAITEM “<n> T”− Specifies DataItem is 8-byte ASCII text string− Should use MATH 2 (replace text)− Should not use AVG or SUM aggregate controls

NonStopAsap.com 22

Futures: ASAP 2.5

NonStopAsap.com 23

ASAP 2.5 – Alert-based Actions• Allows ASAP to automatically take actions when

objectives are not met

• Actions are defined with the associated RANK objective

• Can be specified for any objective and object (both system and user applications)

• Can control how often each action is invoked

NonStopAsap.com 24

ASAP 2.5 – Alert-based Actions• Actions can invoke any NSK command interpreter

(TACL, FUP, …), macro, or user program

• Token indicators in action definition allow entity name, state, domain, attribute, value, last state, operation, and goal of alerting object to be passed to CI, macro, or program

• Examples:RANK FILE $SYSTEM.SYSTEM.USERID, RWEP = "OOOO“

ACTION "FUP SECURE <#DOMAIN>, '<#GOAL>‘ "

RANK CPU, STATUSACTION "TACL RELOAD <#DOMAIN>, PRIME" WAIT 15 REPEAT

RANK MYAPP TRADING, TRANSRATE > 100ACTION “MYAPPMGR <#DOMAIN>, <#ATTR>, <#VALUE>, <#GOAL>”

NonStopAsap.com 25

ASAP 2.5 – Historical Data Management• New DBMaxDays option controls how much data is

retained• Different retention periods can be specified for

each entity type (e.g. save 30 days of CPU data, 6 months of APP data, …)

• Works in conjunction with SET RETAIN ROLLOVER• ASAP automatically manages historical data file

sets, and purges file sets that are no longer needed

SPOOL

COMM

APP FILE

CPU

CPU

TMFTMF

TMF

APP

APPAPP

NonStopAsap.com 26

ASAP 2.5 – Email Notifications• ASAP alerts can

optionally trigger email notifications

• Can send specific alert info (cell phones, pagers)

ASAP Critical Alert: \Newyork\Disk\$Data15 Mirror Down

• Can send full state summary and state change info (email, management reporting)

NonStopAsap.com 27

ASAP 2.5 – Email Notifications• Emails may be issued when state changes occur,

at preset times/intervals, and/or at every sample

• Can be sent to one or multiple recipients

• Content dictated by ASAP client Object State Change (OSC) configuration

• Because email notifications are controlled by the client, each operator can customize when notifications are sent and what they contain

NonStopAsap.com 28

ASAP 2.5 – Enhanced Communications Monitoring• Extends monitoring to:

• Gathers all state and performance data for each line

• Automatically discovers and monitors all lines (unless configured otherwise)

• Objectives may be set on all key attributes• Fully integrated with ASAP infrastructure

SNAX X.25 ATP6100

Envoy EnvoyACP AM3270

MultiLan CP6100

NonStopAsap.com 29

ASAP 2.5 – File ZLE• Hierarchical File Grouping

− Allows logical groups of files− E.g. group all files for the trading application, or group all spooler

data files

• Aggregation/Propagation to Group Levels− Provides aggregate summaries and alerts− Combines detail data into an aggregate record

• Configuration Options− 5 Levels of hierarchical naming− Aggregate domains at any level− Control of aggregate/propagate function− Monitor OSS files by their OSS path names

• Aggregate Only Records− Real-time historical data reduction− Write ONLY the aggregate record to the ASAP database− Store the data from hundreds or thousands of files in a single

historical record− Detail records can still be seen; FILE, MEMORY option retrieves

detailed files information directly from ASAP memory

NonStopAsap.com 30

ASAP 2.5 – Process ZLE, The Sequel• Can now monitor processes by object file name• Support for wildcards in process names• OSS processes can be monitored by specifying the

OSS path/object file name• ASAP displays the OSS path/object file name for

OSS processes, instead of the Guardian file name

$ABC/bin/sort

$SYSTEM.SYSTEM.

SQLCI

• New attributes to track number of processes in a set, and the number alerting

NonStopAsap.com 31

ASAP 2.5 – EDL Synchronization• ASAP entity definition (EDL) files reside on both the

server and client• Changes to server EDL must be synchronized with

client EDL• Prior versions of ASAP detected version

mismatches between client and server, but permitted manual synchronization only

• New version will allow automated download of server EDL to client

• Simplifies use and maintenance, greatly reduces likelihood of errors

NonStopAsap.com 32

ASAP 2.5 – Enhanced Event Reporting• Additional tokens now provided in every alert

event message:− ZASP-TKN-ASAP-OBJECTNAME contains the object name − ZASP-TKN-ASAP-STATE contains the current state of the

object− ZASP-TKN-ASAP-LASTSTATE contains the previous state of

the object

• Can associate specific event numbers with an objective via the RANK command; one for object transition to exception state (objective not met), one for object transition to normal state− RANK MYAPP TRADING, TRANSRATE > 100 CRITICAL 7001-

7002

NonStopAsap.com 33

ASAP 2.5 – OpenView Integration• Enables ASAP data to be added to OpenView• Supports all object types (system, application,

third-party)• Shares object state, availability, performance, and

service level data• Utilizes both OpenView Services and OpenView

Nodes Views• Integration is optional; OpenView is not required in

order to deploy ASAP• Included in base ASAP product

NonStopAsap.com 34

ASAP 2.5 – OpenView Integration• ASAP data can be accessed via OpenView GUI

Console or OpenView Web Interface

NonStopAsap.com 35

ASAP 2.5 – Other Enhancements• Client GUI configuration optionally

password-protected• SGP CPU placement will allow SGPs to run

in any CPU• New “Node Down” Alert can be issued

when ASAP detects that a node is down or unreachable

• Disk monitoring allows threshold to be set on largest fragment

NonStopAsap.com 36

ASAP 2.5 – Release Information

• Official release in fall of 2004

• EAP this summer

• New Basic Operations Bundle (SA29v1) for S-series will contain this version update upon release

NonStopAsap.com 37

Futures: ASAP and Web ViewPoint• Future integration of ASAP and Web

ViewPoint will provide:

− Web interface to ASAP data

− NonStop server accessible from any browser

− Object-based view of ASAP alerts within Web ViewPoint object manager

NonStopAsap.com 38

For More Information

• Background information on ASAP, white papers, downloads of trial versions, and more available at:

http://www.NonStopASAP.com

• Questions? Email us at:support@NonStopASAP.com

NonStopAsap.com