53
© 2014 IBM Corporation Smart integration of IBM System Automation for z/OS with other products Hans Geissler

Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

  • Upload
    others

  • View
    24

  • Download
    2

Embed Size (px)

Citation preview

Page 1: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

© 2014 IBM Corporation

Smart integration of IBM System Automation for z/OS with other products

Hans Geissler

Page 2: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Copyright and Trademarks

© Copyright IBM Corporation 2014The following names are trademarks of the IBM Corp. in USA and/or other countries and may be used throughout this presentation:

CICS, DB2, IBM, IMS, ITM, NetView, OMEGAMON, RMF, RACF, S/390, Tivoli, VTAM, WebSphere, z/OS, zSeries, System z, Linux on System z

Other company, product and service names may be trademarks or service marks of others.

Page 3: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

System Automation

for z/OS

IBM PowerHAVeritas

MSCS

z/OS Linux, AIX, Windows

Agentless Adapter

IBM only vendor to provide end-to-end, cross-platform Automation 

Unclustered nodes

System Automation for Multiplatforms

Linux, AIX, Windows

System Automation Application Manager

AutomationControlfor z/OS

IBM System Automation family works together Enterprise-wide

Page 4: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Customization Dialog (on ISPF)

AutomationPolicy

(PDS containing ISPF Tables)

AutomationControl

File(PDS containing

configuration Data)

Operator

Administrator

Operation (on NetView)

custo

mize

load/refresh

build

SA z/OS

Applications Automation (SYSOPS)✔ Automate repetitive and complex tasks

✔ Policy-based automation administration: Reduces automation implementation time, coding and support effort

✔ Goal-driven automated operation: According to your policy. Monitors, applications, messages, and alerts Increase availability and performance through proactive automation

Hardware Operation (PROCOPS)✔ Policy-based automation administration

✔ Automate and control hardware operations enterprise wide

✔ Power on/off and reset CPC

✔ Perform system IPL for z/OS, Linux and VM

✔ Automate LPAR settings, e.g. weights and capping

BEST PRACTICE policies

System Automation for z/OS - overview

Page 5: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

SA z/OS

EnterpriseAutomation

SA Application Manager

Workload Scheduling TW

S

Disaster Recovery

GDPSActive/Active

Monitoring

OMEGAMONTEP

EventsNotification

OM

NIb

us,

TEC

, TSR

M

HWAutomation Middleware

IMS, DB2, CICS

SA IOM

Alerting

Characteristics

■ Enterprise wide automated operations

■ High-availability

■ In combination with GDPS disaster recovery solution for metro and global distances

■ Part of strategic GDPS Active/Active solutions

■ Hardware and SYSPLEX automation

■ Many point-to-point integrations with major z/OS middleware and system management functions

PROCOPSSYSPLEX

SA z/OS is an integral part of today's datacenter landscape

Page 6: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

EnterpriseAutomation

SA Application Manager

Workload Scheduling TW

S

Disaster Recovery

GDPS

Monitoring

OMEGAMONTEP

EventsNotification

OM

NIb

us,

TEC

, TS

RM

HWAutomation

SA IOM

Alerting

PROCOPSSYSPLEX

SA z/OS

Middleware

IMS, DB2, CICS

SA z/OS is an integral part of today's datacenter landscape

Page 7: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

• Recover IMS components• Recover transactions and/or programs• Monitor critical resources

• Monitors number of available OLDS and excessive switching • Monitors number of available RECON datasets• Monitors VTAM Application ID availability

and the enablement of logons

• Start/stop IMS resources fast and reliable• Dependencies fulfilled: IMS and all connectivity actually works

• Resolve alert messages or escalation to TEP and OMNIbus• Proactive automation through OMEGAMON integration• Multiple IMS start types. Three standard shutdown types• Internal IMS messages can be automated• Sysplex-wide automation

IMS integration and automation main topics

Page 8: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Considerations: ➢ Which transactions should be recovered? ➢ At which error threshold level should recovery be stopped? ➢ Which ABEND codes needs special handling? ➢ Which recovery procedure (command, routine, notifications) should be done?

Example: Application transaction abends→ IMS issues message DFS554A to the master terminal

SA z/OS issue recovery to restart the the transaction

IMS1

Tran1Tran2

SA z/OSIMS1 Policy

Msg: DFS554A

Tran1: EXCLUDETran2: INCLUDE

Tran2

DFS554A

Msg: DFS554A

Tran1: EXCLUDETran2: INCLUDETran1 Tran1: EXCLUDE

Problem: IMS transactions or program abendsSolution: SA z/OS recovers failing transactions / programs

*IMS Best Practice: Recovery of IMS transactions and programs

Page 9: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

© 2013 IBM Corporation9

*IMS Best Practice: Monitor of critical IMS resourcesMonitor of ...• Recovery Control Data Sets (RECON)

• SA z/OS monitors RECON data sets and reports health state. Appropriate recovery action can be defined in the policy.

• Spare 'Online Log Data Sets' (OLDS)• SA z/OS monitors number of spare OLDS

and starts spare OLDS if necessary. Further actions can be defined in the policy.

• 'Online Log Data Sets' (OLDS) switching frequency• SA z/OS passively monitors the OLDS switch frequency.

User actions can be defined in the policy.

• LOGON capability to IMS • SA z/OS monitors the VTAM ACB.

Appropriate recovery action can be defined in the policy. Monitors resources (MTR) and recovery definitions are available in BESTPRACTICE policy

Page 10: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

IMS integration for Operators (INGIMS) • Allows operators (or automation tasks) to issue

IMS console commands• Any console-enabled IMS type-1 command• Any IMS type-2 command if an IMSPlex name is provided • Send commands to one / more / all members of an IMSPlex • Auditing of IMS commands

• Multiple commands can be issued with a single invocation

• To broadcast messages to all or selected IMS users

• To issue a list of pre-defined transactions and view the output

• Usage: As fullscreen operator dialog or programmable API

Uses Common Service Layer (CSL) of IMSPlex

Page 11: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

✔ Applications and group definitions

✔ Support FDR (Fast Database Recovery)

✔ Monitor capabilities✔ DC✔ OLDS✔ OLDS switch ✔ RECON

✔ Diagrams in PDF format: /usr/lpp/ing/doc/policies

*IMS Best Practice Policy

Page 12: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

➢ Critical event monitoring resulting in recovery cmdsProblem: DSNP007I (Data set could not be extended)

Solution: SA z/OS execute cmds as defined in PDB.

DSNP007I

● Write msg to log● Code match for msg details● Execute command defined in PDB

➢ INGDB2 automation utility➔Invoked by automation table or operators➔Start or stop a DB2 table space➔Terminate active DB2 threads➔Inform TSO users that their thread is about to be terminated➔Check for indoubt DB2 threads

➢ *DB2 BEST PRACTICE policy➔ Contains DB2 light start (only if normal mode not possible)

DB2 integration and automation

Page 13: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

© 2013 IBM Corporation13

Problem: Failing connection between IMS and DB2Solution: SA z/OS monitors the connection between IMS and DB2

(active with 'DISPLAY SUBSYS' cmd)User action can be defined in the policy.

Return Code Health Status Description

1 BROKEN No DB2 connection is defined in IMS

2 FAILED DISPLAY SUBSYS db2_id command failed

3 NORMAL IMS is connected to DB2

4 WARNING The IMS connection to DB2 is pending

5 MINOR IMS is not connected to DB2

6 CRITICAL No DB2 connection is defined in IMS, but recovery is outstanding

IMS DB2

BEST PRACTICE policy contains all

required definitions

IMS – DB2 Connection monitoring

Passive monitoring for messages: ● DSNM001I IMS/TM xxxx CONNECTED TO

SUBSYSTEM y● DSNM002I IMS/TM xxxx DISCONNECTED FROM

SUBSYSTEM yyyy RC=rc● DSNM003I IMS/TM xxxx FAILED TO CONNECT TO

SUBSYSTEM yyyy RC=rc

Page 14: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

➢ Transaction recovery

➢ Which transactions should be recovered? ➢ At which error threshold level should recovery be stopped? ➢ Which ABEND codes needs special handling? ➢ Which recovery procedure (cmd, routine, notifications) should be done?

➢ Monitoring of CICS DB2 connections➢ using command CEMT INQUIRE DB2CONN➢ Usage of SA z/OS Monitor resource (MTR) - relationships & status msgs

Return Code Health Status Description

1 BROKEN No DB2 connection is defined in CICS

2 FAILED CEMT INQUIRE DB2CONN command failed

3 NORMAL CICS is connected to DB2

5 MINOR CICS is not connected to DB2

Problem: CICS transaction abendsSolution: SA z/OS recovers CICS transaction dependend on PDB definitions

➢ CICS Operator command (INGCICS)➢ Issue any console-enabled CICS transaction➢ Broadcast messages to all or selected CICS users➢ Issue a list of defined transactions and view the output (output of CICS transactions in full-screen or pipeable line mode)

CICS integration and automation

Page 15: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

➢ Event monitoring➢ Monitoring of events issued by CICSPlex System Manager (CICSPlex SM)

➢ Long running task that subscribes for exceptional events for all monitored objects➢Eg. CICS1.TDQ.FOO1

➢ Event severity mapped to MTR health status → trigger of appropriate cmds ➢Allows you to monitor Links, Files, Xactions, TDQs, Journals, etc

➢ Health Monitoring➢ Use concept of status programs

➢executed periodically and sets CICSPLex SM severities which can be monitored by event monitoring

➢ Link Monitoring➢ Monitoring of intersystem communication (ISC) or interregion communication (IRC) links

➢ Done by registering for events that are issued by CICSPlex SM in case of a problem

➢ Mapping event severity to health status of MTR resource

CICS automation – CICSPLex SM

Page 16: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

✔ Applications and group definitions

✔ CICS Connection Monitor

✔ CICS DB2 Connection Monitor

✔ DB2 lite start

✔ Diagrams in PDF format: /usr/lpp/ing/doc/policies

*DB2 and *CICS Best Practice Policy

Page 17: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

IMS, DB2, CICS

EnterpriseAutomation

SA Application Manager

Disaster Recovery

GDPS

Monitoring

OMEGAMONTEP

EventsNotification

OM

NIb

us,

TEC

, TS

RM

HWAutomation

SA IOM

Alerting

PROCOPSSYSPLEX

SA z/OS

Middleware

SA z/OS is an integral part of today's datacenter landscape

Workload Scheduling TW

S

Page 18: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Update TWS current plan via INGTWS

Send SA z/OS commands Return SA z/OS status via Special Resource

TWS SA z/OS

NetView log

TWS logEQQMLOG

Batch I/F

Special Resource

TWS does the scheduling➔ Complex schedules➔ Holidays➔ First/last day stuff

SA z/OS does the starting / stopping➔ Predefined start/stop command sequences➔ Move of resources ➔Thresholding➔ Automated recovery

Check result

Execute any accepted SA z/O

S (NetView) cmd

SA z/OS integration with TWS - overview

Page 19: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

➢ TWS schedules the customer's repetitive workload and manages interdependencies between concurrent and successive jobs.

➢ The SA z/OS -TWS automation ties TWS’s sophisticated calendar capabilities to SA z/OS and exploits and controls it in SA z/OS environment.

➢ Automation of normal operation and recovery of TWS components.

➢ TWS can issue any SA z/OS (Netview) command as part of TWS operation without intervention of the Automation Team

➢ SA z/OS Applications (APL), Application Groups (APG) and Monitor resources (MTR) can be handled

➢ Supports that multiple commands can be processed simultaneously

➢ Provides Security / Audit Trail

TWS - SA z/OS integration background

Page 20: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

TWS plan: 1) Stop IMS resources

2) Execute batch jobs

3) Start IMS resources

SA z/OS: INGREQ STOP IMS/APG

INGREQ START IMS/APGPost operation complete

Trigger SA z/OS cmd

Post operation complete

Trigger SA z/OS cmd

Any SA z/OS cmd can be defined in TWS dialogs (all cmds accepted under Netview)

Update of TWS operations in TWS dialogs - without intervention of Automation team .

SA z/OS – TWS integration usage example

Definitions in the TWS Dialogs:

Page 21: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Manage current TWS plan from SA z/OS e.g. update operation status

All TWS operation status including details are available in SA z/OS

TWS integration: Operator interfaces on SA z/OS SA z/OS command interface:

Page 22: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Any command accepted by NetView

(MVS, SA z/OS, NetView)

Write output to specified dataset (redirect output)

Note: SAMPLE JCL available in sample library (SINGSAMP) member EVJSJ001

● Interface allows SA z/OS cmds to be submitted from a batch job (any NetView accepted cmd) ● JCL coding required● Allows control of APLs, APGs and MTRs

1.Collect SA z/OS statistical data2. Stop IMS1 group3. MVS command4. Refresh SA z/OS config5. Reset statistical data

SA z/OS Batch interface

Page 23: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

✔ Application and group definitions

✔ Controller, data store, communication, status observer, tracker, …

✔ Multiple TWS Controller support

✔ Diagrams in PDF format: /usr/lpp/ing/doc/policies

*TWS Best Practice Policy

Page 24: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

IMS, DB2, CICS

Workload Scheduling TW

S

Disaster Recovery

GDPS

Monitoring

OMEGAMONTEP

EventsNotification

OM

NIb

us,

TEC

, TS

RM

HWAutomation

SA IOM

Alerting

PROCOPSSYSPLEX

SA z/OS

Middleware

SA Application Manager

EnterpriseAutomation

SA z/OS is an integral part of today's datacenter landscape

Page 25: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

System Automation

for z/OS

IBM PowerHAVeritas

MSCS

z/OS Linux, AIX, Windows

Agentless Adapter

Unclustered nodes

System Automation for Multiplatforms

Linux, AIX, Windows

System Automation Application Manager

SA z/OS integration with SA Application Manager

Page 26: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Diagrams in PDF format: /usr/lpp/ing/doc/policies

● High Availability and Disaster Recovery management of composite business applications (Cross-Cluster + Cross-System + Cross-Site)

● Integrates various High-Availability Clusters (e.g. SA z/OS, SA MP, PowerHA, MSCS, Veritas Cluster Server)

● Automates startup, shutdown and resynchronisation of composite business applications running on different platforms.

● WEB-based user interfaces as single point of control (SPOC) ● BEST PRACTICE policy *E2E shipped with SA z/OS

➢ E2E_ADPT - E2E Automation Adapter ➢ E2E_EAS - Event Automation Service ➢ SAAM - System Automation Application Manager ➢ SADCM - System Automation GDPS Agent ➢ SAHWA - System Automation Hardware Adapter

System Automation Application Manager overview

Page 27: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

SA AM: Dash based Operations ConsoleSystem Automation Application Manager offers WEB-based user interfaces as single point of control

Truly manage the availability of all resources hosted by different OS/Platforms managed by different Automation Products in one Interface

Flyover Context menue

Visualization of the dependency

graph

Actual injected requests

Page 28: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Middleware

IMS, DB2, CICS

EnterpriseAutomation

SA Application Manager

Workload Scheduling TW

S

Monitoring

OMEGAMONTEP

EventsNotification

OM

NIb

us,

TEC

, TS

RM

HWAutomation

SA IOM

Alerting

PROCOPSSYSPLEX

SA z/OS

GDPS

Disaster Recovery

SA z/OS is an integral part of today's datacenter landscape

Page 29: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

BEST PRACTICE policy *GDPS shipped with SA z/OS

Required setup for GDPS automatically triggered during SA z/OS initialization via NetView TOWER functionality

Command definitions Autooperators Automation Table GDPS initialization procedure

GDPS enviromental descriptions and 'takeover' scripts defined in SA z/OS policy

Automatically loaded/refreshed by SA z/OS

SA z/OS and GDPS integration

Page 30: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

System Shutdown 'real' work done by SA z/OS (INGREQ) SA z/OS terminates entire system SA z/OS terminates 'itself' SA z/OS informs GDPS about progress

Shutdown possible either from GDPS or via INGREQ ALL on SA z/OS

SA z/OS and GDPS integration - handshaking

Page 31: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

IMS, DB2, CICS

EnterpriseAutomation

SA Application Manager

Workload Scheduling TW

S

Disaster Recovery

GDPS

EventsNotification

OM

NIb

us,

TEC

, TS

RM

HWAutomation

SA IOM

Alerting

PROCOPSSYSPLEX

SA z/OS

Middleware

OMEGAMONTEP

Monitoring

SA z/OS is an integral part of today's datacenter landscape

Page 32: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

OMEGAMON● Usage of OMEGAMON gathered

data e.g. Address Space Bottleneck Table (CPU_Loop_Index)

● Seamless access to data and status

SA z/OS● Out-of-the-box automation scenarios

(BEST PRACTICE policy *ITM) ● policy-based automation and

resolution

OMEGAMON

NetView /SA z/OS

Policy

z/OSCICSIMS

DB2

MQ

… Performance Specialists

Operators & Automation Administrator

IBM System z Automation and Monitoring

Status

Notifications

MonitoringData

Automation rules

Integration of SA z/OS and OMEGAMON

Page 33: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

SA z/OS

Situation

● The overall z/OS system utilization and also the utilization of individual started tasks / jobs is understood for normal and peak hours

Problem

● Detect when started tasks / jobs show abnormally high CPU utilization

● Prevent that these types of work can dominate the system

Solution

● OMEGAMON XE for z/OS data is analyzed by SA z/OS for high CPU utilization

● SA z/OS can categorize different types of work and allows to define various recovery actions through policy – not programming!

OMEGAMON XE for z/OS

Monitor

Categorize / Recover

Ignore

NotifyReset

Policy

Looping candidates

✔Ignore✔Warn✔Diagnose✔Stop/Cancel✔Reset

✔Ignore✔Warn✔Diagnose✔Stop/Cancel✔Reset

Recovery Options

NewOA43571

OMEGAMON looping address space suppression

Page 34: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Situation● Incidents happen all the time but luckily,

with SA z/OS, they can also be recovered very quickly

Problem● Customers using TEP as primary console

for operations rely on every incident being reported as a situation

● However, short-lived incidents may remain undetected by situation sampling routines and hence are invisible to the operations team

Solution● SA z/OS exploits ITM pure event

situations

● So, in combination with its monitoring agent, SA z/OS allows to send selected exceptional messages immediately to TEP by means of policy

Push inreal-time

TEP

SYSA SYSB

SYSC

SA z/OSMonitoring Agent

SA z/OS

Incident

NewOA43571

Immediate message reporting on Tivoli Enterprise Portal (TEP)

Page 35: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

➢ SA z/OS adds its operational views to the TEP presented side by side with availability data

➢ Capabilities✔ Resource overview, health and status item workspaces

✔ Situation monitoring and visualization

✔ Graphical summaries combined with detailed tabular views

✔ Context-sensitive linking between workspaces

✔ Display critical messages and WTORs

✔ Display topology view

Integration with operator interface TEP

Page 36: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

➢ Focused Resource and Request Mode used as filter criteria➢ Start, Stop, Group -or- All

➢ Relationships shown by different colors Visualization

of the dependency graph

Integration with operator interface TEP

Page 37: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

SA IOM

IMS, DB2, CICS

EnterpriseAutomation

SA Application Manager

Workload Scheduling TW

S

Disaster Recovery

GDPS

Monitoring

OMEGAMON

HWAutomation

PROCOPSSYSPLEX

SA z/OS

Middleware

Alerting

OM

NIb

us,

TEC

, TS

RM

EventsNotification

SA z/OS is an integral part of today's datacenter landscape

Page 38: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Exceptional events could lead to …

➢ Event: A software message indicating that something has happened

➢ Alarm: A device that gives an audible or visual warning of a problem or condition

➢ Ticket: File in a tracking system containing detailed information about a reported incident

Exceptional events are forwarded by SA z/OS in the following way ...

● Event on the Netcool/OMNIBus or Tivoli Enterprise Console (TEC)

● Alarm/Alert notification through System Automation Integrated Operations mgmt (SA IOM)

● Trouble Ticket within Tivoli Service Request Manager (TSRM) / SmartCloud Control Desk

SA z/OS integration regarding exceptional events and Alerting

Page 39: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

SysplexSA z/OS

AgentNetView

E/ASOMNIbusEIF Probe

PPI TCP/IP

Message Format File

Event Notification for Tivoli Netcool/OMNIbus ● SA z/OS sends EIF-events to OMNIbus using the message adapter of the

NetView Event/Automation Service (E/AS)● Single E/AS may be used within an SA-sysplex, at least one E/AS is connected to

the EIF-probe● Event notification may occur at pre-defined alert points or custom alert points

based on SA z/OS policy settings (Informlist = EIF)● Take advantage of BEST PRACTICE *IBMCOMP

for event correlation “out of the box”

SA z/OSAgent

SA z/OSAgent

OMNIbusObjectServer

Rules

Event List

Event routed within sysplex if E/AS not active locally

EIF-event

alerts.status

Integration with Tivoli Netcool/OMNIbus

Page 40: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Opening a trouble ticket - Trouble Ticket Systems

● TSRM notification may occur at pre-defined alert points or custom alert points based on SA z/OS policy settings (Informlist = TTT)

●Tivoli Directory Integrator (TDI) used to provide data to TSRM➢ TDI provides an excellent concept to synchronize and transform data of various different systems (e.g. files, databases, folders, message queues, web services,...)➢ The process is triggered by different kind of events (e.g. email, HTTP, TCP,SNMP,...)➢ Flexible use of TDI in a variety of scenarios

● Tivoli Service Request Manager (TSRM)➢ Tivoli Process Automation Engine➢ Workflow based problem and change management

SA z/OS TDI

Configuration

TSRM

Peregrine

Integration with TDI - TSRM

Page 41: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

IMS, DB2, CICS

EnterpriseAutomation

SA Application Manager

Workload Scheduling TW

S

Disaster Recovery

GDPS

Monitoring

OMEGAMON

EventsNotification

OM

NIb

us,

TEC

, TS

RM

HWAutomation

PROCOPSSYSPLEX

SA z/OS

Middleware

SA IOM

Alerting

SA z/OS is an integral part of today's datacenter landscape

Page 42: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

➢ Notification / Alerting via SA IOM Pager, SMS, email, Voice Escalation process by SA IOM

➢ Predefined set of SA z/OS „alert“ situations Automation failure (e.g. PROBLEM state)

and critical threshold exceeded messages Highly customizable

➢ Immediate notification and escalation of built-in SA z/OS and user-defined issues

➢ Improved productivity by eliminating manual callout procedures for common alerts

➢ Web-based SA IOM user interface to define escalation policy Acknowledge alerts Watch active alerts

SA z/OS SA IOM

Alert Integration with SA IOM

Page 43: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

➢Escalation timeframe and levels configured by the owner

➢Notification delivered via individual notification preference

➢A group contains list of support technicians for a particular application or system

➢An alert must be positively acknowledged, or it will escalate to the next level

Alerting via SA IOM - Concepts

Page 44: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

IMS, DB2, CICS

EnterpriseAutomation

SA Application Manager

Workload Scheduling TW

S

Disaster Recovery

GDPS

Monitoring

OMEGAMON

EventsNotification

OM

NIb

us,

TEC

, TS

RM

SA IOM

Alerting

SA z/OS

Middleware

PROCOPSSYSPLEX

HWAutomation

SA z/OS is an integral part of today's datacenter landscape

Page 45: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

➢ Managing Couple Data Sets (CDS)➢ Automated creation / recovery of alternate CDS➢ INGPLEX CDS: simplifies management of CDS

➢ Managing System Logger➢ Resize primary/alternate LOGR CDSs in case of directory shortage➢ Notification if wrong share options for log streams

CDS are crucial importance for Parallel SYSPLEX → if system cannot access CDS, corresponding SYSPLEX function is impacted

Integration with SYSPLEX functionality

Page 46: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

➢ Managing Coupling Facilities (CFs)➢ Rebuild of allocated structures➢ Duplex structures → increase availability/usability➢ Removing / Integration of CFs

Integration with SYSPLEX functionality (contd.)

Page 47: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

➢ Display SYSPLEX informations ➢ system name, status SSUM & WLM details➢ Console information (master console name, # queued msgs, awaiting mounts...)

➢ Display & update DUMP options & slip traps➢ Support of Multi-system SVC dumps

➢ Display & compare IPL informations➢ Particular PARMLIB member

Integration with SYSPLEX functionality (contd.)

Page 48: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

➢ SA z/OS recovery actions➢ Resolving WTO(R) Buffer shortages

➢ extend buffer ➢ modify console characteristics ➢ cancel of jobs causing the WTO(R)s

➢ Handling Long-Running Enqueues (ENQs)➢ Recover of long-running enqueue ➢ “HUNG” commands ➢ command flooding

➢ Managing System Removal➢ Automation of messages IXC102A & IXC402D

➢ Recovering Auxiliary storage shortage➢ Dynamically allocating spare local page datasets

➢ Hardware validation➢ Cross-validation of HW configuration with SA z/OS definitions

Integration with SYSPLEX functionality (contd.)

Page 49: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

SA z/OSProcOps

4. Single Point of Control for operator and automated actions

3. NOT limited to Sysplex boundaries

2. Focal Point Concept supporting Backup Focal Point

5. ACTIVATE / DEACTIVATE: - CPC (Target HW) - LPAR (Target LPAR) - Image (Target System incl z/VM) - (Sub-)Group of Images- Initialization from POR to IPL according to Activation Profile.- Termination of active sessions, Z EOD, Quiesce, POWER-OFF.

Other cmds: LOAD, SYSRESET,START, STOP, RESTART, EXTINT, ....

8. List - CPCDATA: Target HW report of configuration and status information 7. Capacity control

- CBU: Capacity Backup Temporarily increase CP capacity in case of an unplanned capacity shortage - OOCOD: On/Off Capacity On Demand

6. Activation Profile control: - PROFILE: List and Change - CCNTL: Assign RESET Profile to CPC

1. Monitors and Controls: System z HW and Images

9. Power Mode - POWERMOD Runs z196 in power saving mode if full performance is not required.

10. Support of zEnterprise – zBX

CFCF

CF

CF

Integration with the Hardware (Processor Operations)

Page 50: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Summary● System Automation for z/OS integration points ...

– Middle ware (IMS, DB2, CICS)– Workload scheduling (TWS)– Alerting (SA IOM)– Event notification (OMNIbus, TEC, TSRM)– Disaster Recovery (GDPS, Active/Active)– Monitoring (OMEGAMON - TEP)– Enterprise Automation (SA Application Manager)– Hardware automation (PROCOPS)– ...

… using SA z/OS BEST PRACTICE Policies

SA z/OS

Page 51: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

51

Th ank You

Merci

Thanks

Gracias!

Obrigado

Danke

Japanese

English

French

Russian

German

Spanish

Brazilian PortugueseArabic

Traditional Chinese

Simplified Chinese

Thai

Page 52: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Visit our home pages atAutomation Control for z/OS: http://www-03.ibm.com/software/products/us/en/ibm-automation-control-for-zosSystem Automation for z/OS: http://www-01.ibm.com/software/tivoli/products/system-automation-zos/index.html

Visit our Wikis atAutomation Control for z/OS:

Link to Automation Control WikiSystem Automation for z/OS: Link to System Automation for z/OS Wiki

our Community at IBM Service Management Connect https://www.ibm.com/developerworks/servicemanagement/z/index.html

or our User forums at Automation Control for z/OS: IBM Service Management Connect https://www.ibm.com/developerworks/community/forums/html/forum?id=29e48eab-e754-4aec-ad15-a3d01d4035bd The purpose of this group is to discuss technical issues related to IBM Automation Control for z/OS with your peers.

System Automation for z/OS: Yahoo http://groups.yahoo.com/group/SAUSERS/ The purpose of this group is to discuss technical issues related to IBM Tivoli System Automation for z/OS with your peers

Page 53: Smart integration of IBM System Automation for z/OS with ... · System Automation for z/OS IBM PowerHA Veritas MSCS z/OS Linux, AIX, Windows Agentless Adapter IBM only vendor to provide

Acknowledgements and Disclaimers:

© Copyright IBM Corporation 2014. All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract

with IBM Corp.

IBM, the IBM logo, ibm.com, Tivoli, CICS, DB2, GDPS, IMS, NetView, OMEGAMON, System z, WebSphere, z/OS, zEnterprise and zSeries are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtmlOther company, product, or service names may be trademarks or service marks of others.

Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates.

The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.