Upload
md-fasiuddin
View
215
Download
0
Embed Size (px)
Citation preview
8/20/2019 Event Moniter Service
1/76
Using the Event Monitoring Service
Manufacturing Part Number:B7612-90009
March 1999
© Copyright 1999 Hew lett -Pa ckard Company
8/20/2019 Event Moniter Service
2/76
2
Legal Notices
Th e in for m a t ion con ta i ned in th is d ocu m en t is sub ject to ch a n ge w i th ou t
notice.
H ewl ett-Packar d m akes no warr anty of any ki nd wi th r egard t o this
manu al, in clu di ng, but not l i mi ted to, the im pl i ed w arr ant i es of
mer chan tabil i t y and fitness for a par ti cul ar pur pose. Hewlett-Packardsha ll not be liable for errors cont a ined herein or direct, in direct, specia l,
incidental or consequential damages in connection with the furnishing,
performa nce, or use of th is ma teria l.
Copyright © 1999 Hewlett-Packard Company.
This document conta ins informat ion which is protected by copyr igh t . Al l
rights a re reserved. Reproduction, ada pta tion, or tra nslat ion w ithout
prior w ritt en permission is prohibited, except a s a llow ed under t hecopyright laws.
Corporate Offices:
Hewlett-Packard Co.
3000 Hanover St.
Palo Alto, CA 94304
U se, duplica tion or disclosure by the U.S. G overnm ent D epart ment of
Defense is subject to res t r ict ions as set for th in paragraph (b)(3)(i i) of theRigh t s in Technical D a t a a nd S oftw a re claus e in FAR 52.227-7013.
Rights for non-DOD U.S. Government Depart ments a nd Agencies ar e as
set fort h in FAR 52.227-19(c)(1,2).
U se of this m a nua l a nd fl exible disc(s), compa ct disc(s), or ta pe
cart ridge(s) supplied for t his pa ck is restr icted to t his product only.
Ad dit i on a l copies of th e pr ogra m s m a y be m a d e f or secu r it y a n d ba ck-u p
purposes only. Resa le of th e progra ms in t heir present form or w ith
a ltera tions, is expressly prohibited.
A copy of th e speci fi c w a r r a n ty ter m s a p pl ica b le to y ou r Hew let t -Pa c ka r d
pr od uct a n d r epla cem en t pa r t s ca n be ob ta in ed f rom y ou r loca l Sa l es a n d
Service Offi ce.
8/20/2019 Event Moniter Service
3/76
Contents
3
1. Understanding the Event Monitoring Service
U nderst a nding E MS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
EMS Configur a tion Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
E MS R esource Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
U nderst a nding Monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
U sing EMS in a High Availability Environment . . . . . . . . . . . . . . . . . . .17
E MS w ith MC /ServiceGua rd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
E MS w ith IT/Operat ions or Net work Node Man a ger. . . . . . . . . . . . . .17
EMS wit h Your Choice of Syst em Mana gement Softwa re . . . . . . . . . .18
2. Installing and Using EMS
Ins ta lling EMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Removing E MS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22
St a rting t he Event Monitoring Service . . . . . . . . . . . . . . . . . . . . . . . . . .23
3. Selecting Resources to Monitor
Selecting Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
View ing R esource Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4. Defining a Monitoring Request
St a rt ing a Monitoring Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
Specifying When t o Send a n E vent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Det ermining t he Frequ ency of Event s . . . . . . . . . . . . . . . . . . . . . . . . . . .36
Set ting th e Polling I nt erva l. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37
Selecting P rotocols for S ending E vents . . . . . . . . . . . . . . . . . . . . . . . . . .38
opcmsg (ITO) Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
TCP a nd U DP Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39
8/20/2019 Event Moniter Service
4/76
4
Contents
SN MP Tra ps Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Em a il Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Console Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Sy slog Option. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Textlog Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Adding a Notifi cat ion Comment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5. EMS Operations
Copying Monitoring Requ ests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Modifying Monitoring Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Removing Monitoring Request s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Viewing Monitoring Request s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
6. Configuring MC/ServiceGuard Package Dependencies
7. Troubleshooting
EMS Directories a nd Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Logging a nd Tra cing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
EMS Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
H igh Ava ilability Monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
EMS Tra cing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Performa nce Considera tions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Syst em Performa nce Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Netw ork Performa nce Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Testing Monitor Request s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Testing D isk Monitor Request s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Testing C luster Monitor Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Testing Net w ork Monitor Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Testing S yst em Resource Monitor Requ ests . . . . . . . . . . . . . . . . . . . . 68
8/20/2019 Event Moniter Service
5/76
Contents
5
Ma king Su re Monitors a re Runn ing . . . . . . . . . . . . . . . . . . . . . . . . . . .69
Glossary
8/20/2019 Event Moniter Service
6/76
6
Contents
8/20/2019 Event Moniter Service
7/76
7
Printing History
This edition documents ma terial r elat ed to insta lling a nd confi guring
Event Monitoring S ervice.
The printing da te chan ges wh en a n ew edition is printed. (Minor
corrections and updates which are incorporated at reprint do not cause
th e d a te to ch a n ge. ) Th e pa r t n u mber is r ev ised w h en ex ten s ive tech n ica l
changes are incorporated.
New editions of this ma nua l will incorpora te a ll mat erial updat ed since
th e previous edit ion.
HP Printing Division:
High Performance Systems Division
Hewlett-Packard Co.
19111 Pruneridge Ave.
Cupertino, CA 95014
Table 1
Printing Date Part Number Edition
March 1999 B7612-90009 Edition 1
8/20/2019 Event Moniter Service
8/76
8
8/20/2019 Event Moniter Service
9/76
9
Preface
This gu ide describes how to use E vent Monitoring Service (EMS ) alone,
a nd in conjunction with high a vailabili ty softw a re such a s
MC/Ser viceGu a rd a nd ent erprise ma na gement products like IT/O. The
cha pters a re a s follows:
• U ndersta nding the E vent Monitoring S ervice
• Insta l ling and U sing EMS
• Selecting Resources to Monitor
• Defi ning a Monitoring Request
• EMS Opera tions
• Confi guring MC /ServiceGua rd P a ckage Dependencies
• Troubleshooting
RelatedPublications
The follow ing document s cont a in a dditiona l relat ed informa tion:
• U sing H igh Avai lab i l i ty Moni tors (HP Part Number B5736-90012)
• M anaging M C/ Serv iceGuard (HP Part Number B3936-90026)
• EM S Hardwar e Event M oni tor i ng User’s Guid e (HP Pa rt NumberB 6191-90011)
• EM S Har dw ar e M oni tors Refer ence M anual (HP Pa rt Number
B 6191-90012)
• http://docs.hp.com/hpux/ha Web site for information about
Hewlett-Packard’s high-availability technologies where you can find
documents
• www.software.hp.com Web site for designing a nd building a n E MS
monitor. Select H igh Ava ilability t hen E vent Monitoring S ervice
Developer’s Kit.
• P et er Weyga nt , Clusters for H igh Avai l abi l i ty: A Pri mer of HP-UX
Solut ions (IS B N 0-13-494758-4). H P P ress: P rent ice Ha ll, In c., 1996
• Configuri ng OPS Clu sters wi th M C/ L ockM anager (HP Pa rt Number
B 5158-90019)
• M anagin g Systems and Work groups (HP Part Number B2355-90157)
8/20/2019 Event Moniter Service
10/76
10
• Tom Ma dell, Di sk and F il e M anagement Tasks on H P-UX (ISB N
0-13-518861-X). H P P ress; P rent ice Ha ll, I nc., 1997
• HP OpenView I T/ Operat i ons Adm in istrators Task Gui de (P /N
B 4249-90003)
• M anaging H ighly Avai l able NFS (HP Part Number B5125-90001)
ProblemReporting If you have a ny problems wit h t he softwa re or documenta tion, plea secont a ct your local Hew lett-Pa cka rd S a les Offi ce or Customer Service
Center.
8/20/2019 Event Moniter Service
11/76
11
1 Understanding the EventMonitoring Service
The E vent Monitoring S ervice (EMS ) is a fra mew ork for resourcemonitoring. It provides fa cilities to monitor sys tem resources including
8/20/2019 Event Moniter Service
12/76
12 Chapter1
Understanding the Event Monitoring Service
confi guring, checking resource sta tus, a nd sending notificat ion wh ere
confi gured condit ions a re met.
This chapter describes the following:
• Understan ding EMS
• U sing EMS in a High Availability E nvironment
8/20/2019 Event Moniter Service
13/76
Chapter 1 13
Understanding the Event Monitoring Service
Understanding EMS
Understanding EMS
E M S ca n be con fi g ur ed t o poll a loca l s ys t em or a p plica t i on r es ou rce a n d
send messa ges wh en events occur. It can a lso be confi gured t o listen for
event messa ges from a synchronous monitors. An event is a ny a ction or
sta te you want to know a bout. For example, you ma y wa nt t o be alerted
w h en a d is k fa i ls or w h en a v a ila b le fi l es ys t em s pa ce fa l ls b elow a cer t a in
level. EMS a llow s you to confi gure w ha t you consider an event for anymonitored syst em r esource.
EM S sen d s even t n ot i fi ca t i on to a w i de va r iet y of softw a r e u sin g m u lt i ple
protocols (opcmsg, S NMP, TCP, U DP, sys log, console, t extlog, ema il). For
exa m pl e, y ou ca n con fi gu r e EM S to sen d a m essa ge to M C /Ser viceGu a r d
a nd I T/Opera tions w hen a disk fa ils. These a pplica tions can th en use
tha t messa ge to trigger package failover and t o send a messa ge to a n
a dministra tor to fi x the disk.EMS opera tes w ith a registry fra mework, a collection of monitors, and a
confi gura tion interface tha t r uns under S AM (HP -U X System
Administra tion Mana ger). EMS sta rts a nd stops the monitors, stores
informa tion used by the monitors, a nd directs monitors where to send
events.
Fig ur e 1-1 illustra tes th ese EMS r elat ionships.
Figure 1-1 Event Monitoring Service Operations
Target, such asIT/Operations,MC/ServiceGuard
Registrar
Framework
System andapplication
MONITORS
API
Resource
dictionary
Client, such asEMS the SAM interface to monitors,MC/ServiceGuard package configuration
resources
8/20/2019 Event Moniter Service
14/76
14 Chapter1
Understanding the Event Monitoring Service
Understanding EMS
E M S w or ks in a h ig h a v a ila b ilit y en vir on men t t o h elp y ou q u ick ly d et ect
a nd eliminat e single points of fa ilure, a nd to monitor resources tha t canbe used a s MC/ServiceGua rd pa cka ge dependencies. EMS can a lso be
u sed ou t sid e a h igh a va i la b ili t y en vi ron m en t for m on itor in g th e s t a tu s of
syst em r esources.
EMS Configuration Requirements
Th is b ook d ocu men t s t w o w a y s t o a cces s a n d con fi g ur e E M S . You ca n u se
either or both of the following met hods or use your own m onitor client :
• Configure monitor ing requests through the EMS in ter f ace in the
Resource Management area of SAM.
• Confi gure package dependencies in MC/ServiceGuard by:
— u sin g th e Pa c ka ge C on fi gu r a t ion in ter fa ce in th e Hi gh Ava ila b ili t y
Clust ers suba rea of SAM, or
— edit ing the package ASCII confi gurat ion fi le.
The follow ing a re prerequisites t o using EMS :
• Monitored disks have been confi gured using LVM (Logical Volume
Manager).
• Netw or k ca r d s h a ve been con fi gu r ed .
• M ou n ted fi l esy stems h a ve been cr ea ted.
The follow ing a re syst em requirement s:
• All hardw are you intend to monitor, such as disks and LAN cards,
ha ve been confi gured an d t ested prior t o confi guring E MS.
• EMS must be insta l led on an H P 9000 Series 700 or Series 800
syst em run ning H P -U X version 10.20 or la ter.
EMS Resource Classes
Th e EM S fr a m ew or k gr ou ps r esou r ces in to cla s ses i n a h ier a r ch y s im il a r
to tha t of a fi lesystem str ucture, although the resources ar e not a ctua lly
fi les and directories. Some of th e resource class es supplied wit h t his
version of EMS a re listed in Figu re 1-2.
8/20/2019 Event Moniter Service
15/76
Chapter 1 15
Understanding the Event Monitoring Service
Understanding EMS
Figure 1-2 Event Monitoring Service Resource Class Hierarchy
The full pat h of a resource includes th e resource class hiera rchy a nd
inst a nce. An exa mple of a full resource pat h for th e physical volume
sta tus of the device /dev/dsk/c0t1d2 belonging to volume group
vgDataBase, is /vg/vgDataBase/pv_pvlink/status/c0t1d2.
Crea ting a monitoring request mea ns indentifying th e full pa thna me ofth e r esou r ces to be m on itor ed a n d specif yin g th e con d it ion s u n der w h ich
contains all package,
contains network contains all logical volume, disk and PVlink, and volume group
node, and clusterstatus
interface status
contains all jobqueue, user, andfilesystem status
/net
/cluster
/pv_summary
/lv_summary /lv
/pv_pvlink
/package
/localNode / status
/interfaces
/lan
/ status
/jobQueue15Min
/numUsers
/filesystem
/ availMB
/status
/ status
/vgName
/ package_status
/ status
/ copies
/jobQueue1Min
/jobQueue5Min
summary status
/system
/vg
/ LANname
/ clusterName
/ clusterName
/ package_name
/ deviceName / lvName
/ lvName
/ fsName
Bold Italics -Bold -
Instances, replaced with an actual nameResource instance
/ service_status
/ package_name
/ service_name
Resource Cla sses from
Loca tion in Resource Hiera rchy: /
8/20/2019 Event Moniter Service
16/76
16 Chapter1
Understanding the Event Monitoring Service
Understanding EMS
a n event notifi cat ion w ill be sent out by t he monitor.
Understanding Monitors
M on itor s a r e a p plica t i on s w r i t t en to ga th er a n d r epor t in for m a t ion a bou t
specifi c resources on t he syst em. They use sy stem informa tion st ored in
places like /etc/lvmtab and t he MIB da ta base. When you make a
request to a monitor t he follow ing sequ ence of events occurs:
• The monitor polls the resource to collect informa tion
• The monitor sends a message to the f ramework
• The f ramework eva lua tes the da ta to determine i f an event has
occurred
• I f an event ha s occurred , the f ramework sends not ifica t ion in the
appropriate forma t
To obta in ad ditional informa tion about inst a lled monitors:
1. G o t o t he /etc/opt/resmon/dictionary d irectory.
Ea ch monitor registered wit h EMS ha s an informa tion fi le tha t is
stored in t his directory.
2. View t he monitor dict ionary fi le.
The fi le na me is descript ive of each m onitor. The fi le extension is
.dict. For exam ple, the mibmonitor dictiona ry fi lenam e is
mibmond.dict.
A sta nda rd AP I provides a method for a dding new monitors t o EMS.
To crea t e your own monit or, refer t o the Wri t in g M oni tors for the Event
M oni tori ng Ser vice (EM S) Developer’s Kit :
1. Go to the high ava i l ab il ity w eb si te:
http://www.software.hp.com .
2. From the web site select High Availability, then select Event
M onit ori ng Ser vi ce Devel oper s K it .
8/20/2019 Event Moniter Service
17/76
Chapter 1 17
Understanding the Event Monitoring Service
Using EMS in a High Availability Environment
Using EMS in a High Availability
Environment
EMS can detect loss of redunda nt r esources and report informa tion
a bou t i t . I d en t ify in g a n d r epor t in g s in g le poi nt s of f a ilu r e h el ps m a i nt a i n
a proactive approach t o preventing the loss of da ta a nd a vailability.
B eca use EMS observes a system, a nd does not modify t he system, youuse it w ith a dditiona l softwa re tha t can t ake action. Some useful
combina tions a re described in th e follow ing sections.
EMS with MC/ServiceGuard
M C /Ser viceGu a r d ca n be con fi gu r ed w i th EM S to d eter m in e th e h ea l th of
some resources, such as disks, an d ma y fail packag es over based on tha t
informa tion. Confi gure EMS r equests for use with MC /ServiceGua rd
packages either thr ough the Cluster a rea for Pa ckage Confi gura tion in
SAM, or by editing the pa ckage confi gura tion ASC II fi le.
Create requests through E MS:
• To enable a redundant not ifica t ion sys tem
• To monitor events tha t a f fect h igh ava i lab i li ty
• To be a ler ted to the cause of a package fa i lover
MC/ServiceGua rd ma y a lread y be confi gured to monitor the healt h of
n od es, ser vi ces , a n d subnet s , a n d to m a ke f a ilover d ecis ion s ba sed on th e
sta tus of these resources.
EMS with IT/Operations or Network Node Manager
EMS can be confi gured t o send events t o OpenView I T/Operat ions a nd
Netw ork Node Man a ger. Refer t o th e EMS Developer’s Kit . It conta ins
templates w hich a re a vailable for downloading a nd include instructions
for confi guring in Netw ork Node Ma na ger a nd I T/Opera tions.
To help confi gur e your Net w ork Node Man a ger a nd I T/Opera t ions for
EMS, refer to the Wri t in g M oni tors for the Event M oni tor i ng Servi ce
(EMS) Developer’s Kit :
1. Go to the high ava i l ab il ity w eb s ite :
8/20/2019 Event Moniter Service
18/76
18 Chapter1
Understanding the Event Monitoring Service
Using EMS in a High Availability Environment
http://www.software.hp.com .
2. From the web site select High Availability, then select Event
M onit ori ng Ser vi ce Devel oper s K it .
EMS with Your Choice of System Management
Software
EMS can be used with a ny system mana gement softwa re that uses
messa ges sent in a ny of these protocols:
• S NMP t ra ps
• TCP
• U DP messa ges
• E ma il
• Text messages sent to a specia l ized file, including syslog
8/20/2019 Event Moniter Service
19/76
19
2 Installing and Using EMS
This chapter describes the following:
• Insta l ling EMS
• Removing EMS
8/20/2019 Event Moniter Service
20/76
20 Chapter2
Installing and Using EMS
• St a rting t he Event Monitoring Service
8/20/2019 Event Moniter Service
21/76
Chapter 2 21
Installing and Using EMS
Installing EMS
Installing EMS
EMS checks t he resources for loca l syst ems only. Ins ta ll EMS on every
syst em you wish t o monitor.
NOTE EMS is most effective when inst a lled an d confi gured on a ll systems in
your environment. U pdat ed monitors ma y have new sta tus va lues tha tchange the meaning of your monitoring requests.
The EMS bundle (Part Number B7609BA) version A.03.00 contains the
follow ing fi le sets:
• E M S -C or e—E M S fr a m ew or k
• EM S-C on fi g—SAM in ter fa ce to EM S
To insta ll EMS :
• U se swinstall, or
• U s e t h e S oft w a r e M a na g em en t a r ea i n S AM
If you have ma ny syst ems, it m ay be ea sier t o insta ll over the netw ork
from a cent ra l locat ion:
1. Create a network depot according to the instruct ions in Managing
H P-UX Sof tware wi t h SD-UX .
2. rlogin or telnet to the remote host on wh ich you are inst a lling
EMS.
3. Insta l l over the network from the depot .
When monitors a re upda ted, or wh en you re-insta ll a monitor on top ofa n existing monitor, your request s a re reta ined. This is pa rt of the
functionality provided by t he persistence client.
8/20/2019 Event Moniter Service
22/76
22 Chapter2
Installing and Using EMS
Removing EMS
Removing EMS
To remove EM S, use swremove or th e Softw a re Mana gement products
under SAM. Note tha t the monitors a re persistent , tha t is, they a re
always automatically started if they are stopped. Therefore, it is likely
you w ill have wa rnings in your remova l log fi le tha t sa y:
Could not shut down process
or errors tha t sa y:
File /etc/opt/resomn/lbin/p_client could not be removed.
E v en if y ou s ee t h es e w a r n in gs, m on it or s a r e r em ov ed a n d a n y d ir t y fi l es
a re clea ned up on reboot.
CAUTION Wh en y ou r em ove E M S, a l l log fi l es a r e a l so r em oved . I f y ou w is h t o s a vean y of the EMS log fi les, rename t hem or store them off before y ou
remove EMS.
8/20/2019 Event Moniter Service
23/76
Chapter 2 23
Installing and Using EMS
Starting the Event Monitoring Service
Starting the Event Monitoring Service
To star t EMS :
1. St ar t t he graphical version of SAM. From your comma nd line, type:
sam
2. Double-click t he Resource Mana gement icon.3. Double-click on th e Event Monitoring Service icon.
The ma in screen, shown in Fig ur e 2-1, shows a ll requests confi gured
on tha t system.
If th ere are no requests a nd S entinel monitors are not insta lled:
• A m es sa g e d is pla y s:
Currently no resources are being monitored. Use the
action.
• The fi eld a rea of the ma in screen i s empty.
If Sent inel monitors a re insta lled th e screen is simply blank.
Figure 2-1 Event Monitoring Service Main Screen
8/20/2019 Event Moniter Service
24/76
24 Chapter2
Installing and Using EMS
Starting the Event Monitoring Service
8/20/2019 Event Moniter Service
25/76
25
3 Selecting Resources to Monitor
This chapter describes the following:
• Selecting Resources
• Viewing Resource Descriptions
Selecting Resources to Monitor
8/20/2019 Event Moniter Service
26/76
26 Chapter3
Selecting Resources to Monitor
Selecting Resources
Selecting Resources
All resources a re divid ed int o cla sses. To select a resource to monit or:
1. Cl ick on t he Actions menu.
2. Select Add Monitoring Request
The top-level resource classes for all installed monitors are
dyna mically discovered a nd t hen listed a s shown in th e fi gure below.
S ome Hew lett -P a ckar d products, su ch as ATM Ada pter for H P /9000
S ervers, H P OTS 9000 or Su pport Tools Ma na ger (STM) for HP 9000
ha rdwa re monitoring, include their own monitors w ithin t heir
pr od uct h ier a r ch y. I f th ose pr od uct s a r e in s t a lled on th e sys tem , th en
th eir t op-level resource class es a lso appear here.
NOTE After insta l l a t ion some monitors , including the STM hardware moni tors ,must be enabled before th eir resource clas ses appea r in t he EMS Add or
Copy Monitoring R equest screen.
Similarly, top-level resource classes belonging to user-written
monitors, crea ted using the procedures descr ibed in Wr i t i ng M on i t ors
for the Event M on i tor ing Serv ice (EM S) , a r e d iscover ed a n d d ispla y edhere.
To obta in a dditiona l informa tion a bout an y pa rticular monitor:
• R ev iew th e m on itor d ict i on a r y fi l e:
a . From the command l ine, go to the
/etc/opt/resmon/dictionary d irectory.
When a monitor registers w ith E MS, informa tion a bout t he
monitor ca n be stored in fi les in th e .dict d irectory.
b . View t he monitor dict ionary fi le.
The fi le na me is descript ive of each m onitor. The fi le extension
is .dict. For exam ple, the mibmonitor dictiona ry fi lenam e is
mibmond.dict.
• R eview t he m a n pa ge.
Selecting Resources to Monitor
8/20/2019 Event Moniter Service
27/76
Chapter 3 27
Selecting Resources to Monitor
Selecting Resources
The ma n pa ge nam e is referred to in the dictiona ry fi le in t he
MONITOR entry wit h th e monitor’s na me.
• View t he resource class or instance descript ion through EMS.
The EMS Monitoring Request P a ra meters screen ha s a View
Resource Description button that displays additional
informa tion supplied by th e selected r esource. S ee Figu re 3-1
Figure 3-1 EMS Monitoring Request Parameters Screen
3. Double-click on a resource clas s.
When y ou monitor a resource, you actua lly monitor one or m ore
Selecting Resources to Monitor
8/20/2019 Event Moniter Service
28/76
28 Chapter3
Selecting Resources to Monitor
Selecting Resources
specifi c insta nces of th a t resource clas s. The resource inst a nces,
a ssociat ed wit h t he selected resource clas s, display in t he ResourceInst a nce fi eld. See Figure 3-2.
If t he resource clas s ha s subcla sses, those subclas ses displa y in th e
Resource Classes fi eld.
Asynchronous monitors a re event-driven, ra th er t ha n polled. They
genera te messages in a continuous stream a s events occur a nd
w ithout rega rd for rela tive importa nce. Therefore if th e resource
in s t a n ce is f or a n a sy n ch r on ou s m on itor, th e C u rr en t Va l ue fi e ld d oesnot a pply, so the fi eld display s n/a.
Figure 3-2 Add or Copy Monitoring Request Screen
Selecting Resources to Monitor
8/20/2019 Event Moniter Service
29/76
Chapter 3 29
g
Selecting Resources
4. Select a specific insta nce or th e wildcar d (All Insta nces).
The (*) w ild ca r d is a con ven ien t w a y t o cr ea t e m a n y r eq u es t s a t on ce.
Most syst ems have more than one disk or netw ork card, a nd ma ny
ha ve severa l disks. To avoid ha ving t o crea te a monitor request for
each disk, select *(All Instances) in t he Resource Inst a nce box.
The *(All Instances) l isting is alw ays t he last i tem on the list .
See the fi gure above.
Wildca rds a re ava ilable only w hen a ll inst a nces of a su bclas s ar e of
the same resource type and there are multiple instances. Selectingthe w ildcard a pplies the m onitor to a ll the insta nces of tha t resource
type.
Wildca rds a re not ava ilable for r esource clas ses. So, for exa mple, a
wildcar d is available for the sta tus insta nces in the
/sys tem /fi lesyst em/a va ilMb subclas s, but no wildca rd a ppear s for t he
volume group subcla ss un der t he /vg res ource cla ss.
5. C lick OK .
The Monitoring Request Parameters screen displays. See Chapter 4,
“Defi ning a Monitoring Request.”
Selecting Resources to Monitor
8/20/2019 Event Moniter Service
30/76
30 Chapter3
g
Viewing Resource Descriptions
Viewing Resource Descriptions
Resource class a nd resource inst a nce descriptions a re a va ilable for ea ch
resource.
To view a resource cla ss descript ion, click th e View R esource Cla ss
Description but ton from t he Add or Copy Monitoring R equest screen.
To view a resource inst a nce description, click eith er t he View ResourceInst a nce Description butt on from t he Add or Copy Monitoring Req uest
screen or t he View Resource Description but ton from t he Monitoring
Request Pa ra meters screen.
8/20/2019 Event Moniter Service
31/76
31
4 Defining a Monitoring Request
This chapter describes the following:
• St ar ting a Monitoring Request
• Specifying When t o Send a n E vent
Defining a Monitoring Request
8/20/2019 Event Moniter Service
32/76
32 Chapter4
• Determining t he Frequency of Events
• Sett ing the Polling Int erval
• Selecting P rotocols for S ending E vents
• Adding a Notifi cat ion C omment
Defining a Monitoring Request
S i M i i R
8/20/2019 Event Moniter Service
33/76
Chapter 4 33
Starting a Monitoring Request
Starting a Monitoring Request
After y ou ha ve selected a resource to monitor, you ca n specify w hen a nd
how to send event notifi cat ion on the Monitoring Request P ar a meters
screen, (Fig ur e 4-1). The follow ing s ect ions describe th e monitorin g
par a meters a nd provide exam ples of common a pplica tions.
Figure 4-1 Monitoring Request Parameters Screen
Defining a Monitoring Request
Specifying When to Send an Event
8/20/2019 Event Moniter Service
34/76
34 Chapter4
Specifying When to Send an Event
Specifying When to Send an Event
When you creat e a request , you specify th e conditions un der w hich you
receive an a lert. While the monitor ma y be polling disks every fi ve
minutes, for example, you may only wa nt to be alerted wh en something
ha ppens tha t requires your a tt ention. S pecify these conditions in the
Notify a r ea of t h e M on it or in g R eq u es t P a r a m et er s s cr een . H er e a r e t h e
terms under w hich you ca n be notified:
When value is ... You define the conditions under which you wish to be
notified for a particular resource. Choose an operator
(=, not equal, >, >=,
8/20/2019 Event Moniter Service
35/76
Chapter 4 35
Specifying When to Send an Event
To set a n event tr igger:
• Select f rom the l is ted opt ions in the Notify a rea (When value
is..., When value changes, or At each interval).
Asynchronous m onitors a re event-driven, ra th er t ha n polled. They
generate messa ges in a continuous stream as events occur a nd
wi thout regard for rela t ive impor tance. Therefore, i f the request i s for
a n a synchronous monitor, only t he When value is... option is
available.
NOTE U pd a ted m on it or s m a y h a ve n ew s ta t us va lu es t ha t ch a ng e t he m ea n in gof your monitoring requests, or generat e new a lerts. For exam ple,
a ssume you ha ve a r equest for notifi ca tion if sta tus > 3 for a resource
w it h a v a lu es r a n ge of 1 t h rou gh 7. You r eceiv e a l er t s ea ch t im e t h e v a lu e
eq ua ls 4, 5, 6, or 7. I f t he u pd a ted ver sion of t h e m on it or h a s a n ew s ta t us
va lue of 1 th rough 8, you a lso see alerts w hen t he resource equa ls 8.
Defining a Monitoring Request
Determining the Frequency of Events
8/20/2019 Event Moniter Service
36/76
36 Chapter4
Determining the Frequency of Events
Determining the Frequency of Events
If you select th e When value is... from t he list in t he Notify a rea ,
th e O pt i on s a r ea d ispl a ys th r ee ch oices . Sel ect on e or m or e of th ese th r ee
options:
Init ia l U se t his opt ion to est a blish a ba seline w hen monitoring
resources such as available filesystem space or system
loa d. I t can also be used to test w hether newlyrequested events a re being sent.
Repea t U se t his opt ion for urgent a ler t s. The Repea t opt ion
sends an a lert a t ea ch polling interval a s long a s the
notify condition is met. U se this option w ith cau tion;
there is a r isk of high CP U use or fi ll ing log fi les a nd
alert windows.
Ret urn U se t his opt ion t o t ra ck w hen emergency sit ua t ionsreturn t o normal.
To set t he frequ ency of th e tr igger:
• C l ick one or more but tons on the li st in the Options a rea (Initial,
Repeat, and Return).
Asynchronous monitors a re event-driven, ra th er t ha n polled. They
genera te messages in a continuous stream a s events occur a ndwi thout regard for rela t ive impor tance.Therefore i f the resource is an
a synchronous monitor, the va lues in the Options a rea defaul ts to
Repeat an d cannot be cha nged.
This Options a r ea d oes n ot d ispla y if y ou h a ve selected When value
changes or At each interval from t he list in t he Notify a rea . In
these cases t he options default t o Initial a nd cannot be changed.
Defining a Monitoring Request
Setting the Polling Interval
8/20/2019 Event Moniter Service
37/76
Chapter 4 37
Setting the Polling Interval
Setting the Polling Interval
The polling int erva l specifi es how often th e resource monitor checks t he
resource value. The polling interva l is the ma ximum a mount of ela psed
time before a monitor knows about a cha nge in sta tus for a pa rticula r
resource.
The shorter th e polling interva l, the m ore likely you a re t o have recent
da ta . However, depending on the monitor, a short polling interva l mayuse more CP U a nd syst em resources. You need to weigh the a dvant a ges
a nd disadva nta ges betw een being able to quickly respond t o events a nd
ma inta ining good syst em performa nce. Some considera tions include:
• The minimum polling interval depends on the monitor ’s a bil ity t o
process quickly. For most resource monitors the minimum is 30
seconds. Disk monitor requests ca n be a s short a s 10 seconds.
• MC/ServiceGua rd monitors resources every few seconds. You may
wa nt to use a short polling int erva l (30 seconds or less) w hen it is
critica l tha t you ma ke a q uick failover decision.
• You m ay w a nt a pollin g in ter va l of 5 m in ut es or so for m on it or in g less
critical resources.
• You may w ant to set a very long pol ling interval (4 hours) to monitor
failed disks tha t a re not essential t o the system, but w hich should be
replaced in the next few da ys.
Asynchronous monitors a re event-driven, ra th er th a n polled, th ey
generate messages in a continuous stream a s events occur a nd w ithout
rega rd for r elat ive import a nce.Therefore if t he resource is a n
asynchronous monitor, the Polling Interval field displays n/a.
To set th e Polling I nt erva l:
1. Speci fy the quant i ty of t ime in the numbered fi eld .
2. Select th e unit of time from the unit of measur e field list (seconds,
minutes, hours, day). The m a ximum va lue is one (1) da y.
Defining a Monitoring Request
Selecting Protocols for Sending Events
8/20/2019 Event Moniter Service
38/76
38 Chapter4
g g
Selecting Protocols for Sending Events
Through the Notify via a r ea specif y th e pr otocol y ou w a n t th e m on itor
to u se to sen d even t s. Th e opt i on s a r e d escr ibed in th e f ol low i ng sect ion s.
opcmsg (ITO) Option
This option sends messages t o ITO a pplicat ions via t he opcmsg da emon.I T O per a t ion M a na g ed N od e S oft w a r e 3.x or 4. x m us t be in st a l led on t h e
resource server running HP-UX version 10.20, for this option to display.
This opt ion is not current ly ava i lab le on systems running HP -UX vers ion
11.0.
The I TO messa ge severity options a re:
• Map from value (th is is not a va ilable on a ll monitors)
• Critical
• Major
• Minor
• Warning
• Normal
A specifi ed severity oth er tha n Norma l is retur ned under t he follow ingconditions:
• The When value is . . . condit ion chan ges from FALS E t o
TRUE.
For example if a d isk is being monitored, you wa nt n otifi cat ion w hen
th e disk is down. The When value is condition is FALS E w hile the
disk is up a nd runn ing correctly. The condit ion is TRU E, mea ning
a ction needs to be ta ken, when t he disk is dow n or not opera ting
correctly.
• The When value changes wh en the return va lue does not ma tch
the previous retur n va lue
Cert a in opcmsg monitoring request s ca n ma p directly t o severity level.
Select Map from value option from t he list in t he Severity a rea . This
is not a vailable with a ll monitors.
Defining a Monitoring Request
Selecting Protocols for Sending Events
8/20/2019 Event Moniter Service
39/76
Chapter 4 39
Within t he opcmsg, EMS sets t he follow ing fi elds,:
• I TO a pplica t i on g r ou p to EM S(HP )
• m essa ge gr oup t o H A
• object t o the full pa t h of the resource being monitored
S ee HP OpenView I T/ Operat i ons Adm in istrators Task Guid e (Pa rt
Num ber B 4249-90003) for m ore inform a tion.
Templat es for confi guring IT/Operat ions a nd N etw ork Node Ma na ger t odisplay monitored events can be found on the Hewlett-Packard High
Ava i lab i li ty public web page a t http://www.software.hp.com . C lick on
High Availability, then Event M onit ori ng Ser vi ce Devel oper ’s K i t .
To set t he opcmsg prot ocol for ITO:
1. Speci fy t he notifi ca t ion t ype from t he l is t in t he Noti fy area .
2. Select the opcmsg (ITO) option from the list in t he Notify a rea.3. Select the severity f rom the l is t in the Severity a rea , (Map from
value, Critical, Major, Minor, Warning, Normal).
TCP and UDP Options
Th is s en ds TC P or U D P en cod ed even ts t o t h e t a r get h os t n a me a n d por t
indicat ed for th a t requ est. Thus t he messa ge ca n be directed t o a
user-w ritt en socket progra m.
To set th e TCP or UD P conditions:
1. S elect th e TCP or UDP option, as a ppropriat e, from the list in t he
Notify via a rea .
2. Speci fy t he tar get host na me and the port in t heir respective fields.
SNMP Traps Option
Th is sen d s m essa ges to a p pl ica t i on s u si ng SNMP tr a p s, such a s Netw or k
Node Ma na ger. See HP OpenVi ew U sin g Netw ork N ode M anager (P /N
J 1169-90002) for more informa tion on confi guring SN MP tr a ps.
The follow ing tr a ps ar e used by EMS :
EMS_ENTERPRISE_OID "1.3.6.1.4.1.11.2.3.1.7"
EMS_NORMAL_OID "1.3.6.1.4.1.11.2.3.1.7.0.1"
Defining a Monitoring Request
Selecting Protocols for Sending Events
8/20/2019 Event Moniter Service
40/76
40 Chapter4
- Normal Event
EMS_ABNORMAL_OID "1.3.6.1.4.1.11.2.3.1.7.0.2"- Problem Event
EMS_REBOOT_OID "1.3.6.1.4.1.11.2.3.1.7.0.3"
- Reboot restart Event
EMS_RESTART_OID "1.3.6.1.4.1.11.2.3.1.7.0.4"
- Restart Event
EMS_NORMAL_SEV_OID "1.3.6.1.4.1.11.2.3.1.7.0.5"
- Problem Event w/Normal Severity
EMS_WARNING_SEV_OID "1.3.6.1.4.1.11.2.3.1.7.0.6"
- Problem Event w/Warning Severity
EMS_MINOR_SEV_OID "1.3.6.1.4.1.11.2.3.1.7.0.7"
- Problem Event w/Minor Severity
EMS_MAJOR_SEV_OID "1.3.6.1.4.1.11.2.3.1.7.0.8"
- Problem Event w/Major Severity
EMS_CRITICAL_SEV_OID "1.3.6.1.4.1.11.2.3.1.7.0.9"
- Problem Event w/Critical Severity
The Severity area options for SNMP tra ps are:
• Map from value, (this is not a vailable with all monitors)
• Critical
• Major• Minor
• Warning
• Normal
A specifi ed severity oth er tha n Norma l is retur ned under t he follow ing
conditions:
• The When value is . . . condit ion changes f rom FALSE to TRUE
For example if a d isk is being monitored, you wa nt n otifi cat ion w hen
th e disk is down. The When value is condition is FALS E w hile the
disk is up a nd runn ing correctly. The condit ion is TRU E, mea ning
a ction needs to be ta ken, when t he disk is dow n or not opera ting
correctly.
• The When value changes wh en the return va lue does not ma tch
the previous retur n va lue
Defining a Monitoring Request
Selecting Protocols for Sending Events
8/20/2019 Event Moniter Service
41/76
Chapter 4 41
Certa in SNMP monitoring requests can ma p directly t o severity levels.
Select the Map from value option from the list in t he Severity a rea .To set t he SNMP tra p:
1. Speci fy the notifi ca t ion type from the l is t in the Notify a rea .
2. Select the SNMP tra p option from the l is t in the Notify via a rea .
3. Select the severity f rom the l is t in the Severity a rea , (Map from
value, Critical, Major, Minor, Warning, Normal).
Email Option
This sends event notificat ion t o the email a ddress indicat ed for t ha t
request.
To set for a n ema il notifi cat ion:
1. S elect th e Email option from the list in t he Notify via a rea .
2. Speci fy t he ful l email address in the Ema il Address fi eld .
Console Option
This sends event notifi cat ion t o the syst em console.
To set for a console notifi cat ion:
• S elect the Console option from the list in t he Notify via a rea .
Syslog Option
This sends event notifi cat ion t o th e system log.
An a bnorma l event message (error) is retu rned un der t he follow ing
conditions:
• The When value is . . . condit ion changes f rom FALSE to TRUE
• The When value changes wh en the return va lue does not ma tch
the previous return value
For a n a bn or m a l even t , a sys tem loggin g level of errorwi ll be associa ted
wit h t he logged message.
To set for a syst em log notifi cat ion:
Defining a Monitoring Request
Selecting Protocols for Sending Events
8/20/2019 Event Moniter Service
42/76
42 Chapter4
• S elect t he Syslog option from the list in t he Notify via a rea .
Textlog Option
This sends event notificat ion t o the fi le indicat ed for t ha t r equest.
To set for a text log notifi cat ion:
1. S elect t h e Textlog option from t he list in t he Notify via a rea .
2. Speci fy the fi lename and path in the Fi le Pa th fi eld .
The default path, /var/opt/resmon/log/event.log, displays w hen
th e Textlog option is select ed.
Defining a Monitoring Request
Adding a Notification Comment
8/20/2019 Event Moniter Service
43/76
Chapter 4 43
Adding a Notification Comment
The notifi cat ion comment is useful for sending ta sk reminders t o th e
recipient s of an event. For example, if you ha ve a disk m onitor request
tha t reports a n a ler t t ha t an entire mirror ha s fa i led, when tha t event
shows u p in IT/Operat ions, for exam ple, you may wa nt it t o have th e
na me of th e person to conta ct if disks fail.
Defining a Monitoring Request
Adding a Notification Comment
8/20/2019 Event Moniter Service
44/76
44 Chapter4
8/20/2019 Event Moniter Service
45/76
45
5 EMS Operations
This chapter describes the following:
• Copying Monitoring Request s
• Modifying Monitoring Request s
8/20/2019 Event Moniter Service
46/76
46 Chapter5
• Removing Monitoring Request s
• Viewing Monitoring Request s
8/20/2019 Event Moniter Service
47/76
Chapter 5 47
Copying Monitoring Requests
There a re tw o way s to use th e copy function:
• To cr ea te r eq u es t s f or mul t i p l e resources using t he same monitoring
parameters .
• To cr ea te req u es t s f or th e same resource using different monitoring
parameters .To crea te req uests for mu ltiple resources using t he sa me monitoring
parameters :
1. From the Event Monitoring Service main screen, select t he
monitoring request w hose par a meters y ou wish t o copy.
You need to ha ve confi gured a t leas t one similar request for a simila r
instance.
2. Select Actions menu: Copy option.
The Add or Copy Monitoring Request screen displays.
3. From the Add or Copy Monitoring Request screen, select a different
resource insta nce and click OK.
The Monitoring Request Parameters screen displays.
4. C lick OK in t he Monitoring Request P ar a meters screen.
A message displays indicat ing th e new request ha s been added. The
E vent Monitoring Service ma in screen displa ys.
To crea te req uests for t he sa me resource using different monitoring
parameters :
1. From the Event Monitoring Service main screen, select t he
monitoring request w ith t he insta nce for w hich you wish to havemult iple monitoring request s.
You need to ha ve confi gured a t lea st one request for th e insta nce.
2. Select Actions menu: Copy option.
The Add or Copy Monitoring Request screen displays.
3. C lick OK in t he Add or Copy Monitoring R equest screen.
The Monitoring Request Parameters screen displays.
8/20/2019 Event Moniter Service
48/76
48 Chapter5
4. Modify the para meters a s desired in the Monitoring Request
Parameters screen.
5. C lick OK.
A message displays indicating the new request has been added. The
Event Monitoring S ervice main screen displa ys.
8/20/2019 Event Moniter Service
49/76
Chapter 5 49
Modifying Monitoring Requests
To chang e the monitoring par a meters of a requ est:
1. From the Event Monitoring Service main screen, select t he
monitoring request w hose par a meters you wish t o modify.
2. Select Actions menu: Modify option.
The Monitoring Request Parameters screen displays.
3. Modify the para meters as desired in the Monitoring Request
Parameters screen.
4. C lick OK.
A messa ge display s indica ting t he request ha s been modifi ed. The
E vent Monitoring Service ma in screen displa ys.
8/20/2019 Event Moniter Service
50/76
50 Chapter5
Removing Monitoring Requests
The Remove Monitoring Requests functions w ith m ultiple requests a s
w ell as single request s. To remove monitoring requ ests:
1. From the Event Monitoring Service main screen, select t he
monitoring request you wish t o remove.
To select contig uous mult iple req uest s, hold the Shift key a nd click.
To select individua l mult iple requests, h old t he Ctrl key a nd click.
2. Select Actions menu: Remove option.
A Confirmation screen displays.
3. C lick OK.
A messa ge displa ys indicat ing how ma ny req uest(s) ha ve been
removed. The E vent Monitoring S ervice ma in screen displays.
4. To s t a rt m on it or in g t h e r es ou rce a g a in y ou m us t r ecr ea t e t h e r eq u es t ,
either by copying a similar r equest for a sim ilar r esource or by
re-entering the da ta .
8/20/2019 Event Moniter Service
51/76
Chapter 5 51
Viewing Monitoring Requests
To view t he para meters for a m onitoring req uest:
1. From the Event Monitoring Service main screen, select t he
monitoring request you wish t o view an d either:
• D ou ble-C lick , or
• S elect Act ion s m en u: View
The View Monitoring Request Pa ra meters s creen display s. The
para meters l isted here mat ch th e pa ra meters specified for th e
monitoring request.
2. To modify the para meters of this request, click the Modify
Monitoring Request option.The Monitoring Request P a ra meters
screen display s. P roceed as described in “Modifying Monitoring
Requests.”
3. To close the View Monitoring Req uest screen, click OK.
8/20/2019 Event Moniter Service
52/76
52 Chapter5
8/20/2019 Event Moniter Service
53/76
53
6 Configuring MC/ServiceGuardPackage Dependencies
This cha pter describes how t o use SAM to crea te pa cka ge resource
dependencies on E MS r esources. Tha t is how t o crea te a monitoring
t t b th E MS d t tif MC/S i G d
8/20/2019 Event Moniter Service
54/76
54 Chapter6
request to observe the E MS r esource and to notify MC/ServiceGua rd
w hen th a t resource rea ches a critica l user-defi ned level.
MC/ServiceGuard wi ll then fa i lover the package. The fol lowing are some
examples of how E MS migh t be used:
• I n a clust er w her e on e copy of da t a is sh a red bet w een t w o n odes (bot h
con fi gu red w it h E MS ), y ou m a y w a n t t o fa il a pa cka g e over w h en , for
exa m ple, t he L AN or S C S I h os t a d a pt er fa ils on t h e n od e r un nin g t h e
packag e. MC/ServiceGua rd compa res t he resource “U P ” va lues on
other confi gured nodes, a nd fa ils the package over to the node th a t
ha s th e correct resources ava ilable.
• In a c lus ter where each node has i t s own copy of da ta , you may wa nt
to failover a package t o a nother node for a ny nu mber of rea sons:
• host ada pter, bus, controller, or disk fa i lure
• unprotected data (the number of copies is reduced to one)
• degraded performance because one of the P V l inks has fa i led
This informat ion for crea ting r equests is a lso va lid for E MS sold wit h
other products (ATM, OTS, or STM hardware monitors , for example) and
for user-wri t ten monitors wri t ten according to developer specificat ions in
Wri t in g M oni tors for the Event M oni tor i ng Servi ce (EMS) .
NOTE Creat e the sa me requests on all nodes confi gured for a n
MC/ServiceGua rd packag e.
A packag e can depend on a ny resource monitored by E MS. To crea te
package dependencies:
1. H a l t t h e clu st er. I n clu de a for ce opt ion t o s t op a l l pa ck a ges, b y t y pin g:
cmhaltcl -f
To a d d a n EM S r esou r ce, th e M C /Ser vi ceGu a r d clu ster m u st be d ow n .
You ca n m odify existing EMS resources th rough MC /ServiceGua rd
while the cluster is running.
2. From your command l ine, s tar t SAM, by typing:
sam
3. Double-click the C lusters icon.
4 Double click th e High Availability Clusters icon
8/20/2019 Event Moniter Service
55/76
Chapter 6 55
4. Double-click th e High Availability Clusters icon.
5. Double-click on the P ackage Configur at ion icon.
The High Ava ilability Clust ers screen shows a ll requests current ly
confi gured on t ha t syst em. If there are no requests currently
confi gured, the fi eld a rea is empty. Figu re 6-1 shows the High
Ava ilability C lusters screen wit h t he Actions menu d isplayed.
Figure 6-1 High Availability Clusters Screen
8/20/2019 Event Moniter Service
56/76
56 Chapter6
6. F rom t h e Act ion s m en u, s elect eit h er t h e Create/Add a Package or
Modify Package Configuration option.
I f you select Create/Add a Package, a s cr een s im ila r t o Figure 6-2,
displays.
Figure 6-2 Create/Add a Package Screen
I f h l d f
8/20/2019 Event Moniter Service
57/76
Chapter 6 57
7. I f y ou h a ve n ot pr ev iou sly d on e s o, s pecify a Package Name and Node
a nd specify a Package SUBNET Address. Then click on SpecifyPackage Resource Dependencies... to add EMS resources a s
packag e dependencies. A screen similar to Figu re 6-3 displays.
Figure 6-3 Package Resource Dependencies Screen
8 T k k g d d t EMS li k Add
8/20/2019 Event Moniter Service
58/76
58 Chapter6
8. To make a pa ckage dependent on an EMS resource, click Add
Resource.... The Resources: fi eld lists a ll the insta lled resourcesdiscovered by MC /S erviceG ua rd. S ee Figu re 6-4.
Figure 6-4 Add Resources Screen
9. C lick th r ou gh th e Resource Classes a nd Resource Names to select
th e entity you wish t o monitor. Click OK. A Resource Parameters
screen, simila r t o Figu re 6-5 displays.
The example in Figure 6-5 shows the poss ib le va lues for pv_summary.
Different resources show different available UP values.
Figure 6-5 Resource Parameters Screen
8/20/2019 Event Moniter Service
59/76
Chapter 6 59
NOTE Ma ke sure you a lwa ys select UP in the Resources Up Values fi eld .MC/ServiceGua rd creat es an E MS request t ha t sends a n event if the
Resources Up Value fi eld is not equa l to the UP value.
I f you select UP, t h e pa cka g e fa ils over if t h e va lu e is a n yt h in g bu t UP.
I f you select UP a nd PVG-UP, t he pa cka g e fa ils over if t h e pv_summary
value is not equa l to UP or PVG_UP; in other w ords, if pv_summary is
SUSPECT or DOWN.
The polling interva l determines th e ma ximum a mount of elapsed
t im e b efor e t h e m on it or kn ow s a b ou t a ch a n ge in r es ou rce s t a t us. For
critica l resources, you may w a nt t o set a short polling interva l, such
a s 30 seconds, but t his could a dversely a ffect sy stem performa nce.
With longer polling interva ls you gain sy stem performa nce, but y ou
risk n ot detecting problems soon enough.
8/20/2019 Event Moniter Service
60/76
60 Chapter6
You ca n a l so a d d r esou r ces a s pa cka ge d epen d en cies by m od if yin g th epacka ge confi gura tion fi le. The default fi lena me is
/etc/cmcluster/pkg_name.ascii. See Manag ing
M C/ Serv iceGuard for d et a ils on h ow t o m od ify t h is fi l e. A exa m ple of
the synta x is:
RESOURCE_NAME /vg/vg01/pv_summary
RESOURCE_POLLING_INTERVAL 60
RESOURCE_UP_VALUE = UP
RESOURCE_UP_VALUE = PVG_UP
MC/ServiceGuard automat ica l ly d is t r ibutes th is in format ion to every
node in the cluster w hen the pa ckage confi gura tion fi le is a pplied t o
th e cluster confi gura tion (see the cmapplyconf comma nd).
8/20/2019 Event Moniter Service
61/76
61
7 Troubleshooting
This section gives hint s on testin g your monitoring request s, an d gives
y ou s om e in for ma t i on a b ou t log fi l es a n d m on it or beh a vior t h a t w ill h elp
you determine the cause of problems. For informat ion on fi xing problems
detected by monitors, see th e list of relat ed publicat ions in t he P reface.
Troubleshooting
This cha pter h a s t he follow ing sections:
8/20/2019 Event Moniter Service
62/76
62 Chapter7
• EMS Directories a nd F iles
• Logging a nd Tra cing
• Performance Considerations
• Testing Monitor Request s
Troubleshooting
EMS Directories and Files
8/20/2019 Event Moniter Service
63/76
Chapter 7 63
EMS Directories and FilesEMS fi les a re locat ed in /etc/opt/resmon a nd /opt/resmon. Ta ble 7-1
lists fi les a nd directories th a t might h elp you determine the cause of
some problems:
Table 7-1 EMS Directories and Files
/etc/opt/resmon/config
This file sets how often EMS checks that monitors are persistent
(have not died).
/etc/opt/resmon/dictionary
This directory contains resource dictionaries for the various
monitors. The disk monitor resources are listed in
diskmond.dict and the cluster, network, and system resource
monitors are in the mibmond.dict. If you were writing your own
monitor, the dictionary would go in this directory.
/etc/opt/resmon/lbin
This is the directory where all the monitor daemons live. Some
important daemons in the directory:
p_client restarts any failed monitors based on information in the
config file.
registrar passes monitoring requests to the correct monitors.
Troubleshooting
EMS Directories and Files
Table 7-1 EMS Directories and Files
8/20/2019 Event Moniter Service
64/76
64 Chapter7
/etc/opt/resmon/log
This is a directory of log files used by EMS:
• client.log stores calls made by clients, such as
MC/ServiceGuard or the SAM interface to EMS.
• api.log stores api calls made by monitors.
• registrar.log contains errors found when reading the
resource dictionary.
• emsagent.log is the SNMP subagent responsible for sending
EMS events through an SNMP trap.
/opt/resmon/resls
This command lists the latest polled status of the specified resource
on a specified system.
This can be used to discover and get details on resources available
for monitoring.
/opt/resmon/resdata
This is used to get additional information from EMS regarding
events or monitor restarts.
Troubleshooting
Logging and Tracing
8/20/2019 Event Moniter Service
65/76
Chapter 7 65
Logging and TracingU se logging for most t roubleshooting a ctivities. B y default th e monitors
log t o api.log. Logging t o /var/adm/syslog/syslog.log is ON by
default for the disk monitor an d OFF by default for the remaining
monitors. Tra cing sh ould only be used w hen inst ructed t o do so by HP
support personnel. This is not a va ilable wit h a ll monitors.
EMS Logging
As m en t ion ed in th e pr ev iou s sect i on , l og fi l es i n /etc/opt/resmon/log
/cont a in informa tion logged by t he monitors.
Look at the client.log if you seem to be ha ving a problem wit h t he
SAM, or a ny oth er client, int erfaces to E MS or MC /ServiceGua rd. With
th e d ef a u lt l evel of l oggin g, on ly a u d it a n d er r or m essa ges a r e logged . An
example of a n a udit message is:
User event occurred at Thu Jul 31 16:13:31 1997
Process ID: 10404 (client)
Log Level:Audit
+ /vg/vg00/lv/copies/* (8 instances)
If (
8/20/2019 Event Moniter Service
66/76
66 Chapter7
Ent ries in /var/adm/syslog/syslog.log ar e marked with t he monitor
daemon name, for example diskmond or fsmond, fol lowed by the resource
na me a nd logging da ta . Additions, deletions, notificat ions, a nd chan ges
in resource sta tes a re logged. Er rors explaining w hy a resource is not
a va ila b le for m on itor in g, or w h y th e m on itor ca n n ot a ccess a r esou r ce a r e
a lso logged there.
Look at the registrar.log if you are ha ving tr ouble fin ding resourcesth a t you suspect exist on your syst em. This log cont a ins a ny errors th a t
w er e en cou nt er ed w h en t r yin g t o r ea d t h e d ict ion a r y. I f a d ict ion a r y w a s
cor ru pt ed in a n y w a y, t he r eg is tr a r w ou ld n ot be a ble t o r ea d it , a n d E MS
would not be a ble to fi nd t he resources a ssociat ed with tha t dictionary.
EMS Tracing
Som e m on itor s pr ov id e t r a cin g w h i ch ca n be u sed f or d ebuggin g m on itorcode.
Use the -d option to tur n on tr a cing for EMS . Tra cing should only be
used at the request of your H P support personnel when t rying to
determine if there ma y be a problem wit h EMS . To tu rn on tra cing,
modify the .dict fi le in /etc/opt/resmon/dictionary and add -d t o
th e monitor you w ould like to tra ce:
MONITOR: /etc/opt/resmon/lbin/diskmond -l -d
Kill the monitor process. The monitor will a utoma tically rest a rt w ith
tr a cing ena bled. To speed up monitor rest a rt , use the resls comma nd
w ith t he top level of th e resource class a s a n a rgum ent, for exa mple,
resls /vg.
Tra cing is customar ily logged t o
/etc/opt/resmon/log/monitor_name.log. The monitor_name usually
ma tches the na me used for th e monitor in the dictiona ry fi le. Forexample, the MIBmonitor uses mibmond.dict a nd mibmond.log.
Troubleshooting
Performance Considerations
8/20/2019 Event Moniter Service
67/76
Chapter 7 67
Performance ConsiderationsMonitoring your system, is importa nt to ma inta in high a vailabili ty, but
monitoring consumes system resources. You must carefully consider your
per for ma n ce n eed s a g a in st y ou r n eed t o k now a s s oon a s pos sib le w h en a
failure threat ens ava ilabili ty.
System Performance Issues
The prima ry performa nce impact w ill be relat ed to the polling interva l
a nd t he num ber of resources being monitored. You need to ba lan ce your
need to quickly detect failures w ith y our need for syst em performa nce
a n d a d ju st th e n u mber of r esou r ces y ou m on itor a n d th e pol lin g in ter va l s
accordingly.
For example, pv_pvlink/status resources include th e pv_summary
resource. You may wa nt to creat e only one pv_summary monitoringrequest ra ther th a n monitoring both pv_summary a nd
pv_pvlink/status for all disks.
For polling interva ls, you ma y w an t to set a short interval, such as 30
secon d s, for r esou r ces th a t r eq u ir e q u ick r espon se a f t er f a ilu r e, a n d set a
lon ger polli ng in ter va l , such a s 5 m in u tes or m or e, f or a l l oth er r esou r ces.
Network Performance IssuesAlthough m onitoring is not likely t o affect n etw ork performa nce, you
ma y wa nt t o ma ke sure tha t only necessary messages a re being sent.
Ma ke sure your monitor request s ar e confi gured so you a re notifi ed only
for importa nt events.
Troubleshooting
Testing Monitor Requests
8/20/2019 Event Moniter Service
68/76
68 Chapter7
Testing Monitor RequestsTo tes t th a t even t s a r e bei ng sen t , u se th e I NI TI AL opt i on a va ila b le w i th
conditiona l notifi cat ion w hen creat ing a monitoring request . This option
sends notifi cat ion on sta rtup. Exa mine it t o make sure your request is
properly confi gured a nd showing up in the correct system m an a gement
tool.
An a lternat ive is to use the “At ea ch interva l” notifi ca tion t o test tha t
events a re being sent in t he correct sys tem ma na gement t ool. Once you
es t a b lish th a t even t s a r e bein g sen t pr oper ly, y ou ca n m od if y th e r eq u es t .
Testing Disk Monitor Requests
C on fi g ur in g t h e I N ITI AL opt ion m a y b e en ou gh . H ow ever, if y ou w a n t t o
test tha t events a re sent w hen a disk fa i ls , you may wa nt t o detach the
bus or power down t he stora ge devices a nd see if events a re sent t o th eproper a pplica tion, or if MC/ServiceGua rd fa ils over t he a ppropriat e
package. This is only recommended on clusters th a t a re off-line, and not
being a ccessed by users.
To test /vg/vgName/lv/copies a nd /vg/vgName/lv/status, use the
vgchange com m a nd t o d ea ct iv a t e a n d a ct iv a t e t h e v olu me g rou p a n d s ee
if the proper alerts w ere sent.
Testing Cluster Monitor Requests
Use the cmviewcl -v command to display detailed information about
th e current sta tus of the clust er a nd pa cka ges on t he cluster. The EMS
cluster monitor should return the sa me values a s th is comma nd.
Testing Network Monitor Requests
I f y ou w a n t t o t es t w h et h er ev en t s a r e s en t in ca s e of n et w or k fa ilu re, u se
the /usr/bin/ifconfig LANname down com ma n d t o br in g a ca r d d ow n ,
an d examine the event t o make sure it shows up in t he correct system
ma na gement t ool.
Testing System Resource Monitor Requests
Use the uptime comma nd t o verify the number of users a nd t he system
Troubleshooting
Testing Monitor Requests
loa d. The EMS syst em resource monitor should return th e sam e values
tha t t his comma nd does.
8/20/2019 Event Moniter Service
69/76
Chapter 7 69
Making Sure Monitors are Running
Monitor daemons aut oma tically sta rt w hen you creat e a m onitoring
request. B ecause monitoring is designed to w ork in a high a vailabili ty
environment, monitors ar e writt en to aut oma tically resta rt i f anyt hing
ca uses them t o fail .
A daemon called p_client resta rts a ll appropriat e monitors using themonitor restart interval defin ed in /etc/opt/resmon/config.
Therefore, a monitor cann ot be perma nent ly stopped or st a rt ed by a
human.
Beca u se th e m on itor s a r e per s is t en t , m on itor in g r eq u es t s a r e kept w h en
you inst a ll a new m onitor or upda te a n existing monitor. If a condition,
s uch a s “ st a t us > 3” is bein g m on it or ed for a r es ou rces t h a t h a s a r a ng e of
1-7, and new version of monitor is installed tha t supports a new st a tus
value, such a s “8”, you ma y sta rt seeing notifi cations for “sta tus= 8”.
Troubleshooting
Testing Monitor Requests
8/20/2019 Event Moniter Service
70/76
70 Chapter7
Glossary
8/20/2019 Event Moniter Service
71/76
71
A-H
alert An event. A messa ge sent t owa rn a user or a pplicat ion w hen
certain conditions are met.
asynchronous monitor Amonitor that monitors resource
inst a nces (or resource class)
a synchronously. It is event driven
a n d sen d n ot i fi ca t i on s w h en even t s
occur. It does not keep tr a ck of th e
current st a te or value of each
resource it monitors.
client The applica t ion tha t createsor ca ncels requ ests t o monitor
par ticular resources. The
consum er of a r esource sta tus
messa ge. A user of th e Resource
Monitor f ramework. This user may
browse resources, request status,
and make requests t o have
resources monitored. Examples are
MC/ServiceGua rd a s i t st a rts a
packag e or th e SAM interfa ce to
EMS.
dictionary See Resource
Dictionary.
EMS (Event Monitoring Service)The int erface betw een resource
monitors, the client a nd t a rget
applications.
EMS Framework A set of AP Istogether wit h the Registra r
process a nd t he resource
dictiona ry, w hich a llow client
a pplica tions t o request t ha t
resources be monitored a nd a
ta rget a pplicat ion be notified.
event An alert.
HA H igh Ava ilability.
I-L
ITO H P OpenView IT/Opera tions,formerly known as Opera tions
Center
logical extent The ba sica l loca t i on u n it f or a log ica l volu m e
is called a logical extent . For
mirrored logical volumes, either
tw o or th ree physica l extents a re
ma pped for each logica l extent,
depending on w hether you ar e
using 2-wa y or 3-wa y m irroring.
logical volume A collection ofdisk spa ce from one or more disk s.
E a ch collection a ppea rs t o theopera ting syst em as a single disk.
Like disks, logical volumes can be
u sed t o h old fi l e sy st em s, r a w d a ta
areas, dump areas, or swa p areas.
U nlike disks, logica l volumes ca n
be given a size when they a re
creat ed, an d a logica l volume ca n
8/20/2019 Event Moniter Service
72/76
72
l a ter be expanded or reduced. Also,
log ica l vol um es ca n be spr ea d over
mult iple disks.
LUN (Logica l U nit Numbers) Alogica l disk device composed of one
or more physica l d isk mechanisms,ty pica lly confi gured int o a R AID
level.
LVM (Logical Volume Manager)Ma na ges disks in volume groups,
a nd a llow s you to crea te logica l
a nd phys ica l volume groupings.
M
MIB (Man agement Informa tionB a se). A document th a t describes
objects to be ma na ged. A MIB is
created using a gramma r defined
in “Str ucture of Man agement
Informa tion” (SMI ) forma t. This
gra mma r concisely defi nes the
objects being ma na ged, the dat a
ty pes these objects ta ke,
descr ipt ions of how the objects can
be used, wheth er th e objects a re
rea d-only or rea d-w rite, a nd
identifi ers for the objects.
MIB II (MIB2) A MIB t hatdefines informa tion a bout t he
system, the netw ork interface
car ds it conta ins, routing
informa tion it cont a ins, the TCP
an d UD P sockets it conta ins and
their stat es, and various sta tistics
r ela ted to er r or cou n t s. Th is M IB is
widely a dopted a nd is served by
most I P -a ddressed devices. Most
system a nd n etwork resources
m a n a ged by E M S H A M on it or s a r e
ta ken from this MIB.
monitor S ee resource monitor.
N-P
notification See alert.
physical extent LVM divideseach physical disk int o a ddressed
units called physical extents.
physical volume A disk tha t ha sbeen initia lized by LVM.
polling The process by w hich amonitor obta ins th e most r ecent
sta tus of a resource.
PVG (physical volume group)
A grouping of phy sical d evices
(host ada pters , busses, controllers ,
or disks), th a t a llow LVM to
ma na ge redunda nt l inks ormirrored disks an d a ccess the
redundant ha rdwa re when the
primary hardw are fa i ls.
8/20/2019 Event Moniter Service
73/76
73
PV links A met hod of LVMconfi gura tion tha t a llows you to
provide redundant SCSI inter faces
a nd buses to disk arra ys, thereby
protecting a ga inst single points of
failure in SC SI car ds an d cables.
Q-R
registrar The r egistra r processprovides the l ink between resource
sta tus consumers (client s) a nd
resource sta tus providers (resource
monitors). The cent ra l par t of the
resource monitor framework which
uses th e resource dictiona ry t o act
a s a n intermediary between client
syst ems an d resource monitors.
resource May be any enti ty amonitor application developer
na mes. Exa mples include a
netw ork interfa ce, CP U st a tistics,
a MIB object, or a netw ork service.
resource class A cat egory ofresources useful dur ing
confi gura tion. For exam ple,
/net /in t er fa ces/la n/st a t us is
provided a s a resource class.
resource dictionary A fi ledescribing t he hiera rchy of
resources tha t can be monitored
a n d th e pr ocesses th a t per for m th e
resource monitoring.
resource instanceThe actua lresources tha t can be monitored.
For example,
/net /int erfa ces/la n/st a t us/la n0 ma y
refer to a pa rticular netw ork
interface installed on t he
monitored system.
resource monitor A frameworkfor selecting resources of interest
a nd monitoring t hem a ccording t o
th e user's criteria . When t he
resource value ma tches th e user's
criteria , a notificat ion is sent
a ccording t o the user'sinstructions.
The process th a t is used t o obta in
the st at us of a resource an d send
event notificat ions if a ppropriat e.
A monit or checks resources on t he
local system. The resource monitor
ma ps the phys ica l resource int o asta nda rd interfa ce understood by
EMS.
S-T
SNMP (Simple Netw orkMan agement P rotocol) St an dar d
protocol for net w ork ba sed
retr ieval of informa tion about
syst em resources.
state The curren t v a lue of aresource (UP or DOWN). For some
resource inst a nces, a monitor ma y
8/20/2019 Event Moniter Service
74/76
74
need to maint a in a hist ory of past
events or condit ions in order to
know th e resource va lue. In t his
case, a monitor is said t o be
maintaining state information.
S t a te less m on itor s d o n ot keep a n y
history of past conditions.
target The ta rget a pplica tion isnot ified when a moni tored resource
rea ches t he condition for w hich
notificat ion w a s requested. For
example, a ta rget appl ica t ion could
be MC/S erviceG ua rd or
IT/Oper a t ions (ITO).
U-Z
volume group In LVM, a set ofphysical volumes w hose extent s
ar e grouped together and then
ma de ava ilable to users as logical
volumes. A volume group ca n be
activat ed by only one node at a
time unless you a re using
MC /LockMa na ger.
MC/ServiceGua rd can a ctivat e a
volume group when it st ar ts a
package. A given disk can belong
to only one volume group. A logica l
volume can belong t o only one
volume group.
Index
8/20/2019 Event Moniter Service
75/76
Index 75
A
a pi.log fi le, 65asynchronous
monitors, 37a synchronous monitors, 28
C
classes , 28client.log file, 65cluster , 54cluster monitor request
test ing, 68console
notifi cation option , 41copying requests , 47creating a m onitoring request ,
33creating package dependencies,
54
D
dictionary, 63disk monitor request
test ing, 68
E
emailnotifi cation option , 41
E MSfi les and directories, 63logging, 65restart ing , 69SAM interface, 13, 23star t ing, 23testing monitoring requests,
68tracing, 66
EMS, high a vai labi li ty, 17event
notification , 34notifi cation frequency , 36
notification protocol options,38
opcmsg I TO option , 38SNMP notifi cat ion, 39
F
fi les and directories conta iningEMS monitors, 63
H
H Abasic, 13install ing , 21MC/Serv iceG ua rd , 17prerequisites, 14s yst em management s of t wa r e,
18hierarchy , 28high a vai labi li ty, 17
I
informationa bout monitors, 26
initial notification , 36install ing
over netw ork, 21instal l ing H A, 21IT/Oper a t ions , 39
send event , 17ITO
severity options, 38
L
log fi les, 63, 64logging, 65
M
MC /ServiceG ua rd , 17, 54creat ing package
dependencies, 54HA Monitors, 17
moidfying requ ests, 49monitor , 16
AP I , 16asynchronous, 28, 37fi nding informa tion , 26view informa tion, 16
monitor daemons, 63monitor persistence, 69monitoring request
copying , 47creating , 33crea ting comment s, 43modifying , 49polling interva l, 37removing , 50
test ing, 68viewing, 51monitors
updat ing, 35
N
Network Node Managersend event , 17
notification , 34event frequency , 36
notifi cation comment , 43notifi cation option
console, 41email, 41syslog, 41TCP , 39textlog , 42U D P , 39
notifi cation options, 36
notification protocol, 39SNMP , 39
Noti fy at each interval, 34Notify when value cha nges, 34Notify wh en va lue is. . ., 34
O
opcmsg
Index
8/20/2019 Event Moniter Service
76/76
76 Index
event notification , 38options for notifi cation, 36
P
p_client, 69package confi gurat ion fi le, 60package dependencies, 60
creating , 54performace, 67persistence, 35, 69polling interva l, 37, 60, 67protocol
event notifi cation options , 38protocol, sending notifi cat ion , 39
Rregistra r. log fi le, 66removing requ ests, 50repeat not ifi cat ion , 36resource
a s MC/Serv iceG ua rddependency, 54
selecting, 26UP va lues, 59
viewing descriptions, 30resource cla sses, 14, 28resource dictionary , 63resource monitor , 16restart ing EMS , 69return not ificat ion , 36
S
SAM interface
E MS , 13SAM interface to EMS , 23SAM interface to
MC/Serv iceG ua rd , 57selecting a resource, 26sending
events, 34severity options
ITO, 38
SNMP , 39SNMP
severity options, 39SNMP t raps, 39star t ing
E MS , 23
syslognotifi cation option, 41
system load, 67system requirements, 14
T
TCPnotifi cation option, 39
testing monitoring requests, 68
textlognotifi cation option, 42
tracing, 66
U
U D Pnotificat ion option, 39
UP v a lue, 59updating monitors, 35
V
viewingresource descriptions, 30
viewing requests, 51
W
wildcard, 29