HP ArcSight ESM 24/7 · 2014. 9. 9. · Forward-looking statements This document contains forward looking statements regarding future operations, product development, product capabilities

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

HP ArcSight ESM 24/7 Aparna Varanasi, Sr. Software Engineer Bill Alexander, Sr. Software Engineer #HPProtect

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3

This is a rolling (up to three year) Roadmap and is subject to change without notice.

Forward-looking statements

This document contains forward looking statements regarding future operations, product development, product capabilities and availability dates. This information is subject to substantial uncertainties and is subject to change at any time without prior notification. Statements contained in this document concerning these matters only reflect Hewlett Packard's predictions and / or expectations as of the date of this document and actual results and future plans of Hewlett-Packard may differ significantly as a result of, among other things, changes in product strategy resulting from technological, internal corporate, market and other changes. This is not a commitment to deliver any material, code or functionality and should not be relied upon in making purchasing decisions.


This is a rolling (up to three year) Roadmap and is subject to change without notice.

HP confidential information

This Roadmap contains HP Confidential Information. If you have a valid Confidential Disclosure Agreement with HP, disclosure of the Roadmap is subject to that CDA. If not, it is subject to the following terms: for a period of 3 years after the date of disclosure, you may use the Roadmap solely for the purpose of evaluating purchase decisions from HP and use a reasonable standard of care to prevent disclosures. You will not disclose the contents of the Roadmap to any third party unless it becomes publically known, rightfully received by you from a third party without duty of confidentiality, or disclosed with HP’s prior written approval.


ESM 24/7

Next ESM release • What is ESM HA (High Availability)?

– HA features overview – HA architecture

• HA features – Installation – Monitoring

• Failover demo • Key takeaways

This is a rolling (up to 3 year) roadmap and is subject to change without notice


What is ESM HA?

ESM HA (High Availability) is a two server installation of the ESM product for improved reliability and availability. ESM HA is an active/passive cluster. • Primary – the server running ESM. • Secondary – the other server (on hot standby).



HA features overview

• Secondary backs up Primary disk • Automatic Failover

− System failures are automatically detected − Secondary switches to primary and runs ESM

• Monitoring − Audit events and notifications − Console Content − arcsight_cluster script – monitoring and

maintenance • Simplified installation


ESM HA

Connectors Clients


HA architecture


Server room layout

Interconnected pair of ESM servers • Primary and Secondary connected by one or more 1G or 10G

Ethernet cables. • Optional HP iPDU (Intelligent Power Distribution Unit)

– Only external device that ESM HA supports for forced reboot of servers. • To reboot a server, commands are sent to the iPDU to

turn power on and off. – Optionally redundant power supplies supported.

• Switch(es) provide network access. – Connectors – ESM Clients – iPDUs

HA architecture

iPDU

Primary

Secondary

Network Switch



Software architecture

• Cluster Control (Pacemaker/Heartbeat) – Monitors software and communications – Determines which software runs on which server. – Restarts software when needed

• Disk Mirroring (DRBD) – Makes disk available on primary – Sends changes made on primary to secondary

• ESM, Service IP, and File System – only on primary – File System containing ESM installation only

mounted on primary. – Service IP – ESM IP Address – dynamically moves

between servers. – ESM runs where its files are mounted.

HA architecture Intranet

Primary Secondary

eth0 eth0

eth1 eth1 Disk Mirroring

Disk Mirroring

Cluster Control

ESM

File System

Service IP

Cluster Control



HA architecture

STONITH (Shoot The Other Node In The Head)

Enabling technology for failover • Needed when primary is crippled and will not release resources

– Communication problems – primary cannot receive stop request – Software problems (e.g. out of memory or other resources)

• Ideally STONITH mechanism should be independent of primary hardware/software

– Power control like iPDU – In some clusters cutting the server off from the network (I/O fencing) is used.

• Default SSH based fallback reboot control far from ideal.

– Will only work if SSH to server, reboot is possible.



Installation


Installation overview

• Designed to be easy – Installation questions via Wizards – Run installation on primary • Secondary installation done automatically

• Major data inputs covered in the next slides

Installation

Hopefully this won’t be too bad …



Cluster setup parameters

• Shared disk – mount point for ESM installation (/opt or /opt/arcsight)

• Metadata volume – small partition with disk sync status

• Service hostname – ESM hostname or IP – moves between servers.

• Secondary hostname – hostname of secondary server.

• Primary cable IP – IP address of primary via interconnect cable.

• Secondary cable IP – IP address of secondary via interconnect cable.

Installation


All product views are illustrations and might not represent actual product screens


Cluster configuration parameters

• Preferred primary – this one will be selected as primary.

• Connected hosts – hosts to ping to see if this machine is connected to the internet.

• Connectivity down timeout – how long the communication between the network and the primary should be down before failover.

• Time between failovers – if a failover has occurred recently, wait this long before failing over again.

• Ping timeout – how long to wait before considering a ping to have failed.

• Ping attempts – how many pings to try before concluding cannot reach this host.

Installation




iPDU parameters Installation



Only fill out this screen if you have iPDU • iPDU hostnames – hostname of iPDU(s) • Wait time for reboot – reboot is accomplished by

doing power off, followed by power on. The iPDU should wait this long to turn on power after turning it off.

• Primary iPDU outlets – the outlet(s) to which the primary is connected.

• Secondary iPDU outlets – the outlet(s) to which the secondary is connected.

• iPDU login – user to log into iPDU. • iPDU password – password to log into iPDU


HA monitoring

“Trust, but verify” Ronald Reagan 40th President of the United Statesrerson’s name, title and)


HA monitoring

• HA audit events and notifications • HA monitoring and maintenance script • HA content



HA audit events and notifications



Audit event and notification types • HA:100 – Primary Manager Started • HA:200 – HA Status Failed

– Secondary is offline – Disk is not syncing or possible disk failure on

secondary – Network communication on primary or secondary

is down • HA:300 – Sync in Progress • HA:400 – iPDU Status Failed • HA:500 – HA Status OK

Frequency • HA:100 – Created on change of HA state • HA:200, HA:300, HA:400 - Created on change of

HA state and/or about every 5 mins, if the same state exists. Notifications sent as well

• HA:500 – Created on change of HA state. Notification sent



HA audit events and notifications This is a rolling (up to 3 year) roadmap and is subject to change without notice



Configuration


HA properties in server.properties highavailability.monitor.on=true • Turn on/off the HA Notification feature. True is On and False is Off.

highavailability.notification.interval=300 • Set notification interval for failure conditions. It is configured in seconds and the default is 5 mins.

whine.check.interval.HASubsystemChecker=30 • Set the polling interval of the tracker/checker that checks arcsight_cluster status. It is configured in seconds

and the default is 30s.



HA monitoring and maintenance script


arcsight_cluster script

HA monitoring and maintenance script

/usr/lib/arcsight/highavail/bin/arcsight_cluster Options • status • prefer • offline • online • diagnose • clusterParameters • increaseDisk • tuneDiskSync



arcsight_cluster status output This is a rolling (up to 3 year) roadmap and is subject to change without notice



HA content


HA Monitoring Use Case - All Use Cases/ArcSight Administration/ESM/HA Monitoring

HA content

• Active Channel – HA Monitoring

• Dashboard – ESM HA Status

• Query Viewers and Queries – System Status Changes – Last

24 hours – Current Primary Server – System Status Changes – Current Primary

• Report – ESM HA Status Updates –

last 7 days • Filter

– ESM HA Status



HA Monitoring Use Case - All Use Cases/ArcSight Administration/ESM/HA Monitoring

HA content

• Rules – ESM SystemStarted – Alert – HA Status Change • Notification sent

• Active List – Current Primary System

• Data Monitor – Last 10 HA Status Changes – ESM HA Status

• Field Set – HA Management

• Session List – Current Primary System Status

Change



HA content




Failover demo


How is a failover done?

1. Cluster determines failover is necessary. 2. Cluster brings down resources on old primary

a. Stops ESM b. Unconfigures Service IP c. Unmounts Disk d. Puts Disk Mirroring Software into Secondary Mode

3. If any of the steps in 2 fail, STONITH (reboot) the primary. 4. Cluster brings up resources on new primary

a. Puts Disk Mirroring Software in Primary Mode b. Mounts Disk c. Configures Service IP d. Starts ESM



Cover title over with Failover video This is a rolling (up to 3 year) roadmap and is subject to change without notice



Key takeaways


Key takeaways

• ESM HA can help by – Protecting ESM from hardware failures – Minimizing downtime for scheduled maintenance

• Steps to a successful implementation – System is only as reliable as weakest link – improve reliability everywhere • Power • Network

– Use application monitoring software to catch cases when both Primary and Secondary go down • HP Operations Manager, NAGIOS

– HA Best Practices • Use Logical Volume Manager (LVM) to simplify creating, resizing partitions. • Bonded Interfaces for speed, reliability in interconnect. • Use iPDU for cleaner failovers



For more information

Attend these sessions

• TT3058, Building a highly available HP ArcSight solution

Visit these demos

• HP ArcSight ESM

After the event

• Contact your sales rep

Your feedback is important to us. Please take a few minutes to complete the session survey.


Please fill out a survey. Hand it to the door monitor on your way out.

Thank you for providing your feedback, which helps us enhance content for future events.

Session TB3069 Speakers Aparna Varanasi and Bill Alexander

Please give me your feedback


Thank you

Documents

HP ArcSight ESM 24/7 · 2014. 9. 9. · Forward-looking statements This document contains forward looking statements regarding future operations, product development, product capabilities