S N58: ArcS ight, Monitor Thyself€¦ · ArcSight Infrastructure What to monitor? Availability – Monitor critical devices – Monitor ArcSight connectors, appliances, ESM Performance

1www.arcsight.com © 2010 ArcSight Confidential

© 2010 ArcSight, Inc. All rights reserved. ArcSight and the ArcSight logo are trademarks of ArcSight, Inc. All other product and company names may be trademarks or registered trademarks of their respective owners.

1

S N58: ArcS ight, Monitor Thys elf

K en MermoudS oftware Development Manager

R as haad S tewardP rinc ipal E nterpris e S pecialis t - P ublic S ector S ervices

September, 2010


Overview

Monitor ArcSight Infrastructure

ArcSight Internal Events Configuration and Forwarding

ArcSight Monitoring Content


Monitor ArcS ight Infras tructure


ArcSight InfrastructureWhat to monitor?

Availability– Monitor critical devices

– Monitor ArcSight connectors, appliances, ESM

Performance– CPU Usage– Memory Usage

Network– Current EPS, EPS over time

– Inbound/Outbound traffic

Disk and Storage– Monitor disk usage on Logger, ESM– Monitor disk free space on Logger, ESM


ArcSight InfrastructureMonitor components individually

Device– Connect to the device

– Make sure it’s up and running

Connector– Connect to ESM or Connector Appliance– Check status, check logs

Appliances (Logger, Connector Appliance, NSP)– Connect to the Appliance

– Check CPU, disk usage, EPS

ESM– Connect to ESM, ArcSight Status Monitoring (manage.jsp)– Check status, check logs


ArcSight InfrastructureMonitor components from ESM

How?– Leverage ArcSight internal events– Forward internal events to ESM– Use ESM Rules, Reports, Trends, Dashboards

Why?– Centralized view– Overview Summary– Allows drill-down and further investigation


ArcS ight Internal E vents


ArcSight Internal EventsDefinition

Events generated by ArcSight products internally

Events can be local or forwarded to ESM Two types of internal events

– Status Monitor Events• Statistics about system health (CPU Usage, EPS, DB free space)

• Generated periodically

– Audit Events• Reports an action (User authentication, activity, resource modification)

• Generated for every action (real-time)


ArcSight Internal EventsStatus Monitor Events

Example:– CPU Usage

– EPS

– Storage Free Space


ArcSight Internal EventsAudit Events

Example:– User Authentication

– User Group Modification

– Resource Creation


ArcS ight Internal E ventsB y P roduct


ArcSight Internal EventsConnector and Device

Device Statistics– Last event received

– Total number of events– Event count since last call

Connector Flow Statistics– Event Rates

– Cache Size

Connector Audit Events– Start/Stop– Heartbeat

– Cache Statistics


ArcSight Internal EventsAppliances (Logger, Connector Appliance, NSP)

CPU Statistics– Current Value

Disk Statistics

– Disk Space– Read/Write

Event Statistics

– EPS (Receiver, Forwarder)– Event Count (Receiver, Forwarder)

Memory Statistics– JVM Memory

– Platform Memory

Network Statistics– Inbound usage

– Outbound usage


ArcSight Internal EventsESM

Resource Statistics– Open resource count

– Queries/Evaluations per second

Resource Framework Statistics– Inserts– Updates

– Deletes

Rules Engine Statistics (CPU, memory)– Events in rule engine– Events matching rules

– Rate of correlated events


ArcSight Internal EventsESM (2)

Event Broker Statistics– Event Count– Insert Time– Retrieval Time

Main Flow Statistics– EPS (Count since last monitor event)– Events (Count since startup)

Side Table Statistics– Size– Insert– Cache (misses/hit rate)

Database Statistics– Free Space– Read/Write


C onfiguration and F orwarding


ArcSight Internal EventsConfiguration

How to forward these events to ESM?– Device

• Modify a property on the connector to enable device status monitoring events

– Connector• Direct connection to ESM

• Connection through Connector Appliance

• Connection through Logger

– Appliance (Logger, Connector Appliance, NSP)• Configure a connector to forward internal events to ESM

– ESM• Single-tier: events are already on ESM

• Multi-tier: use forwarding connector


Internal Events Forwarding ConfigurationConnector and Device

Connector Device Status Events– Events sent by the connector to ESM– Forwarding can be enabled on the

connector– Provides status about connector and device

• Timestamp of the last time the Connector received an event

• Count of events sent by a device since last check

• Total count of events sent by a device

Configuration Steps– Select the connector– Right-Click -> Configure– Default Tab -> Content– Processing– Enable Device Status Monitoring (in

millisec)


Internal Events Forwarding ConfigurationConnector Appliance


Internal Events Forwarding ConfigurationConnector Appliance (Summary)

Configuration Steps:1.Upload ESM Certificate

• Upload Certificate to Connector Appliance

2.Add ESM Certificate• Associate Certificate to Container

3.Add Syslog Connector• Type: Syslog

• Destination: ESM

– Enable Status Monitor Events1. Preserve System Health Events (Status Monitor Events)

• Enable Device Status Monitoring

1.Forward Audit Events• Select Connector


Internal Events Forwarding ConfigurationLogger


Internal Events Forwarding ConfigurationLogger (Summary)

Configuration Steps:1. Upload ESM Certificate

• Upload Certificate to Logger Appliance

2. Add ESM Destination• Create a Connector

• Point it to ESM Manager

3. Add Forwarder• Type: ArcSight ESM (CEF) Forwarder

• Query: cef:0\|ArcSight\|Logger (Status Monitor Events)

• Destination: ESM Destination

4. Forward Audit Events• Select ESM Destination


Internal Events Forwarding ConfigurationESM

Single-Tier ESM– No extra configuration needed

– Internal events are already present

Multi-Tier ESM– Configure Forwarding Connector– Parameters

• Connector Name

• Source Manager (host, port, user/password)

• Destination Manager (host, port, user/password)


Internal Events in ESMConnector Appliance – Status Monitor Events


Internal Events in ESMLogger – Audit Events


ArcS ight Monitoring C ontent


ArcSight Monitoring ContentOverview

From field services an “Advance” Monitoring Content Example– All Inclusive Connector/No Connector Caching State

– We are working on improving the stock ESM content based on feedback/research done in real deployments by field services team


All Inclusive Connector/No Connector Caching StateUse Case

Objective – To provide a single icon

representation last state data view for all Connector/No Connector Caching State

– Allows you to easily identify connectors caching in your infrastructure -especially if you have many connectors in your environment

– Support the topics of this UC session - shows how to leverage internal ArcSight events to produce advance content


All Inclusive Connector/No Connector Caching StateOverview

All Inclusive Connector/No Connector Caching State– Content will be available in future ESM Foundation Content– Will be part of the ArcSight Administration Package– Content will be located in:

• /All */ArcSight Administration/Connectors/System Health/

Configuration– Clear Infrastructure Connectors Currently Caching and Infrastructure Connectors

Caching Active Lists entries upon initialization– Tweak the Infrastructure Connectors Currently Caching Active List TTL based on your

preference on how long a connector can cache before you are alerted (e.g. every 30 minutes, every 2 hours)

– Ensure Infrastructure Number of Connectors Caching Active List entry has File Name = Infrastructure Connectors Caching and Counter = 0 upon initialization

Content– Rules (Several Rules have Dependent Var iables) :

• Infrastructure Connectors Cache - Connector Caching - Rule 1• Infrastructure Connectors Cache - Failed - Rule 2• Infrastructure Connectors Cache - Failed Increment Counter - Rule 3


All Inclusive Connector/No Connector Caching StateOverview Continued

Content– Rules (Several Rules have Dependent Var iables) :

• Infrastructure Connectors Cache - Success Decrement Counter - Rule 4• Infrastructure Connectors Cache - Number of Connectors Cache Active

List Checker - Rule 5• Infrastructure Connectors Cache - Red or Green Determinant - Rule 6• Infrastructure Connectors Cache - Red - Rule 7• Infrastructure Connectors Cache - Green - Rule 8• Infrastructure Connectors Cache - Connector Cache Emptied - Rule 1a

– Active Lists: • Infrastructure Connectors Currently Caching• Infrastructure Connectors Caching• Infrastructure Number of Connectors Caching

– Filters: • Infrastructure Connector Cache Counter Check Filter• Infrastructure Connectors Cache Status


All Inclusive Connector/No Connector Caching StateOverview Continued

Content– Notification: If critical connector is caching for more than X minutes/hours– Dashboard:

• All Inclusive Connector/No Connector Caching status icon• Query viewer to list connector(s) caching if all inclusive icon is red

Sources– Connector Caching Framework & Internal ESM Manager Events


All Inclusive Connector/No Connector Caching StateContent Description

Active List: Infrastructure Connectors Currently Caching– Stores the l i st of all the connectors currently caching– Active list entries expire after connector has constantly cached for

2 hours or more (by default TTL=2)

Active List: Infrastructure Connectors Caching– Stores the l i st of all the connectors that have been constantly

caching for 2 hours or more– Active list entries never expire - cleared when connector cache is

emptied and rule fire action occurs

Active List: Infrastructure Number of Connectors Caching– Stores the t ot a l number of all connectors constantly caching for 2

hours or more– Active list entries never expire - cleared when connector cache is

emptied and rule fire action occurs


All Inclusive Connector/No Connector Caching StateActive Lists Entr ies Examples

Example:– Infrastructure Connectors Currently Caching Active List (TTL=2 hours)

– Infrastructure Connectors Caching Active List (TTL=0)

– Infrastructure Number of Connectors Caching Active List (TTL=0)


All Inclusive Connector/No Connector Caching StateContent Description Continued - Rule 1

Rule: Infrastructure Connectors Cache - Connector Caching - Rule 1– Fires when a connector is caching and is NOT already listed as an entry in

the “Infrastructure Connectors Currently Caching” active list– Conditions around internal event monit or : 113 set to make rule fire



Rule: Infrastructure Connectors Cache - Connector Caching - Rule 1– Desired fields File Name (connector name) and File Path (connector URI)

added to active list “Infrastructure Connectors Currently Caching”


All Inclusive Connector/No Connector Caching StateContent Description Continued - Rule 1a

Rule: Infrastructure Connectors Cache - Connector Cache Emptied -Rule 1a– Fires when a connector’s cache is cleared and if connector was

previously listed in “Infrastructure Connectors Currently Caching” or “Infrastructure Connectors Caching” active lists

– Conditions around internal event monit or : 113 set to make rule fire


All Inclusive Connector/No Connector Caching StateContent Description Continued - Rule 1a

Rule: Infrastructure Connectors Cache - Connector Cache Emptied -Rule 1a– Removes entries from “Infrastructure Connectors Currently

Caching” and “Infrastructure Connectors Caching” active lists



Rule: Infrastructure Connectors Cache - Failed - Rule 2– Fires when a connector constantly caches for more than 2 hours and falls off

the “Infrastructure Connectors Currently Caching” active list producing internal event act ivel ist : 104 with piped delimited value of expired active list entry

– Conditions around internal event act ivel ist : 104 set to make rule fire



Rule: Infrastructure Connectors Cache - Failed - Rule 2– Rule uses dependent variables– 6 variables (IndexOf, Substring, LengthOf, Add, LengthOf and Substring) used

to retrieve name of connector and connector resource URI for caching connector identified in active list entry expired internal event act ivel ist : 104in deviceCustomString4 piped delimited field



Rule: Infrastructure Connectors Cache - Failed - Rule 2– Fields set to aggregate

on so may be used in the Actions tab later

– *variables created and used in the Actions tab need to be added to the identical Aggregate field



Rule: Infrastructure Connectors Cache - Failed - Rule 2– Desired variable fields set to ESM schema fields to be added to active list

“Infrastructure Connectors Caching”– *Send Notification: If critical connector is caching for more than X

minutes/hours - you can add notification action here or leverage custom email templates to do the work



Rule: Infrastructure Connectors Cache - Failed Increment Counter - Rule 3– Fires when Infrastructure Connectors Cache - Failed - Rule 2 adds the details of

the connector which has been constantly caching for more than 2 hours to “Infrastructure Connectors Caching” active list

– Conditions around internal event act ivel ist : 101 (ent r y added t o AL) set to make rule fire



Rule: Infrastructure Connectors Cache - Failed Increment Counter - Rule 3– Rule uses dependent

variables– 2 variables

(getALCounterValue) used to retrieve values for “Infrastructure Connectors Caching” entry in “Infrastructure Number of Connectors Caching” active list

– (incrementALCounter) used to Add (1) to Counter field value retrieved for “Infrastructure Connectors Caching” entry in “Infrastructure Number of Connectors Caching” active list



Rule: Infrastructure Connectors Cache - Failed Increment Counter - Rule 3– Fields set to aggregate





Rule: Infrastructure Connectors Cache - Failed Increment Counter – Rule 3

Desired field and variable field set to ESM schema fields to be added to active list “Infrastructure Number of Connectors Caching”– Increments the count of the total number of connectors caching

Set flexNumber1 to the variable incrementALCounter–Variable is an increment value to be added to the total count of the number of connectors caching for more than 2 hours



Rule: Infrastructure Connectors Cache - Success Decrement Counter - Rule 4– Fires when Infrastructure Connectors Cache - Connector Cache Emptied - Rule

1a action removes a previously noted caching connector entry from “Infrastructure Connectors Currently Caching” and “Infrastructure Connectors Caching” active lists

– Conditions around internal event act ivel ist : 102 (ent r y r emoved f r om AL) set to make rule fire



Rule: Infrastructure Connectors Cache -Success Decrement Counter - Rule 4– Rule uses dependent

variables– 2 variables

(getALCounterValue) used to retrieve values for “Infrastructure Connectors Caching” entry in “Infrastructure Number of Connectors Caching” active list

– (decrementALCounter) used to Subtract (1) to Counter field value retrieved for “Infrastructure Connectors Caching” entry in “Infrastructure Number of Connectors Caching” active list



Rule: Infrastructure Connectors Cache -Success Decrement Counter - Rule 4– Fields set to aggregate





Rule: Infrastructure Connectors Cache - Success Decrement Counter –Rule 4

Desired field and variable field set to ESM schema fields to be added to active list “Infrastructure Number of Connectors Caching”–Decrements the count of the total number of connectors caching

Set flexNumber1 to the variable decrementALCounter–Variable is a decrement value to be subtracted from the total count of the number of connectors caching for more than 2 hours



Rule: Infrastructure Connectors Cache - Number of Connectors Cache Active List Checker - Rule 5– Fires when Infrastructure Connectors Cache - Failed Increment Counter - Rule 3

or Infrastructure Connectors Cache - Success Decrement Counter - Rule 4 increments/decrements (modifies) Counter field value entry in “Infrastructure Number of Connectors Caching” active list

– Conditions around internal event act ivel ist : 103 (ent r y changed in an AL) set to make rule fire



Rule: Infrastructure Connectors Cache - Number of Connectors Cache Active List Checker - Rule 5– Rule uses dependent variables– 7 variables (IndexOf, Substring, LengthOf, Add, LengthOf, Substring and

Convert_String_To_Long) used to retrieve modified (act ivel ist : 103) values presented in deviceCustomString4 piped delimited field for entries in “Infrastructure Number of Connectors Caching” active list

– *Convert_String_To_Long variable is used to convert second value in DCS4 from string to long to be evaluated later as a long value



Rule: Infrastructure Connectors Cache -Number of Connectors Cache Active List Checker - Rule 5– Fields set to aggregate





Rule: Infrastructure Connectors Cache - Number of Connectors Cache Active List Checker - Rule 5

Desired variable fields set to ESM schema fields to be evaluated later by Infrastructure Connectors Cache - Red or Green Determinant - Rule 6

Set fileName to getSubstringOfFirstString - the string value of “Infrastructure Connectors Caching” retrieved from DCS4 variable work

Set flexNumber1 to convertSecondSubStringToLong - the long value retrieved from DCS4 variable work for current number of Connectors Caching



Rule: Infrastructure Connectors Cache - Red or Green Determinant - Rule 6– Fires when Infrastructure Connectors Cache - Number of Connectors Cache

Active List Checker - Rule 5 and File Name = Infrastructure Connectors Caching conditions are met



Rule: Infrastructure Connectors Cache - Red or Green Determinant - Rule 6– Rule uses dependent variable– 1 variable (Filter_Based_Condition_Function) used to evaluate if number of

Connectors Caching (flexNumber) is > 0



Filter: Infrastructure Connector Cache Counter Check Filter– Evaluates Infrastructure Connectors Cache - Number of Connectors Cache

Active List Checker - Rule 5 fire and its conditions– Base on the conditional evaluation a string field will be set to either Daily RED

(flexNumber1>0) or Daily GREEN (flexNumber=0)



Rule: Infrastructure Connectors Cache - Red or Green Determinant - Rule 6– Fields set to aggregate





Rule: Infrastructure Connectors Cache - Red or Green Determinant - Rule 6 Desired variable field set to ESM schema fields to be evaluated later by

Infrastructure Connectors Cache - Red - Rule 7 & Infrastructure Connectors Cache -Green - Rule 8

Set flexString2 to conditionalEval - the string value of “Daily RED” or “Daily GREEN” retrieved from Filter_Based_Condition_Function in Infrastructure Connector Cache Counter Check Filter variable work



Rule: Infrastructure Connectors Cache - Red - Rule 7– Fires when Infrastructure Connectors Cache - Red or Green Determinant - Rule 6

and Flex String2 = Daily RED conditions are met



Rule: Infrastructure Connectors Cache - Red - Rule 7 Set deviceCustomString2 to “Connector Cache Status” to be used as key field

declaration in last state data monitor “Infrastructure Connector Cache Status” -allows only one icon last state to populate in dashboard for Connectors Caching

Set priority to 10 indicating connector(s) have been caching for 2 hours or more (remember the TTL=2 hours is configurable)

*Rule Fire Name will be used in data monitor Mapping: Name -> Status to set value of last state all inclusive Connector Cache icon to RED



Rule: Infrastructure Connectors Cache - Green - Rule 8– Fires when Infrastructure Connectors Cache - Red or Green Determinant - Rule 6

and Flex String2 = Daily GREEN conditions are met



Rule: Infrastructure Connectors Cache - Green - Rule 8 Set deviceCustomString2 to “Connector Cache Status” to be used as key field

declaration in last state data monitor “Infrastructure Connector Cache Status” -allows only one icon last state to populate in dashboard for Connectors Caching

*Rule Fire Name will be used in data monitor Mapping: Name -> Status to set value of last state all inclusive Connector Cache icon to GREEN


All Inclusive Connector/No Connector Caching StateContent Description Continued - Last State Data Monitor Filter


All Inclusive Connector/No Connector Caching StateContent Description Continued - Last State Data Monitor


All Inclusive Connector/No Connector Caching StateContent Description Continued - Query Viewer

Query Viewer: Queries “Infrastructure Connectors Caching” active list every (1) minute to list name of connector(s) caching


T he Whole Enchilada - Putting It All T ogetherAll Inclusive Infrastructure Connectors State Status Dashboard


Your F eedback B uilds a B etter C onference!

Download s es s ion replays after the c onferenc e:

https : //protec t724.arc s ight.c om/c ommunity/protec t10

Excellent Good Fair Poor

Rate the speaker a b c d

Rate the content e f g h

Please provide comments: (*) enter any comments/feedback

Text to 32075 (US A & C anada) or 447786204951 (Non-US A)

Type AR C S <s pac e> 58 and the letter to eac h res pons e

S MS body exam ple: ARCS 58ae*your comments


Use Case Strategy Contact Information

For More Information about Use Case strategy or ArcSight Enterprise Specialist (AES) Professional Services

Rashaad Steward: [email protected] Inc.: www.arcsight.com


ArcSight, Inc.Corporate Headquarters: 1 888 415 ARST

EMEA Headquarters: +44 (0)844 745 2068Asia Pac Headquarters: +65 6248 4795

www.arcsight.com

Documents

S N58: ArcS ight, Monitor Thyself€¦ · ArcSight Infrastructure What to monitor? Availability – Monitor critical devices – Monitor ArcSight connectors, appliances, ESM Performance