How to Write HP ArcSight FlexConnectors

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

How to Write HP ArcSight FlexConnectorsAaron Kramer, CISSP, CEHJune 25, 2014Aaron.Kramer@hp.com

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2

Agenda

• Logistics• SmartConnector Capabilities• FlexConnectors• Sources of Help• FlexConnector Toolkit Demo• Questions and Answers

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.3

Logistics

• Lots of Material• Submit Questions via the Questions section in this Virtual Room• My background

– Over 20 years in Network, Application, and Computer Security (CISSP, CEH)– currently a Global Field Support Engineer for HP ArcSight– >9 years of ArcSight experience– Presented at previous ArcSight Protect User Conference on FlexConnectors, Logger,

ESM– Regular contributor to the Protect724 User Community

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

SmartConnector Capabilities

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.5

SmartConnector Event Collection

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.6

SmartConnector Event Collection

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.7

Connectors: Robust Collection

Follows NIST 800-92 Log Aggregation Guidelines

HP ArcSight Logger HP ArcSight ESM/Express

ArcSight Connector

Centralized Updates/Upgrades

Bandwidth ManagementHeartbeat Connection

Encrypted & Compressed

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.8

My device or Application or Source is NOT one of the 350+ listed Supported sources?

Why FlexConnectors?

Enter the HP ArcSight FlexConnector

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

HP ArcSight FlexConnectors

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.10

HP ArcSight FlexConnectors

• Has same capabilities as SmartConnectors (Caching, Batching, Compression, etc)

• The FlexConnector Developer Toolkit is the same Toolkit that Developers use to write the 350+ SmartConnectors

• The Toolkit is a fully-supported and documented offering

• The FlexConnector Development Kit is a licensed item; must be purchased.

• GOAL: Produce the Properties File

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.11

HP ArcSight FlexConnectors can be written for various Files and Formats and Sources

FlexConnector Types

• Regularly-formatted Files

• Files processed better by use of Regular Expressions

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.12

FlexConnector Types

JSON Files XML Files

HP ArcSight FlexConnectors can be written for various Files and Formats and Sources

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.13

FlexConnector Types

Read from Databases Various sources in a Syslog Stream

HP ArcSight FlexConnectors can be written for various Files and Formats and Sources

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.14

FlexConnector Types

SNMP Over a RESTful API

• REST API endpointshttps://abc.com/events?created_after=<>&maxEvents=<>...

• JSON output• OAuth2

HP ArcSight FlexConnectors can be written for various Files and Formats and Sources

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.15

Where do FlexConnectors Run?

• Windows• Linux• Solaris• AIX• Connector Appliance

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.16

ArcSight FlexConnector Wizard on Connector Appliance

Step 1 Upload

the Device Log File

Step 2 Parse the

Device Log File

Step 3 Generat

e the FlexConnector

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.17

flexagentwizard regex

Flex Connector Helpers

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.18

ArcSight Flex Connectors• GOAL: Produce the Properties File, with 3 sections– Parsing– Tokens, types, and formats– Mapping

• Delimited File: Delimiter, Tokens, Mappings• Regular Expression: Words, Tokens, Mappings• Database: Query, Tokens, Mappings•… and so on

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.19

HP ArcSight FlexConnector Further Capabilities• Follow File Rotations and Follow Folders• Consume Multiline events• Advanced functions to parse, manipulate, convert• __concatenate(String1,String2….)

• __extractNTDomain(“AMERICAS\WABC123”) yields AMERICAS

• __regexToken(TOKEN,regex)

• __simpleMAP(TOKEN,Case1,Case2…)

• __safeToLong(TOKEN)

• Lots n lots more

• Pull pieces of filename or filepath

• Chaining – where one type of Flex Connector calls another– A text file of events, where each event has a field that is XML in structure

• Can read compressed files directly

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.20

Sources of Help

• Documentation– FlexConnector Developer’s Guide– REST FlexConnector Developer’s Guide

• http://Protect724.HP.Com Forums– Question and Answer– Previous HP Protect Content

• Tech Support – supporting the FlexConnector Developer Toolkit, not the FlexConnector itself

• HP Partners• HP ArcSight Education

– 3-day course• HP ArcSight User Gatherings

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

How To Write a FlexConnector

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.22

How To Write a FlexConnector

• Confirm that your organization is licensed for the FlexConnector Developer Toolkit• Research to see if a FlexConnector was already written by someone else, somewhere

else• Gather Sample Log events and/or files• Decide on which FlexConnector is best (File reader, database reader, Syslog subagent)• Consult the FlexConnector Developer Guide for step-by-step example

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Switch to Live Demo

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Questions and Answers

For more information, and future webinars, please visit:

https://protect724.hp.com/community/events/enterprise-security-webinars

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Thank you