October 17, 2017
WarrantyThe only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statementsaccompanying such products and services. Nothing herein should be construed as constituting an additional warranty.Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.
The information contained herein is subject to change without notice.
The network information used in the examples in this document (including IP addresses and hostnames) is for illustrationpurposes only.
HPE Security ArcSight products are highly flexible and function as you configure them. The accessibility, integrity, andconfidentiality of your data is your responsibility. Implement a comprehensive security strategy and follow good securitypractices.
This document is confidential.
Restricted Rights LegendConfidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use or copying.Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and TechnicalData for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.
Copyright Notice Copyright 2016, 2017 Hewlett Packard Enterprise Development, LP
Follow this link to see a complete statement of copyrights and acknowledgements:https://community.saas.hpe.com/t5/Discussions/Third-Party-Copyright-Notices-and-License-Terms/td-p/1589228
Phone A list of phone numbers is available on the HPE Security ArcSight Technical SupportPage:https://softwaresupport.hpe.com/documents/10180/14684/esp-support-contact-list
Support Web Site https://softwaresupport.hpe.com
Protect 724 Community https://community.saas.hpe.com/t5/ArcSight/ct-p/arcsight
HPE Connectors Page 2 of 241
10/17/2017 l Updated "Set Global Parameters" section to include encryption parameters.
l Updated information for downloading SQL Server JDBC drivers.
l In Appendix E: ArcSight Built-in Event Field Mappings, the ArcSight Mappingsfields have been changed to camel case.
l Only non-blocking I/O is available for syslog connectors; therefore, thetcpmaxidletime, tcpsetsocketlinger, and tcppeeerclosedchecktimeout parametersare no longer relevant and have been removed from the Advanced Parametersappendix.
l As flexString fields are for the use of customers, examples have been updated toshow deviceCustomString or deviceCustomNumber fields rather than flexStringfields.
05/15/2017 l Added a notice about ODBC connections not being supported after release 7.2.1 tothe "ArcSight FlexConnector ID-Based Database", "ArcSight FlexConnector MultipleDatabase", and "ArcSight FlexConnector Scanner Database" sections.
02/15/2017 l Added JSON to the list of available extra processors. See "Extra Processors".
l Clarified the configuration file names and locations for vulnerabilities, open ports,and URIs for scanner FlexConnectors for normal text reports. See "GettingVulnerabilities for Scanned Hosts", "Getting Open Ports on Scanned Hosts", and"Getting OS and Applications (URIs) on Scanned Hosts".
11/30/2016 l Updated installation procedure for setting preferred IP address mode. UpdatedFlexConnector information for IPv6-aware parsers.
08/30/2016 l Reorganized and expanded content for increased usability.
l Updated the "Configure the JDBC Driver and Windows Authentication section.
l In "Advanced Parameters", updated information regarding preservestateparameters.
06/30/2016 l Added parameters to Parameters Common to all SmartConnectors.
HPE Connectors Page 3 of 241
05/16/2016 l Updates and clarifications in the Log Rotation Types section. Added a section onthe unparsed events detection feature: Unparsed Events Detection.
l In "Advanced Parameters", clarified the descriptions of several advancedparameters.
03/31/2016 l Added advanced parameters to customize connector behavior as Chapter 5.
l In "Advanced Parameters", noted that for Syslog connectors, thepersistenceinterval parameter must be a positive integer to enable persistence.
l In "Advanced Parameters", noted that the rawlogfolder and usefilequeueparameters cannot be applied to Syslog Pipe/File Connector.
l Added configuration properties for JSON Folder Follower FlexConnector.
l Noted that only one question mark is supported for time-based databaseFlexConnector queries.
l Removed agents[x].maxfilesize parameter.
02/15/2016 l End of life for FlexConnector SNMP (install the SmartConnector for SNMP Unified).
l Added the new feature to detect and log unparsed events.
l Updated the time format for __parseMutableTimeStamp function.
l Updated the wildcard parameter default value to use *.
HPE Connectors Page 4 of 241
Chapter 1: Overview 13
FlexConnector Development 13IPv6-Aware Parsers 14
Event Fields 14Operations 14Developer Considerations 15
Folder Structure 15Key Files 16
FlexConnector Management 16ArcSight Connector Appliance 16ArcSight Management Center 17
Chapter 2: Choose a FlexConnector Type 19
FlexConnector Types 19
Event Data Format Examples 21Log File FlexConnector 22ID-Based Database FlexConnector 22JSON Folder Follower FlexConnector 22Multiple Database FlexConnector 23Regex FlexConnectors (Variable-Format File FlexConnectors) 24Scanner FlexConnector 24SNMP FlexConnector 25Syslog FlexConnector 27Time-Based Database FlexConnector 27XML File FlexConnector 28
Chapter 3: Install and Configure the FlexConnector 30
FlexConnector Installation 30Install Core Software 30Set Global Parameters (Optional) 31Select Connector and Add Parameter Information 32
ArcSight FlexConnector File 33ArcSight FlexConnector ID-Based Database 34ArcSight FlexConnector JSON Folder Follower 36
HPE Connectors Page 5 of 241
ArcSight FlexConnector Multiple Database 37ArcSight FlexConnector Multiple Folder File 40ArcSight FlexConnector Regex File 41ArcSight FlexConnector Regex Folder File 42ArcSight FlexConnector REST 43ArcSight FlexConnector Scanner Database 44ArcSight FlexConnector Scanner Text Reports 47ArcSight FlexConnector Scanner XML Reports 49ArcSight FlexConnector XML File 50ArcSight FlexConnector Simple Network Management Protocol (SNMP Unified) 51ArcSight FlexConnector Syslog 52
Select a Destination 53Complete Installation and Configuration 53
Additional Configuration for Database Connectors 54Install SQL Server JDBC Driver 54Install MySQL Driver 56Add a JDBC Driver to the Connector Appliance/ArcSight Management Center 57Configure the JDBC Driver and Windows Authentication 57Oracle 8i Support 58Troubleshooting Duplicate Events 59
Example 1: ID-based Database Connectors Only 59Example 2: ID-based and Time-based Connectors 60Example 3: Complex Main Query with a Join 60
Chapter 4: Create a Configuration File 62
Parser File Locations and Names 62
Example Parser File 63
Parser File Structure 64Token Declarations 65Token Types 66Event Mapping 66RequestUrl Event Field 66Operations Table 67Severity Mapping 68
Examples 68Extra Processors 69Key-Value Parsers 71
FlexConnector Creation Wizard for Delimited Log Files 72
HPE Connectors Page 6 of 241
Regex Tool for Regex FlexConnectors 75
Start the FlexConnector 78
Chapter 5: Configuration File Examples 79
Configuration Properties for a Log File FlexConnector 79
Configuration Properties for all Regex FlexConnectors 80
Configuration Properties for a Time-based Database FlexConnector 81Version 81Query 82Timestamp 83UniqueID 83
Configuration Properties for an ID-based Database FlexConnector 83Version 84MaxID 84Query 84ID 85UniqueID 85Query Limit 85
Configuration Properties for an SNMP Connector 85
Configuration Properties for an XML FlexConnector 87Namespace 88Hop Nodes 88Trigger Nodes 88Token Mappings 89
Examples of Token Mappings 89Extra Events 90
Configuration Properties for a JSON Folder Follower FlexConnector 90Trigger Node 92Token Location and Mappings 92JSON Parsers for Complex Event Schemas 92
Working with Hierarchical Schemas 92Representing a JSON Array with a Key Element 94Representing a Token Value in URI Format 95Sample JSON Array 96
Configuration Properties for Scanner FlexConnectors 96Scanner FlexConnectors for Normal Text or XML Scan Reports 96How Scanner FlexConnectors Parse Scan Reports 97
HPE Connectors Page 7 of 241
Parser Files for Normal Text Reports 97Getting a List of Hosts 98
Ignore or Include Line 99Regular Expression and Token Mappings 99Use IP 100Invalid Vulnerabilities 100Extra Events 100
Getting Vulnerabilities for Scanned Hosts 101Token Mappings 102Event Mappings 102Severity Mappings 103Ignore or Include Line 103
Getting Open Ports on Scanned Hosts 104Token Mappings 105Event Mappings 105Ignore or Include Line 105
Getting OS and Applications (URIs) on Scanned Hosts 106Token Mappings 106Event Map