HP ArcSight ESM and HP IDOL for Security Analytics

Table of contentsAnalyzing massive datasets in the security domain 2

HP ArcSight ESM and the Common Event Format 2

Sample use case Correlating structured and unstructured data using HP ArcSight ESM and HP IDOL 2

Social media monitoring for negative sentiment and insider threat 3

Social media monitoring for hacktivist threats 5

About HP 6

Technical white paper

HP ArcSight ESM and HP IDOL for Security Analytics

2

Analyzing massive datasets in the security domainThe ability to analyze massive amounts of datamdashboth structured and unstructuredmdashhas quickly become all but mandatory for large organizations that must effectively protect themselves against malicious activity This exponential growth in available data has created many challenges particularly in terms of analysis and actionable intelligence in the security domain It has also created a unique opportunity for developers to look beyond the basic functionality of their application and develop apps that are integrated more seamlessly into the world of big data

Todayrsquos security analysts typically rely on tools such as Security Information and Event Management (SIEM) systems and log management solutions both of which focus primarily on the collection and correlation of real-time audit logs from network devices operating systems and applications By utilizing the tools available in the HAVEn platform such as the Common Event Format developers can greatly enhance the usefulness of their applicationrsquos audit data to help enable security analysts to detect threats

However even with increased interoperability there is a growing need to complement these solutions with more extensive analytics to identify anomalies and other suspicious activity as attacks become increasingly sophisticated Combining an SIEM like HP ArcSight ESM with HP IDOLrsquos unstructured data analytics capabilities together with applications that have been designed with interoperability in mind allows organizations to gather the actionable security intelligence necessary in todayrsquos complex threat landscape

HP ArcSight ESM and the Common Event FormatHP ArcSight ESM is the premier security event manager that enables organizations to store analyze and correlate millions of events for security event monitoring from compliance and risk management to security intelligence and operations ArcSight ESM sifts through millions of log records correlates them and provides identity and asset context to find the critical events that matter in real time via dashboards notifications and reports enabling you to accurately prioritize security risks and compliance violations

The Common Event Format (CEF) is an open log management standard created to simplify log management challenges It uses a standardized format allowing you to easily collect and aggregate data for analysis by an enterprise log management system CEF is an extensible text-based high-performance format designed to support multiple device types and applications in the simplest manner possible Specifically CEF defines a syntax for audit log records comprised of a standard header and a variable extension formatted as key-value pairs This format contains the most relevant event information making it easy for event consumers to parse and make use of the data

Sample use case Correlating structured and unstructured data using HP ArcSight ESM and HP IDOLOne of the many sources of unstructured data available to an organization comes from social media outlets such as Twitter Facebook Instagram LinkedIn GlassDoor and more Traditionally these data streams have been inaccessible by most SIEM platforms however much of this information could provide useful insight into pending threats especially when combined with structured audit data from network devices operating systems and applications

In this use case IDOL is used to analyze unstructured social media data to detect negative sentiment towards an organization When it finds an email or social media thread for example containing negative sentiment IDOL generates an event in the Common Event Format and sends it to ArcSight ESM IDOL can also analyze information to provide additional business context regarding communications going in or out of an organization For example IDOL can analyze the body of an email attachments and recipients to provide additional context beyond the subject line

In addition the integration between ArcSight ESM and HP IDOL is bi-directional An analyst using ArcSight can ask IDOL for additional information regarding an event they have seen in ArcSight and IDOL can send CEF events to ESM This can be a powerful tool in facilitating an analystrsquos investigation into an incident

Technical white paper | HP ArcSight ESM and HP IDOL for Security Analytics

Unstructured data Structured data

HP IDOL HP ArcSight ESM

Alerts ESM to target negative sentiment communications

and threat intelligence

IDOL provides additional business context for suspicious

communications

Display to analyst the full content of communications and

threat intelligence HTTPS HTTPS

API queryCEF

CEF

CEF

Security devices (FW IDS etc)Identity and Access Management

Applications

Email filesSocial media chat sessionsWebsites audiovideo

HP ArcSight ESM and HP IDOL

3

Architecture

Social media monitoring for negative sentiment and insider threatIn this scenario a current employee has posted a review on an employer review website expressing dissatisfaction with their current employer and the possibility of leaving to go work for a competitor IDOL has detected this post and sent an event in CEF to ESM where a correlation rule is triggered and an analyst is alerted to the incident

IDOL CEF EventCEF0|IDOL|IDOL|10|event0001|IDOL Category Event|0|cfp1=7642 cfp1Label=Weight cs2=Recipe cs2Label=IDOL Category cs3=c6c7f0054a7fa385817c13a9605656b6 cs3Label=IDOL MessageID cs4=Negative cs4Label=Sentiment cs5= cs5Label=Sendmail MessageID cs6= cs6Label=FROM cs7= cs7Label=TO cs8= cs8Label=Handle cs9= cs9Label=FacebookProfile cs11=httpwwwemployerreviewcomReviewsEmployee-Review-KobaltSystems-1783htm cs11Label=URL

The analyst can access the URL from the event within ArcSight and pull up the post for more information


4

Unfortunately at this point the analyst does not know who posted this as these reviews are anonymous Using HP ArcSight Identity View and logs collected by ArcSight from the proxy it can be determined that at approximately the same times as this post was created the user jsmith who is a presales engineer visited this exact post

In addition to this IDOL has sent ArcSight an alert identifying a potentially suspicious email containing source code Upon further investigation within ArcSight using logs collected from Exchange the analyst determines that this same user emailed this source code to his personal account Given his earlier posting to the employer review site it may be that this employee is looking to take some code with him if he potentially leaves and joins a competitor Using these tools an analyst can detect this and avoid a potentially devastating scenario


5

Social media monitoring for hacktivist threatsIn this scenario IDOL is configured to crawl a social media feed for threads containing an organizationrsquos name and malicious activities When a thread is detected a CEF event is sent to ArcSight ESM alerting an analyst of this potential DDOS attack along with the posterrsquos handle and the sentiment evaluation of the post

Sample IDOL Configuration File [SocialMedia_Search0] Query=Kobalt Systems LuaConfigureScript=CAutonomyIDOL10SocialMediaConnectorlua_scriptsconfigure_socialmedia_searchlua ConnectionLibrary=connectionSocialMediadll

type=popular

USER=someUser CONSUMER_KEY=xbp0rUbSxM8kSWuSKV1234 CONSUMER_SECRET=OGGnKPVuLinR00GIvEcyrBIvwSzg69Dg3qPb5c1234 ACCESS_TOKEN=891363228-pyKqh02EfuqNgfyu16iUlLDqkr1EbHthjky65vWv ACCESS_TOKEN_SECRET=0F4AXpYJxl3Yhqt8yIApCMiHtOZlVsyGry3YRZf3b4

IndexDatabase=SocialMedia

IDOL CEF Event CEF0|IDOL|IDOL|10|event0001|IDOL Category Event|0|cfp1=9018 cfp1Label=Weight cs2= DDOS cs2Label=IDOL Category cs3=1e2c814ab194af80e0743fac572ba26e cs3Label=IDOL MessageID cs4=Negative cs4Label=Sentiment cs5= cs5Label=Sendmail MessageID cs6= cs6Label=FROM cs7= cs7Label=TO cs8=hakdplnt cs8Label=SocialMediaHandle cs9= cs9Label=FacebookProfile cs10= cs10Label=User Internal Email

The analyst can access the URL from the ArcSight console and see the thread


copy Copyright 2013 Hewlett-Packard Development Company LP The information contained herein is subject to change without notice The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services Nothing herein should be construed as constituting an additional warranty HP shall not be liable for technical or editorial errors or omissions contained herein

20131120_PI_WP_HP_ArcSight_ESM_IDOL_for_Security_Analytics

Share with colleaguesSign up for updates hpcomgogetupdated


Additionally the poster has identified a vulnerable system and posts this information as well

By combining the ability to analyze unstructured data using HP IDOL the powerful correlation engine of ArcSight ESM and the enhanced interoperability made possible by the Common Event Format the analyst now has actionable information to use to potentially block this attack This level of analysis has traditionally not been available to organizations where they relied primarily on SIEM solutions alone but it is now possible using the HP HAVEn platform

About HPHP creates new possibilities for technology to have a meaningful impact on people businesses governments and society The worldrsquos largest technology company HP brings together a portfolio that spans printing personal computing software services and IT infrastructure to solve customer problems

More information about HP (NYSE HPQ) is available at hpcom

2

Analyzing massive datasets in the security domainThe ability to analyze massive amounts of datamdashboth structured and unstructuredmdashhas quickly become all but mandatory for large organizations that must effectively protect themselves against malicious activity This exponential growth in available data has created many challenges particularly in terms of analysis and actionable intelligence in the security domain It has also created a unique opportunity for developers to look beyond the basic functionality of their application and develop apps that are integrated more seamlessly into the world of big data

Todayrsquos security analysts typically rely on tools such as Security Information and Event Management (SIEM) systems and log management solutions both of which focus primarily on the collection and correlation of real-time audit logs from network devices operating systems and applications By utilizing the tools available in the HAVEn platform such as the Common Event Format developers can greatly enhance the usefulness of their applicationrsquos audit data to help enable security analysts to detect threats

However even with increased interoperability there is a growing need to complement these solutions with more extensive analytics to identify anomalies and other suspicious activity as attacks become increasingly sophisticated Combining an SIEM like HP ArcSight ESM with HP IDOLrsquos unstructured data analytics capabilities together with applications that have been designed with interoperability in mind allows organizations to gather the actionable security intelligence necessary in todayrsquos complex threat landscape

HP ArcSight ESM and the Common Event FormatHP ArcSight ESM is the premier security event manager that enables organizations to store analyze and correlate millions of events for security event monitoring from compliance and risk management to security intelligence and operations ArcSight ESM sifts through millions of log records correlates them and provides identity and asset context to find the critical events that matter in real time via dashboards notifications and reports enabling you to accurately prioritize security risks and compliance violations

The Common Event Format (CEF) is an open log management standard created to simplify log management challenges It uses a standardized format allowing you to easily collect and aggregate data for analysis by an enterprise log management system CEF is an extensible text-based high-performance format designed to support multiple device types and applications in the simplest manner possible Specifically CEF defines a syntax for audit log records comprised of a standard header and a variable extension formatted as key-value pairs This format contains the most relevant event information making it easy for event consumers to parse and make use of the data

Sample use case Correlating structured and unstructured data using HP ArcSight ESM and HP IDOLOne of the many sources of unstructured data available to an organization comes from social media outlets such as Twitter Facebook Instagram LinkedIn GlassDoor and more Traditionally these data streams have been inaccessible by most SIEM platforms however much of this information could provide useful insight into pending threats especially when combined with structured audit data from network devices operating systems and applications

In this use case IDOL is used to analyze unstructured social media data to detect negative sentiment towards an organization When it finds an email or social media thread for example containing negative sentiment IDOL generates an event in the Common Event Format and sends it to ArcSight ESM IDOL can also analyze information to provide additional business context regarding communications going in or out of an organization For example IDOL can analyze the body of an email attachments and recipients to provide additional context beyond the subject line

In addition the integration between ArcSight ESM and HP IDOL is bi-directional An analyst using ArcSight can ask IDOL for additional information regarding an event they have seen in ArcSight and IDOL can send CEF events to ESM This can be a powerful tool in facilitating an analystrsquos investigation into an incident







communications



API queryCEF

CEF

CEF


Applications



3

Architecture





4




5



type=popular



















communications



API queryCEF

CEF

CEF


Applications



3

Architecture





4




5



type=popular














4




5



type=popular














5



type=popular






















Data & Analytics

HP ArcSight ESM and HP IDOL for Security Analytics