© Copyright 2014 Hewlett -Packard Development Company, L.P ... · 4© of 36 . Experian Public....

© 2014 Experian Information Solutions, Inc. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Information Solutions, Inc. Other product and company names mentioned herein are the trademarks of their respective owners. No part of this copyrighted work may be reproduced, modified, or distributed in any form or manner without the prior written permission of Experian. Experian Public.

Flight of the flightless bumblebee Chad Roberts chad.roberts@experian.com Protect724 ID: deathbywedgie

Anthony Morris anthony.morris@experian.com Protect724 ID: awmorris

Preliminaries

INTENDED AUDIENCE

This *is* an advanced session. We presumed the majority of the audience will have some previous ArcSight development experience.

Greatest benefit will be realized by ArcSight ESM content developers who understand basic Python and regex.

Coding expertise is NOT a prerequisite for benefitting from this session.

QUESTIONS

We have planned this presentation to allow for a dedicated question time at the end. If you have a question during the session, feel free to ask and if it needs a longer answer, we’ll answer at the end.

Feel free to catch us in the hall, over drinks, or email after the conference.

About Us

ABOUT EXPERIAN Experian is a global leader in consumer and business credit reporting and

marketing services. We support clients in more than 80 countries and employ more than 17,000 people in 40 countries. We have 3 instances of ArcSight managers deployed globally as primary/fail-over pairs. We have 2 additional application specific managers and 1 development manager.

ABOUT CHAD Chad is the manager of the Defensive Security Services team in Experian. He

has been in the information security field for more than 10 years, has been using ArcSight since version 4.0 (2008), and is a leading contributor on Protect724.

ABOUT ANTHONY Anthony is a Sr. Security Analyst on the Defensive Security Services team in

Experian. He has a Masters degree in Network Security, CISSP and has been developing content in ArcSight since version 4.5 (2009).

Agenda

TOPICS TO BE DISCUSSED INCLUDE:

Emulating $Now with a global variable (Package available)

Aggregating over long periods of time

Using an active list as a properties file (Flex Connector and package available)

Using actions to find random generated domain names (Code available)

How to use evaluateVelocityTemplate variable

BONUS: Managing multiple ESM consoles (Code available)

Emulating $Now with a global variable

THE PROBLEM

ArcSight ESM has a certain internal event that is generated once every minute. Using this event as a trigger, we used a lightweight rule to store the manager receipt time to an active list. We then created a global variable that reads that value from the active list. This effectively simulates the $Now variable and makes it accessible to our rules/queries.

When manipulating variables inside of queries and rules, ArcSight limits acceptable choices to standard fields or global variables. This makes the $Now variable inaccessible in some situations where we needed it.

HOW WE SOLVED THE ISSUE

https://protect724.hp.com/docs/DOC-11266 Package available at:

HYPOTHETICAL SITUATION We store the case number and create time in an active list when the case is first created. We now want to know anytime someone modifies a case that is 3 or more days old.

Emulating $Now with a global variable Slide 2 of 3

Emulating $Now with a global variable Slide 3 of 3

WAYS TO GO TO THE NEXT LEVEL Increase accuracy by choosing a different, more frequent event for populating

the active list

► This comes with a resource cost as the rule fires and the list is updated more frequently

Aggregating events over long periods of time

THE PROBLEM Aggregation counting restarts when the manager is restarted, when a rule is disabled, and when a rule is modified. Additionally, using a very long time frame in rules adds to the number of partial matches.

Lightweight rule adds to multi-

mapped active list

ArcSight event for active list addition

Triggers rule that counts events in

the list

EXAMPLE PROBLEM Find 50 failed logins with the same username in last 7 days

THE MULTIMAPPED ACTIVE LIST

Aggregating events over long periods of time Slide 2 of 7

THE LIGHTWEIGHT RULE

Lightweight so ArcSight won’t create a correlated event for every matching base

event.

THE LIGHTWEIGHT RULE (cont’d)

THE RESULTING ACTIVE LIST

THE SECOND RULE- TYING THINGS TOGETHER

THE SECOND RULE- TYING THINGS TOGETHER (cont’d)

WAYS TO GO TO THE NEXT LEVEL Create a dashboard to see counts approach the threshold. Note- this is a

benefit you receive when you aggregate with lists rather than inside a rule.

Figure out how to accommodate events where aggregated event count > 1. We haven’t discovered a clean way to do this yet.

THE PROBLEM We have multiple ESM managers in a global deployment. We wanted to use content synchronization to ensure all content was synchronized across all managers. At the same time, not ALL content pertained to every manager. Was there a way that we could keep our content synchronized but still have it act differently on different managers?

HOW WE SOLVED THE ISSUE Maintain an active list and populate the list with values that tell each manager how to act. The values in the list could be maintained:

Manually

Reading a configuration file through a flex connector

Reading a configuration file and injecting the events via CEF

Using an active list as a properties file Slide 1 of 4

THE ACTIVE LIST SAMPLE ENTRIES ON MGR 1

SAMPLE ENTRIES ON MGR 2

DEFINE GLOBAL VARIABLES TO READ THE VALUES FROM THE ACTIVE LIST

USE THE GLOBAL VARIABLES TO MAKE RULES BEHAVE DIFFERENTLY

Here is the global variable that is being

read from our active list.

WAYS TO GO TO THE NEXT LEVEL Modify the FlexConnector to directly manage the “Content Parameters” active list.

► Manage list without any rules

► See presentation “SN10: Tips and Tricks in Rules & Dynamic Mapping in Connectors” by Raju Gottumukkala from Protect 2010

https://protect724.hp.com/docs/DOC-11452

FlexConnector and content package can be found at:

https://protect724.hp.com/docs/DOC-1405

THE PROBLEM Many viruses and pieces of malware use dynamically or random generated domain names. We wanted a way we could detect use of these domains in our environment.

First rule identifies ‘interesting domains’ and calls action to

Python script

Python script calculates

‘entropy’ and returns via CEF

Second rule triggers on thresholds

Using actions to detect random generated domains Slide 1 of 5

FIRST RULE

Domains that don’t have a category

assigned by the proxy

ASSUMPTIONS Typical romantic languages (English, Spanish, French) only have a limited

amount of consecutive non-vowel characters. Long strings of consonants are atypical within strings of words.

Random domain names would not be comprised of known dictionary words.

THE ALGORITHM Step 1: Calculate the longest character string without vowels. Step 2: Remove all known words 4 characters or more (and numbers). Step 3: Calculate the new length and determine ratio to original length.

Original string www.yellowbitsfroggedmyhorn.com www.35hwe89dcaubjkwlcvrij8.com

Longest consonant string www.yellowbitsfroggedmyhorn.com [4] www.35hwe89dcaubjkwlcvrij8.com [8]

Remove known words > 3 chars and numbers www.yellow bits frogged my horn.com www.35hwe89dcaubjkwlcvrij8.com

Ratio of reduced string to original string round(13 / 31 * 100) = 42 round(25 / 30 *100) = 83

EXAMPLE

HOW THE LOGIC PERFORMS IN THE REAL WORLD (Ratio>80, Consonants>3)

https://protect724.hp.com/docs/DOC-11434 Starter Python script can be found at:

WAYS TO GO TO THE NEXT LEVEL For this use case:

► Improve the algorithm to measure entropy.

► Tie the domain activity back to the original activity

● Create thresholds so analysts don’t investigate individual results (e.g. A single user querying multiple high entropy domains)

► Perform other functions (e.g. look up date registered, registrar, etc.)

● Requests to resolve high entropy domains that aren’t registered

● Multiple high entropy domains that resolve to the same IP etc.

Don’t limit use cases to domain information only:

► Download executable files and compare the MD5 against VirusTotal

How to use evaluateVelocityTemplate variable Slide 1 of 7

THE PROBLEM Traditional variables can be tedious, since most types of variables perform only a single function. Collecting the data you need may take a series of many variables, which is often time-consuming and complex.

Familiarize with Java methods and use the variable type “evaluateVelocityTemplate” to add a great deal of flexibility while reducing the number of variables.

From the ArcSight ESM 6.5c User’s Guide (though available in earlier releases too):

Protip: LOTS of tools have embraced Apache Velocity, so learning how to use this is a great investment!

To parse large strings, use the evaluateVelocityTemplate function:

Comparing traditional variables with the same example

4 variables:

1 variable:

Stacking methods to do more in a single variable

It is possible to append additional java methods in a single evaluateVelocityMacro variable to do even more!

Make uppercase:

$name.replaceAll('.*?to=<(.+)>.*','$1').toUpperCase()

Replace Characters:

$name.replaceAll('.*?to=<(.+)>.*','$1').toUpperCase().replace('@','_')

(and so on…)

More power!

Functionality not available with other variable types

Many variables do not work correctly when a field contains multiple lines of data. Use “replaceAll” method with \r and \n to manipulate multiline input and process as a single continuous line

► $deviceCustomString5.replaceAll('\r','').replaceAll('\n','')

► Notice in this example we again chain functions together.

$deviceCustomString3.hashCode().toString().replaceFirst('^0$','')

WAYS TO GO TO THE NEXT LEVEL

Consider all the Java methods available to you

► Start here for a list of available “String” methods: http://docs.oracle.com/javase/7/docs/api/java/lang/String.html

Get crazy with it

► #set( $recipient = $name.replaceAll('.*?to=<(.+)>.*','$1') ) #set( $recipientHash = $recipient.hashCode() ) #set( $delay = $name.replaceAll('.*?delay=(.+?),.*','$1') ) Recipient: $recipient Delay: $delay Recipient Hash: $recipientHash

BEFORE

THE PROBLEM Running multiple ArcSight ESM managers to multiple systems can be confusing because the program name is truncated on the system bar. We need the most significant part of the manager name to be first. Better yet, we wanted a friendly name that matched our conversation.

BONUS: Managing multiple ESM consoles Slide 1 of 4

To use, simply enter the manager name into the properties file and the preferred common name next to it.

Run the executable (or Python script) to enjoy the results.

How it was accomplished and how can you do it too?

► Using ctypes in Python, we gained access to the USER32.DLL functions.

● ctypes is a foreign function library for Python. It provides C compatible data types, and allows calling functions in DLLs or shared libraries.

● Ref: https://docs.python.org/2/library/ctypes.html

► Main functions called from USER32.DLL

● EnumWindows

● GetWindowText/SetWindowText

► If you wish to use this yourself, the Python code has been uploaded to the Protect724 website.

● Located at https://protect724.hp.com/docs/DOC-11085

WAYS TO GO TO THE NEXT LEVEL

The script provided only works on Windows. Consider making the script aware of the operating system so it works cross-platform.

Thank you!

QUESTIONS?

Chad Roberts chad.roberts@experian.com Protect724 ID: deathbywedgie

Anthony Morris anthony.morris@experian.com Protect724 ID: awmorris

Your feedback is important. Please take a few minutes to complete the session survey.

Emulating $Now

Aggregating via multi-mapped lists

Using an active list as a properties file

Actions to external scripts that return CEF

Using evaluateVelocityTemplate variable

Python to rename the console window

All packages/code can be found on Protect724 by searching for “TB3111”

Tonight’s party

Time 7:00 – 10: 00 pm Shuttles run between hotel’s Porte Cochere (Terrace Level, by registration) and Newseum from 6:30 - 10:00 pm Questions? Please visit the Info Desk by registration

@ Newseum Enjoy food, drinks, company, and a private concert by Counting Crows

Please fill out a survey. Hand it to the door monitor on your way out.

Thank you for providing your feedback, which helps us enhance content for future events.

Session TB3111 Speakers Chad Roberts and Anthony Morris

Please give me your feedback

© Copyright 2014 Hewlett -Packard Development Company, L.P ... · 4© of 36 . Experian Public....

Documents

©2012 Experian Limited. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Limited. Other products

Data Sets Guide - Experian · Datasets Guide | Page 13 Data sets guide Experian National Business Database Experian National Business Database Business data The Experian National

©2012 Experian Information Solutions, Inc. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian

©2014 Experian Information Solutions, Inc. All rights reserved. Experian Confidential

© 2011 Experian Information Solutions, Inc. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian

© 2010 Experian Information Solutions, Inc. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian

©2015 Experian Information Solutions, Inc. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian

Explaining Experian - Experian plc - Home

#DeathToPasswords - Aventri€¦ · Emerging need to recognize PEOPLE and their PROXIES = 26 ©Experian 4/19/2017 Experian Public Vision 2017 …broadcasting who and where we are,

How to Read an Experian Credit Report - Find Answers · How to Read an Experian Credit Report June 2014 FADV0036 Introduction An Experian Credit Report provides information about

Experian Annual Report 2020 - Experian plc - Home

© 2009 Experian Limited. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Limited. Other

Experian is a registered trademark of Experian Information Solutions, Inc. © © Experian Information Solutions, Inc. 2001 Confidential and proprietary -

State of the Automotive Finance Market Third Quarter 2015©2015 Experian Information Solutions, Inc. All rights reserved. 8 Experian Public. Leasing and consumer credit 718 715 626

© Experian Limited 2008. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Limited. Other

© Experian Information Solutions, Inc. 2007. All rights reserved

© Experian CheetahMail 2007. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian CheetahMail

Roma, 31 Gennaio 2008 - e-meffea.net file© Experian Limited 2007. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Limited

1 ©2014 Experian Limited. All rights reserved. Experian Public. Department for Business, Innovation and Skills Analysis of Cash Retention Behaviour of

Test First yes we can - ASQF€¦ · © 2012 Experian Limited. All rights reserved. 1 Experian Public. Test First – yes we can ! ASQF Regionalgruppentreffen Agilität NRW Köln,