IBM Security Guardium Tech Talk · 2016-04-07 · © 2015 IBM Corporation 2 Agenda Overview Deployment basics Sample policy showing what’s new in V10 Demo

© 2016 IBM Corporation

Guarding against insider threats for Hadoop: What’s new in Guardium

Sundari Voruganti

Big Data QA Lead and Solutions Architect

IBM Security Guardium

Kathy Zeidenstein

Evangelist and Community Advocate

IBM Security Guardium

IBM Security Guardium Tech Talk

2© 2015 IBM Corporation

Agenda

Overview

Deployment basics

Sample policy showing what’s new in V10

Demo


Organizations are Jumping into Big Data with Both FeetAre you ready for the headache?

• Departmental projects

• Rogue IT teams

• Using production data

• Loose user controls

• Difficult to audit

http://www.upperleftedge.com/wp-content/uploads/2012/11/us-capitalism.jpg

http://www.upperleftedge.com/wp-content/uploads/2012/11/us-capitalism.jpg


Security challenges

• Insider threats

• Large attack surface with

multiple entry points

• Emerging applications offer new

breach opportunities

• Complex security models to

support need-to-know

• Inconsistent tools across

vendors

Data marts, operational

data stores, etc

Big data platform

Privileged insiders

Hackers/organized

crime

Diverse users,

applications


Insider threat: What’s on the inside counts

Damaging security incidents involve loss or illicit modification or destruction of sensitive data

Many security programs only focus on what’s happening beyond the

perimeter

**Source: 2Q15 X-Force Report

55% of all attacks are caused by insider threats**

https://www-01.ibm.com/marketing/iwm/dre/signup?source=swg-WW_Security_Organic&S_PKG=ov36741&S_TACT=C406003W&S_OFF_CD=10000203&lang=en_US


Complexity

•Many avenues to access

•Security and authentication is

evolving

•Complex software stack with

significant log data from each

component

•Security and audit viewed in

isolation from rest of data

architecture


Simplify Data Security with IBM Security Guardium

Guardium Data Collection

IBM Guardium S-TAP Probes

Protect critical data

from a single console

Big Data

• Support both traditional and

disruptive technologies

(Hadoop, NoSQL and Cloud)

• Expand security capability from

mere compliance to

comprehensive data protection

• Reduce costs and improve

results using a single, scalable

infrastructure


Data security and protection with IBM Security Guardium

Clients

Dashboards

Anomalies

Audit reports

• Deep visibility into Hadoop activity from any entry point

• Who, what, when, and where

• Exceptions, such as authorization and access control

failures

• Real-time alerts reduce time to breach discovery

• Built-in reports to speed compliance

• Dynamic data masking to protect data privacy

• Block suspicious users to prevent breaches

• Common policy-based platform, normalized audit data

• Separation of duties

Privileged users

Guardium Data Collection

IBM Guardium S-TAP Probes

Big Data

Alerts


Guardium helps support the most complex of IT environments …

Examples of supported databases, Big Data environments, files…

Applications Databases

DB2Informix

IMS

Data Warehouses

NetezzaPureData for AnalyticsDB2 BLU

CICSWebSphere

SiebelPeopleSoftE-Business

Database Tools

Enterprise

Content

Managers

Big Data

Environments

Files

VSAMz/OS Datasets

FTP

DB

Cloud

Environments

Windows, Linux,

Unix


DEPLOYMENT BASICS


Hadoop Deployment Guide

For those that are seriously looking at Hadoop deployments with Guardium, we have a deployment guide that is in constant state of change

– Includes much material from this tech talk plus more details and will continue to evolve

If you want a copy, send Kathy Zeidenstein an email with your request. [email protected]

mailto:[email protected]


Questions to ask

Which nodes are the services running on

– Which nodes are masters and which are slaves

Which ports do the components use?

– Are you using default ports?

Which Hadoop distribution?

Are you using kerberos?

Are you using SSL?


Standby NN

S-TAP placement in a Hadoop cluster

DB storeDistributed data processingMapReduce/Yarn

Distributed queryprocessingDistributed data storage

HDFS

Ma

ste

rsS

lave

s

HBase Master

Job Tracker/Resource ManagerNameNode

S-TAP

Clients

S-TAP required on the region servers/compute nodes only for monitoring HBase table operations, Impala, and Big SQL traffic.

Data NodeNode ManagerHBase RegionImpala DaemonBig SQL Worker




Hive Server

TezHAWQ

Big SQL

Landing

zone

Ed

ge

No

de


Which nodes?

Hadoop Component STAP on

Master?

STAP on

Slaves?

Comments

HDFS Yes No

HBase Yes Yes For put, scan, list

Hive Yes No

Impala (Cloudera) Yes Yes For Impala Daemons

Yarn Yes No

Big SQL(IBM

BigInsights)

Yes Yes To be fully secure

Solr Yes Yes If using SolrCloud

Plus S-TAP on Edge Node for any components used there.

Example: If monitoring only HDFS and Hive, only S-TAP on Master and Edge node

needed

Example: If adding Impala, add S-TAP to each data nodes/slave as well.


Hadoop

Client

Guardium

CollectorSniffer

Hadoop fs –mkdir /user/data/sundari

Hadoop fs –mkdir ….

Sessions

Commands

Objects

Read Only Hardened Repository

(no direct access)

Hadoop commands

mkdir

Joe /user/data/sundari

Parse commands

then log

Joe

Namenode

S-TAP

Hadoop fs –mkdir …

Hadoop fs –mkdir/user/data/sundari

Under the covers


Partial list of inspection engine configurations

DB Protocol Port Node What is being monitored?

HADOOP 8020 Master HDFS

WEBHDFS 50070 Master WebHDFS

HIVE 10000 Master HiveServer2

HADOOP 9083 Hive Master Hive metastore

HIVE 21050 Master For Impala from Hue

Hadoop 60000 HBase Master Hbase commands that go through

master

Hadoop 60020 HBase Region

Servers

HBase table commands

IMPALA 21000 Data Impala shell

HUE 1521 Master Oracle Hue Backend

HUE 3306 Master MySQL Hue Backend

HUE 5432 Master PGSQL Hue backend

See Deployment Guide for more, including BigSQL, Solr, etc


One possible strategy for testing/deployment

Mandatory to monitor HDFS- you’ll need to determine what else you want to

monitor based on your auditing requirements

HDFS is simplest to configure and validate (STAP only needed on NameNode and

secondary NN)

Add workload and monitor impact on the collectors. Use filtering when possible.

– No Ignore Session policy rule for Hadoop! .

Add more inspection engines gradually, validate results, and monitor impact on the

collectors.


Sizing – rule of thumb!

Current rule of thumb based on deployments that are not high

volume in terms of what is being audited

– 10 management/server nodes per collector,

– 20+ datanodes per collector, assuming STAPs are needed for the data

nodes (They are not needed for all components)

– Possibly even more nodes per collector if if physical appliances are used

Your sizing may vary.


SAMPLE POLICY: HIGHLIGHTING V10 ENHANCEMENTS


Example policy


Failed Logins - Inspection Engine – Oracle backend

Defining the inspection

engine for the Hue

metastore.

Exception report

showing failed logins

from Hue.


Full visibility into activity

Details are captured in

Guardium.

Data displayed in Excel

Creating a query using ODBC

connection to Hive


Hive Beeline queries (JDBC)

Command line query

Details captured in Guardium


Prevent brute force attack

svoruga@rh6-cli-06:>Hadoop fs –mkdir /user/dgundam/test

mkdir: Permission denied: user=svoruga…

Capture login failures and set

alerts when they exceed policy

Capture HDFS permission

failures


Block inappropriate access

Policy: Block privileged user access to customer data through Hive1

2Privileged user attempts to read customer data and is blocked

3Access attempt is reported as a policy violation

Send real time alerts to your favorite SIEM – QRadar, Splunk, HP Arcsight…


Protect data privacy

Masked Hive data in Hue/Beeswax

Masked Hive data command line

Guardium policy examines

result data for sensitive data

patterns


Pre-built Reports


Analytics and search

Enterprise search – centralize analysis of audit data with federated

searches

– facilitates gathering insights on data traffic from across the enterprise.

Investigation Dashboard – Pivot like facility to extract data activity insights

from heat maps

– focus on any specific context :specific data source, user, date, etc

– reveal patterns, anomalies, and relationships across your data

Outlier Detection – new scorer and performance improvements

Activity Animation - graphically play back access traffic

Find a needle in a haystack Normalized views

Extends our lead in providing the most comprehensive analytical tools for data security

Replay last

48 hours

Size = amount of data

Detail drill down


Outlier detection

Outliers show up as triangle and square

icons against the volume of activity. Hover

on an outlier to get more detail.

The outliers tab in the

detail area of this screen

includes the anomaly score

and why something was

labelled an outlier

Drilldown are available to view the detailed

activities, full SQL commands, error details

and to navigate to related pivot-views


Guardium for Big Data

Gain visibility and insight while protecting sensitive data

No other vendor offers the breadth of support for Hadoop and NoSQL data management systems

Platform/Feature Hadoop MongoDB Cassandra

Activity monitoring

and reporting

Yes Yes Yes

Real-time alerting Yes Yes Yes

Data redaction Yes* Yes Yes

Blocking Yes* Yes Yes

Outlier detection Yes Yes Yes

Quick search (ad-hoc

analysis)

Yes Yes Yes

*For Hive, Impala, and BigSQL

Visit: ibm.com/guardium


DEMO


Q&A


133 countries where IBM delivers

managed security services

20 industry analyst reports rank

IBM Security as a LEADER

TOP 3 enterprise security software vendor in total revenue

10K clients protected including…

24 of the top 33 banks in Japan,

North America, and Australia

Learn more about IBM Security

Visit our web page

IBM.com/Security

Watch our videos

IBM Security YouTube Channel

Read new blog posts

SecurityIntelligence.com

Follow us on Twitter

@ibmsecurity

IBM.com/Security/CISO

IBM.com/Security

https://www.youtube.com/channel/UClAgZm2OXFpX8WoMsOpWoXA

http://securityintelligence.com/

http://www.twitter.com/ibmsecurity

© Copyright IBM Corporation 2016. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any

kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor

shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use

of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and / or

capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product

or feature availability in any way. IBM, the IBM logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries

or both. Other company, product, or service names may be trademarks or service marks of others.

Statement of Good Security Practices: IT system security involves protecting systems and information through prevention, detection and response to improper access from within and outside

your enterprise. Improper access can result in information being altered, destroyed, misappropriated or misused or can result in damage to or misuse of your systems, including for use in attacks

on others. No IT system or product should be considered completely secure and no single product, service or security measure can be completely effective in preventing improper use or access.

IBM systems, products and services are designed to be part of a lawful, comprehensive security approach, which will necessarily involve additional operational procedures, and may require other

systems, products or services to be most effective. IBM DOES NOT WARRANT THAT ANY SYSTEMS, PRODUCTS OR SERVICES ARE IMMUNE FROM, OR WILL MAKE YOUR ENTERPRISE

IMMUNE FROM, THE MALICIOUS OR ILLEGAL CONDUCT OF ANY PARTY.

THANK YOUwww.ibm.com/security

http://www.ibm.com/security

http://www.ibm.com/security


Backup

Documents

IBM Security Guardium Tech Talk · 2016-04-07 · © 2015 IBM Corporation 2 Agenda Overview Deployment basics Sample policy showing what’s new in V10 Demo