HoneySpider Network 2.0detecting client-side attacks the easy way

Paweł Pawlinski

CERT Polska / NASK

24th Annual FIRST Conference21 June 2012

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 1 / 23

Outline

1 Introduction

2 Architecture

3 Services

4 Demonstration

5 Future plans

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 2 / 23

Introduction

Outline

1 Introduction

2 Architecture

3 Services

4 Demonstration

5 Future plans

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 3 / 23

Introduction

Origins of HSN 2.0

Joint projectCERT PolskaNCSC-NL (GOVCERT.NL)

Started in 2011Successor to HoneySpider Network version 1.x

used in production by CERTswe gained experience in scanning web pages automatically

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 4 / 23

Introduction

Project goals

Detect attacks on client applicationsweb pagesfiles

Apply multiple analysesPDF, SWF, JavaScript, . . .low and high interaction honeypots

Configurable (processing details)Scalable (crawling)Open architecture

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 5 / 23

Introduction

Project goals

Detect attacks on client applicationsweb pagesfiles

Apply multiple analysesPDF, SWF, JavaScript, . . .low and high interaction honeypots

Configurable (processing details)Scalable (crawling)Open architecture

version 1

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 5 / 23

Architecture

Outline

1 Introduction

2 Architecture

3 Services

4 Demonstration

5 Future plans

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 6 / 23

Architecture Overview

HSN: 1.x vs 2.0

Framework

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 7 / 23

1.x 2.0

Architecture Overview

Architecture overview

Reporting

Web GUI

Alerts

CLI

Report DB

Operational

Framework

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 8 / 23

Architecture Overview

Technical foundations

Network communicationAdvanced Message Queueing ProtocolGoogle Protocol Buffers

StorageCouchDBJSON documentsoperational data + flexible mapping → persistent reports

Programming languagesJavaPython(C++)

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 9 / 23

Architecture Configurability

Sample workflow

Jobstart

parameter A= "some value"

...

accepted

rejected

yesno

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 10 / 23

Architecture Configurability

Sample workflow

Jobstart

parameter A= "some value"

...

accepted

rejected

yesno

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 10 / 23

Architecture Configurability

Sample workflow

Jobstart

parameter A= "some value"

...

accepted

rejected

yesno

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 10 / 23

Services

Outline

1 Introduction

2 Architecture

3 Services

4 Demonstration

5 Future plans

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 11 / 23

Services Implemented services

Web client emulators

HtmlUnit-based custom browser emulatorimplemented in Javauses Rhino enginecomplete control over all behaviors(requests, redirects, frames)link extraction

Thug (low interaction honeypot)implemented in Pythonuses V8 engineless controldetects common attacks

These are not crawlers!

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 12 / 23

Services Implemented services

Analyzers

Static JavaScript analyzerport from version 1n-grams + Bayes classifier

SWF analyzer (NASK)Shellcode detection (scdbg)Cuckoo SandboxCapture-HPC

high-interaction honeypotused in HSN 1.xnew features and stability fixes

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 13 / 23

Services Implemented services

Utilities

Feederfile with URLssearch engine results. . .

URL normalizerReporter (persistent data)

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 14 / 23

Services Razorback integration

Razorback: short introduction

Modular IDSData acquisition decoupled from offline analysesDispatcher: routes dataNuggets (services)

collection (Snort, SMTP, . . . )analyzersenrichment (DNS, . . . )

SQL databaseGUI

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 15 / 23

Services Razorback integration

Razorback: short introduction

Modular IDSData acquisition decoupled from offline analysesDispatcher: routes dataNuggets (services)

collection (Snort, SMTP, . . . )analyzersenrichment (DNS, . . . )

SQL databaseGUI

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 15 / 23

Services Razorback integration

Razorback analyzers

Universal Razorback-to-HSN 2.0 adapterOnly recompilation required, no changes to source codeTested nuggets:

swfScannerpdfFoxclamavNuggetofficeCatvirusTotalarchiveInflate

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 16 / 23

Services Extensibility

Extensibility

Open communication protocolWell-defined data contract for each serviceOpen technologies: AMQP, protobuf, REST, JSONLibraries provided for Java and Python

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 17 / 23

Demonstration

Outline

1 Introduction

2 Architecture

3 Services

4 Demonstration

5 Future plans

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 18 / 23

Demonstration

Demonstration

. . .

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 19 / 23

Future plans

Outline

1 Introduction

2 Architecture

3 Services

4 Demonstration

5 Future plans

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 20 / 23

Future plans

Current state of HSN 2.0

All essential components implementedframeworkstorageweb client

Growing set of analyzersFunctional web interfaceMore tests and stabilization needed

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 21 / 23

Future plans

Future plans

Release as open source (soon!)Improve management of the whole systemMore analyzers

integrate existing toolsanalysis of sandbox dataalternative web clients (high-interactive?)looking for more ideas!

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 22 / 23

Thank you for your attention.

Questions?

Paweł Pawlinski (CERT Polska / NASK) HoneySpider Network 2.0 FIRST 2012 23 / 23