Latest advance in Suricata IDPS - RMLLschedule2012.rmll.info/IMG/pdf/2012_rmll_suricata.pdf ·...

Latest advance in Suricata IDP S

Éric Leblond

July 9th 2012

Éric Leblond (OISF) Latest advance in Suricata IDP S July 9th 2012 1 / 44

1 IntroductionIntroductionGoals of the projectEcosystem

2 FunctionnalitiesList of functionnalitiesSignatures

3 Advanced functionalities of SuricatalibHTPFlow variables

4 Suricata 1.3Extraction et inspection of filesTLS Handshake parser

5 The futureThe roadmapMore information

Suricata ?

(C) Jean-Marie Hullot, CC BY 3.0

Suricata ?

(C) Jean-Marie Hullot, CC BY 3.0

Introduction

Éric LeblondInitial and lead developer of NuFWNetfilter Contributor (mainly ulogd2 and userpace interaction)Suricata core developer (IPS, multicore optimisation, . . . )Independant Open Source et security consultant. . .

About OISF

Open Information Security Foundationhttp://www.openinfosecfoundation.org

Non-profit foundation organized to build a next generation IDS/IPSengineFunded by US Governement (DHS, Navy)Development of an Open Source IDS/IPS:

Developers financementFinancial support of related projects (barnyard2)Board who defines big orientationRoadmap is defined in public reunion

About OISF

Developers financement

Financial support of related projects (barnyard2)Board who defines big orientationRoadmap is defined in public reunion

About OISF

Developers financementFinancial support of related projects (barnyard2)

Board who defines big orientationRoadmap is defined in public reunion

About OISF

Developers financementFinancial support of related projects (barnyard2)Board who defines big orientation

Roadmap is defined in public reunion

About OISF

Developers financementFinancial support of related projects (barnyard2)Board who defines big orientationRoadmap is defined in public reunion

About OISF

Consortium membersHOST program: Homeland Open Security TechnologyPlatinium level: BAE systemsGold level: Npulse, Endace, Emerging ThreatsBronze level: SRC, Everis, Bivio networks, Nitro Security, Marasystems, . . .

Technology partner: Napatech, NvidiaDevelopers

Leader : Victor Julien

Developers: Anoop Saldanha, Gurvinder Singh, Pablo Rincon,William Metcalf, Eric Leblond, . . .

BoardMatt JonkmannRichard Bejtlich, Dr. Jose Nazario, Joel Ebrahimi, Marc Norton,Stuart Wilson. . .

About OISF

Consortium membersHOST program: Homeland Open Security TechnologyPlatinium level: BAE systemsGold level: Npulse, Endace, Emerging ThreatsBronze level: SRC, Everis, Bivio networks, Nitro Security, Marasystems, . . .Technology partner: Napatech, Nvidia

DevelopersLeader : Victor Julien

About OISF

DevelopersLeader : Victor Julien

About OISF

DevelopersLeader : Victor JulienDevelopers: Anoop Saldanha, Gurvinder Singh, Pablo Rincon,William Metcalf, Eric Leblond, . . .

About OISF

DevelopersLeader : Victor JulienDevelopers: Anoop Saldanha, Gurvinder Singh, Pablo Rincon,William Metcalf, Eric Leblond, . . .

Bring new technologies to IDSPerformance

Multi-threadsHardware accelerationhttp://packetchaser.org/index.php/opensource/suricata-10gbps

Open sourceSupport of Linux / *BSD / Mac OSX / Windows

Similar projects

BroDifferent technology (capture oriented)Statistical study

SnortEquivalentCompatibleFrontal concurrenceSourcefire has felt endangered and has been aggressivehttp://www.informationweek.com/news/software/enterprise_apps/226400079

Suricata vs Snort

SuricataDrived by a foundationMulti-threadedNative IPSAdvanced functions(flowint, libHTP)PF_RING support, CUDAsupportModern and modular codeYoung but dynamic

SnortDeveloped by SourcefireMulti-processIPS supportSO ruleset (advanced logic+ perf but closed)No hardware accelerationOld code10 years of experience

Independant study:http://www.aldeid.com/index.php/Suricata-vs-snort

Suricata with snort ruleset

Not optimisedDon’t use any advanced feature

Suricata with dedicated ruleset

Use Suricata optimised matchsUse Suricata advanced keywordsCan get one from http://www.emergingthreats.net/

Fonctionnalities

Ipv6 native support

Multi-threadedNative hardware acceleration (PF_RING, Napatech, Endace,Myricom)Numerous options for performance optimisationOptimized support of IP only testsIPS is native (inline mode)Protocol detection

Fonctionnalities

Ipv6 native supportMulti-threaded

Native hardware acceleration (PF_RING, Napatech, Endace,Myricom)Numerous options for performance optimisationOptimized support of IP only testsIPS is native (inline mode)Protocol detection

Fonctionnalities

Ipv6 native supportMulti-threadedNative hardware acceleration (PF_RING, Napatech, Endace,Myricom)

Numerous options for performance optimisationOptimized support of IP only testsIPS is native (inline mode)Protocol detection

Fonctionnalities

Ipv6 native supportMulti-threadedNative hardware acceleration (PF_RING, Napatech, Endace,Myricom)Numerous options for performance optimisation

Optimized support of IP only testsIPS is native (inline mode)Protocol detection

Fonctionnalities

Ipv6 native supportMulti-threadedNative hardware acceleration (PF_RING, Napatech, Endace,Myricom)Numerous options for performance optimisationOptimized support of IP only tests

IPS is native (inline mode)Protocol detection

Fonctionnalities

Ipv6 native supportMulti-threadedNative hardware acceleration (PF_RING, Napatech, Endace,Myricom)Numerous options for performance optimisationOptimized support of IP only testsIPS is native (inline mode)

Protocol detection

Fonctionnalities

Ipv6 native supportMulti-threadedNative hardware acceleration (PF_RING, Napatech, Endace,Myricom)Numerous options for performance optimisationOptimized support of IP only testsIPS is native (inline mode)Protocol detection

Global architecture

Chained treatment modulesEach running mode can have its own architecture

Architecture of mode "pcap auto v1":

Fine setting of CPU preferencesAttach a thread to a CPUAttach a threads family to a CPU setAllow IRQs based optimisation

Global architecture

Chained treatment modulesEach running mode can have its own architectureArchitecture of mode "pcap auto v1":

Global architecture

Chained treatment modulesEach running mode can have its own architectureArchitecture of mode "pcap auto v1":

Entry modules

IDSPCAP

live, multi interfaceoffline support

AF_PACKETPF_RING: mutltithreadhttp://www.ntop.org/PF_RING.html

Capture card support: Napatech, Myricom, Endace

IPSNFQueue:

Linux: multi-queue, advanced supportWindows

ipfw :FreeBSDNetBSD

Entry modules

IDSPCAP

live, multi interfaceoffline support

AF_PACKETPF_RING: mutltithreadhttp://www.ntop.org/PF_RING.html

Capture card support: Napatech, Myricom, Endace

IPSNFQueue:

Linux: multi-queue, advanced supportWindows

ipfw :FreeBSDNetBSD

Output modules

FastlogUnified log (Barnyard 1 & 2)HTTP log (log in apache-style format)Prelude (IDMEF)

Suricata Ecosystem

Signatures

Support almost all snort ruleset featuresExclusive features used by VRT ou Emerging Threats rulesets

alert tcp any any -> 192.168.1.0/24 21 (content: "USER root"; msg: "FTP root login";)

Action: alert / drop / pass IP parameters Motif Other parameters

Signatures

Action: alert / drop / pass

IP parameters Motif Other parameters

Signatures

Action: alert / drop / pass

IP parameters

Motif Other parameters

Signatures

Action: alert / drop / pass IP parameters

Other parameters

Signatures

Action: alert / drop / pass IP parameters Motif

Other parameters

libHTP

Security oriented HTTP parserWritten by Ivan Ristic (ModSecurity, IronBee)Flow trackingSupport of keywords

http_bodyhttp_raw_urihttp_headerhttp_cookie. . .

Able to decode gzip compressed flows

Using HTTP features in signature

Signature example: Chat facebook

a l e r t h t t p $HOME_NET any −> $EXTERNAL_NET $HTTP_PORTS \(msg : "ET CHAT Facebook Chat ( send message ) " ; \flow : es tab l ished , to_server ; content : "POST" ; http_method ; \content : " / a jax / chat / send . php " ; h t t p _ u r i ; content : " facebook . com" ; ht tp_header ; \classtype : po l i cy−v i o l a t i o n ; reference : u r l , doc . emerg ingthreats . net /2010784; \reference : u r l ,www. emerg ingthreats . net / cgi−bin / cvsweb . cg i / s igs / POLICY / POLICY_Facebook_Chat ; \sid :2010784; rev : 4 ; \

This signature tests:The HTTP method: POSTThe page: /ajax/chat/send.phpThe domain: facebook.com

Flow variables

ObjectivesDetection of in-multiple-step attackVerify condition on a flowModify alert treatmentState machine inside each flow

Flowbitsboolean conditionSet a flag

FlowintDefine counterArithmetic operation

Flow variables

Suricata 1.3

New hardware support: Myricom, Endace, NapatechTLS/SSL handshake parserBetter performancesOn the fly MD5 calculation and matching for files in HTTP streamsScripts for looking up files/file md5s at Virus Total and others(contributed by Martin Holste)http_user_agent keyword for matching on the HTTP User-Agentheader

More scability

Flow engine: removal of a contention point and better hashfunction.Thresholding and Tag engines: fine locking instead of global one.

In the engine

Less memoryNew ac-bs algorithmFrom 35G with ac-full to less than 4G for ac-bsHandling 1Gb/s with 7000 rules

Rule engine improvementReload ruleset without breaking the flow analysisRule analyser

Extraction et inspection of files

Get files from HTTP downloads and uploadsDetect information about the file using libmagic

Type of fileOther details. . .

A dedicated extension of signature language

Dedicated keywords

filemagic : description of content

a l e r t h t t p any any −> any any (msg : " windows exec " ; \f i l e m a g i c : " executable f o r MS Windows " ; sid : 1 ; rev : 1 ; )

filestore : store file for inspection

a l e r t h t t p any any −> any any (msg : " windows exec " ;f i l e m a g i c : " executable f o r MS Windows " ; \f i l e s t o r e ; sid : 1 ; rev : 1 ; )

fileext : file extension

a l e r t h t t p any any −> any any (msg : " jpg claimed , but not jpg f i l e " ; \f i l e e x t : " jpg " ; \f i l e m a g i c : ! "JPEG image data " ; sid : 1 ; rev : 1 ; )

filename : file name

a l e r t h t t p any any −> any any (msg : " s e n s i t i v e f i l e leak " ;f i lename : " sec re t " ; sid : 1 ; rev : 1 ; )

Examples

Files sending on a server only accepting PDF

a l e r t h t t p $EXTERNAL_NET −> $WEBSERVER any (msg : " susp ic ious upload " ; \flow : es tab l ished , to_server ; content : "POST" http_method ; \content : " / upload . php " ; h t t p _ u r i ; \f i l e m a g i c : ! "PDF document " ; \f i l e s t o r e ; sid : 1 ; rev : 1 ; )

Private keys in the wild

a l e r t h t t p $HOME_NET any −> $EXTERNAL_NET any (msg : " outgoing p r i v a t e key " ; \f i l e m a g i c : "RSA p r i v a t e key " ; sid : 1 ; rev : 1 ; )

Disk storage

Every file is stored on diskwith a metadata file

Disk usage limit can be setScripts for looking up files / file md5’s at Virus Total and others

Actual limit of files extraction

Limited to the HTTP protocolStorage limit are suboptimalMS Office files are not decoded

TLS Handshake parser

TLS is an application in Suricata wayAutomatic detection of protocol

Independent of portMade by pattern matching

Dedicated keywordsUsable in the signatures

Other supported applications

HTTP :keywords: http_uri, http_body, http_user_agent, . . .

SMTPFTP

keyword: ftpbounceSSH

keywords: ssh.softwareversion, ssh.protoversion

DCERPCSMB

A TLS handshake parser

No traffic decryptionMethod

Analyse of TLS handshakeParsing of TLS messages

A security-oriented parserCoded from scratch

Provide a hackable code-base for the featureNo external dependency

Contributed by Pierre Chifflier (ANSSI)With security in mind:

Resistance to attacks (audit, fuzzing)Anomaly detection

A handshake parser

The syntax

a l e r t tcp $HOME_NET any −> $EXTERNAL_NET 443

becomes

a l e r t t l s $HOME_NET any −> $EXTERNAL_NET any

Interest:No dependency to IP paramsPattern matching is limited to identified protocol

Less false positiveMore performance

TLS keywords

TLS.version: Match protocol version numberTLS.subject: Match certificate subjectTLS.issuerdn: Match the name of the CA which has signed thekeyMore to come

TLS.fingerprint: Match the fingerprint of the certificate

TLS keywords

TLS.version: Match protocol version numberTLS.subject: Match certificate subjectTLS.issuerdn: Match the name of the CA which has signed thekeyMore to comeTLS.fingerprint: Match the fingerprint of the certificate

Example: verify security policy (1/2)

Environnement:A company with serversWith an official PKI

The goal:Verify that the PKI isused

Without working toomuch

The goal:Verify that the PKI isused

Without working toomuch

The goal:Verify that the PKI isusedWithout working toomuch

Let’s check that the certificates used when a client negotiate aconnection to one of our servers are the good one

The signature:

a l e r t t l s any any −> $SERVERS any ( t l s . issuerdn : ! "C=NL, O=Staat der Nederlanden , \CN=Staat der Nederlanden Root CA" ; )

Let’s check that the certificates used when a client negotiate aconnection to one of our servers are the good oneThe signature:

a l e r t t l s any any −> $SERVERS any ( t l s . issuerdn : ! "C=NL, O=Staat der Nederlanden , \CN=Staat der Nederlanden Root CA" ; )

Example: detect certificate anomaly

Google.com is signed byGoogle Internet Authority

Not by an other CA

(Diginotar by example)

If it is the case, this is bad!Let’s block that!

Signature:drop t l s $CLIENT any −> any any ( \

t l s . sub jec t= "C=US, ST= C a l i f o r n i a , L=Mountain View , O=Google Inc , CN=∗. google . com" ; \t l s . issuerdn =! "C=US, O=Google Inc , CN=Google I n t e r n e t A u t h o r i t y " ; )

What! KPN has been hacked too!Let’s get rid of the Dutch!drop t l s $CLIENT any −> any any ( t l s . issuerdn="C=NL" ) ;

Google.com is signed byGoogle Internet AuthorityNot by an other CA

(Diginotar by example)If it is the case, this is bad!Let’s block that!

Google.com is signed byGoogle Internet AuthorityNot by an other CA(Diginotar by example)

If it is the case, this is bad!Let’s block that!

Google.com is signed byGoogle Internet AuthorityNot by an other CA(Diginotar by example)If it is the case, this is bad!

Let’s block that!

Google.com is signed byGoogle Internet AuthorityNot by an other CA(Diginotar by example)If it is the case, this is bad!Let’s block that!

What! KPN has been hacked too!

Let’s get rid of the Dutch!drop t l s $CLIENT any −> any any ( t l s . issuerdn="C=NL" ) ;

Actual limit

Keywords apply only to first certificate of the chain.Impossible to do check on chained certificatesSupported by the parser but not by the keywords.

Some keyword are missing and will be addedused cryptographic algorithmKey sizeDiffie-Hellman parameters

Statistical study and certificate storage

Roadmap

IP and DNS reputationSCADA Preprocessor (thanks to Digital Bond)Keyword geoip

How to test it fast and easy?

Already available in Debian, Ubuntu, Gentoo, FreebsdLive distribution:

SIEM live (Suricata + Prelude + Openvas) : https://www.wzdftpd.net/redmine/projects/siem-live/wiki

Smooth-Sec (Suricata + Snorby) :http://bailey.st/blog/smooth-sec/

Questions

Do you have questions ?

Big thanks:Pierre Chifflier : http://www.wzdftpd.net/blog/The whole OISF team and especially Victor Julien

Related read:OISF website: http://www.openinfosecfoundation.org/Planet suricata: http://planet.suricata-ids.org/Suricata devel site:https://redmine.openinfosecfoundation.org/

Join me:Mail: eric@regit.orgTwitter: RegitericBlog: https://home.regit.org

Latest advance in Suricata IDPS - RMLLschedule2012.rmll.info/IMG/pdf/2012_rmll_suricata.pdf ·...

Documents

Integrate Suricata - EventTracker › EventTracker › media › ...open source-based Intrusion Detection System (IDS), Suricata is a high performance Network IDS, IPS and Network

Suricata User Guide

Suricata: caso di studio di Intrusion Detection and ...computerscience.unicam.it/marcantoni/tesi/Suricata... · ché Suricata sia un software multipiattaforma, alcune funzionalità

FIAT7IFTA @ dubai 2013 SURICATA PAPER

Suricata User Guide - Read the Docs · CHAPTER 2 Installation Before Suricata can be used it has to be installed. Suricata can be installed on various distributions using binary packages:

Performance Testing Suricata The Effect of Configuration Variables On Offline Suricata

State of Suricata - inliniac.net · 10 years of Suricata Name “Suricata” came from the suggestion of using the Meerkat as mascot Latin Genus name: “Suricata” Late July 2009

How with Suricata you save the world - NDH2K14

Suricata IDPS and Linux kernel - NetDev conf · 1 Suricata Introduction Streaming Performance 2 Suricata and Linux kernel AF_PACKET NFQUEUE 3 Suricata and ofﬂoading Interest of

Suricata 2.0, Netfilter and the PRC - stamus-networks.com · Suricata 2.0, Netﬁlter and the PRC Éric Leblond Stamus Networks April 26, 2014 Éric Leblond (Stamus Networks) Suricata

Eric Leblond IDS-suricata

Suricatayaml - Suricata ion Security Foundation - Suricata.yaml... · Suricata.yaml Suricata uses the Yaml format for configuration. The Suricata.yaml file included in the source

Meerkat (Suricata suricatta)

Hardware-based Flow Ofﬂoad in Suricata - ntop Flow Ofﬂoad in Suricata Alfredo Cardigliano cardigliano@ntop.org @acardigliano © 2017 - ntop.org About ntop • ntop develops open

Suricata, the Terminator of IDS/IPS world · Suricata, the Terminator of IDS/IPS world Éric Leblond OISF July 9, 2013 Éric Leblond (OISF) Suricata, the Terminator of IDS/IPS world

Meerkats (Suricata suricatta) fail to prosocially …...Meerkats (Suricata suricatta) fail to prosocially donate food in an experimental set-up Federica Amici1,2,3 • Montserrat Colell

Suricata & AWS - 2020 SuriCon in Boston presented by OISF › wp-content › uploads › 2019 › 11 › ...Suricata & AWS Pre and Post Session Mirroring $ cat about.txt - Overview

ntopng and Suricata: Merging Network Visibility and Security€¦ · Motivation: Unify Visibility and Security [1/2] • Suricata is a great tool for dissecting selected protocols,

Advances in Suricata - DeepSec · 2012-02-02 · Advances in Suricata ... Suricata can efficiently load them all and match with low overhead. Advanced HTTP inspection and logging

Suricata and the Shark: suriwire · Suricata and the Shark: suriwire É. Leblond Stamus Networks July. 03, 2018 É. Leblond (Stamus Networks) Suricata and the Shark: suriwire July