How Visualization Makes IT Possible?

Sebastien Tricaud

Picviz Labs CTO

Honeynet Project

About Me

• Founder & CTO of Picviz Labs (www.picviz.com)

• Honeynet Project CTO

• Twitter @tricaud

• 15 years of various IDS implementations

• Contributor of Prelude IDS, OSSEC, Sancp, Linux PAM, …

CURRENT SITUATION

Securing a classical infrastructure

• Anti-virus

• Web Application Firewall (blocking of course non-trivial JS tricks)

• Intrusion Detection (& Prevention) Systems

• Firewall

• Security Information Event Manager (SIEM)

Results from this security

• Known stuff are (maybe) blocked

Conclusion

• Securing this infrastructure has not laid off security teams

• Worse, some media entertainment companies had to hire a CSO

Practical Security of hundred Machines

• Is your NTP really synchronized?

• Logs are sent to a centralized database

Log eater

Log eater

Amazing Database !

Amazing Database !

Amazing Database !

Amazing Database !

EASY FIX:

START YOUR

CERT !

Facebook amazing infrastructure!

2010: > 60 000 web servers About 10% of Internet traffic is for them

Our Current Problem

• (buzzword) APT: Advanced Persistent Threats

• Unknown attacks remain unknown until they are known

• Slow Response Time

• Limited Metrics (is a top 10 enough to handle thousand machine?)

• We have found only one way to handle the signal/noise ratio (signatures)

FIND ATTACKS

(some of the) Tools Used

• Bandwidth trigger

• OSSEC

• Netflow

• Wireshark

• Honeypot

• Grep (ahah)

• …

Google Summer of Code: Wireviz

Google Summer of Code: WireBrowse

Google Summer of Code: WireShnork

Google Summer of Code

• The wireshark improvements was amazing.

• Thank you Jakub Zawadzki & Guillaume Arcas!

• It is one among 12 other great projects (http://honeynet.org/gsoc/slots)

• Thank you Google!

Tools issues

• They hardly interact with each other

• They are often limited

• At least they work well, so they are used

• No one does better

Scenario: we want the .ru TLD from our proxy logs

• Easy:

$ grep "\.ru" squid.log

Scenario: we want the .ru TLD from our proxy logs

[...] "[28/Feb/2011:00:13:02 +0100]" XXXX GET

http://pixel.quantserve.com/pixel;r=1869975797;fpan=0;fpa=P0-

1991180462-1298650127845;ns=1;url=http%3A%2F%2Foptimized-

by.rubiconproject.com%2Fa%2F3346%2F3 [...]

[...] "[28/Feb/2011:00:14:32 +0100]" xgbj352 GET

http://eco.rue89.com/2011/02/25/oui-les-militaires-meritent-

leur-reduction-de-75-a-la-sncf-192164?page=0 HTTP/1.1 500

1120 500 505 TCP_NC_MISS 567 12 1103 10.33.37. […]

• Easy:

$ grep "\.ru" squid.log

Scenario: we want the .ru TLD from our proxy logs

• Solution:

grep –e with regex:

'http(s)?\://[a-zA-Z0-9\-\:\.]+\.(ru)/'

[...] "[28/Aug/2011:02:13:50 +0100]" XXXX GET

http://www.ritb.ru/XXXX [...]

[...] "[28/Aug/2011:02:13:50 +0100]" XXXX GET

http://nx.ritb.ru/XXXX [...]

[...] "[28/Aug/2011:02:13:49 +0100]" XXXX GET

http://tas.mb.ran.ru/XXXX [...]

Scenario: we want the .ru TLD from our proxy logs

• Solution? NO

• It will still mess up

[...] "[28/Feb/2011:02:02:08 +0100]" XXXX GET

http://www.facebook.com/plugins/like.php?href=http:/

/slon.ru/articles/XXXX […]

Scenario: we want the .ru TLD from our proxy logs

Pie Charts

Histograms

Treemaps

SIEM

Log Management

SERIOUS VISUALIZATION

Serious Visualization

• Handling large scale incidents

• Make you close to your data

• As many events as logs may have

• As many dimensions as logs may have

There is only Parallel Coordinates

From a log to a picture

Adding a new input

• Demo time: input SQUID logs

Find attacks on your Apache logs

• Demo time (again!)

Find behaviors

• Demo time

Find OpenVPN tunnels

• Demo time! yeah!

Conclusion

• Usual Visualization is often a failure when it comes to practice

• There is still hard work to do

• Parallel Coordinates enables you large scale analysis

• Finding the unknown goal is reached

• The better you know your logs, the easiest it will be

Questions?