1. HIGH SPEED NSM NETWORK SECURITY MONITORING MICHA PURZYSKI

2. WHY ARE WE DOING IT? Were under a constant attack like theres no tomorrow 3. WHAT ARE THE TARGETS? Bugzilla RelEng - code RelEng - builds RelEng - signing infrastructure 4. Help us in the incident response providing a lot of useful context 5. Protect crucial infrastructure such as RelEng or Bugzilla 6. Understand the methods people use to attack us What Where How 7. Detect advanced and targeted attacks 8. Return observations to the community 9. HOW ABOUT PRIVACY? Entire project under the Privacy team review Thats only our infrastructure 10. TRADITIONAL IDS Can just scare you with something like this MALWARE CNC COMMUNICATION 11. HAVE YOU BEEN COMPROMISED? 12. ENTER THE NSM WHAT IF YOU COULD... 13. DEMO - NSM SESSION DATA CAPTURED 14. FULL PACKET CAPTURE 15. STORE FOR A FEW DAYS ROTATE 16. USE Bro Snort Sguil Snorby Wireshark Network Miner and a lots more ON ALL YOUR TRAFFIC 17. Encrypted - you dont have the key for SSH, IPSEC Encrypted - you can capture plaintext SSL - get it after the SSL termination Public (a stream of Firefox) or a bulk od data (SCP again) Sensitive (personal data) WHAT NOT TO STORE? 18. BUT... THERES A TONS OF TRAFFIC N x 10Gbit / datacenter over 10Gbit/sec in peak after ltering 19. HOW DO WE FEED OUR SENSORS? Everydays sensors care and feeding. A quick walkthrough. 20. CAPTURE CARDS pooling - no IRQ zero copy card -> application kernel bypassed Kernel Buffer Application Got packets? 21. SENSORS CARE AND FEEDING Optical Taps where possible - rewall core Span port on switch 22. Internet Load Balancers Backend systems xDirector Filtering Dropping Load balancing Snort cluster Bro cluster Pcap cluster Firewalls Trafc - mirror Trafc - original 23. SECURITY ONION NSM distribution - BRO, Snort, Netsni-ng included Ubuntu 12.04 LTS Does a LOT of the heavy lifting for you Easy to customize and extend - client/server architecture 24. WHAT WORKED FOR US? - HARDWARE Netoptics xDirector and optical taps Xeon E5-2670 - 2.6Ghz, 8 core Lots of ram - 64GB per sensor Local storage - 900GB RAID1, SAS 10k rpm Myricom capture cards 25. WHAT WORKED FOR US? - SOFTWARE Security Onion Bro 2.2 git/beta Snort Myricom Snier10G XFS 26. WHAT DIDNT WORK? pfring without and with DNA Cards other than Intel and Myricom Too slow and too fast (expensive) CPUs 16GB RAM Multi purpose sensors 27. WHAT DIDNT SCALE? Argus Prads P0F 28. WHAT DID WE USE INSTEAD? Argus -> Bro, Nfsen Prads -> Bro, Elsa (only in SO) P0F -> Bro (not 100%) 29. DEMO - BRO 30. WHY BRO ISNT JUST THE IDS? Network ow scripting language. Example? 1. extract les crossing the network 2. calculate the hash 3. compare against the MHR 25 lines of code 31. WHY BRO ISNT JUST THE IDS? nd bro-2.2-beta/build/html/scripts/base -iname *event* | wc -l 44 Understands and generates events for 44 protocol types Hook them 32. OK - WEVE BUILT THE NSM NOW WHAT? MOUNTAINS OF DATA 33. HOW DO I STORE LOGS? Elasticsearch to the rescue Bro can write to it natively Over 200GB logs / day Good to have other teams and share resources :) 34. WHATS ON THE WIRE? L2 - MAC src/dst L3 - IP src/dst, Port src/dst, Proto, bytes L4 - L7 headers 35. WHATS ON THE WIRE? Timestamps SSL - certs, sizes, types, expire Server and Client application versions Protocol level data 36. WHAT TO DO WITH IT? Signature based warnings - Snort, Bro Blacklisted IP communication Evil DNS queries Services detection 37. WHAT TO DO WITH IT? Passive vulnerability management no need to on the host agents even network hardware Surprising protocols 38. 1379084400.225996 10.22.24.213 - HTTP::BROWSER Firefox 23 0 - - - Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/ 20100101 Firefox/23.0 39. 1374687810.544717 10.22.111.53 22 SSH::SERVER OpenSSH 4 3 - - - OpenSSH_4.3 40. 1374626058.615864 10.22.74.73 3128 HTTP::SERVER Microsoft-IIS 7 0 - - - Microsoft-IIS/7.0 41. DOES IT STILL WORK? Attack yourself, automate it Make sure you dont lock yourself out Source attacks from a single known IP? 42. MORE TO COME STAY EXCITED!