Upload
owasp-russia
View
632
Download
1
Embed Size (px)
Citation preview
Large enterprise SIEM: get ready for oversizeSvetlana/Mona ArkhipovaQiwi
OWASP Meetup, Moscow, 28 Feb 2015
What are we talking about?• Log collecting != Security Information and
Event Management• Systems monitoring is not enough• Logs as a ‘Big Data’•
WTF is qRadar?Administrator’s nightmare:
• Frontend: Java+Tomcat• Backend: Java daemons• DB: Ariel for collected+ indexed data, PostgreSQL for ‘static’ data• Painful performance metrics and load
balancing
To log or not to logGuides/best practices• https://www.owasp.org/index.php/Logging_Cheat_Sheet • http
://www.syslog.org/logged/logging-and-syslog-best-practices/
• http://sniperforensicstoolkit.squarespace.com/storage/logging/Windows%20Logging%20Cheat%20Sheet%20v1.1.pdf
• https://zeltser.com/media/docs/security-incident-log-review-checklist.pdf
• …
To log or not to logHuston, we got a problem:• Standard syslog message size (RFC 5424)• Windows security logs permissions on
W7/2008+• Database audit – what to log?• Log files on FS (IIS and so on)• In-house developed apps
To log or not to logStandard sources: Windows
• Event collectors vs. agents• Extended system audit• Non-English logs:
To log or not to log
Standard sources: *nix, network devices
• Syslog as a standard• TCP syslog+network issues=pain
(google: “TCP is not reliable”)• UDP syslog message size• Auditd – what to log?
To log or not to log
Standard sources: Databases
• Is login history enough?• Syslog vs DB connection
To log or not to logNon-Standard sources:
• Exotic network devices• In-house developed apps• 1C (OMG…) and other specific apps• Integration with other security systems (NGFW,
DBFW, AV, Security scanners…)
Normalizing/indexingEvent at a glance• Standard properties: timestamp, src IP, dst IP, log
source identifier and so on• Custom event properties – KISS principle• No search – no property.
Indexing• Standard properties – index, index, index!• Custom event properties indexing: with great
power comes great responsibility…• BTW, watch your index size.
Over(sizing)Current Qiwi SIEM metrics:
• 1800 log sources• 10 000 - 24 000 RAW events per second (EPS)• ~11 600 network flows per second (FPS),
~700 000 flows per minute(FPM)
SIEM system: 39 virtual servers, 2 hardware servers with Napatech 2x10G cards, 1 archive server
Over(sizing)
Expectations (sizing) RealityvCPU 140 160vRAM 272 Gb 521 GbvHDD 15 TB 61 TB
Once upon a time in a far far galaxy we decided to build our own SIEM…
Online/offline storageDaily stats:• 67-145 Gb raw event logs per day• 37-53 Gb network communication events per
day
• Online storage – fast access (realtime + some previoius data)
• Offline – archive storage
Internal security scanners“Normal paranormal” activity inside and outside.
• Butthurt :(• Log or drop events?• Custom rules set for nodes• Keep an eye on credentials!• Balancers NAT/SNAThttps
://f5.com/resources/white-papers/load-balancing-101-nuts-and-bolts
Questions?Svetlana/Mona Arkhipova
Lead information security expertQIWI infrastructure security team
mona.sax m0na_sax