View
784
Download
1
Category
Tags:
Preview:
Citation preview
Copyright 2013 BalaBit IT Security Ltd.
Why proper logging is important...
...in all phases of development?
Péter Czanikcommunity manager
Copyright 2013 BalaBit IT Security Ltd.
About me
• Peter Czanik from Hungary
• community manager at BalaBit: syslog-ng upstream
• BalaBit is an IT security company with HQ in Budapest, Hungary with 100+ developers
• part of the openSUSE testing team• openSUSE syslog-ng package maintainer
Copyright 2013 BalaBit IT Security Ltd.
Topics
• no, it is not about cutting trees :-)
• what is syslog? and syslog-ng?• who uses syslog-ng?• what to log?• free-form messages against name-value pairs• the new buzzword: journal• standardization efforts: CEE/Lumberjack• name-value pairs at work: ELSA
Copyright 2013 BalaBit IT Security Ltd.
What is syslog?
• logging: recording events• syslog:
- application: collecting events- data: the actual log messages- protocol: forwarding events
• history:- originally developed as a logging tool for sendmail- quickly many other apps started to use it
• syslog-ng: “next generation” syslog server- since 1997
- focus on central log collection
Copyright 2013 BalaBit IT Security Ltd.
What is syslog-ng
• “Swiss army knife” of logging• OSE vs. PE
• high performance• more input sources (files, programs, and so on)• more destinations (databases, encrypted net,
etc.)• better filtering (not only priority, facility)• processing (rewrite, parse, correlate, and so on)• JSON output and parser• AMQP
Copyright 2013 BalaBit IT Security Ltd.
Who uses syslog-ng?
• syslog-ng is the default logging solution in SLES since SLES 10- Uses 2.0, an ancient version
• syslog-ng is the default logging solution in Gentoo• syslog-ng is available in openSUSE (package is
maintained by me :-) )
• ...and?
Copyright 2013 BalaBit IT Security Ltd.
Who uses syslog-ng?
Copyright 2013 BalaBit IT Security Ltd.
What to log?
• what: everything :-)• in more detail: SANS Top5 log reports:
- authentication, change, resource access, etc.
• during development logging is often an afterthought- some/many of the above is missing- aids just coding- difficult to debug or audit in production
• logging should be an integral part of development- think also about production :-)- consult with operators → DEVOPS!!!- use a similar logging environment, as in production
Copyright 2013 BalaBit IT Security Ltd.
How to log?
• short answer: centrally• long: centrally, because:
- ease of use: one place to check instead of many- availability: even if the sender machine is down- security: logs are available even if sender machine is compromised
Copyright 2013 BalaBit IT Security Ltd.
Free form log messages
• most log messages are: date + hostname + textMar 11 13:37:56 linux-6965 sshd[4547]: Accepted keyboard-interactive/pam for root from 127.0.0.1 port 46048 ssh2
• text = English sentence with some variable parts• easy to read by a human
Copyright 2013 BalaBit IT Security Ltd.
Why it does not scale?
• few logs (workstation) → easy to find information• many logs (server) → difficult to find information• information is presented differently by each
application• difficult to process them with scripts
• answer: structured logging- Events represented as name value pairs
Copyright 2013 BalaBit IT Security Ltd.
Solution from syslog-ng: PatternDB
• syslog-ng: name-value pairs inside- date, facility, priority, program name, pid, etc.
• PatternDB parser:- can extract useful information into name-value pairs- add status fields based on message text- message classification
• example: an ssh login failure:- user=root, action=login, status=failure- classified as “violation”
Copyright 2013 BalaBit IT Security Ltd.
Journal
• the logging component of system• name-value pairs inside:
- message- trusted properties- sny additional name-value pairs
• native support for name-value pair storage
• persistent log storage can be disabled• logs can be forwarded to syslog-ng through a
socket• syslog-ng can filter, process logs and forward
them to central log server
Copyright 2013 BalaBit IT Security Ltd.
Journal: the enemy?
• FAQ: Q: is journal the enemy? A: No!• Journal is local only (syslog-ng: client – server)• Journal does not filter or process log messages• Journal is limited to Linux/systemd (syslog-ng: all
Linux/BSD/UNIX)
Copyright 2013 BalaBit IT Security Ltd.
CEE
• Journal, syslog-ng, Windows eventlog, rsyslog, auditd, and so on are based on name-value pairs
• All use different field names• Standardization is a must: CEE → Common Event
Expression• Events: name-value pairs instead of free-form text- Taxonomy: name-value pairs to describe events (example: status)- Dictionary: name-value pairs for event parameters (example: user)
• PatternDB can turn free-form messages into CEE
Copyright 2013 BalaBit IT Security Ltd.
Name-value pairs in action: ELSA
• ELSA: Enterprise Log Search and Archive• based on syslog-ng, PatternDB and MySQL• simple and powerful web GUI• extreme scalability• patterns focused on network security:
- firewalls: Cisco, iptables- IDS: Snort, Suricata, Bro- HTTP, Windows logs, etc.
Copyright 2013 BalaBit IT Security Ltd.
Search
Copyright 2013 BalaBit IT Security Ltd.
Graph
Copyright 2013 BalaBit IT Security Ltd.
Map
Copyright 2013 BalaBit IT Security Ltd.
So, why syslog-ng?
• 15 years of open source development• high performance log management• flexible configuration• excellent documentation• PatternDB message parsing
Copyright 2013 BalaBit IT Security Ltd.
Questions? (and some answers)
• Questions?
• Some useful syslog-ng resources:- Syslog-ng: http://www.balabit.com/network-security/syslog-ng- SANS top5 essential log reports extended: http://chuvakin.blogspot.hu/2010/08/updated-with-community-feedback-sans_06.html - Many books at http://oreilly.com/- ELSA: http://code.google.com/p/enterprise-log-search-and-archive/- My blog: http://czanik.blogs.balabit.com/
Copyright 2013 BalaBit IT Security Ltd.
Thank You!Péter Czanik
community managerpeter.czanik@balabit.com
Recommended