Embed Size (px)
DESCRIPTION
Log management can be overwhelming! This talk provide an overview of log management. You will learn about different kinds of logs, structured/unstructured logs, retention, archival, and more. We will also briefly introduce several common tools in the log management space.
Citation preview
Log ManagementAn Introduction
Lennart Koopmann Jordan Sissel
What is a Log?
time + data
What is a Log?
37.5.55.31 - - [08/Apr/2014:15:31:30 -0400] "GET /images/web/2009/banner.png HTTP/1.1" 304 - "http://semicomplete.com/style2.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0"
What is a Log? time + data
120707 0:40:34 4 Connect root@localhost on 4 Query select @@version_comment limit 1 120707 0:40:45 4 Query select * from mysql.user
What is a Log? time + data
?????? ???????
Kinds of Logs
Trace and Debug
Kinds of Logs
Accounting
Kinds of Logs
Transaction
Kinds of Logs
Problems
Difficult to Access
Problems
Too Many Logs
Problems Difficult to Access
Too Many Servers
Problems Difficult to Access
No Permissions :(
Problems Difficult to Access
Difficult to Consume
Problems
Unstructured
Problems Difficult to Consume
Requires Expertise
Problems Difficult to Consume
Requires Maintenance
Problems
Configuration
Problems Requires Maintenance
Log Retention
Problems Requires Maintenance
Bad Tooling
Problems
grep, ssh, awk
Problems Bad Tooling
Life of a Log
Life of a Log
Record
Transport
Search & Analyze
Archive
Delete
Sources of Logs
Vendor Hardware
Sources of Logs
Routers, VPNs, Printers, Phones, AWS CloudTrail, etc
Vendor Software
Sources of Logs
Nginx, Wordpress, Jira
In-house Software
Sources of Logs
Your company controls it
Solutions(Open Source!)
Solutions
Logstashfrom Elasticsearch
Graylog2from Torch
![RebeccaA. Koopmann arXiv:1604.08862v2 [astro-ph.GA] 9 May …](https://img.pdfslide.us/doc/110x75/621aa18e26e82d71f1014145/rebeccaa-koopmann-arxiv160408862v2-astro-phga-9-may-.jpg)
