Embed Size (px)
DESCRIPTION
Citation preview
©2010 Improving Enterprises, Inc.
Logging For Fun and Profit
Jane Prusakova
http://softwareandotherthings.blogspot.com
Improving Enterprises
AA.com
2013
©2010 Improving Enterprises, Inc.
Why
How-to
Using the results
©2010 Improving Enterprises, Inc.
Why logging?
Enquiring minds want to know!
Avoid programming-by-coincidence
Trace what the users do
See effects of Integration
Multithreading
High load
©2010 Improving Enterprises, Inc.
When to use logging?Greenfield development
Brownfield developmentwhat came before
the effect of changes
Maintenance and troubleshootingtrace bugs
BonusEvolve apps based on users behavior
©2010 Improving Enterprises, Inc.
Logging [vs Debugging]
Set up once, always ready to use
Minimum disruption to the flow
Works locally and in the cloud
Faster data collection
Collected data is persisted
©2010 Improving Enterprises, Inc.
How-to
Using the results
©2010 Improving Enterprises, Inc.
Yes, I want to setup logging!
Pick a framework
Proper configuration
Consider ways to parse logs
Logging will change with the application
©2010 Improving Enterprises, Inc.
Log4J SLF4J
Apache MIT license
Logging framework Logging facade
Configuration, log levels and categories, rotate log files, thread-safe logging.
Support: online documentation, tutorials, online forums.
©2010 Improving Enterprises, Inc.
Where to write logs?
Console
File system
DB
Queue mechanisms
Combination
©2010 Improving Enterprises, Inc.
Database Files
Easy to parse, hard to evolve
Very hard to parse, easy to change
Uses DB connections
Requires FS access
Can cause app crash, loss of logs,
slow down app
Can fill up space
©2010 Improving Enterprises, Inc.
Configuring Log4J
Java properties-style
XML
Programmatically
Hard-coded
Takes preference
©2010 Improving Enterprises, Inc.
Log Message
What to log with the message
Importance level
When
Where in the code
Data being processed
Thread information
©2010 Improving Enterprises, Inc.
Example layout
log4j.appender.stdout.layout.ConversionPattern= %d %5p [%t] (%F:%L) - %m%n
- Date- Log Level - Thread name- File and line (slow to retrieve)
©2010 Improving Enterprises, Inc.
Enough and but not too much
Logging should support in code
correctness
readability
performance
Consider making logs easier to use
©2010 Improving Enterprises, Inc.
Best Practices
Relate each message to generating point
Code
Data
Format messages for readability
Use .ToString on objects
©2010 Improving Enterprises, Inc.
More Best Practices
Input and output Errors and exceptionsTransitionsIntegration pointsThread rendezvous
©2010 Improving Enterprises, Inc.
Log input and output
public int calculateCost(String user, Flight flight) {
log.debug(“calculateCost: user “ + user + “: “ + flight);
int cost = … ;
… // cost calculation
log.debug((“calculateCost: cost for “ + user + “ on “ + flight + “: “ + cost);
return cost; }
©2010 Improving Enterprises, Inc.
Log errors and exceptions
// do dangerous workresult = DoDangerousWork(user, flight);
}
catch (Exception e){
log.error(“methodName: exception in DoDangerousWork for user “ + user + “ on “ + flight, e);
// recover or re-throw }
©2010 Improving Enterprises, Inc.
More
Log destination, parameters, outcomesDB calls
SOAP calls
Wait timeThread joins
Available memoryMemory-intensive operations
Intermediate calculation resultsFor complicated calculations
©2010 Improving Enterprises, Inc.
Practices to avoid
Little information… // step 1
Logger.info(“did something”);
… // ste 2
Logger.info(“did more work”);
foreach (…) {
… // useful work
Logger.info(“working hard”);
}
©2010 Improving Enterprises, Inc.
Watch for logging-caused bugs
Side effects
Logger.info(count++ “: did something”);
Errors and exceptionsLogger.info(m.GetValue());
©2010 Improving Enterprises, Inc.
More practices to avoid
DRY principle applies
©2010 Improving Enterprises, Inc.
Using the results
©2010 Improving Enterprises, Inc.
You’ve got logs!
Rotate logs: by size, by time
Retention policy
2-level storage
Provide access - read only!
Transfer and analysis
Warning: large datasets
©2010 Improving Enterprises, Inc.
Security and performance
Moving and processing
can impact performance
Absolutely no sensitive
data
©2010 Improving Enterprises, Inc.
Learning from the logs
Never rely on eye-balling
WYS is not WYG
Aggregate data
Distinguish common and
rare
©2010 Improving Enterprises, Inc.
Tools
Process datasets:
Line by line grep, Perl, awk, cut, tail, head
Load entire file to eyeballvi, notepad++
Better way to eyeballmore, less
©2010 Improving Enterprises, Inc.
Learning from the logs
> grep userName MyApplication.log
grep methodName MyApplication.log
grep threadName MyApplication.log
tail -2000 VeryImportant.log | grep Exception
grep FATAL MyApplication.log
©2010 Improving Enterprises, Inc.
Querying logs using AwkGNU Awk www.gnu.org/software/gawk
create histogramsWait times
Number of calls
Exceptions per time period or per user
Compare and relate events Exception stacks traces
Outcomes by caller or thread
©2010 Improving Enterprises, Inc.
Logging For Fun and Profit
Jane Prusakova
http://softwareandotherthings.blogspot.com
Improving Enterprises
AA.com
2013
