Logs as Data: Using Logs to track Web Application Performance

by RevelOps

Logs as Data:

Using Logs to track Web Application Performance

May 2014

by RevelOps

A Heritage in Advanced Research • Founders hold PhDs in Distributed Systems

• Former Scientists at the IBM Center for Advanced Studies

• Built Log Management Solution for IBM (RTCE)

Global SaaS Log Management Service• More than 1,000 customers in over 100 countries, processing

over 10B events daily

• Locations in Boston, Dublin, Prague

• Raised more than $11M in venture funding

Trevor Parsons Formerly a scientist at IBM's Center for advanced studies with a PhD in Computer Science.

Andrew BurtonProven software executive, with over 16 years of experience in start-ups & enterprise companies.

Brendan DillonExperienced software engineering & security executive, formerly CTO at Adaptive Mobile.

Viliam HolubOver a decade developing enterprise software, with a PhD in Computer Science.

Leadership Team

Leigh MerriganSaaS marketing, communications and branding executive, formerly with AppNeta.

President & CEO

Co-founder, Chief Scientist

VP, Engineering

Co-founder, CTO

VP, Marketing

About Logentries

by RevelOps

Developers, IT Operations and Business Analysts from…

Some of Our Customers

by RevelOpsOver 25k Users across 100 Countries

by RevelOpsWhat We Do?

by RevelOpsThe Humble Log

by RevelOpsMaking Log Data Accessible

by RevelOpsLogs As Data

by RevelOpsTop 10 Customer Use Cases

Production Monitoring

Production Troubleshooting

Debugging During

Development

Web Application Analytics

Support

Real User Monitoring

Security

Quality Assurance & Testing

Business Analytics

Mobile Application Analytics

All Other Responses

71%

66%

41%

34%

32%

28%

22%21%

10%

6%

5%

© 2014 Logentries. All Rights Reserved. Confidential.

by RevelOps

Benefits of Logs for Performance Monitoring

MACHINE GENERATED LOG DATA

#1: You can’t always instrument the cloud

by RevelOpsBenefits of Logs for Performance Monitoring

#2 Logs Maintain the Evidence

by RevelOpsBenefits of Logs for Performance Monitoring

#2 Logs Maintain the Evidence

by RevelOpsBenefits of Logs for Performance Monitoring

#2 Logs Maintain the Evidence • Even with ephemeral servers

by RevelOpsBenefits of Logs for Performance Monitoring

#2 Logs Maintain the Evidence • When you have performance issues on ephemeral servers

by RevelOps

Benefits of Logs for Performance Monitoring#3 Logs can contain symptoms of failures

by RevelOpsBenefits of Logs for Performance Monitoring

#4 Events, App Performance, Resource Usage

by RevelOps

Benefits of Logs for Performance Monitoring#5 Top to Bottom and End to End

by RevelOpsA Study across 25k users: Some Common Issues

Community Insights - Strength in Numbers

• Categories:

• Generic patterns across all web logs

• Tomcat specific patterns

• IIS logs i.e. running a ASPX/ASP web app

• Logentries Community Insights: logentries.com/doc/community-insights

by RevelOps

Investigating Performance Issues w/ Logs#1 Set Up Real Time Alerts: e.g. Response Time>500ms

by RevelOpsInvestigating Performance Issues w/ Logs

#2 Understand what single user behavior caused the issue

by RevelOpsInvestigating Performance Issues w/ Logs

#2 Understand if group user behavior caused the issue

by RevelOps

Investigating Performance Issues w/ Logs#3 What did resource usage look like?

by RevelOps

Investigating Performance Issues w/ Logs#4 Was performance effected?

by RevelOps

Investigating Performance Issues w/ Logs#4 Was performance effected? (Real User Monitoring)

by RevelOpsInvestigating Performance Issues w/ Logs

#5 What part of the application code caused the issue?

by RevelOpsInvestigating Performance Issues w/ Logs

#5 What part of the application code caused the issue? (Tracing Transactions)

by RevelOpsWhat part of the app code caused the issue?

• Step 1: Collate your logs

by RevelOpsWhat part of the app code caused the issue?

• Step 2: Refine the log entries

by RevelOps

• Step 3: Narrow the time range

What part of the app code caused the issue?

by RevelOpsWhat part of the app code caused the issue?

• Step 4: View the Transaction

by RevelOpsSummaryBenefits of Logs for Production Monitoring

1. You can’t always instrument the cloud2. Logs Maintain the Evidence3. Logs can contain of symptoms of more serious issues4. Logs combine event data, resource usage, performance metrics5. Top to Bottom and End to End

Investigating Performance issues w/ Logs

6. Setup Real Time Alerts7. Understand user behaviour8. Inspect Resource Usage9. Was Performance Effected10. What Code caused the issue

by RevelOpsGive it a Try for Free

• Get started in minutes with the free Logentries service!

• Lots of examples of using Logs to Monitor Application Performance on our blog: http://blog.logentries.com

• Contact us directly! • Trevor.parsons@logentries.com• Support@logentries.com

by RevelOps

May 2014