View
192
Download
2
Category
Tags:
Preview:
DESCRIPTION
Wie oft sind Sie schon am Wochenende im Büro gesessen, um kurzfristig aufgetretene Probleme zu lösen? 80 Prozent der Probleme werden immer und immer wieder durch dieselben Ursachen ausgelöst (wie z. B. die fehlerhafte Verwendung/Konfiguration von Resource Pools, beim Deployment "vergessene" Änderungen oder Dateien, Memory Leaks, datenabhängige Performanceprobleme). In dieser Session zeigen wir, wie Sie diese „Stolpersteine“ in Ihrer Produktionsumgebung identifizieren können, welche Informationen Ihre Entwickler benötigen, um sie effizient beheben zu können, und wie Ihre Tester die Behebung automatisiert verifizieren können.
Citation preview
1 1
Wolfgang Gottesheim
2
3
4
5
Testing is Important – and gives Confidence
6
But are we ready for “The Real” world?
7
Measure Performance during the game
Ball Possession: 40 : 60 Fouls: 0 : 0 Score: 0 : 0
Minute 1 - 5
8
Measure Performance during the game
Minute 6 - 35
Ball Possession: 80 : 20 Fouls: 2 : 12 Score: 0 : 0
9
Deep Dive Analysis
10
Options “To Fix” the situation
11
Not always a happy ending
Minute 90
Ball Possession: 80 : 20 Fouls: 4 : 25 Score: 3 : 0
12
Frustration
13
How does that relate to
Software?
14 14
From Deployment to…
Deploy Promotion/Event Problems Ops Playbook War Room
15 15
The “War Room” – back then 'Houston, we have a problem‘ NASA Mission Control Center, Apollo 13, 1970
16 16
The “War Room” – NOW Facebook – December 2012
17 17
Problem: Unclear End User Problem Descriptions
18 18
Problem: Unclear Ops Problem Descriptions
19 19
Status Quo: Problem – “War Rooms” create Tension
20 20
THE APP LIFECYCLE
21 21
Status Quo
Integration Testing
Unit Testing
Load/Performance Testing
Production Development QA
Business
Requirements
Change Stability
Friction
22 22
Operating in Silos: Different tools, different goals
Integration Testing
Unit Testing
Load/Performance Testing
Production Development QA
Logs, Exceptions, Stack Traces
Throughput, Response Time
CPU, Memory, Disk, Network
Business
Requirements
$$$
26 26
Feedback cycles – slow and tedious?
Integration Testing
Unit Testing
Load/Performance Testing
Production Development QA
Business
Requirements
$$$
XML Report
Log Files
Reports, Log Files
PDF Reports
27 27
Status Quo
Integration Testing
Unit Testing
Load/Performance Testing
Production Development QA
Business
Requirements
Change Stability
Confidence
Feedback
Friction
28 28
Wouldn‘t we love one single solution?
Production
Development QA
Business
29 29
What Devs would like to have
30 30
What Devs would like to have
Top contributor is save of Mage_Core_Model_Abstract
70% of that time comes from Sales_Model_Quote
Reverse call stack allows me to identify the entry point, e.g. index.php
31 31
Problem: Perception of each other …
What Ops thinks about Dev & Test …
…and how Ops looks for Dev and Test
32 32
Problem: “Walls” results in more Tension and Ignorance
33 33
Problem: Very “expensive” to work on these issues
~80% of problems
caused by ~20% patterns
YES we know this
BUT
34 34
TOP PROBLEM PATTERNS
• Focus on Web and PHP
35 35
Top Problem Patterns: Does this sound familiar?
• Resource Pool Exhaustion
– Misconfiguration or failed deployment, e.g: default config from dev
– Actual resource leak -> can be identified with Unit/Integration Tests
36 36
Top Problem Patterns: Does this sound familiar?
• Resource Pool Exhaustion (same as before – just different Pool)
– Using the same deployment tools in Test and Ops can prevent this
– Testing with real load can detect that
37 37
Top Problem Patterns: Does this sound familiar?
• Deployment Issues leading to heavy logging resulting in high I/O and CPU
– Using the same deployment tools in Test and Ops can prevent this
– Analyzing Log Output per Component in Dev prevents this problem
38 38
Top Problem Patterns: Does this sound familiar?
• Too many and too slow Database Queries
– Dev and Test need to have “production-like” database – Updates on a “Sample Databases” won’t show slow updates
– Access patterns can be identified with Unit Tests
39 39
Top Problem Patterns: Does this sound familiar?
• Too much data requested from Database
– Dev and Test need to have “production-like” database – Otherwise these problem patterns can only be found in prod
– Educate Developers on “the power of SQL” – instead of loading everything in memory and performing filters/aggregations/… in the App
40 40
Top Problem Pattern: Oversized Pages
• Not following WPO (Web Performance Optimization Rules)
– Non optimized content, e.g: compression, merging, …
– Educate developers and automate WPO checks
17! JS Files – 1.7MB in Size
Useless Information! Even might be a security risk!
41 41
Top Problem Pattern: Caches
• Not leveraging Browser-side Caching
– Misconfigured CDNs or missing cache settings -> automate cache configuration deployment
– Educate developers; Educate testers to do “real life” testing (CDN, …)
62! Resources not cached
49! Resources with short expiration
42 42
Top Problem Pattern: 3rd Party Content
• Slow or failing 3rd party content
– Impacts page load time; Ops is required to monitor 3rd party services
– Educate devs to optimize loading; Educate test to include 3rd party testing
43 43
Lots of Problems that could have been avoided
• BUT WHY are they still making it to Production?
44 44
Missing Focus on Performance
45 45
Different Goals for Dev and Ops
46 46
Disconnected Teams despite “Shared Responsibility”
50 50
51 51
How to make the Enterprise Crew happy?
52 52
53 53
Solution: DevOps + Performance Focus
Culture “Shared Responsibility”
Agile Process for ALL Teams Performance as Key Requirement
X-Team Collaboration and Education
Automation Measurement, Collaboration and Deployment
Automate Performance and
Architectural Problem Detection
Measurement “Visible” KPIs for each Team
Focus on Performance, Architectural
and Deployment Measures
Sharing Expertise, Tool and Data Sharing
“Easy” sharing of Performance, Deployment
and Production Data
http://www.opscode.com/blog/2010/07/16/what-devops-means-to-me/
54 54
Culture: EXTEND Requirements with …
Performance Scalability
Testability
Deployability
Deployability
55 55
Sharing: DON’T EXCLUDE anyone from Agile Process
Stand-Ups Sharing Tools
Feedback
56 56
Measurement: Define KPIs accepted by all teams
# of SQL Executions
# of Log Lines
MBs / Uses
Time for Deployment
Time for Rollback
Response Times Perf Test Code Coverage
57 57
AUTOMATION, AUTOMATION, AUTOMATION
Performance Scalability
Shared Tools Automatic Feedback
58 58
DevOps Collaboration – TODO LIST FOR YOU!!
Access to Production Data
Shared Reporting and Task Management
Diagnostic Tools
Shared Performance KPIs and Tooling
Known How Exchange
59 59
Feedback cycles – fast and precise!
Production Development QA
Business
60 60
Feedback cycles – fast and precise!
Production Development QA
Business
61 61
Recap – Problem – Root Cause – Solution - Result
DevOps + Performance Culture
Automation
Measurement
Collaboration
62 62
TIPS FOR DEVS
67 67
DevOps Automation in Action
• Most problems can be identified in CI
12 0 120ms
3 1 68ms
Build 20 testPurchase OK
testSearch OK
Build 17 testPurchase OK
testSearch OK
Build 18 testPurchase FAILED
testSearch OK
Build 19 testPurchase OK
testSearch OK
Build # Test Case Status # SQL # Excep CPU
12 0 120ms
3 1 68ms
12 5 60ms
3 1 68ms
75 0 230ms
3 1 68ms
Test Framework Results Architectural Data
We identified a regresesion
Problem solved
Lets look behind the
scenes
Exceptions probably reason
for failed tests Problem fixed but now we have an
architectural regression
Problem fixed but now we have an
architectural regression
Now we have the functional and
architectural confidence
68 68
Performance Focus in Test Automation
Analyzing All Unit / Performance Tests
Analyzing Metrics such as DB Exec
Count
Jump in DB Calls from one Build to the
next
69 69
Performance Focus in Test Automation
Cross Impact of KPIs
70 70
Performance Focus in Test Automation
Embed your Architectural Results in Jenkins
71 71
Performance Focus in Test Automation
Here is the difference!
Compare Build that shows BAD Behavior! With Build that shows GOOD Behavior!
72 72
Performance Focus in Test Automation
CalculateUserStats is the new Plugin that causes
problems
73 73
Remember – DevOps requires Cultural Change
Share Integrate
Collaborate Performance
74 © 2011 Compuware Corporation — All Rights Reserved © 2011 Compuware Corporation — All Rights Reserved
74
Participate in Compuware APM Discussion Forums apmcommunity.compuware.com
Like us on Facebook facebook.com/CompuwareAPM
Join our LinkedIn group Compuware APM User Group
Follow us on Twitter twitter.com/CompuwareAPM
Read our Blog About:Performance
Watch our Videos & product Demos youtube.com/Compuware
www.compuware.com/APM
Thank You
75 © 2011 Compuware Corporation — All Rights Reserved
Simply Smarter
http://tinyurl.com/wpc13eval
Recommended