Doing DevOps? Better base it on performance - DevNexus 2014

1   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

Stephen  Wilson  

• apmblog.compuware.com  • @steve0wilson  (0  as  in  zero)  

• LinkedIn  • stephen.wilson@compuware.com  

2   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

80

60

150

% of a Development Budget is spent on ID and fixing defects

3   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

4   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

80

60

150

% of a Development Budget is spent on ID and fixing defects

B dollars is the cost of defects that reach production

5   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

6   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

80

60

150

% of a Development Budget is spent on ID and fixing defects

B dollars is the cost of defects that reach production

X the cost to fix a problem in production vs. in development

7   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

8  8  

DOING  DEVOPS?    WHY  YOU  NEED  TO  FOCUS  ON  PERFORMANCE  Andreas  Grabner,  Technology  Strategist  @  Compuware/dynaTrace  

9   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

10   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

11  11

12   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

13  13

14   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

Development  

TesZng  

ProducZon  Release  

The Waterfall Process

15  15

Then came Agile…

16   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

Rapid Acceleration…

17   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

Production Problems

18  18

Produc>on  (local,  remote,  outsourced)  

•  Cloud  load  tesZng  •  Monitoring  

19   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

20   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

buzzword? process?

agile ops? a way of life?

21   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

Seeking  Knowledge  vs.  Applying  Knowledge  

22   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

BUSINESS  ALIGNMENT  

TRADITIONS  

ORGANIZATIONAL  TRUST  

CHALLENGES  NATURAL  FRICTION  

LEGACY  ATTITUDES

23   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

ReacZve   ProacZve   PredicZve  

24   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

25   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

26   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

27   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

28   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

Development  (local,  remote,  outsourced)  

Test/QA   Produc>on  

•  Ops  tools  •  Network  Tools  •  Log  scraping  •  App  Monitoring  

•  Load  TesZng  •  App  Monitoring  

•  Jprobe  •  Eclipse  •  Debuggers    

29   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

Problem:  Unclear  Ops  Problem  Descrip>ons  

30   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

31   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

Unclear  End  User  Problem  Descrip>ons  …  

32   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

…  when  you  really  want  to  have  this  …  

33   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

…  and  this  

Top  Contributor  is  related  to  String  handling  

99%  of  that  Zme  comes  from  RegEx  Pabern  Matching  

Page  Rendering  is  the  main  component  

34   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

35   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

STOP  THE  BLEEDING  

COMMUNICATE  THE  WINS  

IDENTIFY  KEY  PLAYERS  

36   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

37   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

38   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

39   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

40   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

41   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

Performance  Focus  in  Test  Automa>on  

Embed  your  Architectural  Results  in  Jenkins  

42   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

Performance  Focus  in  Test  Automa>on  

Build-­‐level  Performance  Metrics    

43   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

STANDARDIZE  THE  PROCESS  

DEFINE  PERFORMANCE  DRIVERS  

CREATE  A  CONSTITUTION  

44   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

45  45

How?  Performance  Focus  in  Test  AutomaZon  

12   0   120ms  

3   1   68ms  

Build  20   testPurchase   OK  

testSearch   OK  

Build  17   testPurchase   OK  

testSearch   OK  

Build  18   testPurchase   FAILED  

testSearch   OK  

Build  19   testPurchase   OK  

testSearch   OK  

Build  #   Test  Case   Status   #  SQL   #  Excep   CPU  

12   0   120ms  

3   1   68ms  

12   5   60ms  

3   1   68ms  

75   0   230ms  

3   1   68ms  

Test  Framework  Results   Architectural  Data  

We identified a regresesion

Problem solved

Lets look behind the scenes

Exceptions probably reason for failed tests

Problem fixed but now we have an architectural regression

Problem fixed but now we have an architectural regression

Now we have the functional and architectural confidence

46  46

How?  Performance  Focus  in  Test  AutomaZon  

Analyzing  All  Unit  /  Performance  Tests  

Analyze  Perf  Metrics  

IdenZfy  Regressions  

47  47

How?  Performance  Focus  in  Test  AutomaZon  

Cross  Impact  of  KPIs  

48   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

49   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

DRIVE  FEEDBACK  

AUTOMATE  THE  PROCESS  

PERFORMANCE  BASED  DECISIONS  

50   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

ROME  WAS  NOT  BUILT  IN  A  DAY  

SPONSORSHIP  

DON’T  BE  AFRAID  TO  FAIL  

TALENT  KNOWLEDGE  

51   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

Stephen  Wilson  

• apmblog.compuware.com  • @steve0wilson  (0  as  in  zero)  

• LinkedIn  • stephen.wilson@compuware.com  

52  © 2011 Compuware Corporation — All Rights Reserved © 2011 Compuware Corporation — All Rights Reserved

52

ParZcipate  in  Compuware  APM    Discussion  Forums  apmcommunity.compuware.com  

Like  us  on  Facebook  facebook.com/CompuwareAPM  

Join  our  LinkedIn  group  Compuware  APM  User  Group  

Follow  us  on  TwiTer  twiber.com/CompuwareAPM  

Read  our  Blog  About:Performance  

Watch  our  Videos  &    product  Demos  youtube.com/Compuware  

www.compuware.com/APM  

Thank  You  

53  © 2011 Compuware Corporation — All Rights Reserved © 2011 Compuware Corporation — All Rights Reserved

53   COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE