IBM Innovate - Uderstanding DevOps

Understanding DevOps Session IDR-1988

Sanjeev Sharma IBM WorldWide Lead – DevOps Technical Sales [email protected] DevOps Blog: http://bit.ly/sdarchitect @sd_architect Author: DevOps For Dummies – http://ibm.co/devopsfordummies Eric Minick IBM DevOps Evangelist [email protected] @EricMinick Author: Application Release and Deploy For Dummies

© 2013 IBM Corporation

Please note IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

Agenda

•  Why DevOps? •  What is DevOps? •  Adopting DevOps

o  People o  Process o  Technology

4

CEOs identify technology as the most important external force impacting their organizations – again

Source: IBM Institute for Business Value, The Global CEO Study 2013. Question: “What are the most important external forces that will impact your organization over the next 3 to 5 years?”

2004 2006 2008 2010 2012 2013 1

2

3

6

4

5

7

8

9

1

2

3

6

4

5

7

8

9

1

2

3

6

4

5

7

8

9

1

2

3

6

4

5

7

8

9

1

2

3

6

4

5

7

8

9

Technology Factors

Market Factors

Macro-economic Factors

People Skills

Regulatory Concerns

Socio-economic Factors

Globalization

Environmental Issues

Geopolitical Factors

2

3

6

4

5

7

8

9

1

CEO Studies 2004–2013

External forces that will impact the organization

Software delivery Intelligent/

Connected Systems Software component in smart products driving increased value and

differentiation

Big Data Insights on new products by more efficiently interpreting massive quantities of data

Cloud Demand for apps requires fast, scalable environments for dev and test, as well as production

Instrumented Products Industry requirements demand faster response to regulations and standards, with traceability

and quality

Social Business Broader set of stakeholders

collaborates to deliver continuous innovation

and value

Mobile Modern workforce expects constantly

updated software to connect to enterprise

systems

Software delivery is at the heart of today’s top technology trends

HOW DO WE GO FASTER?

6

Agile?

Agile and Continuous Integration 90s - 2004

2001 2003 2005

Prac

tices

/

Met

hodo

logy

To

ols

1999

Some Agile Focused Work Item Tracking

2005-2010: To the Enterprise!

Agile in 2006-2011

Agile (Scrum) • Larger teams • Cross functional • Standardized • Distributed

… but Agile didn’t help with Ops

A lack of continuous delivery impacts the entire business enterprise in the new reality of “Systems Of Interaction”

>45% of customers experience

production delays

>50% of outsourced projects fail to

meet objectives

>70% of budgets devoted

to maintenance and operations

4-6 weeks

to deliver even minor application changes

to customers

Systems of Interaction

Continuous client

experience Partner

value chain Cloud-based

Services

Systems of Engagement Systems of Record

Operations Rapid app releases impact

system stability and compliance

Suppliers Delivery in the context of

agile

Development/Test Speed mismatch between faster moving front office and slower moving back office systems, delaying time to obtain feedback

Line-of-business Takes too long to introduce or make

changes to mobile apps and services

HR

DB ERP

MF iSeries

CRM

Agenda



DevOps is a Philosophy

Source: http://virtualplatosacademy.blogspot.com/p/great-philosophers.html

Getting Started - The Critical Measure of DevOps Success The Hidden Factory Opportunity

Overhead Productive

80%

20%

50%

50%

Hidden Factory= additional value you could create if you eliminated waste and redirected those resources at innovation

DevOps Transformation

The Big Sources of Wasted Efforts: Find the Hidden Factory Forrester Research study finds an average 3x project capacity after implementing DevOps

Type of Waste Create Feature Deliver Feature

Unnecessary Overhead

Communicating ideas/knowledge Supporting Artifacts

Communicating between development and operations

Unnecessary Re-work

Tasks assigned back to developers from testing and usage

Tasks assigned back to developers from production rollbacks

Over-production Unnecessary functionality produced Unnecessary hardware, data center, personnel

Forrester Consulting: Total Economic Impact of IBM DevOps The study analyzed several development teams and found that the group, on average, went from delivering 10 projects in Year 1 before implementing IBM DevOps, to 20 projects in Year 2, to 30 projects in Year 3 — all without having to add additional developer headcount. And the developers were happier, less stressed, and had a greater feeling of accomplishment.

DevOps approach: Apply Lean principles accelerate feedback and improve time to value

Line-of-business

Customer

1

3

2

1.  Get ideas into production fast 2.  Get people to use it 3.  Get feedback

Adopt DevOps approach to continuously manage changes, obtain feedback and , deliver changes to users

Change speed must be an asset, Not an anchor

Non-Value-added waste Value-added production work

Lean Transformation

Business Outcomes and Operational Objectives

§ Faster Time to Value –  Reduced cycle times –  Industrialization

§ Improved predictability – Operational excellence – Uncertainty reduction

§ Improved customer satisfaction – Customer alignment – Transparency in real time

§ Improved efficiency – Waste reduction, reduced overhead, rework, overproduction – Capex / Opex optimization

IBM DevOps Adoption Model Practices, tools and services to plan and execute a staged adoption of DevOps to improve business outcomes

Feedback Cycles

Productive Waste

Efficiency

Steer Product-based

Agile

Automated

Collaborative

Optimizing

More Predictable

More Transparent

More Continuous

Process-based

Process-heavy

Manual

Silo-ed

Develop/Test

Deploy

Operate

Inefficient Leaner Leaner and Smarter

Map your Bottlenecks

Idea/Feature/Bug Fix/ Enhancement

Production

Development Build QA SIT UAT Prod

PMO

Requirements/ Analyst

Developer

Customers Line of Business

Build Engineer

QA Team Integration Tester User/Tester Operations

Artifact Repository

Deployment Engineer

Release Management

Code Repository

Deploy

Get Feedback

Infrastructure as Code/ Cloud Patterns

Feedback

Customer or Customer Surrogate

Metrics - Reporting/Dashboarding

Tasks

Artifacts

Key software delivery bottlenecks we must eliminate

20

Lack of alignment between IT and business goals

Complexity of languages, tools, and platforms

Poorly integrated teams across the application lifecycle

Lack of automated and scalable testing methods

High cost of setting up and maintaining test environments

Application release process is still error-prone and slow

Lack of centralized incident management and monitoring

Slow or unresponsive to customer feedback

Deploy

Develop - Test

Steer

Operate

Bottlenecks impact delivery cycles, cause rework, and waste resources

The IBM DevOps Adoption Model

IT Org leader Team member

One-on-one guidance on how to improve, from where you are.

How do I determine the best place to

start?

I know where I want to start.

What’s the fastest way to

success?

Inefficient Leaner Leaner and Smarter

Collaborative Silo-ed More Continuous

Process-based

Process-based Agile More

Predictable

Manual Automated More Transparent

Steer

Develop/Test

Operate

Deploy

Optimizing Outcome-based

…

…

…

…

Lean Self-Assessment: http://ibm.biz/devops-lean-assessment

DevOps Fitness Desk in Solution Center EXPO

Practice Self-Assessment: http://ibm.biz/devops-practices-assessment

Consulting Cafe

Agenda



People

23

24

Building a DevOps Culture •  Everyone is responsible for Delivery

•  Common measures of Success

•  Right People are needed

Product Owner

Team Member

Team Lead

Team Member

Team Member

Senior Executives

Users Domain Experts

Auditors

Gold Owner Support Staff

External System Team

Operations Staff

It’s all about the People/Culture

•  Organizational Change

‘Shift Left’ – Operational Concerns Build ‘Application aware’ Environments

Environment Sprints

Adopting DevOps in the Enterprise: People/Culture

Agenda



Adopting DevOps in the Enterprise: Process

27

•  DevOps as a Business Process •  A Process to get Capabilities from Ideation to Value

•  Apply Lean Thinking to Processes

Key Capabilities 1.  Collaborative Development & Continuous

Integration

2.  Continuous Business Planning

3.  Continuous Release and Deploy

4.  Continuous Testing

5.  Continuous Feedback


1. Collaborative Development and Continuous Integration

http://bit.ly/PRQ4a7

Mobile App Developent

Teams

Enterprise Services

Developent Teams


2. Continuous Business Planning 3. Continuous Release and Deploy 4. Continuous Testing 5. Continuous Feedback

http://bit.ly/PRQ4a7


Agenda



•  Infrastructure as Code/Software Defined Environments

package "apache2" do package_name node['apache']['package'] end service "apache2" do case node['platform_family'] when "rhel", "fedora", "suse" service_name "httpd" # If restarted/reloaded too quickly httpd has a habit of failing. # This may happen with multiple recipes notifying apache to restart - like # during the initial bootstrap. restart_command "/sbin/service httpd restart && sleep 1" reload_command "/sbin/service httpd reload && sleep 1"

/* REXX */ /* REXX BIND processor sample */ trace o Arg PACKAGE DBRM rcode = 0 /* Set BIND options */ SYSTEM = 'DSN9' i = Pos('(', DBRM) len = Length(DBRM) LIBRARY = Substr(DBRM, 1, i - 1) MEMBER = Substr(DBRM, i + 1, len - i - 1) OWNER = 'DEVDBA' ACTION = 'REPLACE' VALIDATE = 'RUN' ISOLATION = 'CS' EXPLAIN = 'NO' QUALIFIER = 'DEVDBA' Call Bind_it Exit rcode Bind_it: /* Create a bind control statement as a single long line. Then */ /* queue that into a FIFO stack */ DB2_Line = "BIND PACKAGE("PACKAGE")" ||, " LIBRARY('"LIBRARY"')" ||, " MEMBER("MEMBER")" ||, " OWNER("OWNER")" ||, " ACTION("ACTION")" ||, " VALIDATE("VALIDATE")" ||, " ISOLATION("ISOLATION")" ||, " EXPLAIN("EXPLAIN")" ||, " QUALIFIER("QUALIFIER")" /* Write the bind control statement to the data queue and execute */ /* DB2I to perform the bind. */ queue DB2_Line queue "End" Address TSO "DSN SYSTEM("SYSTEM")" rcode = RC Return

Adopting DevOps in the Enterprise: Technology

Adopting DevOps in the Enterprise: Technology

33

•  Common Collaboration Tools •  Common Work Item Management Tool

•  Dashboards to show status/progress

Develop / Test Deploy Steer Operate

IBM DevOps – Broad set of DevOps capabilities Address bottlenecks across the application delivery lifecycle

Ra#onal Focal Point Ra#onal DOORS NG

Ra#onal Team Concert Ra#onal Lifecycle Integra#on Adapters

Ra#onal Quality Manager Ra#onal Test Workbench Ra#onal Test Virtualiza#on Server InfoSphere Op#m Test Data Management

IBM UrbanCode Release IBM UrbanCode Deploy SmartCloud Orchestrator

SmartCloud Applica#on Performance Management

SmartCloud Control Desk

Continuous Business Planning

Collaborative Development

Continuous Testing

Continuous Release and Deployment

Continuous Monitoring

Continuous Customer Feedback & Optimization

IBM Tealeaf IBM Digital Analy#cs

“IBM has a more comprehensive end-to-end DevOps and ALM

testing integration tool strategy, and the strongest market momentum.” –

Diego Lo Giudice, Forrester Research, January 27, 2014

Implementing a DevOps toolchain

SCM

Build / CI Server

Unit testing Test Automation Test Stubbing

Delivery Pipeline

Environment Configuration

Automated Monitoring

Asset Repository

UrbanCode for Release Automation

IBM UrbanCode Deploy automates the deployment of applications, databases and configurations into development, test and production environments, helping to drive down cost, speed time to market with

reduced risk.

IBM UrbanCode Release is an intelligent collaboration release management solution that replaces error-prone manual spreadsheets

and streamlines release activities for application and infrastructure changes.

https://www.ibmdw.net/urbancode/

Drive down cost Remove manual effort and wasted resource time with push button deployment processes Speed time to market Simple, graphical process designer, with built-in actions to quickly create deployment automation Reduce risk Robust configuration management, coordinated release processes, audits, and traceability

Continuous Delivery Pipeline

Databases Internal Messages

Third-party Services

virtual components

Simultaneously test across

multiple test stages

Dev QA

IBM Rational Test Virtualization

Server

Deploy

Provision

Test

Continuous Testing: Deploy what is ready, Virtualize the rest

IBM UrbanCode Deploy

IBM Rational Test Workbench

{ integrated with

Test Environments

Dynamic Infrastructure

Deploy what is ready, virtualize the rest

Continuously test in production-like env.

Test using real world network conditions n Network

Virtualization

Cloud Portability with Patterns

Cloud Resources (DEV) Cloud Resources (PROD)

Environments

QA ... PROD

World-Wide Banking Application war ddl mq


IBM Virtual System Pa2erns OpenStack HOT templates


DEV

UrbanCode Deploy Cloud

ApplicaAon Component

Tomcat MySQL JMS WebSphere Liberty DB2 WSMB

SmartCloud Orchestrator

IBM PureApplicaAon System

IBM UrbanCode Deploy

Private Clouds Hybrid Public Clouds

VM VM VM

Using Patterns to define ‘Full Stack’ Environments

What’s required… What the business wants… What a pattern automates…

Patterns of Expertise:

Proven best practices and expertise learned from decades of client and partner engagements

•  Pre-defined architecture of an application or Cloud service

•  Captures best practices for complex tasks •  Optimized into a deployable form for private

or public cloud •  Repeatable deployment with full lifecycle

management reducing operational costs

•  Organic environments

•  Version-aware

•  Full-stack engineering

•  Composable content

•  Cloud agnostic

IBM UrbanCode Deploy with Patterns

HOT document editor & Environment lifecycle management tool

A full-stack engineering solution for designing, deploying, and managing environments

Components

Resources

Heat Templates (HOT) Building

Blocks

HOT Executes on

Next generation pattern development and lifecycle management

OpenStack Heat

Tool

Engine

Platforms

o  Provide Software Components right along side infrastructure

o  Represent these resources in HOT documents

o  Once deployed update either from Heat or UrbanCode Deploy

o  Will support Chef, Puppet, etc over time

IBM UrbanCode Deploy & Heat Templates

Improve Delivery Lifecycle with IBM Virtual System Patterns

SCM Build Automation

IBM Endpoint Manager

QA PROD

Provision platform Execute application deployment and manage settings

across environments

Patch/update

Publish build Pull

changes

SmartCloud Orchestrator

Platform Config Management

Environments

Deploy early and often to ensure

higher quality and faster releases

using repeatable, reliable, and

managed automation

DEV DEV DEV DEV

IBM PureApplicaAon System

44

§  DevOps Lean Assessment (Beta):

http://ibm.biz/devops-lean-assessment §  IBM DevOps Page: http://ibm.com/DevOps §  DevOps For Dummies Book:

http://ibm.co/devopsfordummies §  Release and Deploy For Dummies Book:

http://ibm.co/1bplaQV §  IBM DevOps YouTube Playlist: http://bit.ly/1fiDOtl

Resources

45

Thank You!

Your Feedback is Important!

Access the Innovate agenda tool to complete your session surveys from your smartphone, laptop or conference kiosk.