Upload
ibm-urbancode-products
View
2.122
Download
1
Tags:
Embed Size (px)
DESCRIPTION
"Shift Left" is a DevOps practice that provides an effective means to perform testing with or in parallel to development activities. When shifting left, development, test and operations work together to plan, manage and execute automated and continuous testing to accelerate feedback to developers and improve the quality of changes early in the life-cycle. The rate of the accelerated feedback is determined by an organization’s desired outcomes for velocity of changes and capacity for feedback.
Citation preview
© 2013 IBM Corporation
Shifting Left Approach and Practices
Paul Bahrs Chief Architect, Emerging Technologies, IBM Software, Rational
Nov 6, 2014
Who is this guy?
• Paul is the Chief Architect for emerging technologies, focused on new or acquired solutions and accelerating early client successes
• Today he works with IBM clients with their adoption of DevOps, Cloud, and Quality solutions to meet market and business demands
• Paul is the published on DevWorks, ibm.com, a presenter at industry/IBM conferences, patent author and supported open standards
Paul Bahrs
Market shifts are fundamentally changing what and how we do IT
3
Software Development Customers
Development Testing Deployment
Macro Business Environment Volatile economic and changing regulatory
environments require businesses to adapt
quickly to changing market conditions
Empowered Users The exponential increase
in empowered users drives
expectations for higher quality
customer experiences and the
need for continuously updated
software
Technology Trends Mobile, cloud, big data, social,
agile and delivery analytics
increase application complexity
while promising better
accessibility, flexibility
and agility
Business Owners
Develop / Test
Deploy Steer Operate
IBM DevOps – most comprehensive capabilities Addresses bottlenecks and waste across the delivery lifecycle
Continuously plan, measure and
bring business strategy and
customer feedback into the
development lifecycle.
Enable collaboration between
business, development, and QA
to deliver innovative, quality
software continuously.
Reduce the cost of testing while helping
development teams balance quality and
speed.
Deliver software to customers
and internal users faster and
more frequently with better
quality, lower cost, and reduced
risk.
Understand and accommodate the
user perspective to achieve service
levels with better visibility and
continuous feedback across the
entire software lifecycle.
Continuous
Business Planning Collaborative
Development
Continuous
Testing
Continuous Release
and Deployment Continuous
Monitoring
Continuous
Customer Feedback
& Optimization
Provide the visual evidence and full
context for analyzing customer behavior
and pinpointing pain points.
© 2013 IBM Corporation
Some Observations
6-12 Month Delivery Cycles Are Still the Norm
Where do Testers want to spend their time?
Creating Automated Tests
Performing Exploratory Tests
Executing Automated Tests
Designing Tests
Planning Tests
Reviewing Test Results
Running Pre-defined Tests
Maintaining Automated Tests
I want to spend MORE time…
I want to spend LESS time…
Creating Reports
Maintaining Automated Test Tools
Configuring Test Tools
Creating Test Data
Re-running tests
Investigating, Documenting and Submitting Defects
Setting Up Test Labs
What are deployment engineers doing?
*Data based on UrbanCode customer survey
Bottlenecks affecting Test
Delays for
Business Scope
Provision,
Deploy, Test,
Delivery,
Integration, Build
Production
Releases
Continuous Changes
from Business
Water Fall Projects
Biz-Dev-Test-Ops are organizational aligned and interact formally
Integration test phase immediately follows Development to “enable testing”
Testing follows INT and is usually impacted by delays or bottlenecks
– Time delays to scope project
– Time to move changes through test to production
Change from business is continuous throughout the project
Feature deliveries
weekly or bi-weekly
Integrations are resource
intensive and manual
QA and beyond require
formal builds
Build and Deploy process are
governed but usually not
automated nor continuous.
Managed or virtual environments
are not necessarily impacted
Deployments are performed by
developers in Dev/INT and more
formally (by organizations) in other
environments.
Feedback to features / stories
may still be delayed by weeks
or provided by the developer
The need for INT is relative to
the testing performed during
iterations. QA is still formal
test within interface context
Water-Agile-Fall Projects
Biz-Dev-Test-Ops are organizational aligned and interact formally except for construction
Integration test phase may still follow Development to “enable testing”
Iterations achieve planning flexibility, but most testing still occurs after development
– Some departments may use periodic iterations to integrate, build, deploy, test
– Developers are usually burdened with doing their own testing during iterations
Change from business is continuous throughout the project
Feature / story
deliveries during
iterations
Integrations may still be
resource intensive but
performed more often
Iteration testing requires
periodic builds to verify
but may not test
Build and Deploy process are
either distinct or heavily
interdependent
Managed environments require 2-3
weeks to 2-3 months to prepare
Deployments are performed by
multiple organizations with different
processes/technologies
Feedback to features may be
delayed by weeks or months
or not at all due to time
INT and QA are first time to
perform test within interface
context
Agile Projects
Biz-Dev-Test are usually collaborative for Dev purposes. Ops may still be organizational
aligned and interact formally
Integration and QA test phase needs are reduced if testing performed during construction
Bottlenecks begin at
– Business planning to support iterative or accelerate release cycles
– Ability of environments, application deployments to keep up with agile teams
– Test team to support iteration testing – early and expand testing to exploratory/performance/loading
Change from business is continuous throughout the project
Stories delivered
during iterations
Integrations are ongoing
between team members,
components
Builds services may be
provided for individual,
team and applications
Build and Deploy process are
run often to verify and to run
test/scanning during process
Environment availability is
dependent on complexity of
changes and capacity of ops
Time through the “post construction”
pipeline is determined by quality of
code, risk and velocity of pipeline.
Feedback to developers to
dependent on periodicity, type
and quality of testing
Testing periodicity is
dependent on the provision,
deploy process capacity
On to Shifting Left
What is Shift Left Testing?
"Shift Left" is a DevOps practice that provides an effective means to perform testing with
or in parallel to development activities.
– Development, test and operations work together to plan, manage and execute automated and
continuous testing to accelerate feedback to developers
– The rate of the feedback is determined by an organization’s desired velocity.
– Technology is used to automate processes and virtualize components.
Testing may be performed as a part of the development process or as a service running in
parallel to development activities. In either case, shifting left accelerates feedback to
developers and improves the quality of code delivered to QA.
Shift left is an essential capability for Agile teams but may be successfully leveraged by all
development teams.
What Shifts Left?
Perspectives
– Testing: Begin test execution earlier
– Development: Act on feedback to achieve value
– PM: Resource allocation to address feedback
– Infrastructure: Shift left environments available
early, capacity drives availability
– Deployment: Accelerated capacity to meet test
periodicity
Activities:
– plan, create and automate integration test
cases in advance
– support a periodic cycle of integration testing
for the components/applications under
change
– prioritize, process, and resolve feedback by
development teams within the feedback
period
– plan, automate and deploy components,
applications and virtual services
– plan, create and virtualize component
services relevant to the development
activities
– execute delivery, integration, build, deploy,
testing, feedback and correction during the
defined periodicity.
Whatever testing you can perform shifts left - Integration testing could be
your first step as a means to provide a means to prepare for a formal QA
and reduce bottlenecks associated with Integration testing post
development.
Why Some Clients Shift Test Left
Move Defect Curve to left – Reduce costs because defects are cheaper earlier (Dev/Test teams cannot test
adequately)
– Change enough to find more defects early (Dev/Test teams have a long wait time to
get the right environment)
– No change to project management processes (We have too many configurations to
be tested. There is no consistent way to spin the right environments in time without
errors)
Reduce Issues in Production – Dramatic quality improvements earlier in pipeline to reduce risk to production
– Continue quality maturing through to production
– Support across technology
– Promotion criteria should ensure quality is met
Prepare for Continuous Delivery – Reduce bottleneck in QA and set stage for continuous delivery
– Start QA on delivery of first feature
– Ensure automation supports usage for Developer, Team, Application, Project
– Ensure the proper SLA’s and Reuse are supported
Support for Agile teams – Ability for Agile teams to test daily
– Ability to provide and act on feedback daily
– Ability to scale as Agile adoption scales
Where should you change?
Environment
The “What” The “How” The “Where” The “Verb”
• Application Development
• Application Deployment
• Environment Provision • All types of Testing
Steer Develop / Test Deploy Operate
Sc
ale
d
Reli
ab
le
Rep
ea
tab
le
Pra
cti
ced
Practice improvements
Define release with business objectives
Measure to customer value
Optimize applications
Use enterprise issue resolution procedures
Standardize and automate cross-enterprise
Automate patterns-based provision and deploy
Manage data and virtual services for test
Deliver and integrate and build continuously
Link objectives to releases
Centralize Requirements Management
Measure to project metrics
Link lifecycle information
Deliver and build with test
Automate testing
Embed Quality Reporting
Document objectives locally
Manage department resources
Manage Lifecycle artifacts
Schedule SCM integrations and automated builds
Test following construction
Plan and manage releases
Standardize deployments
Monitor resources consistently
Collaborate Dev/Ops informally
Plan and source strategically
Dashboard portfolio measures
Automate problem isolation and issue resolution
Optimize to customer KPIs continuously
Improve continuously with development intelligence
Test Continuously
Leverage Quality Tends
Manage environments through automation
Provide self-service build, provision and deploy
Adopting IBM’s approach: https://ibm.biz/BdDaMe
Plan departmental releases and automate status
Automated deployment with standard topologies
Monitor using business and end user context
Centralize event notification and incident resolution
Considerations
Overall
– Identify feedback velocity and means to
measure it
– Scope of automation pipeline (build, deploy,
test)
– Decide which changes will improve team’s
success (versus introduce functionality)
– Continuously improve and plan for next steps
(they will be needed)
Software Configuration Management
– Delivery and integration processes velocity and
measures
– Complexity that causes bottlenecks and delays
Build
– Time to build – number of application level
builds
– Accessibility for developer, team, application
– Periodicity of builds and disposition of results
Deploy
– Time to deploy
– Rate of deployment
– Scalability to extend to post deployment
actions
– Cross platform, approved versions and virtual
services
Test
– Provide complete or virtual environments
each event
– Only start testing early only then improve
– Test automation should follow immediately
Lifecycle Measurements 2008 2010 2012 – 2014 Total
Improvement
Project Initiation 30 days 10 days 2 days 28 days
Groomed Backlog 90 days 45 days On-going 89 days
Overall Time To Development 120 days 55 days 3 days 117 days
Composite Build Time 36 hours 12 hours 5 hours 700 %
BVT Availability N / A 18 hours < 1hour 17 hours
Iteration Test Time 5 days 2 days 14 hours 4 days
Total Deployment Time 2 days 8 hours 4 hours -> 20
minutes
2 days
Overall Time To Production 9 days 3 days 2 days 7 days
Time Between Releases 12 Months 12 Months 3 Months 9 Months
Innovation / Maintenance 58% / 42% 64% / 36% 78% / 22% +20% / -20%
Double-digit revenue growth, increased client adoption, improved client satisfaction
How IBM Rational Cloud Hosted Products have improved!
A Waterfall Client’s Experiences with Shift Left
Focus was on shifting integration test left, but not boiling the ocean on change. The
practices that would be affected included code changes, delivery and integration, build
automation, deployment automation. Test practices would merely execute earlier
Outcomes
Rework reduction: First time in 3 years to impact defect rate going into QA – 90%
reduction.
Practice impact: Demos of UI and retrospective improved requirements understanding and
business acceptance. No pilot project ever reported as medium or high risk during pilot.
Work reduction: Over 50% of testing was executed first week in QA almost 100% success
(normally done in weeks with almost 50% defects)
Collaboration: Qualitative improvement of end to end testing. Faster issue resolution and
cross team coordination. Automated build, deploy, unit test/scan and virtual service
activation processes.
First Steps – Integration Test Feedback
Environment
The “What” The “How” The “Where” The “Verb”
• Application Dev
• Code coverage
• Static Scans
• Code Reviews
• Security Scans
• Two/Screen
• Code deliveries
• Change Integration
• Binary management
• Documentation
• Technology maintenance
• Process execution
• Extensibility
• Speed, repeatability
• Scalability
• Environment consistency
• Levels of Delays
• Resource requirements
• Error prone processes
• Documentation
• Agility
• Availability
• Manual processes
• Automated processes
• Unit testing
• Regression testing
• Integration testing
• Requirements coverage
• Code coverage
• Test Data
• Virtual services/stubs
Shift Left Planning Workshop
Goals and
Assessment
•Client Capabilities: Current Status
•Business Goals for Improvement
•Solution Capability Oriented
•People, Practices, Technology, information
Capability
Priorities
•IBM Best Practices for Solution Capabilities
•Discussions to refine Objective Capability
•Prioritize Incremental Capabilities
•Vision, User Value, Pain and Complexity
Executive
Sponsor
Review
•Review Outcomes
•Assumptions & Risks
•Gain concurrence
•Identify next steps
Solution
Improvement
Roadmap
•IBM Best Practices for Adopting Solution
•Identify Key Milestones
•Roadmap activity to define actionable plan
•Define Quick Win Pilot
Da
y 1
D
ay 2
D
ay 3
D
ay 4
Current State
Assessment
Objective & Prioritized
Capabilities
Adoption Roadmap
Draft Results
Detailed 1-3 month roadmap
for initial win. High level
roadmap for remainder of
initiative. Executable week
following the workshop
Theme Activities Objective
Yes, we have tools that help
Rational Team Concert –Continuous Integration
UrbanCode Deploy –Application Deployment Automation
Rational Test Virtualization Server –Service Virtualization
Now what?
Check out our Practices Self Assessment Tool to assess your current
practices in context to Shift Left
– Ibm.biz/devops-practices-assessment
Review our Shift Left Practices @ Jazz.net
–Jazz.net/shift_left_practices
Contact us to discuss our workshop to plan your Quality improvements
Reach out to me for more discussion
–@paulbahrs
25
Next step - Shift Monitoring Left
26
© 2013 IBM Corporation
© Copyright IBM Corporation 2014. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
www.ibm.com/devops