Upload
rosalind-radcliffe
View
629
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
DevOps for the Mainframe Leveraging Continuous Integration, the cloud, and beyond to deliver z/OS applications Session SC-2402 David Myers, Product Manager, Cross-Brand DevOps [email protected] @Dave_Does Rosalind Radcliffe, Distinguished Engineer, Enterprise Modernization Solution Architect [email protected]
Please note
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
3
Social, Local, Mobile Smart Infrastructure Analytics
ERP Legacy DB
CRM HR
Manage workloads and maintain security
Rapid innovation in the cloud
User experience and mobile management
New era systems integrate existing operational systems with rapid delivery of new client-facing apps
Evolving customer and market expectations
Capabilities and User Experience Today Emerging
Primary Workload Types Systems of Record Transactional
Systems of Engagement (+ Record) Big Data, Analytics, Mobile/Social Channels
Time to Value Planned Opportunistic
Delivery Model Planned Incremental (DevOps)
Development and Operations Team Sizes 100s and Costly 10s with built-in DevOps automation
Release Frequency Months to Years Hours to Days, based on business opportunity
Integration Frequency Weeks Continuous
Service Sourcing Develop Consume and Assemble (Public and Private)
Operational Model Systems Management Built in to application, Recovery Oriented Computing, Continuous Availability
Infrastructure Deployment Days Minutes
Risk Profile Big-Bang (High Risk) Incremental
DevOps is… A set of principles and values that facilitate collaboration across disciplines to…
1. Enable rapid evolution of deployed business services
2. Reduce risk, decrease cost, and improve quality across the portfolio
DevOps Principles
§ Collaborate across disciplines
§ Develop and test against a production-like system
§ Deploy frequently using repeatable and reliable processes
§ Continuously monitor and validate operational quality characteristics
§ Amplify feedback loops
People
Process
Tools
Solution: A Continuous Delivery Pipeline
Development Testing Staging Production Business Owners Customers
• Validate on more production-like conditions earlier
• Automate hand-offs/promotions to increase velocity through the different stages
• Standardization on processes and assets between Dev and Ops
• Automated monitoring and dashboarding of quality and performance against service level agreements at multiple stages
Ensure applications are production-ready throughout the lifecycle and can be released at any time while minimizing rollback due to quality issues
Mainframe Delivery Pains…
§ Multiple teams working across restricted dev and test capacity lead to conflict, delays, or bad test results in shared environments
§ Complex and manual management and configuration tasks result in errors and delays
§ Too much bad code going into test and production causes crit sits and emergency fixes
§ Bottlenecks due to inefficient communications between disparate platforms and teams (Dev/Test - System Programmers; mobile – distributed-mainframe)
Go on Offense
Play Defense
…solutions from IBM
§ Provide cheap, isolated, development and test environments for project teams
– Rational Development and Test Environment
– Rational Test Virtualization Server – SmartCloud Provisioning – Cloud Ready for Linux on System z
§ Automate consistent build, configure, and deploy processes across all stages
– Rational Team Concert – UrbanCode uDeploy – SmartCloud Orchestrator
§ Enforce base quality standards automatically prior to promotion
– Rational Test Workbench – Rational Quality Manager – SmartCloud Application Monitoring – Omegamon
§ Improve communication and collaboration with cross-platform release planning
– IBM Collaborative Lifecycle Management
– Smart Cloud Control Desk – UrbanCode uRelease
Test Test
Dev Dev
1
DevOps for the common mainframe delivery cycle
Dev Test Staging Prod
Development" Test" System Programmer"
3
2 2
4
10 10
Test LPAR
z/OS
…
Typical z/OS Testing Architecture Organized by project team, vertically scaled, sharing resources, limited automation
Project Team [April Maintain]
Project Team [Prototype SOA]
Project Team [June New Func]
Project Team [Dec Sys Upgrade]
Test Data
App
App
App
Problems Encountered
1. Shared resources combined with overlapping schedules can elicit conflicts, impede innovation and slow code delivery
2. Coordination of environmental changes and releases cause bottlenecks, delays and additional overhead
3. Shared test data is difficult to manage and can lead to over testing or incorrect test results
The ultimate in modern application development for System z
§ Liberate developers to rapidly prototype new applications § Develop and test System z applications anywhere, anytime! § Free up mainframe development MIPS for production capacity § Eliminate costly delays by reducing dependencies on operations staff Note: This Program is licensed only for development and test of applications that run on IBM z/OS. The Program may not be used to run production workloads of any kind, nor more
robust development workloads including without limitation production module builds, pre-production testing, stress testing, or performance testing.
RDz & ISPF user
ISPF user RDz user
RDz user RDz user
COBOL, PL/I, C++, Java, EGL, Batch, Assembler, Debug Tool
x86 PC running Linux
RDz user
IMS
z/OS
WAS
DB2
MQ CICS
Rational Development and Test Environment for System z
Simulate subsystem dependencies – Test impact of latency – Test application response to unresponsive
services Create a stable test environment – Simulate “public” services – Reduce capacity requirements on
infrastructure – Improve security by reducing access from
external services
Improve testing with Green Hat service simulation Decouple subsystem teams using Green Hat
Databases Mainframe applications
Third-party Services
Rational Test Virtualization Server
Application changes being tested
virtualized services
Testing with dependency virtualization
Phase 1 Phase 2 Phase 3
App
Test Case
Virtual Services
3rd Party Call
IMS Data Access
CICS Commarea Call
App
Test Case
Virtual Services
3rd Party Call
IMS Data Access
CICS Commarea Call
App
Test Case
Virtual Services
3rd Party Call
IMS Data Access
CICS Commarea Call
Controlled large system testing by isolating components under test • Easier problem determination • Lower test environment capacity requirements • Improved component quality
Provision new dev / test systems in minutes • Fast VM provisioning for near-instant
deployment of 100s of virtual machines and scale based on business needs
• Rich set of web interfaces into the cloud that can be driven by a user or scripts for complete automation
• Reliable, non-stop cloud capable of automatically tolerating and recovering from software and hardware failures
• Near-zero downtime due to faults, hypervisor/management software upgrades or addition/removal of hardware
Requested VMs will be up and running under a
minute using standard HW
Function Test
Performance Test
§ Fast, dependable, automatic feedback speeds time to market § Lower cost of application testing using off-mainframe z/OS test environment § Enables confidence by automatically tracking and promoting code health
Continuous Integration for z
Developer Frequent, rapid feedback earlier to improve quality
Rapid Feedback
Code Check In + Build + Deploy +
Run Automated Tests
Change Request
Automated, continuous testing with isolated development and test environment
Reusable integration tests simplify testing across layers
Promote To QA
Higher quality going into QA
Reduced delivery time, end-to-end visibility of test activities, safer and faster upgrades (V2V) Fail fast and drive better quality downstream
System z Continuous Integration for System z Scenario
16
SCM/CI Server
(Rational Team Concert)
Test Environment (RD&T or LPAR)
Test Execution Manager
(Rational Quality Manager)
Test Automation
(Rational Test Workbench) 1. Check-in code
2. Build code and zUnit tests
3. Deploy build results and test data to Test Environment
4. Execute zUnit Tests
<Deploys App+zUnit Tests> <D
rives>
5. Kick-off Automated Test Plan 6. Run automated
interface tests against Test Environment
7. Mark execution records Pass/Fail in Test Execution Manager
<Starts Tests>
8. Report test results in dashboard/build results/defect records in CI server.
<Results>
§ Each tool came with its own • UI - Web and desktop
presentations of views and tasks
• Logic – Workflow, process, search, query, scale, security and collaboration
• Storage – Availability, traceability
• Privacy, backup/archive § Resulting in...
• Brittle integrations • Silos everywhere • High cost to maintain and
administer • Proprietary API's
Tool E SCM
UI
LOGIC
DB
UI
LOGIC
DB
UI
LOGIC
DB
UI
LOGIC
DB
UI
LOGIC
DB
Tool B Release Planning
Tool D Incident
Management
Tool C Reporting
Tool A Dev work Items
Separate tools makes collaboration inefficient or difficult...
Tie development release to operational release Rational Team Concert Release Plan and work items
SmartCloud Control Desk
19 19
Delivery Organized for Flexibility and Speed Organized by application team, horizontally sliced, dedicated resources, highly automated
Problems Encountered 1. Shared resources
combined with overlapping schedules can elicit conflicts, impede innovation and slow code delivery
2. Coordination of environmental changes and releases cause bottlenecks, delays and additional overhead
3. Shared test data is difficult to manage and can lead to over testing or incorrect test results
4. Provisioning, managing, and synchronizing project test environments including data
QA
/Inte
grat
ion
Test
LPA
R
z/OS LPAR
…
Project Team [April Maintain]
Project Team [Prototype SOA]
Project Team [June New Func]
Project Team [Dec Sys Upgrade]
Integr. Test Data
LPAR
Data
RD&T
Data
RD&T
Data
LPAR
Data
What are we working on in the lab…
Go on Offense
Play Defense
…solutions from IBM
§ Provide cheap, isolated, development and test environments for project teams
– Rational Development and Test Environment
– Rational Test Virtualization Server – SmartCloud Provisioning – Cloud Ready for Linux on System z
§ Automate consistent build, provision, configure, and deploy processes across all stages
– Rational Team Concert – SmartCloud Continuous Delivery – SmartCloud Orchestrator*
§ Enforce base quality standards automatically prior to promotion
– Rational Test Workbench – Rational Quality Manager – SmartCloud Application Monitoring – Omegamon
§ Improve communication and collaboration with cross-platform release planning
– IBM Collaborative Lifecycle Management
– Smart Cloud Control Desk
Cloud-style deploy of zLinux and z/OS dev and test patterns on RD&T or
zHardware
Infrastructure as Code
Source Control Management
Source Artifacts
.cbl
.asm
.cpy
CLIST .rexx
Provisioning using Cloud
#!/usr/bin/env ruby
class DevopsDeployer def initialize(build_url, build_id) @log = Logger.new(LOG_FILE) @log.level = LOG_LEVEL @iaas_gateway = IaasGateway.new(HsltProvider.new(),LOG_FILE, LOG_LEVEL) @server_instance = nil rtc_build_system_provider = RtcBuildSystemProvider.new(RTC_REPOSITORY_URL, RTC_USER_ID, RTC_PASSWORD_FILE) @build = rtc_build_system_provider.resolve_build(build_url, ENV['buildResultUUID'], build_id) @build_system_gateway = BuildSystemGateway.new(rtc_build_system_provider, LOG_FILE, LOG_LEVEL) end def add_build_stamp template_file = WEB_APP_ROOT + "/app/templates/pages/page.html" @log.info "Adding build ID stamp #{@build.id} to \#{template_file}" # Read in the file's contents as a string, replace # the build_id, then overwrite the original contents # of the file text = File.read(template_file) new_text = text.gsub(/\{\{ build_id \}\}/,"<a href=\"#{@build.uri}\">#{@build.id}</a>") File.open(template_file, "w") { |file| file.puts new_text } end
# ...
Delivery Pipeline Fundamentals
Deploy
Environment Running System
Source Control Management
Source Artifacts
.cbl
.asm
.cpy
CLIST .rexx
Deployable Artifacts
Build, Package, & Unit Test Application Binaries & Platform Configuration
Library
Consolidate, standardize, simplify deployment
COBOL
Java
Copybooks
WSDL
DB2 DDL IMS DBD/
PSB
HTML
MQ Queue
Def
DB2 Plans
Procs
JCL
Deploy
Environment Running System Deployable Artifacts
Build, Package, & Unit Test Application Binaries & Platform Configuration
Library
Enterprise Applications have thousands of disparate parts • Currently maintained in separate systems, if maintained at all • Limited linkage between systems for application and configuration
dependencies • Missing assets and information is rampant DevOps force linkages, automation, and standardized packaging…
Source Management
Create, secure, and deploy test data
2TB
-Subset
Production or Production Clone
-Compare -Refresh
Create “right-size” production-like environments
for application testing De-identify sensitive information with realistic but fictional data for testing & development purposes
De-identify sensitive information with realistic but fictional data for testing & development purposes
z/OS
UT RD&T
Automated test data deployment for each
build and test
1 GB
z/OS
UT LPAR
1 GB Masked Test Data
Automated Build & Deploy Test Data
Automated Testing
Optim RTC
Test Data
Standardize z/OS region configurations to ease deployment
• Standard topologies exist today (production LPARs)
• Standardized regions which can be repeatedly deployed are rare
• Standardized/Automated deployment of COMPLETE system is spotty
To adopt the cloud for testing… Standardize and automate provisioning of everything …
Standard Middleware Pattern
CICS 5.1 IMS 11.1
DB2 10.1 Z/OS 1.12
Configuration
Data
Application
RTC
/CIz
Ava
ilabl
e C
apab
ilitie
s
Standardize z/OS region configurations to ease deployment
Standard Middleware Pattern
CICS 5.1 IMS 11.1
DB2 10.1 Z/OS 1.12
Configuration
Data
Application
Pattern/Platform
Application load modules
Test Data
JCL and REXX
Evolving to deploying regions … cloud-style
“Cloud” RD&T Or LPAR
RD&T Or LPAR
RD&T Or LPAR
RD&T Or LPAR
SmartCloud Orchestrator +
<Deployment Automation for Mainframe project>
Region/Middleware Patterns
CICS+DB2
CICS+ IMS DB
Batch
29
Daily Apple TV giveaway § Complete your session surveys online each day at a conference kiosk or on
your Innovate 2013 Portal!
§ Each day that you complete all of that day’s session surveys, your name will be entered to win the daily Apple TV!
§ On Wednesday be sure to complete your full conference evaluation to receive your free conference t-shirt!
30
Acknowledgements and disclaimers
© Copyright IBM Corporation 2013. All rights reserved. – U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
IBM, the IBM logo, ibm.com, Rational, the Rational logo, Telelogic, the Telelogic logo, Green Hat, the Green Hat logo, and other IBM products and services are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml If you have mentioned trademarks that are not from IBM, please update and add the following lines: [Insert any special third-party trademark names/attributions here] Other company, product, or service names may be trademarks or service marks of others.
Availability: References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates.
The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.
© Copyright IBM Corporation 2012. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
www.ibm.com/software/rational