Migrating a Large Scale Legacy IT System to a System with ...conferences.computer.org/stc/2014/papers/5034a039.pdf · Migrating a Large Scale Legacy IT System to a System with Current

Migrating a Large Scale Legacy IT System to a System with Current

Technology

Bill Wood, Mike Gagliardi, Phil Bianco

4/18/2014 Migration Planning 1


Copyright 2014 Carnegie Mellon University and IEEE

This material is based upon work funded and supported by the Department of Defense under Contract No. FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center.

NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN “AS-IS” BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.

This material has been approved for public release and unlimited distribution.

This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at [email protected].

DM-0001020

BackgroundAgency wanted to Migrate two paired and tightly coupled systems

– Both are 24/7/365 and have disaster recovery at multiple sites– Both have many classes of users

1. Service Based System - 15% functionality (going to 40%)– Java, Relational DB, COTS tools, Service-based, J2EE, SAP, Informatica

2. Legacy System - 85% of functionality (going to 60%)– COBOL, Hierarchical DB, mainframe

Migrate to a well defined target reference architecture (TRA) as a basis for a common platform infrastructure (CPI) – developmental, operational, test

• Briefing is focused on the Legacy Migration


Its Complicated

Understanding the Legacy System Architecture• Infrastructure Tools• Application Component Relationships (Data and Code)• Business Process Threads

Understanding the Target System• Target Reference Architecture (TRA)

• SOA; Layered Infrastructure• Designing Services on top of TRA• Business Process threads

Mapping between Legacy and Target in Phases• Architecture mismatches (development, operational, certification, sustainment, COTS)• Operating with dual authoritative data systems, cutover, synchronization• Relationships between Application Components (Legacy versus TRA, COTS)


It’s Worrisome• Is there too much code/ data coupling and

spaghetti code to partition for migration?• Are the current business processes and

screens appropriate?• Is the CPI (and TRA ) stable?• Are COBOL to Java transformation tools up to

the job? • Can we operate with 2 systems overlapping

authoritative data?• Do we have sufficient technology expertize in:

legacy system, target reference architecture, discovery and analysis tools, transformation tools?

• Is the business logic only understandable in the legacy code?

• How can we overcome the lack of architectural documentation?

• Will the entire testing and certification process change?

• Will I be creating a maintenance nightmare?• It’s a 24/7/365 operation- no downtime!


Keep the Right Vision

• The TRA is a good start, but an application architecture with data modeling is needed• Synchronize with phased TRA and CPI infrastructure deployment• Operating with multiple sources of authoritative data• Using new technology (e.g. customized security to infrastructure security)• Capability of transformative toolsets


Don’t Live in a Dreamworld


• Big-Bang changes usually fail• Conducting transactions across networks

and keeping response times satisfied!• Transforming spaghetti code automatically

• Separating presentation from business processing from data access!

• Moving from green screens to windows• Making multiple types of changes

simultaneously!• Edict no changes to the legacy system!

Paul Delvaux

Target Reference Architecture


Appl

icat

ion Presentation Layer

Business LayerData Access Layer

Data Layer

Data

Metadata

Data Warehousing

Transactional Data

Infr

astr

uctu

re

Commodity Hardware

Linux Software

Load BalancingFailover

Integration Layer

Secu

rity

This is a good startBut it is not enough

Need a system and application software architecture• End-state• Each Release

Migration ApproachesAn RFI was conducted and multiple approaches suggested• Lift and Shift (L&S)

– Reduce Operations and Maintenance (O&M) Costs ASAP• Fast retirement of legacy infrastructure

– Move to the CPI• Don’t come close to satisfying the TRA • Make minimal changes- keep COBOL, Hierarchy etc. etc.

• Modernize – Phased transformation to TRA

• Keep legacy architecture• Change technology- Java, SOA• Use Conversion tools?

• Re-engineer– Re-Develop the system in Java on CPI– Developed system and application architecture– Satisfy TRA


Options Considered1. Do nothing (baseline)2. L&S (baseline)3. L&S and modernize4. L&S and re-engineer5. Re-engineer6. Hybrid- L&S, modernize, re-engineer

Its not reasonable to explore every possible option (combination of approaches), and then to explore every possible implementation alternative • Leads to analysis paralysis• Have to make some recommendations

incrementally and choose alternative approaches• Build a high level Target Architecture• Map legacy elements to TA elements

• Code, data, test, configuration• Keep vs Throwaway

• Perform an Architectural Evaluation (and fix)

List the AlternativesConduct Discovery and Analysis of Legacy

Understand TRA and CPIReview Migration Toolsets

Develop Issues for ResolutionDevelop Design Factors

Migration Process

Determine and Score Options

Explore implementation alternatives for options

Build a Roadmap

Build an End State Architecture


List the OptionsDevelop evaluation criteria

Score the optionsMake a selection

• Goals for sequencing• Constraints on Phasing• Approach to Migration

• Data, code, user, business processes ordering• Quality attribute considerations• Migration tooling

• Define phases• Groups

• Legacy: functionality, code, data BP, users• Tiers/layers• Transient code in Legacy and TA• Throwaway

• Align with Infrastructure Roadmap

0.00

5.00

10.00

15.00

20.00

25.00

30.00

Do N

othi

ng

Lift

&Sh

ift (L

&S)

L&S

and

Mod

erni

ze

Re-E

ngin

eer

L&S

and

Re-E

ngin

eer

L&S

and

Hybr

id

PerformanceCostRisk

Legacy Relationships

Architecture Documentation

Subject Matter Experts

System usage measurements by elementPerformance, Interfaces, usersScoping parameters

Static relationships using toolsCode, data, configurations, interfaces

Important Operational Quality Attributes (QA)Recovery- failures, disastersSecurity

Important Developmental and Sustainment QAs

Business Process ThreadsUsers, code , data, interfaces

Element

Platform

Language

Database

User Interface (UI)

Transactions

Business Process

Architecture Doc

Standards

Quality Attributes

Developmental Operational Sustainment

Infrastructure

Goal- Examples

Reduce operation and maintenance (O&M) costsLicensing, obsolescence, electric, space, peopleVendor lock

Create technical uniformity across organizational boundaries

Take advantage of new technology

Determine and Score OptionsUnderstand Migration

Goals

Create and Evaluate Options

Recommend an Option




Build a Roadmap


Choose an Option

Best Overall Measure - cost, risk, performance

Determine ordering: sequential, concurrently

Options

6 mentioned previously

Initial crude evaluation against TRA

Detailed evaluation against Cost, risk, performance factors (23)

We started with the team’s architectural knowledge• 40 or so factorsWe discovered the OMB factors -19 factors• Merged them into the initial factors- 50 or soWe started to weight the factors• Too many factors had an equivalent distribution of weightsWe combined factors with like distributions- 23 factors

Evaluation FactorsCost Performance Risk

0.0010.0020.0030.00


W Factor 1 2 3 4 5 6

3 How is data organized wrt TA : relational, normalized, distributed, partitioned

1 1 5 8 8 8

• Partitioning of software and data for L&S causes cumbersome migration

• Are there tool factors that make the schedule unpredictable: learning curve, tool underperforms requiring extended manual intervention

• What is the risk of technical obsolescence of the end state?

• Extent of external interfaces (OGA, Commercial)

• How solid is the architectural foundation, supported by documentation, at the end state?

• How much will be the Impact on Users Interface functionality?

• How well can the two migrations be coordinated?

• Risk of creating a monopoly for future procurements

• How complex will the program management effort be?

• How much business functionality will be negatively affected?

• Budget/ Cost/Schedule profile goes out of alignment

• How much will be the migration labor cost?

• How much recovery of investment cost after cancellation

• How early will cost savings happen- license and support during migration?

• How much will be the on-going legacy license and/or support costs post migration?

• How readily can new functionality be put into production during the migration process- e.g. adaptive maintenance, changing user workflows

• How well does the end-state system satisfy the TRA architectural structures (application layers, understanding, tiers, SOA) for developmental and sustainment attributes (extensibility, maintainability, reuse across applications, standards)

• How well does the system meet the operational attributes: satisfy end-user response under maximum workload, scalability, manageability, load balancing, reliability, availability)

• How is data organized wrt TRA : relational, normalized, distributed, partitioned

• How many internal interfaces changes will be needed

• How much will be the level of effort for testing to make it production ready

• How well are the TRA security conditions satisfied: easily changed, no unauthorized access, no data corruption

• What will be the impact for data synchronization, parallel run and cutover?

• How modernized are the user screens wrt the TRA

There was a separate ROM cost team• Developed a ROM for each option• We provided them with data as a basis for estimation• Their ROM sizing was about the same as our cost factor sizing

0.00

5.00

10.00

15.00

20.00

25.00

30.00

Selected Hybrid Option- Example


Legacy Phase 3Phase 1 Phase 2

Two Systems with Shared Authoritative Data

Modernize

Lift and Shift

Re-engineer

How to partition: L&S and modernize; and Re-engineer• Significant Business Process Changes – re-engineer• Layered stable legacy code- L&S and modernize• Experiment with modernization tools

Legacy

L&SRe-engineered

Modernize

# Description

1 Move the Data then move the Code

2 Move the Code then move the Data

3 Move by groupings of Code and Data and Users and End-to-End Business Threads

4 Clean up legacy in situ first (or not)

Assumptions • The hybrid Analysis showed the business processes, screens,

software and data elements to be Lifted and Shifted.• It is preferred that the L&S be done in stages



Build a Roadmap


Design Factors

Explore Implementation Alternatives-Lift and Shift


Discovery and Analysis

Alternatives

Issues for Resolution # Factors

1 Cutover

2 Rollback

3 Synchronize

4 Data Copy

5 Performance

# Issues for Resolution

1 Functionality/Data may be too entangled to group effectivelySuch that authorized data split can take place

2 What toolset is best for converting data structure, copying data, modifying APIs & JCL, writing adaptors?

3 Replacing transactional support

4 Should we do some cleanup before L&S

Finds the relationships between• The subsystems and their programs• The Data Stores and their files• The subsystems and the data stores• End-to-end business threads/User classes/subsystems/data stores• Identify Dead code and dead data• Legacy API’s usage• Legacy transactions usage • Batch usage

What commercial toolsets are best to do this?


Develop a Basis

Build a Target Architecture

Evaluate Target Architecture



Explore alternativesfor options

Build a Roadmap


Conduct an Architecture Evaluation• Identify risks and risk themesUpdate the architecture to mitigate the risks• Keep a risk register

• TRA and CPI constraints on End State Architecture• Legacy System Architecture Findings• Relationships between Legacy and Target COTS

products• Identify technical challenges and risks

Identify the Target Application Elements• Services, data models• References to Legacy elements• Interfaces to external actors• TRA elements usedBuild an Application Target Architecture• Connections between Application elements• Business process flows through the elements• User screen contents (appropriately mirror the legacy)• Map to TRA elements used

Build an Architectural RoadmapDevelop a Basis

Define a Pilot

Define Next Phase

Evaluate Consistency

Define Migration Scenarios• Developmental, operational, sustainmentEvaluate the roadmap against the scenarios• Capture risks, gaps, sensitivities, tradeoffsUpdate the roadmap• Establish Risk repository

Issues not AddressedTasks involving transformation of legacy artifacts to TATools used in the transformation

Phase selection can be done in a number of ways• Business processes, users, services, authorized data Define the tools being used for migration• Data, code, screens Define the TA contents of the phase • Business processes, screens• Tables, data sources, services and end-points, screens• User impacts• Workarounds and Technical DebtDefine the TRA • Infrastructure, COTS tools, platforms, networksDefine the Legacy system artifacts being migrated• Subsystems, tables, dataDefine the quality attribute considerations• Cross –coupling between systems • Data transfer, cutover, synchronization,• Recovery from failure- routine and disastrousConfirm that all events synchronize



Explore alternativesfor options

Build a Roadmap


Goals• Low hanging fruit, budgeting, scheduling, early success• Reduce legacy sustainment costsExternally induced schedules• aligning phased capabilities with external milestonesOrganizational boundaries • Infrastructure• data model, applications, UI• certification testing- per release?Constraints on Phasing (Releases)• User impact, security, site rollout, authoritative data• Platform and networks and COTS productsTarget System Infrastructure• in-place, build epics during migrationApproach to migration• Data first, code first, code/data grouping, BP firstIdentify technical challenges and risksTools to assist migration

Architectural views for every major release, that correlate to the End State Architecture and the Legacy Architecture. • Shows how the legacy elements are replaced at each stage• Shows what end-state elements are introduced at each stage• Describes the correlation between the legacy and the end-state• Describes how the users interact

Select a Pilot Migration for First Phase• Lightweight, low risk, few challenges • Gain target and transformation tool experience• Include data extraction, analysis, and reporting• Perhaps no cutover• Happy path functionality

• But include some quality attribute drivers• Aligned with TRA infrastructure development schedule• Identify target architectural elements needed

• Business Process Threads, Screens• Services, data models, COTS tools

• Identify legacy architectural items being migrated• Screens, Reports• Subsystems (partial), tables (partial), data sources

Management • Changing political environment• Drift and Re-direction can become a way of life

Summary


After The Party

Technical• Criteria for selecting between options

• OMB guidelines were good• Plan, but don’t overdo it• Architect

• Understand Legacy• Include quality attributes• High Level End State

• Roadmap• Detailed architecture at each phase

Documents

Migrating a Large Scale Legacy IT System to a System with ...conferences.computer.org/stc/2014/papers/5034a039.pdf · Migrating a Large Scale Legacy IT System to a System with Current