Optimizing Cost of Quality in Legacy - HCL Technologies services provide an end-to-end solution to optimize the ... spent on Appraisal and dealing with Failures. ... Optimizing Cost

Optimizing Cost of Quality in Legacy An investment to increase ROI

Executive Summary

Quality processes cannot be followed only because someone else is following them, but return on following them has remarkable impact as projects mature. The cost of poor quality can result in raising the total project cost significantly. Most businesses do not know what their quality costs are because they do not keep reliable statistics. Finding and correcting mistakes consume tremendously large portion of resources. Typically, the cost of elimination of a failure in the production stage is five times greater than it is at the development stage. Effective quality management decreases production costs because the sooner an error is found and corrected, the less costly it will be to rectify it. As a legacy system ages, the optimization of cost of quality contributors gets complex. With more and more job abends, less/non SME availability, etc., the cost to maintain good quality goes higher. Eventually it can become very challenging to manage the system. HCL-MMS services provide an end-to-end solution to optimize the factors which are impacting the cost of quality for a legacy system. It not only focuses on the visible parameters impacting the cost of quality, but also the concealed and interlinked parameters, giving a holistic solution.

Purpose

The purpose of this paper is to epitomize how to optimize the CoQ for a legacy system and its benefits in the long run. It also highlights HCL Mainframe and Midrange Services‟ capabilities to optimize the cost of quality in a legacy system.

Background

The theory behind the Cost of Quality (CoQ) Model is that, the more time that is spent on Prevention activities, the less time will need to be spent on Appraisal and dealing with Failures. Prevention activities build quality into the system. Both Appraisal and Failure are “after the event”, and therefore are too late and add cost. The costs associated with quality are divided into two categories: costs due to poor quality, and costs associated with improving quality. Prevention costs and appraisal costs are costs associated with improving quality, while failure costs result from poor quality. An organization‟s primary goal is to maintain high quality services. With a comprehensive understanding of the costs related to quality this goal can be achieved.

2

Optimizing Cost of Quality in Legacy - An investment to increase ROI

© 2011, HCL Technologies. Reproduction Prohibited. This document is protected under Copyright by the Author, all rights reserved.

Figure 1: Cost of Quality classification

Cost to achieve good quality To ensure that customers will receive a product or service that is worth the money they will spend, firms should spend on prevention and appraisal costs. Prevention costs are associated with preventing defects from occurring. The focus of a prevention cost is to assure quality, and minimize or avoid the likelihood of an event with an adverse impact on the product or service. This also includes the cost of establishing a quality system. A quality system should include the following three elements: training, process engineering, and quality planning. Quality planning is establishing a production process in conformance with design specification procedures, and designing of the proper test procedures and equipment. Training is establishing training programs for employees to keep them efficient on emerging technologies, such as updated languages, platforms, and programs. Appraisal costs are direct costs of measuring quality. This includes: testing, reviews, costs associated with assessment for ISO or other quality award assessments.

Cost due to poor quality The cost due to poor quality is accounted from the failure costs, which is divided into two categories: internal and external. Internal failure costs are expenses incurred from internal failure. This includes cost of rework to fix the pre-shipment defects. External failure costs are associated with product failure after the completion of the production process.

Failure costs take many forms: The effort that developers spend for investigating and

diagnosing defects and then reworking designs and code to correct them.

Schedule slippage as testing uncover defects that require rework and re-testing.

The customer support costs, most of which are for helping customers to deal with all of the defects shipped to them, while developers spend even more time in investigation and rework.

3



Cost of Quality

Cost due to

poor quality

Cost to achieve

good quality

Internal

failure cost

External

failure cost

Prevention

cost

Appraisal

cost

Statistics According to a study commissioned by the U.S. Department of Commerce's National Institute of Standards and Technology (NIST), software bugs, or errors, are so prevalent and so detrimental that they cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the gross domestic product,. At the national level, over half of the costs are borne by software users and the remainder by software developers/vendors. The study also found that, although all errors cannot be removed, more than a third of these costs, or an estimated $22.2 billion, could be eliminated by an improved testing infrastructure that enables earlier and more effective identification and removal of software defects. These are the savings associated with finding an increased percentage (but not 100 percent) of errors closer to the development stages in which they are introduced. Currently, over half of all errors are not found until "downstream" in the development process or during post-sale software use.

Figure 2: Cost of inadequate testing

If the defects are not fixed early in the life cycle, the rework effort due to external failure can turn out to be very expensive.

CoQ in a Legacy System

In the previous section, we defined the three major costs of quality and discussed their application. We can now examine how they affect mainframe system. The more an IT project invests in preventive measures, the more they are able to reduce failure costs. Furthermore, an investment in quality improvement benefits the organization image, efficiency, performance and growth. This is summed up by the Ludvall-Juran quality cost model, which applies the law of diminishing returns to these costs. The model shows that prevention and appraisal costs have a direct relationship with quality conformance, meaning they increase as quality conformance increases. Thus, quality conformance should have an inverse relationship with failure costs – meaning as quality conformance increases failure costs should decrease. Understanding these relationships and applying the cost of quality process enables an organization to decrease failure costs and assure that their products and services continue to meet customer expectations.

4



The important reasons to invest in reducing the cost of quality for a legacy system are:

The cost of the application(s) developed on mainframe system is much higher than the cost of the application(s) developed on distributed systems.

The most critical business processes run on the mainframe. The failure costs of the application(s) on the mainframe system have more impact on the organization than the failure costs of the applications(s) on the distributed environment.

The mainframe environment is much complex than the distributed environment.

The various time indices like development time, time for ROI, application life time are much longer for mainframe system.

Mainframe systems, product suites and architectures are entirely different than distributed environment. Moreover, these applications control the critical business flows.

With emerging business needs, mainframe systems are changing continuously – be it SOA, SOI, or Cloud.

The major attributes to legacy system cost of quality are:

Number of unscheduled outages Complex architecture Inefficient planning, release management, processes, etc. Availability of batch window Time to restore the system in case of failure Time to deploy a new application

HCL-MMS Services to Optimize CoQ for Legacy System

HCL‟s Mainframe & Midrange Services (MMS) is 15+ year old practice with partnerships with leading ISVs and vast experience across portfolio of Services. HCL-MMS services provide an end-to-end solution to optimize the factors which are impacting the cost of quality for a legacy system. It not only focuses on the visible parameters impacting the cost of quality, but also the concealed and interlinked parameters, giving a holistic solution.

5



Figure 3: CoQ Iceberg

Optimizing Prevention Cost for mainframe projects Managing preventive costs are the most effective way to keep the cost of quality in control. The cost of preventing the defect is much less when compared to rework cost. ASSeT™ is HCL‟s proprietary framework for Knowledge Acquisition Plan / transition. The major phases involve due diligence, Initiation, Absorption, Replication, Observation, Operation. It also provides governance all across the phases during transition. This helps in building a robust knowledge repository of the entire legacy system. It also helps create cross-references between various components.

Figure 4: ASSeTTM Framework

The HCL-MMS has a defect management methodology developed for the mainframe projects, which provides end-to-end solutions for the defects/failures/abends. A detailed root cause analysis is done for the

6



defects logged in the defect log, from where the action plan to eliminate the defect is arrived at. Role specific training is conducted for the defect prevention coordinator, project manager, team lead, and team members.

Optimizing Appraisal Cost for mainframe projects

The Mainframe Testing Service Offering targets to reduce the cost of quality in the testing phase of the mainframe project. The factors which influence the CoQ of the mainframe project are studied thoroughly and suggestions are given to optimize them. To achieve the above purpose, HCL-MMS has the following proprietary framework of mainframe testing and also providing the in-depth coverage of various testing types. MMS Mainframe testing framework is built based on the industry best practices, proven methodologies, reusable components, and automation tools.

Figure 5: Mainframe Testing Framework

Optimizing Failure Cost for mainframe projects

HCL-MMS service provides an assortment of services to optimize the failure effort. HCL-MMS has the proprietary Analyzer tool which helps in identifying the redundant code, dead code in the mainframe application, thereby optimizing the existing code. This will help in handling the future enhancements efficiently.

HCL-MMS suggested guidelines for optimizing the number of abends/failures of maintenance projects:

7



Plan the operations (weekly/monthly/quarterly schedules) Define procedures/policies for the operations and maintenance

department Define SLAs Define roles & responsibilities Developing / procuring the required tools to analyze the abends Building the database of problems and their solutions/work

around Well planned change management Buy-in from relevant stakeholders

CoQ Optimization for Application Development projects The development project can be broadly classified into the following phases: Initiation, Development, Testing, and Closure. The execution phase deals with carrying out the entire project life cycle. For each of these phases, certain practices are identified to optimize certain aspects of the CoQ.

Initiation Phase: Following are the practices adopted in the initiation phase to optimize the cost of quality.

Sl.

No

Practice Cost of Quality

1 Effective requirements strategy Preventive Cost

2 Deploying of SMEs for Design, Architecture

reviews

Preventive Cost

3 Identifying and deploying the Third Party /

In-house tools for code verification, test data

generation, test case generation, generation

of SQL scripts, effective defect management,

SCM

Appraisal cost

4 Identifying the deployment strategy to

minimize the production failures

Preventive Cost

5 Identifying the Training requirements and

implementing

Preventive cost

6 Identifying and implementing the

industry/organization Best Practices &

Lessons Learnt

Preventive cost

Table 1: Initiation phase practices

8



Development Phase: Following are the practices adopted in the development phase to optimize the cost of quality.

Sl.

No


1 Identifying the existing

programs/modules/sub-routines , test cases,

test data, JCLs, templates for re-usability in

the beginning of the development saves a lot

of effort, there by reduces the effort in

reviews, testing

Appraisal cost

2 Defining and communicating the Coding

standards / Guide lines in the beginning of

development

Preventive cost

3 Using the tools for Code analysis, reduces

code review effort

Appraisal cost

4 Implementing efficient DP plan, risk

management

Preventive cost

Table 2: Development phase practices

Testing Phase: Following are the practices adopted in the testing phase to optimize the cost of quality.

Sl.

No


1 Prioritize testing according to risk (Risk

Based Testing)

Appraisal cost

2 Implementing automation testing tools Appraisal cost

3 Identify repetitive and time consuming

activities in the testing phase and automate

them

Appraisal cost

Table 3: Testing phase practices

9



Closure Phase: Following are the practices adopted in the closure phase to optimize the cost of quality.

Sl.

No


1 Identifying the Best Practices and Lessons

Learnt from the project

Preventive cost

2 Identifying the newly built reusable

components from the project

Preventive cost

Table 4: Closure phase practices

Project Execution Phase: Following are the practices adopted in the execution phase to optimize the cost of quality.

Sl.

No


1 Effective risk monitoring Preventive cost

2 Effective defect monitoring Appraisal cost & Failure

cost

Table 5: Execution phase practices

CoQ Optimization for Application Maintenance & Support projects Usually the efficiency of a legacy application during the maintenance and support period is measured based on the number of job abends, total system failures, problem tickets raised etc. Reduction in these numbers show that the system is working efficiently.

Following are the practices adopted in a maintenance & support project to optimize the cost of quality.

10



Sl.

No


1 Building the database of problems / job

abends with reasons and root cause

analysis

Preventive cost

2 Periodical review of the job abends

database and taking/planning for further

actions

Appraisal cost

3 Plan for enhancements, changes of the

system (if required), well in advance

Preventive cost

Table 6: Maintenance & Support practices

Benefits If a legacy system suffers from a high number of job abends, low quality, and require a great deal of rework effort, then it is unfeasible to accomplish objectives such as faster time-to-market and lower operational and maintenance costs. The MMS CoQ approach allows the customer to identify problem areas which result in poor quality; and also implement a plan of action for optimizing the contributors to the cost of quality. The approach looks at the entire CoQ iceberg giving a holistic solution to address all the aspects.

Appendix In-house Tools used to Optimize CoQ The below table shows some of the HCL-MMS built in-house tools which are used to optimize the cost of quality for a legacy system.

Tool Benefit Impact on CoQ

Interface Analyzer

(AnalyzeIT)

Creates Application level block

diagram depicting cross references.

Allows creating Application blocks

encompassing multiple application

elements of different types.

Reduced appraisal cost

11



Tool Benefit Impact on CoQ

Documentation

Generator

(DocumentIT)

Reports overall Application level and

element level details, metrics,

complexity and cross referencing.

Generates Application Flow,

Program Flow, Job Flow diagrams.

Produces CICS control flows, CRUD

reports for Data elements and data

stores. Produces Code & Data

Redundancy reports, dead code

analysis reports. Produces Data

Model reports, Dataset information

reports. Produces Program logic and

Paraphrased Business Rules.

Reduced appraisal

cost & failure cost

Test Case Generator

(TestIT)

Generates the Test Cases based on

Application Analysis.

Reduced appraisal

cost

Code Standardization

Tool

It is for COBOL programs. Required

coding standards are given as input.

The tool will scan the program and

reports the coding standard

violations.

Reduced appraisal

cost

Data Migration (PortIT) Automated generation of Extract

Jobs, data migration transformation

programs and Load Jobs. Handles

data migration for multiple

databases. Handles migration of data

into normalized databases.

Generates Audit reports and Data

Reconciliation modules to ensure

data validity. Automated Code Page

conversion of Hitachi or Fujitsu code

page to IBM code page.

Reduced appraisal

cost & failure cost

Database Migration

(MigrateIT)

Will be helpful in database migration

like from IDMS to DB2 or IMS to DB2

or ADABAS to DB2. After migration it

will generate the report with the

details of tables migrated and the

control totals to cross check, thereby

reducing the testing effort.

Reduced appraisal

cost

Table 7: HCL-MMS in-house tools

Code Standardization Tool – An Example

HCL-MMS has developed a code standardization tool (CSTOOL) which can assist in verifying the COBOL Coding standards in a Development or Enhancement Project. CSTOOL works on a generic input file called a “Rule File” which defines the coding standards which need to be enforced by the tool.

12



The “Rule File” is easily customizable to cater to customer specific or application specific COBOL coding guidelines. CSTOOL can verify the applications COBOL programs for rules enforced in “Rule File” and report detailed information on Lines which don‟t conform to the enforced guidelines. CSTOOL is panel driven and easy to use, where the input Rule File can be setup in MS Excel and uploaded as a CSV File. A Rule File can be applied to any number of programs in the Application. Also, Specific rule files can be setup for certain programs. Output Report gives non conformance of enforced rule with detailed information like exact line number, exact issue and exact source code snippet.

Figure 6: Code Standardization Tool

Appraisal Vs. Failure Activities The below table shows the majority of time spent in appraisal/failure activities and the effectiveness of detecting defects and cost to diagnose and remove each defect.

Activity Cost of Quality Effectiveness

Structured Personal

Reviews

Appraisal ***********

Formal Software

Inspections

Appraisal **********

Informal Peer Reviews Appraisal *********

Compiling Failure ********

Unit Testing Failure *******

Integration Failure ******


13


Activity Cost of Quality Effectiveness

Beta Testing Failure *****

System Testing (and

performance & other testing)

Failure ****

Acceptance Testing Failure ***

Walkthroughs Appraisal *

Table 8: Appraisal vs. Failure Activities

The 'Cost of Quality' column indicates whether the majority of the time in that activity is spent appraising or dealing with failures. The 'Effectiveness' column indicates both the percentage of defects that are likely to be detected and the total cost in engineer-hours to detect, diagnose, and remove each defect. The placement of Reviews and Inspections in the list is based on best practices. In some cases, their effectiveness is quite low. The order of Compiling, Unit Testing, Integration, Beta Testing, System Testing and Acceptance Testing in the above list really does indicate their relative efficiency in removing defects (not just their lifecycle order). The placement of Unit Testing in the list is based on best practices. Many developers have never been trained in testing, and are ineffective at it. Walkthroughs are more effective for training purposes than for defect removal.

Cost of Quality Model Categories Herb Krasner described the cost of quality model in his research as:

Major Category Sub category Definition Typical Cost

Sub Items

Dealing with non-

conformances

Internal non-

conformances

Quality problems

detected prior to

product

shipment

Pre-release defect

management,

rework, re-reviews

and retesting.

External non-

conformances

Quality problems

detected after

product

shipment

Post release

technical support,

complaint

investigation,

defect notification,

remedial upgrades

and fixes


14


Major Category Sub category Definition Typical Cost

Sub Items

Appraising the

level of quality

Discovering the

condition of the

product

Discovering the

level of non-

conformances

Testing, SQA,

inspections,

reviews

Assuring the

achievement of

quality

Quality control

gating

Product quality

audits, go/no

decisions to

release or proceed

Preventing poor

quality from

occurring

Quality basis

management

Efforts to define

quality, set

quality goals,

standards and

thresholds.

Quality tradeoff

analysis

Defining release

criteria for

acceptance

testing, and

related quality

standards

Project and

process

Oriented

interventions

Efforts to

prevent poor

product quality,

and/or improve

process quality

Training, process

improvements,

metrics collection

and analysis

Table 9: Quality Model Categorization

References http://asq.org/learn-about-quality/cost-of-quality/overview/overview.html http://www.isixsigma.com/ http://thequalityportal.com/ http://www.performance-measurement.net http://www.projectconnections.com/articles/051005-koch.html http://lifelong.engr.utexas.edu/pdf/sqi/cosq-article-new06.pdf www.nist.gov/director/planning/upload/report02-3.pdf Quality Control Handbook, J. Juran and F. Gryna

AUTHORS Shilpa Kerur, ETS Mainframe CoE HCL Technologies, Hyderabad Satya Srinivas Biyyapu, ETS Mainframe CoE HCL Technologies, Hyderabad


15


http://asq.org/learn-about-quality/cost-of-quality/overview/overview.html

http://asq.org/learn-about-quality/cost-of-quality/overview/overview.html

http://www.isixsigma.com/index.php?option=com_k2&view=item&id=937:cost-of-quality-not-only-failure-costs&Itemid=187&tmpl=component&print=1

http://thequalityportal.com/

http://www.performance-measurement.net/

http://www.projectconnections.com/articles/051005-koch.html

http://lifelong.engr.utexas.edu/pdf/sqi/cosq-article-new06.pdf

http://www.nist.gov/director/planning/upload/report02-3.pdf

ABOUT HCL

HCL Technologies

HCL Technologies is a leading global IT services company, working with clients in the areas that impact and redefine the core of their businesses. Since its inception into the global landscape after its IPO in 1999, HCL focuses on „transformational outsourcing‟, underlined by innovation and value creation, and offers an integrated portfolio of services including software led IT solutions, remote infrastructure management, engineering and R&D services and BPO. HCL leverages its extensive global offshore infrastructure and network of offices in 31 countries to provide holistic, multi-service delivery in key industry verticals including Financial Services, Manufacturing, Consumer Services, Public Services and Healthcare. HCL takes pride in its philosophy of „Employees First‟ which empowers our 72,267 transformers to create real value for customers. HCL Technologies, along with its subsidiaries, had consolidated revenues of US$ 3.1 billion (Rs. 14,101 crores), as on 31st December 2010

About HCL Enterprise

HCL is a $5.7 billion leading global technology and IT enterprise comprising two companies listed in India - HCL Technologies and HCL Infosystems. Founded in 1976, HCL is one of India's original IT garage start-ups. A pioneer of modern computing, HCL is a global transformational enterprise today. Its range of offerings includes product engineering, custom & package applications, BPO, IT infrastructure services, IT hardware, systems integration, and distribution of information and communications technology (ICT) products across a wide range of focused industry verticals. The HCL team consists of over 79,000 professionals of diverse nationalities, who operate from 31 countries including over 500 points of presence in India. HCL has partnerships with several leading Global 1000 firms, including leading IT and Technology firms. For more information, please visit www.hcl.com


16


Documents

Optimizing Cost of Quality in Legacy - HCL Technologies services provide an end-to-end solution to optimize the ... spent on Appraisal and dealing with Failures. ... Optimizing Cost