Outsourcing: Eliminating Us vs. Them

March 21, 2008

Agenda

Evolution of outsourcing

Key challenges for buyers and vendors

Using software quality metrics to bring objectivity and transparency

CAST AIP

Outsourcing growing pains

70 percent of participants have had significant negative experiences with outsourcing (Deloitte Survey)

68% of large enterprises say outsourcing is overrated (CIO Insight) 67% say the total cost of outsourcing has been as or more

expensive than doing the same work in-house (CIO Insight) #1 reason for outsourcing failures: “The buyer's unclear

expectations up front as to its objectives” (Outsourcing Center)

Has the industry matured yet?

Outsourcing evolution R

ela

tiv

e V

alu

e t

o C

om

pa

nie

s

Cost FocusCost Focus

Transformation Focus

Transformation Focus

Growth and Innovation Focus

Growth and Innovation Focus

Gen. 1 Gen. 2 Gen. 3

1990 2000 2010

Drivers Labor arbitrage Scale

Sample Functions Helpdesk Infrastructure

Despite the challenges, outsourcing is here to stay. It will become

bigger and play an important role in strategic business decisions.

Drivers Specialization Transformational Capabilities

Sample Functions Application Development Process Streamlining

Drivers New methodologies

and techniques Reduced

development lifecycles

Sample Functions New technology

development New product

development

Agenda

Evolution of outsourcing

Key challenges facing buyers and vendors

Using software quality metrics to bring objectivity and transparency

CAST AIP

Us vs Them Divide – Lack of objectivity and transparency

Great honeymoon… High expectations

Lousy marriage….

Breakdown in… Communication Trust

Lack of visibility into vendors development and pricing

High turnover of personnel Lack of objective metrics to measure

performance

Internal bias against outsourcing Constant questioning of quality Commodity sourcing mentality, low

bidder wins

Buyers Vendors

Industry response

SLAs Functionality Run-time performance levels Bug correction delays / feature delivery delays Maintenance cost decrease rate

Process Focus (CMMI) Process adherence Cost Schedule Productivity Quality Customer satisfaction

Why traditional SLAs are not enough

SLAs do not address: Production issues Increasing number of RFCs outside of the

maintenance baseline More expensive RFCs

In addition, renewal clauses fail to measure the effective workload: Far too technology-dependent Difficult to control objectively

Why CMMI is not enough

"In theory, there is no difference between theory and practice, but in practice, there is.” - Yogi Berra

SEI hypothesis: “The quality of a product is largely determined by the quality of the process that is used to develop and maintain it.”

CMMI contributes to, but does not guarantee excellent software quality since it is not a software quality standard

CMMI addresses two sources of poor quality software weak project management poor development practices

CMMI does not measure other sources of poor application quality inexperienced developers continual system modification external software

Although process is necessary, not sufficient Still have to measure the quality of product

Both SLAs and CMMI are process and project quality focused

Requirements

Development

Testing

Maintenance

ProductDesign

Process Management (CMMi) Process adherence Cost Schedule Productivity Quality Customer satisfaction

SLAs Functionality Run-time performance

levels Bug correction

delays / feature delivery delays

Maintenance cost decrease rate

Process

Service

Agenda

Evolution of outsourcing

Key challenges facing buyers and vendors

Using software quality metrics to bring objectivity and transparency

CAST AIP

Why focus on software quality metrics?

TE

ST

RE

VIE

W

visible symptoms EXTERNAL

QUALITY

INTERNALQUALITY

correctness efficiency

maintenance cost

Invisibleroot

program structure

complexity

coding practices

coupling

testability

maintainability

readability flexibility

reusability

Source: Code Complete, Steve McConell

Software Quality Iceberg

Your hands can’t hit what your eyes can’t see.. ( - Muhammad Ali)

... but what you can’t see can hit you very hard.

Why focus on software quality metrics? (Cont.)

People’s belief about software quality Software solutions – even Business Applications – are stable utilities Black box testing – functionalities, response time, … – at the end of

the production line is enough

Why they are wrong: Source code contains the root causes of

production issues Undetectable through black box testing

(non-exhaustive test coverage) Business applications continue to evolve, with

Increases in time constraints Requests for lower-than-ever costs New teams, unfamiliar with the code

Limited visibility into evolving system assets

It works now, but do you really know what your teams are producing?

Internal and Outsourced

Development Teams

Account Management System

Product Pricing Application

Customer Information System

Is it…

Amateur, shoddy development…

Product Pricing Application

Dangerous and uncertain Hard to change Easy to hack into Expensive to maintain

Dangerous and uncertain Hard to change Easy to hack into Expensive to maintain

…or…

well-engineered and high-quality?

Account Management System

Secure and robust Easy to learn and change Inexpensive to maintain … And faster to build

Secure and robust Easy to learn and change Inexpensive to maintain … And faster to build

Testing, by itself, cannot uncover the risk built in. Testing either fails to . Testing either fails to detect root issues, or it’s too expensive and too latedetect root issues, or it’s too expensive and too late

Outsourcing software quality – Myth vs Reality?

Worse than in-house results, 46%

The same as in-house results, 26%

Unusable or a setback to progress, 14%

Better than in-house results, 8%

Other, 6%

Outsourcer Quality Survey ResultsSoftware Development Magazine, March 2006

How is quality measured?

Software quality metrics can help both buyers and vendors

Takes the guess work out of quality

Improved code acceptance criteria

Objective performance measurements

Justify the quality of work Benchmark applications against

industry and clients applications Better release management

Buyers Vendors

Transparency between client and vendor encourages a healthier and risk-free relationship

What is software quality in Application Development?

Conformance to Specifications?Conformance to Specifications?

Accuracy, adaptability, affordability, availability?

Accuracy, adaptability, affordability, availability?

Satisfying the end user/consumer?Satisfying the end user/consumer?

What if …• The specifications are wrong?• The budget and schedule are way overrun?• The code is inscrutable, difficult to modify?

What if …• These attributes conflict with each other?• Customers can’t specify how much of each they need?• Tradeoffs are impractical to analyze or prioritize?• Attribute levels can’t be accurately measured?

What if …• Customer is constantly unhappy?• End-Users are unhappy?• Maintainers are unhappy, disempowered?• Developers are burned out?

Definition of quality

Code Quality

Code quality is the measure of individual components for standards and best practices

compliance in the context of a specific language

Application Quality

Application quality also measures how well the individual components work together to make

up the overall business system

Good code quality ≠> Good application quality

Agenda

Evolution of outsourcing

Key challenges facing buyers and vendors

Using software quality metrics to bring objectivity and transparency

CAST AIP

The CAST Application Intelligence Platform

Automated analysis of entire application Immediate, unbiased quality assessment Executive level of synthesis & trending Drill down to root cause in the source code

Achieve higher quality at less costAchieve higher quality at less cost

Transparency! Automated.

Oracle PL/SQLSybase T-SQLSQL Server T-SQLIBM SQL/PSMC, C++, C# Pro CCobolCICSVisual BasicVB.NetASP.NetJava, J2EEJSPXMLHTMLJavascriptVBScriptPHPPowerBuilderOracle FormsPeopleSoftSAP ABAP, NetweaverTibcoBusiness ObjectsUniversal Analyzer for other languages

CAST Application Intelligence Platform – how it works

AD GOVERNANCE DASHBOARD

To assess, monitor, and improve applications, development teams and outsourcers

TEC

HN

ICA

L

METR

ICS

APPLICATION HEALTH

APPLICATION SIZE

APPLICATION KNOWLEDGE BASE

Transferability

ChangeabilityPerformance

MaintainabilitySecurity

Robustness

APPLI

CA

TIO

N

META

DA

TA

TECHNICAL INVENTORY

Technical Size Functional Weight

ANALYZERS

DRILL-DOWN TO ACTION

Immediate Impact On-going Impact

Overview…

… to remediation

Sub- metrics

Rules

Portfolio

Applications

Modules

Objects

Health factors

Compliance

MANAGEMENT VISIBILITY

Over 800 rules and

best practices

Deep structural analysis of software quality

Transferability

Changeability

Robustness

Performance

Size

Naming Conventions

Documentation

Architecture

Complexity

Package naming

Class naming

Interface naming

Method naming

Attribute naming

Constant naming

Package comment

Class comment

Method comment

Package size

Class size (methods)

Class size (attributes)

Interface size

Method size

Class complexity (Inh. depth)

Class complexity (Inh. width)

Method complexity (Param.)

Method complexity (control flow)

Maintainability

Security

ProgrammingPractices

File conformity

Dead code

Controled data access

Structuredness

Modularity

Encapsulation conformity

Empty code

Inheritance

Immediate ImpactImmediate Impact

Application Quality

On-Going ImpactOn-Going Impact

Over 800+

architectu

ral and

lang

uag

e-specific co

de ch

ecks

Health Factors Quality Indicators Quality Metrics Subset Application Quality

Taking the entire business application into account

JSP / JSTL / JSF

STRUTS - MVC

Database

Batches CBatches CPro CPro C

EJB – Hibernate - Ibatis

Java Services

Spring

COBOLCOBOL

DatabasesDatabasesFilesFiles

BatchesBatches

Webmethods

ERP

Web

Services

CICS Connector

Enterprise

Applications

Legacy

Applications

Middleware Presentation

Tier

Business

Logic Tier

Data Tier

Only CAST can analyze thisOnly CAST can analyze this

Profile, assess, and benchmark applications and teams

Portfolio view of applications Helps prioritize management

attention

Each application’s quality measured on 5 dimensions

Health factors inspired by ISO 9126-3 and SEI Maintainability

Look inside – one application at a time

Health factors to measure overall application technical quality

Technical inventory Functional weight

estimates

Drill down to the root cause of any quality issue…

Robustness metrics for entire application, shown in order of their impact on overall score

…to turn metrics into action and value

Examine a specific quality metric (e.g., architecture)

Detailed issue identification and action items

Map the issue all the way to the source code

Insights for both buyers and vendors

Management visibility

Guidance for developer

• Ensure teams are working efficiently• Manage stability, security & project risks• Better relationships with outsourcers

• Ensure architectural compliance• Ensure projects are not at risk• Metrics – quality, quantity,

technical

• Immediate feedback regarding code qualityInternal and Outsourced Teams

Division CIO and VP, Apps Delivery

Project Managers, Architects and Quality Assurance

Java developers .NET developers DBAs

Solution Information What IT constituents need

Overall team and application KPIs

Measure of conformance to standards &

architecture

Identify specific application quality

issues

Identify code-level style and quality issues

CAST AIP

Establish a quality monitoring process as part of development cycle

Created management visibility into quality issues and the performance of development teams

Establish a quality monitoring process as part of development cycle

Created management visibility into quality issues and the performance of development teams

Depository Trust & Clearing Corporation

Business Need: The DTCC is outsourcing the development of a business critical trade clearance application that needs to support $60-$70 billion worth of transactions daily. The application is developed offshore and will be approximately 10 million lines of code written in C++ and Mainframe languages. DTCC needed an automated process to mitigate risk, ensuing quality and on-time delivery of this core business application.

CAST’s solution

Benefits

““Thanks to CAST, we were Thanks to CAST, we were

able to mitigate quality able to mitigate quality

and performance risk as and performance risk as

we engaged an off-shore we engaged an off-shore

development partner to development partner to

supplement our internal supplement our internal

resourcesresources.”.”

Significant improvements in quality delivered and the performance of development partners

Path for continual improvement – both internally and for the offshore partners

Significant improvements in quality delivered and the performance of development partners

Path for continual improvement – both internally and for the offshore partners

Create a “software factory” at the FDA Deliver immediate development productivity gains

with eBlueprints and impact analysis Deliver long lasting improvements to quality and

standards enforcement, while lowering maintenance and security costs

Create a “software factory” at the FDA Deliver immediate development productivity gains

with eBlueprints and impact analysis Deliver long lasting improvements to quality and

standards enforcement, while lowering maintenance and security costs

United States Food and Drug Administration

Business Need: The FDA’s application development (AD) efforts comprise eight decentralized development centers that are staffed with in-house developers, with a large number of contractors. FDA was looking to improve the success and efficiency of maintenance and development projects, while meeting OMB-300 business case requirements.

CAST’s solution

Benefits

““The CAST platform has The CAST platform has

allowed us to create a set allowed us to create a set

of software factories, of software factories,

where we can see exactly where we can see exactly

how far we are from how far we are from

delivering our end delivering our end

productproduct.”.”

Unlike PPM tools CAST is not a burden to the development team – the solution is automated and delivers productivity benefits

CAST had the only product on the market that provided the level of visibility across multiple languages that the FDA was looking for

Unlike PPM tools CAST is not a burden to the development team – the solution is automated and delivers productivity benefits

CAST had the only product on the market that provided the level of visibility across multiple languages that the FDA was looking for

Vodafone branch – SFR-Cegetel

Improve the quality and reliability of the deliveries

Enable dialog with outsourcers to be based on tangible elements and fact-based information

Mitigate downtime risks and reduce costs

CAST Standard Enforcement solution implemented on over 90% of the multi-million-line portfolio consisting of COBOL, C/C++, Java, Oracle and DB2 applications

Provide standard enforcement dashboards and detailed exception reports

““With CAST, we With CAST, we

learned that our Quality learned that our Quality

Standard Compliance Standard Compliance

ratio was at only 60%. ratio was at only 60%.

Since then our service Since then our service

providers have improved providers have improved

their QA processes.”their QA processes.”

Business Need: SFR-Cegetel is an $10 billion telecom group owned by Vodafone and Vivendi. The company has 19 million customers in Europe. Application development and maintenance activities are entirely outsourced (2,000 staff), making it critical for the company to preserve a high level of maintainability and quality of its application portfolio over time

CAST’s solution

Benefits

Deutsche Telekom

Reliability and stability of application deliveries have significantly increased

Enable dialog with outsourcers to be based on tangible elements and fact-based information

Better understanding and control of applications to significantly reduce future maintenance costs

Business Need: Deutsche Telekom's Corporate Billing system (480 million invoices per year) is completely outsourced. In order to keep offering competitive Billing Services, DT has established a Quality Service Center for outsourcing management. The goal was to improve the technical quality of outsourcers' delivery - before the applications were delivered - and reduce outsourcing and quality management costs.

CAST’s solution

Benefits

Automatically assess and monitor the quality of applications

Optimize the technical acceptance process with automated quality reviews

Automatically assess and monitor the quality of applications

Optimize the technical acceptance process with automated quality reviews

““Our quality assurance & Our quality assurance &

integration management integration management

department has been department has been

reduced from 170 to 70 reduced from 170 to 70

peoplepeople.”.”

CxO and VP – Can you manage what you can’t see?

Mapping of applications from the selected portfolio according to their TQI (Technical Quality Index) on the horizontal axis to their business value (manual entry).

Display of the number of violations (i.e., defects for rules that have been tagged as “critical” in the Quality Model) split

by application of the selected portfolio, and per Health Factor.

Health Factor grades for each application from the

selected portfolio (including the SEI

Maintainability index).

Evolution of the TQI of applications from the selected portfolio as lines,

against the volume (counted as

kLOCs) of the whole portfolio.

Technical size, functional weight and SW quality statistics about violation ratios (overall, per file, etc.)

…

Visual display of the functional weight split by types of languages

and technologies.

Project Manager – On time, on quality, and at less risk

Overall Health Factor grades, along with the evolution (in %) of the app since baseline, and for just the latest delivery

The most critical violations within the app: Overall, and just those introduced in the last version

The most critical violations split by

Health Factor

Trend analysis of the quality

& quantity delivered

Award Section

Technical characteristic and statistics

on the app

Chief Architect/QA – Enforce Architecture and programming standards

Current Rule Compliance grades,

along with the evolution (in %) of the latest delivery

(since the previous snapshot), and since the baseline (since the first available

snapshot).

List of rules tagged as "critical" with violations, together with the number of these violations and the resulting grade as a visual

gauge.

Heat map that identifies areas where there is a quality issue.

The evolution of the Rule

Compliance grades over time against the volume of the

application (counted as kLOCs)

Vendor Manager – High quality delivery from SIs. Guaranteed.

The number of rules tagged as "critical“ to the quality goals achievements or failures

Shows the compliance objective (in %), the effective compliance with the rule (in %), the gap (in %-points) and

the numbers of violations.

Displays the list of artifacts whose cyclomatic

complexity has increased the most together with their

number of violations

Displays the current and previous distributions of

artifacts according to their complexity and with violations (i.e., not

complying with rules tagged as "critical“ in the

Quality Model)

Developers – Continuous self improvement and faster remediation

Displays a holistic view of the architectural context of the violation, for better understanding and easier

remediation

Displays list of defects, with severity

levels and justification of the

rule, if needed.