Webinar with Melinda Ballou

Program Director , IDC

Big Data, Big Problems: Avoid System Failure with Quality Analysis

CAST Confidential 1

Speakers

Melinda Ballou

Program Director, Application

Life-Cycle Management

IDC

Pete Pizzutillo

Director, Product Marketing

CAST

Sep-13 © 2013 IDC 2

One Long Hot Week in August!!

One week (from Aug 19 to Aug 26)

In the last

two weeks

Sep-13 © 2013 IDC 3

“Process Gap”- High Cost of Inertia

Evolve Beyond Traditional ASQ to Better Address Risk

Cloud Mobile

Development Practices Complex Sourcing Less Budget

Social Analytics

Sep-13 © 2013 IDC 4

Industry Highlights: Disruptive Trends

Diverse deployment demands for mobile, cloud, embedded drive corporate need for architectural impact analysis for application portfolio, business dynamism is enabled by software quality analysis -- & cost prohibitive

Organizations re-invest, seeking to do more with fewer resources with financial and staffing constraints; leveraging efficient approaches to restore and sustain high performing, timely, business-critical software.

Complex sourcing/off-shoring plus use of open source need strong teaming, effective code management, testing, and metrics enabled by SQAM; Services driven environment (SaaS/cloud, Devops emergence)

Global economic competition and local compliance across geographies demand quality, change and portfolio management, adaptability and rigor

Flexible development paradigm with services creation increasingly drive technology and business collaboration – strong agile emergence

Emerging security issues (as driver) and virtualization/cloud (as enabling technology) for SQAM adoption; ad hoc approaches unsustainable

End-user experience and business impact challenges of rich Internet, mobile, embedded, with social media collaboration/community opportunities

Very public software failures increasing

Sep-13 © 2013 IDC 5

“Quality Gap”- High Cost of Failure

Poor Quality = Increased Business Risk

Lost Revenue

($$$$$)

Lost Customers

Lost Productivity

Increased Costs

Lower Profits Damaged Brand

Sep-13 © 2013 IDC 6

• Software Quality Analysis and Measurement: software tools that

enable organizations to observe, measure, and evaluate software

complexity, size, productivity, and risk (including technical &

structural quality, non-functional testing)

• Architectural assessment of design consequences (on software

performance, stability, adaptability, and maintainability)

• Static analysis and dynamic analysis

• Quality metrics for complexity, size, risk, and productivity to establish

baselines and to help judge project progress and resource capabilities

• Application portfolio evaluation through understanding the impact of

architectural flaws and dependencies

• In-phase prevention of additional software problems not easily

observable through typical ASQ tools.

SQAM Definition: Establishing a Strategy

Sep-13 © 2013 IDC

Barriers to Traditional Testing – SQAM Drivers

• Agile velocity demands immediate, frequent, iterative testing

• Lack of system resources constrains testing usage – expense limits

ability to mirror production configurations (mobile issues)

• Lack of architectural and design context for multi-modal deployment need

for management & coordination

• Challenges to test system configuration and impact to performance and

adaptability of design

• Lack of visibility into consequences of poor architecture with significant

impact to business or failed software

• Increasing occurrences of business critical failures are driving

engagement and interest in software quality analysis and measurement

7

Sep-13 © 2013 IDC

ASQ Forecast with IDC Software Quality Analysis & Mgmnt Segment

0

500

1,000

1,500

2,000

2,500

3,000

3,500

2009 2010 2011 2012 2013 2014 2015 2016 2017

$M

ASQ (6/13) SQAM (9/13 est)

19%

29%

Sep-13 © 2013 IDC 9

IT and Business Challenges: Silos, Gaps

Today’s applications are high-visibility, and carry a high cost-of-failure -- customer self-serve, supplier/channel; key internal business applications

“Network effect” – failure in one leads to other failures

The need for SQAM as part of quality life-cycle is key since G2000 organizations are split across groups:

– Business/users stakeholders

– Architects, Designers and Developers

– QA professionals

– Operational staff

Must extend the Quality life-cycle across geographies, life cycle phases and groups

Sep-13 © 2013 IDC 10

Goals of Effective IT/Business Alignment

New Business Value

Reduced Exposure

Innovation: Maximize Upside

Through Technology- Enabled Business

Processes

Compliance: Minimize Downside

Through Risk Management

Sep-13 © 2013 IDC

CIO’s 2013 Personal Agenda

Q. In 2013, which of the following goals will be top of your personal agenda as CIO?

Please select your top 3 goals.

0% 5% 10% 15% 20% 25%

Implement a more rigorous process to evaluatenew ideas for IT to take on

Re-skill existing IT talent

More effectively attract new IT talent

Carve out more IT budget for newprojects/innovative projects

Focus IT organization on better understandingthe requirements of the consumers

Better align IT with the business

Foster a culture within IT where IT more oftenprovides a qualified "yes" to the business

Foster a culture within IT that drives moreinnovation

Focus the IT organization more on businessstrategy than technology strategy

US

WE

Total

n = 70

WE respondents = 21; US respondents = 49

Source: IDC 2013 CIO Agenda Survey, Fall 2012

5

Sep-13 © 2013 IDC

By 2016, LOB executives will be directly involved in 80% of new IT investments

It is Time to Revisit IT Planning, Quality Governance and Portfolio Management Methods

Of the new internal IT projects initiated at your

company this year, what percentage will be led

under the following scenarios?

N = 57

Source: IDC 2013 CIO Sentiment

Survey, Fall 2012

Prediction

58% of new IT investments in 2013 will

involve direct participation by LOB executives

Companies will initiate an average of 40 new

IT projects in 2013 (with or without IT)

Line of business’ participation in IT projects

will grow to 80% in 3 years

The implications are vast on how the CIO

works with the line of business

Situation Assessment

Cloud, social and mobile services are the

great equalizers, the balance-wheel of the

corporate machinery

Notable instances of CEOs and CFOs driving

the migration to Cloud and Managed Services

8%

17%

33%

42%

0% 10% 20% 30% 40% 50%

Project solely led /managed by the LOBs

Project led / managed byLOBs, but subject to

review by IT

Project jointly led /managed by IT and the

LOBs

Project solely led /managed by IT

9

Sep-13 © 2013 IDC

Three Key Challenges for IT

IT must deliver new applications that have greater business value

and higher quality, while managing costs …

in the face of these 3 key challenges:

Increasing criticality of applications to the business

Increasing complexity of software systems

– From web to mobile to embedded… encompassing social systems of

engagement to feed systems of record, performance demand with Big

Data Analytics for business optimization

Increasingly distributed teams with multi-sourcing

– From onshore to offshore to open source

13

Sep-13 © 2013 IDC

Source: IDC CloudTrack Survey, Winter 2012

n=493

Mean rating by

respondents

Q. How concerned are you about cloud...? (1-5 scale; 5 = extremely concerned)

Security Tops Concerns: Risk Major Role

Sep-13 © 2013 IDC 15

Coordinating across the Life-Cycle

• Coordinating architectural design, requirements, software analysis, quality and operational performance is key across emerging technologies

• Failures and slow response time costs prohibitive for business areas

• Organizations should leverage quality automation through design, requirements, unit test, system integration, pre-deployment & application performance testing with emerging cloud / mobile /social platforms

• Evaluating software analysis with automation can help teams react and manage user application experience

• As business requirements change, a cogent life-cycle approach enables adaptive software analysis and responses

• Look to SQAM alternatives initially as an on-ramp to mobile, cloud and multi modal dev – strategize through to deployment

Sep-13 © 2013 IDC 16

IDC Calls to Action

• Across industries, poorly designed and problematic software leads

to brand perception impact above and beyond individual problems –

demand response

• The challenges of increased complexity and high-end development

across diverse platforms increase code problems, increase costs

and drive debilitating consequences resulting from defects pre- and

post-deployment

• Companies must become better educated about the business

consequences and labor costs of poor software design since

optimism masks the need for change

• Organizations should evaluate SQAM tools to supplement

traditional ASQ along with appropriate process and organizational

approaches

Sep-13 © 2013 IDC 17

Summary

Coordinate a Quality Life-Cycle approach that targets pragmatic approaches to SQAM from design through to deployment to obtain benefits

Evaluate your organization’s current strategies for design, application portfolio review, effective quality processes and automated tools adoption

Schisms between business, architects, development, testers and operations must be addressed -- IT groups and the business must build a common language, common metrics, and common tools and practices that include SQAM

Drive towards an effective quality strategy to help cut costs, increase efficiency and business agility, to sustain brand, address competitive challenges

Analyzing and Measuring Software Risks

CAST Confidential 19

Industry starting to pay attention to code quality

But code quality & hygiene, things traditional safe

guards identify are only a small part of the solution.

Sources: Li, et al. (2011). Characteristics of multiple component defects and

architectural hotspots: A large system case study. Empirical Software Engineering

“Tracking programming practices at the Unit Level alone may not translate into the

anticipated business impact,…most devastating defects can only be detected at the

System Level.”

8%

90%

Unit-Level

Flaws

System-

Level Flaws

% of

apps

defects

% of

repair

effort

92%

8%

52%

48%of downtime caused by

8% of system-level defects!

CAST Confidential

Business

Characteristic

Good Coding Practices

@ Unit-Level

Good Architectural Practices

@ Technology/System Levels

RELIABILITY Protecting state in multi-threaded environments

Safe use of inheritance and polymorphism

Resource bounds management, Complex code

Managing allocated resources, Timeouts

Multi-layer design compliance

Software manages data integrity and consistency

Exception handling through transactions

Class architecture compliance

PERFORMANCE

EFFICIENCY

Compliance with Object-Oriented best practices

Compliance with SQL best practices

Expensive computations in loops

Static connections versus connection pools

Compliance with garbage collection best practices

Appropriate interactions with expensive or remote resources

Data access performance and data management

Memory, network and disk space management

Centralized handling of client requests

Use of middle tier components vs. procedures/DB functions

SECURITY Use of hard-coded credentials

Buffer overflows

Missing initialization

Improper validation of array index

Improper locking

Uncontrolled format string

Input validation

SQL injection

Cross-site scripting

Failure to use vetted libraries or frameworks

Secure architecture design compliance

MAINTAINABILITY Unstructured and duplicated code

High cyclomatic complexity

Controlled level of dynamic coding

Over-parameterization of methods

Hard coding of literals

Excessive component size

Duplicated business logic

Compliance with initial architecture design

Strict hierarchy of calling between architectural layers

Excessive horizontal layers

Excessive multi-tier fan-in/fan-out

NUMBER OF ISSUES 90% of violations 10% of violations

BUSINESS IMPACT 52% of repair workload

10% of production downtime

48% of repair workload

90% of production downtime

Industry must focus on the flaws that matter

CAST Confidential

CAST Software Risk Prevention

CAST solutions expose the weaknesses in complex multitier systems by identifying

the high severity engineering flaws undetectable by testing. CAST insures the

confidence that critical systems are free from vulnerabilities, either intentionally

designed into the software or accidentally inserted at anytime during its lifecycle.

1. Define the business-relevant software characteristics:

stability & resilience, performance efficiency, & security

important to your business.

2. Identify structural weaknesses and architectural hotspots

based on initial of applications.

3. Baseline and benchmark key risk indicators

against industry norms.

4. Monitor to ensure system do not degrade over time.

SOFTWARE RISK PREVENTION PROCESS

PEACE OF MIND - FROM THE INSIDE OUT.

CAST Confidential

Analysis strategy for typical IT application portfolio

22

Eff

ort

(M

an

Days/Y

ea

r)

Importance to Business

Highest Lowest

Critical Apps

Entire Application Portfolio

CAST AIP

Deep Structural

Analysis

Risk Detection

Lean Application

Development

Function Points &

Productivity

Vendor Management

Continuous

Improvement

CAST Highlight

Fast Cloud-based

Delivery

No source code

aggregation

Key Metrics on Entire

Portfolio

Size, Complexity and

Risk analytics

Annual/Quarterly

Benchmark

CAST Confidential

Portfolio risk review with Highlight

23

QUICKLY SPOT SHORT TERM RISK – COMPLEX SYSTEMS LIKELY TO FAIL

CAST Confidential

Arc

hit

ec

ture

Co

mp

lia

nce

Enterprise IT applications require depth of analysis

24

Intra-technology architecture

Intra-layer dependencies

Module complexity & cohesion

Design & structure

Inter-program invocation

Security Vulnerabilities

Module Level

Integration quality

Architectural compliance

Risk propagation

simulation

Application security

Resiliency checks

Transaction integrity

Function point & EFP

measurement

Effort estimation

Data access control

SDK versioning

Calibration across

technologies

System Level

Data Flow Transaction Risk

Code style & layout

Expression complexity

Code documentation

Class or program design

Basic coding standards

Program Level

Propagation Risk

Java

EJB

PL/SQL

Oracle

SQL

Server

DB2

T/SQL

Hibernate

Spring

Struts .NET

C# VB

COBOL

C++

COBOL

Sybase IMS

Messaging

Java Web

Services

1

2

3

JSP ASP.NET APIs

CAST Confidential

CAST AIP - well beyond static analysis

Static Analysis

Dependencies Code Pattern

Scanning

Data Flow Rule Engine

Transaction Finder Intelligent

Configuration Content Updater

The architectural assessment

of design consequences (on

software performance, stability,

adaptability, maintainability, and

security vulnerabilities) is an

area in which CAST excels and

successfully differentiates from

static analyzers.”

Architecture Analysis

Behavioral Simulation

Function Points

CAST Confidential 26

Making risk management actionable

Identify and stabilize are the tactical steps

To harden and optimize is a move towards proactive

risk management that requires actionable processes

into the application lifecycle

Quickly spot the riskiest applications

in your portfolio

View overall Technical Quality

Risk Score

View total number of critical

violations discovered.

CAST Confidential

• Upload Source Code

and documentation

• Complete a Technical

Survey

Application Assessment Process T

RA

NS

FE

R

VA

LID

AT

E

INS

IGH

T

• CAST Consultant verifies

completeness of source code ,

artifacts, and technical survey.

• Verifies application boundaries.

• Results are published

to a private, secure portal

• Assessment report delivered

and presented to client

Results by application

Code Quality performance

Benchmark across industry

Day 1 Day 2 – 4

Day 8

AN

ALY

ZE

• CAST Consultant performs the analysis.

• Using highly-sophisticated language

analyzers and more than 1000 industry-

best-practice rules, CAST assessment

identifies weakness in the application and

provides guidance on how to fix them.

• Verifies results with Client application

owner/SME

Day 4 – 7

CAST Confidential

Contact Information

Pete Pizzutillo

p.pizzutillo@castsoftware.com

www.castsoftware.com

blog.castsoftware.com

linkedin.com/company/cast

@OnQuality

slideshare.net/castsoftware