Webcast Presentation: Accelerate Continuous Delivery with Development Testing and Rational Team Concert

Join the Community

www.rational-ug.org

Tweet with Us @RationalUC #rationaluc

www.twitter.com/rationaluc

Join Our Group on LinkedIn: Global Rational User Community

https://www.linkedin.com/groups/Global-Rational-User-Community-GRUC-120486/about

Connect with Us on Google+

https://plus.google.com/+RationalugOrgGlobal/posts

http://www.rational-ug.org/



http://www.twitter.com/rationaluc













Coverity & Rational Team Concert

Jon Jarboe

Senior Technical Manager, Coverity

• Development Testing: What is it and why is it important?

• Coverity’s development testing platform

• Coverity: Who are they?

• Coverity and Rational: An overview

• Demo

• Q&A

Agenda

2 Copyright 2013, Coverity Inc.

Development Testing

What is it, and why is it important?


4

Design DevelopmentQuality

Assurance

Product Release &

Management

Development TestingTransform software testing, from reactive to proactive

10x cost 30x cost

Copyright 2013, Coverity Inc.

5

Fewer defects escape dev

Design DevelopmentQuality

Assurance

Product Release &

Management

Development TestingTransform software testing, from reactive to proactive

10x cost 30x cost


Velocity requires automation

Software Development Evolution

Productivity and ToolsIDE, Compiler, Debugger

Manual testing prevails

Process andGovernanceALM for workflow and traceability

Automated QA testing

Agility and AutomationRise of Agile and Continuous Delivery methodologies

Automated Development testing


Maximize velocity through code intelligence

DevelopersWrite better software

ManagersMake better decisions

Ship 50% faster

Reduce development costs by 25%

Deliver high quality, secure software


Coverity development testing platform


Test smarter and faster

Code Analytics Meets Test Automation

QualityAnalysis

PolicyManagement

TestPrioritization

CodeReview

Security Analysis

CodeIntelligence

Code IntelligencePowered by Coverity SAVE

Interprocedural Data Flow Analysis

SemanticAnalysis

Change ImpactAnalysis

PatternAnalysis


Visibility. Predictability. Accountability.

Code changeimpact

Unit tests to run and write

Quality and security defects to fix

Design

Code

Test

DeployCode

IntelligenceEngine

Measurable testing gate

Tests to run

Measurable testing gate

Customer


Coverity Platform

TestAnalysis

Analysis Packs

Coverity SAVE® Static Analysis Verification Engine

C, C++, C#, Java

SDLC IntegrationsPolicy Manager

CodeAnalysis

Dynamic Analysis

Architecture Analysis

Analysis Integrations

Other Languages

Coverity Connect

Test Execution

Third Party Metrics

Build/Continuous Integration

ALM

IDE

Code Coverage

Defect Tracking

SCM

Proprietary Code | Open Source Code


Code Analysis



Concurrency problemsRace conditions Suspicious lockingProgram hangs/deadlocks

Resource Leaks

Improper use of memoryBuffer overflowsMemory corruptions Illegal access

Null pointer dereferences

Incorrect use of APIs

Incorrect database operations

Class hierarchy inconsistencies

Security problems Insecure data handling Security best practices violationsWeb application security issues

Uninitialized variables

Logic ErrorsArithmetic errorsControl flow issues Incorrect error/exception

handling

Code maintainability issues

Suspicious code

Performance inefficiencies

Issue Classes Identified by Coverity


Accurate

Proven false positive rate of less than 10%

Actionable

Prescriptive remediation

advice

Patent pending security engine

Integrated

IDEDefect trackingSCM, Build/CI

Why Coverity

“Coverity is really great and its web GUI is fun to use, too. I was able to identify and fix resource leaks, NULL pointer issues, buffer overflows and missing checks all over the place.”

-Christian, Python developer


Comparison by Defect Type

Type Coverity FindBugs Shared Defects

Unhandled exceptions

(incl. NULL deref)79 7 5

Resource leaks 86 12 13

Concurrency problems 22 10 9

Critical Defect

Subtotal188 29 27

Coding Standards, Best

Practices, Other9 598 1

Total Bugs 196 627 28

Coverity

79

86

22

187

Coverity identified

4 timesas many critical defects


Test Analysis



SDLC Testing Stage Worst Median Best

Requirements review (informal) 20% 30% 50%

Top-level design reviews (informal) 30% 40% 60%

Detailed functional design inspection 30% 65% 85%

Detailed logic design inspection 35% 65% 75%

Code inspection / static analysis 35% 60% 90%

Unit tests / Regression tests 10% 25% 50%

New Function tests 20% 35% 65%

Integration tests 25% 45% 60%

System test 25% 50% 65%

External Beta tests 15% 40% 75%

Automated testing can deliver 50% defect removal efficiency – but often falls short

Source: Capers Jones


Focus testing time where it matters

And don’t waste time writing testsyou don’t need

Test AnalysisImproving automated testing effectiveness and efficiency

HighRiskCode

HighRiskCode


Powered by Coverity SAVE®

Test Analysis: How It Works

The critical code that must be thoroughly tested

Define

Analyze

Remediate

Govern

Code that has changed and been impacted by changes

Code that has been insufficiently tested

Manage progress to improve test coverage


What code needs to be tested based on your high risk criteria

With patent-pending techniques based on code behavior and change impact

Surface issues in your workflow and efficiently manage to closure

Create a testing stage gate and enforce developer accountability

All code changes for next release …

and code impacted by those changes …

must have 100% coverage …

not counting exception handling and debug code.

Test Analysis: Sample Policy

Define

Analyze

Remediate

Govern


• Change impact analysis enables understanding of the effect of a given code change beyond the place in the code where the change occurs

• Example: changing the behavior of a function might affect the behavior of other functions that call it

• Example: changing the type hierarchy might change the resolution of virtual methods, resulting in a change of behavior where those methods are called

What is Change Impact Analysis?

21

f25

f33 f77

f15 f90

Foo

...

f23f76 f32

f34

f54

...

f89 f67

f87f56

f34

......

... ...

... ...... ...

......

Changed code

Legacy code

Impactedcode

Change“ripple”


Maximizing ROI on Automated Testing

• Improve the efficiency of your testing through focus

• Move from 10-25% efficiency to 50%Focus

• Establish and enforce consistent policies and a process for automated testing Process

• Improve visibility into the testing process

• Establish criteria for when testing is complete/introduce stage gate

Visibility and Control

Copyright 2013, Coverity Inc.22

Coverity: Who are they?


Coverity Overview

Company Facts Financial Facts Customer Facts

• Founded in 2003 at Stanford Labs

• 300 employees across 10 countries

• #1 in software quality analysis – IDC

• Acquired by Synopsys in March 2014

• Over 1,100 world class customers

• Over 75,000 happy developers

• 5 billion lines of code under management

• 30% YOY revenue growth

• Cash flow positive with no debt

• 30% of revenue invested in R&D


Coverity is the Development Testing Leader

2012 Testing Platforms

Market Mover ArrayCoverity Recognized as Transformational

Vendor

Featured in…

#1 vendor for Worldwide Software Quality Analysis and Measurement

Worldwide Software Quality Analysis Measurement 2011-2015 Report

Awards and Leadership



Free cloud-based service for open source community

Coverity Scan

2000 2006 2013

Over 1,600 developers across 600 projects

Over 45,000 defects fixed by the community


Coverity Summary

Pioneer and leader of the development testing disruption

Comprehensive platform for quality and security testing

1,100 market-leading customers across multiple industries

Viral developer adoption within open source community

Built from the ground-up by developers, for developers


Coverity and Rational

An Overview


Release Cycles Are Condensed

Less Time for Formal QA

Greater Risk of Releasing Code with Known Issues

The Challenges of Continuous Delivery


How Coverity Fits Into Rational

TestAnalysis

Analysis Packs

Coverity SAVE®

Static Analysis Verification Engine

CodeAnalysisArchitecture

Dynamic

Third Party/ Custom

Coverity Platform

Test Execution

Third Party Metrics

Build/Continuous Integration

CLM

IDE

Code Coverage

Work Item Tracking

SCM

Rational Platform

Defects, Testing needs

Code, Build, Tests, Coverage


Systems Engineer Development Build

DOORS RhapsodyTeam

Concert

Quality

Manager

Project Manager

Modify requirements and evaluate

impact

Software Development

Submit defect

Create change request

Derive software requirements

Plan tests and link to

requirements

QA

Schedule & execute tests

Continuous Integration

Assess progress

Continuous Delivery with Rational


Quality Dashboards

Central BuildCode Analysis


DOORS RhapsodyTeam

Concert

Quality

Manager

Project Manager


impact





requirements

QA


Coverity

Quality Certification Workflow Automation

DesktopCode Analysis

Work Item Summary


Submit defect

Analysis finds defects early and

often

Defects fed back to developers via

Rational workflow

With Rational/Coverity: Code Analysis




DOORS RhapsodyTeam

Concert

Quality

Manager

Project Manager


impact


Quality Dashboards




requirements

QA


Coverity



Work Item Summary


Submit defect

Enforce testing policy

Prioritize testing runs

With Rational/Coverity: Test Analysis




DOORS RhapsodyTeam

Concert

Quality

Manager

Project Manager


impact


Quality Dashboards




requirements

QA


Coverity



Work Item Summary


Submit defect

With Rational/Coverity

Quality gates ensure efficient, continuous

quality


Accelerate Continuous Delivery Faster time to market

Single workflow, Find defects

earlier, Get more done with less

Lower overall cost, increased

developer productivity

Reduce cycles between

Development and QAIncreased testing efficiency

Reduce risk of defects escaping

to customersReduce cost, maintain reputation

Value to User Business Benefits

Benefits of Rational/Coverity


demo

Coverity in Rational Team Concert


Getting More Information

To learn more about how Coverity can help you maximize your Rational investment,

ASK NOWor contact:

[email protected]

http://www.coverity.com/products/sdlc-integrations/ibm-alm/


mailto:[email protected]

Join the Community

www.rational-ug.org

Tweet with Us @RationalUC #rationaluc

www.twitter.com/rationaluc

Join Our Group on LinkedIn: Global Rational User Community


Connect with Us on Google+





http://www.twitter.com/rationaluc













Documents

Webcast Presentation: Accelerate Continuous Delivery with Development Testing and Rational Team Concert