1

What's the Big Deal About CQCs?

By: Rick Hess, Pat Theeke

2

Code Quality Characteristics (CQCs, or Checks)

• What does your project care about looking for?

• Example source: Goddard Open Learning Design (GOLD) Rules http://standards.gsfc.nasa.gov/gsfc-std/gsfc-std-1000/gsfc-std-1000.html

• Used as Inputs to the Static Code Analysis Method

3

Overview of the Method

• Created to work with the Evidence-based Assurance effort.

• Inputs are CQCs, and a list of the tools that are applicable (can handle the given code language(s), limitations of the tools have been considered).

• Within the method, determine which tool, or combination of tools, provide the most coverage for the CQCs you care about.

• Output from the method is the analysis results from the tools selected.

4

Determining which tools need to be used.

• Determine what you want to analyze, before you run your tools– Regarding CQCs, what does the project want to

look for? – Which tools can I use to provide coverage among

all my CQCs?

5

Verify Software Code Quality using the Static Code Analysis Method – Next Steps

– Add a description about a using formal Capability Matrix based upon what the tools can do/say they can do to assist in picking the specific tool.

– Add on to the current method to include the following: For the CQCs that cannot be covered by Static Code Analysis, the project then needs to determine whether performing other activities will fill in the ‘gap’, and is it worth the cost? (Manual Analysis, for example)

6

Proposed CD Effort

• Create a Capability Matrix to show which CQCs are covered and NOT covered by specific tools.– Tools usually broadcast most of what they CAN do. You never

hear about the functionality that isn’t available or was removed.

– Create a set of validation programs, or scripts• assure that we understand the capabilities and limitations of our

tools• Verify that new tools and new version of existing tools have not

limited or removed existing capabilities• Help to identify when additional/different tools and/or Methods

may be required, and when existing tools no longer meet our needs

7

Questions?

8

Backup Slides

9

Another possible example of CQCs: SWAT Code Defect Categories

SWAT Code Defect Category Not An Issue Withdrawn Accepted % Accepted% That Were Sev

3 or Higher (Accepted)

Arguments passed to function inconsistent with function definition 0 0 2 100.00% 0.00% 0.0% 0.00%Assignment in conditional or conditional instead of assignment 1 0 10 100.00% 70.00% 1.1% 0.00%Catch statement missing/inadequate 1 0 5 100.00% 0.00% 0.0% 0.00%Coding style violation 1 0 8 100.00% 0.00% 0.0% 0.00%Commented out code 3 0 9 100.00% 11.11% 0.2% 0.00%Divide by zero 0 0 1 100.00% 0.00% 0.0% 0.00%Downcast, loss of precision, or loss of sign 0 3 18 85.71% 61.11% 1.7% 0.00%Entity defined more than once with same/different value 1 3 20 86.96% 15.00% 0.5% 0.00%Fixme found in code 2 1 14 93.33% 28.57% 0.6% 0.00%Floats used in equality check 0 0 1 100.00% 0.00% 0.0% 0.00%Infinite loop 1 0 1 100.00% 100.00% 0.2% 0.00%Magic numbers 0 1 9 90.00% 33.33% 0.5% 0.00%Manual: Code performance 0 2 3 60.00% 100.00% 0.5% 66.67%Manual: Code to requirements violation 53 76 515 87.14% 60.00% 48.4% 2.91%Manual: Coding style violation 3 3 14 82.35% 7.14% 0.2% 0.00%Manual: Comments inconsistent with implementation 0 0 1 100.00% 0.00% 0.0% 0.00%Manual: Engineering Observation 4 9 57 86.36% 40.35% 3.6% 1.75%Manual: Heritage Analysis 0 0 2 100.00% 50.00% 0.2% 0.00%Manual: Incorrect code 6 4 36 90.00% 47.22% 2.7% 0.00%Manual: Open file semaphore exhaustion 1 0 3 100.00% 100.00% 0.5% 100.00%Manual: Test to requirements violation 0 0 4 100.00% 100.00% 0.6% 0.00%Memory Leak 1 1 5 83.33% 40.00% 0.3% 0.00%Mismatched or no return type for function 14 3 15 83.33% 46.67% 1.1% 0.00%Missing Code References 0 0 4 100.00% 0.00% 0.0% 0.00%Missing or unused headers 0 0 1 100.00% 0.00% 0.0% 0.00%Multiple files with same name 0 0 2 100.00% 0.00% 0.0% 0.00%Null pointer or dereference of null pointer 10 5 60 92.31% 66.67% 6.3% 3.33%Out of bounds or overrun 26 22 141 86.50% 80.85% 17.9% 3.55%Redundant procedure 1 2 11 84.62% 9.09% 0.2% 0.00%Returned value not checked or not captured or not verified 2 0 7 100.00% 57.14% 0.6% 0.00%Static boolean response 5 3 14 82.35% 28.57% 0.6% 0.00%Streams not closed 0 0 1 100.00% 0.00% 0.0% 0.00%Suspicious operator 0 0 1 100.00% 100.00% 0.2% 0.00%Uninitialized variable or symbol used 10 18 79 81.44% 59.49% 7.4% 0.00%Unnessary break statements 0 0 1 100.00% 0.00% 0.0% 0.00%Unreachable or dead code 4 1 22 95.65% 54.55% 1.9% 0.00%Unused variables or code 14 13 121 90.30% 6.61% 1.3% 0.00%Variable out of range for operator 0 0 1 100.00% 100.00% 0.2% 0.00%