The “Fantastic 4”Metrics You Can’t Ignore When Reducing Application Layer Risk

2

Why application security metrics?

3

Why application security metrics?

Sometimes you need: 1. To communicate to your sponsors what you’re doing with the

money they provided for the program.2. A way to communicate with your development teams that is

anchored in something more than just encouragement.3. A tool to show yourself how much progress you’re making.

4

It’s a hostile environment out there

5

It’s a hostile environment out there

Applications have been a top vector for data breaches over the last five years because they’re

not coded with security in mind. The software industry’s shift to composing applications via

pre-built—some would say “pre-0wned”—components has made it more challenging for

security teams by introducing risk via the software supply chain.

So application security is important, but how do you show progress?

6

Four key metrics to save the day

7

FANTASTIC METRIC #1: POLICY COMPLIANCE

You have to have some way of measuring the quality of applications; it should be aligned with the needs of the business.

A lot of your program measurements are going to be anchored in how well your portfolio does against a policy.

But what sort of pass rate

should you expect?

9

10

FANTASTIC METRIC #2: FLAW PREVALENCE

11

When vulnerabilities are all around you might feel like your world is on fire. Let’s try to get our arms around how common some of these fatal flaws really are.

12

Top vulnerabilities by industry

13

FANTASTIC METRIC #3: FIX RATE

You know that guy, the one who always insists that the hole you’re in isn’t as deep as you think it is…

It turns out that’s true of AppSec. There are a lot of people out there making their applications safer, never accepting “no” for an answer. And it turns out that tracking the flaws fixed can be powerfully motivational.

16

17Source: Veracode State of Software Security vol. 6: https://info.veracode.com/state-of-software-security-report-volume6.html

How? Empower developers• Customers in the financial services and manufacturing verticals are

successfully fixing between 65% and 81% of the flaws found in their applications. Applications undergoing remediation coaching (readouts) reduce application risk 2.5x more than those that don’t, as measured by average flaw density per MB

Source: Veracode State of Software Security vol. 6: https://info.veracode.com/state-of-software-security-report-volume6.html

18

FANTASTIC METRIC #4

19

Which One?

20

Which One?

It depends. Just as there’s no “one” lineup of a super hero team, you may find you need a different set of metrics depending on the goals of your program—developer training

completion, for instance, or percent of applications undergoing automated testing. Ultimately it’s up to you, and the needs of your business.

21

Answers Key Questions for CISOs• Which industries are doing the best job of reducing application-layer risk ?   

• Do I have more serious vulnerabilities than my peers?

• What percentage of vulnerabilities do my peers remediate?

• How many of our applications should pass the OWASP Top 10 when initially assessed?

• What are the Top 10 most common vulnerabilities in our vertical?

• How can I reduce more risk in my organization’s applications?