© 2017 Synopsys, Inc. 1

© 2017 Synopsys, Inc. 2

© 2017 Synopsys, Inc. 3

BSIMM basics

© 2017 Synopsys, Inc. 4

We hold these truths to be self-evident

• Software security is more than a set of security functions.

– Not magic crypto fairy dust

– Not silver-bullet security mechanisms

• Non-functional aspects of design are essential.

• Bugs and flaws are 50/50.

• Security is an emergent property of the entire system (just like quality).

• To end up with secure software, deep integration with the SDLC is necessary.

© 2017 Synopsys, Inc. 5

2006: A shift from philosophy to HOW TO

• Integrating best practices into large organizations’ SDLC (that is, an SSDL)

– Microsoft’s SDL

– Synopsys Touchpoints

– OWASP CLASP

© 2017 Synopsys, Inc. 6

Prescriptive vs. descriptive models

Descriptive Models

• Descriptive models describe

what is actually happening.

• The BSIMM is a descriptive

model that can be used to

measure any number of

prescriptive SSDLs.

Prescriptive Models

• Prescriptive models describe

what you should do.

• SAFECode

• SAMM

• SDL

• Touchpoints

• Every firm has a methodology

they follow (often a hybrid).

• You need an SSDL.

© 2017 Synopsys, Inc. 7

BSIMM: Software security measurement

• 146 firms measured (data freshness)

• BSIMM8 = data from 109 real initiatives

• 321 distinct measurements over time

• 36 over time (one firm 5 times)

• McGraw, Migues, and West

© 2017 Synopsys, Inc. 8

109 firms in BSIMM8 community

© 2017 Synopsys, Inc. 9

Building BSIMM (2008)

• BIG idea: Build a maturity model from actual data gathered from 9 well-known,

large-scale software security initiatives.

– Create a software security framework.

– Interview 9 firms in-person.

– Discover 110 activities through observation (1 removed, 4 added later).

– Organize the activities in 3 levels.

– Build a scorecard.

• The model has been validated with data from 146 firms (109 in BSIMM8).

• There is no special snowflake.

© 2017 Synopsys, Inc. 10

The magic 30

• Since we have data from >30 firms we can perform statistical analysis.

– How good is the model?

– What activities correlate with what other activities?

– Do high-maturity firms look the same?

• BSIMM8 has 109 firms with 321 distinct measurements.

– BSIMM (the 9)

– BSIMM Europe (9 in EU)

– BSIMM2 (30)

– BSIMM3 (42)

– BSIMM4 (51)

– BSIMM-V (67)

– BSIMM6 (78)

– BSIMM7 (95)

© 2017 Synopsys, Inc. 11

Monkeys eat bananas

• BSIMM is not about good or bad ways to eat bananas or

banana best practices.

• BSIMM is about observations.

• BSIMM is descriptive, not prescriptive.

• BSIMM describes and measures multiple prescriptive

approaches.

© 2017 Synopsys, Inc. 12

A software security framework

See informIT article on BSIMM website https://bsimm.com

4 Domains 12 Practices

© 2017 Synopsys, Inc. 13

Example activity

[AA1.2] Perform design review for high-risk applications.

The organization learns about the benefits of architecture analysis by

seeing real results for a few high-risk, high-profile applications. The

reviewers must have some experience performing detailed design

review and breaking the architecture being considered, especially for

new platforms or environments. In all cases, design review produces a

set of architecture flaws and a plan to mitigate them. If the SSG is not

yet equipped to perform an in-depth architecture analysis, it uses

consultants to do this work. Ad hoc review paradigms that rely heavily

on expertise can be used here, though in the long run they do not

scale. A review focused only on whether a software project has

performed the right process steps will not generate expected results.

© 2017 Synopsys, Inc. 14

BSIMM8 measurements

© 2017 Synopsys, Inc. 15

Average percentage of SSG to development of 1.60%

(1 person for every 60 developers)

© 2017 Synopsys, Inc. 16

© 2017 Synopsys, Inc. 17

Earth (109)

© 2017 Synopsys, Inc. 18

BSIMM8 as a measuring stick

© 2017 Synopsys, Inc. 19

BSIMM8 as a measuring stick

© 2017 Synopsys, Inc. 20

BSIMM8 results

Top 12 activities

– purple = good?

– red = bad?

“Blue shift” = practices to

emphasize

© 2017 Synopsys, Inc. 21

Comparing groups of firms

© 2017 Synopsys, Inc. 22

We are a special snowflake (NOT)

© 2017 Synopsys, Inc. 23

© 2017 Synopsys, Inc. 24

BSIMM longitudinal: Improvement over time

• 36 firms measured twice (an

average of 26 months apart)

• We know how firms improve

– An average of 33.4% activity increase

© 2017 Synopsys, Inc. 25

BSIMM by the numbers

© 2017 Synopsys, Inc. 26

© 2017 Synopsys, Inc. 27

BSIMM7 to BSIMM8

• BSIMM8 released September 2017 under Creative Commons.

– https://bsimm.com

• BSIMM is a yardstick.

– Use it to see where you stand.

– Use it to figure out what your peers do.

• BSIMM7→BSIMM8

– BSIMM grew to 146 firms, which we then culled to 109.

© 2017 Synopsys, Inc. 28

Where to learn more

© 2017 Synopsys, Inc. 29

Useful resources

• Participate in the BSIMM Community bsimm.com

• Read the BSIMM FAQ: bsimm.com/about/faq/

• Download the BSIMM8 study: bsimm.com/download/

• View our video introduction to the BSIMM: synopsys.com/bsimm

• Watch the BSIMM webinar: bsimm.com/resources/bringing-science-to-software-security/