Copyright 2013, Cigital and/or its affiliates. All rights
reserved. BSIMM-V THE BUILDING SECURITY IN MATURITY MODEL GARY
MCGRAW, PH.D. CHIEF TECHNOLOGY OFFICER
Slide 2
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. Cigital Providing software security professional services
since 1992 Worlds premiere software security consulting firm o 270
employees o Washington DC, New York, Santa Clara, Bloomington,
Boston, Chicago, Atlanta, Amsterdam, and London Recognized experts
in software security o Widely published in books, white papers, and
articles o Industry thought leaders
Slide 3
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. BSIMM basics
Slide 4
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. We Hold These Truths to be Self-evident Software security
is more than a set of security functions o Not magic crypto fairy
dust o Not silver-bullet security mechanisms Non-functional aspects
of design are essential Bugs and flaws are 50/50 Security is an
emergent property of the entire system (just like quality) To end
up with secure software, deep integration with the SDLC is
necessary
Slide 5
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. 2006: A Shift From Philosophy to HOW TO Integrating best
practices into large organizations SDLC (that is, an SSDL) o
Microsofts SDL o Cigitals Touchpoints o OWASP CLASP
Slide 6
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. Prescriptive vs. Descriptive Models Descriptive models
describe what is actually happening The BSIMM is a descriptive
model that can be used to measure any number of prescriptive SSDLs
Prescriptive Models Descriptive Models Prescriptive models describe
what you should do SAFECode SAMM SDL Touchpoints Every firm has a
methodology they follow (often a hybrid) You need an SSDL
Slide 7
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. BSIMM: Software Security Measurement Real data from (67)
real initiatives 161 measurements 21 (4) over time McGraw, Migues,
& West
Slide 8
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. 67 Firms in the BSIMM-V Community Intel
Slide 9
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. Building BSIMM (2009) Big idea: Build a maturity model
from actual data gathered from 9 well known large-scale software
security initiatives o Create a software security framework o
Interview nine firms in-person o Discover 110 activities through
observation o Organize the activities in 3 levels o Build scorecard
The model has been validated with data from 67 firms There is no
special snowflake
Slide 10
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. The Magic 30 Since we have data from > 30 firms we can
perform statistical analysis (Laurie Williams from NCSU is doing
more of that now) o How good is the model? o What activities
correlate with what other activities? o Do high maturity firms look
the same? We now have 67 firms with 161 distinct measurements o
BSIMM (the nine) o BSIMM Europe (nine in EU) o BSIMM2 (30) o BSIMM3
(42) o BSIMM4 (51) o BSIMM-V (67) data freshness emphasized
Slide 11
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. Monkeys Eat Bananas BSIMM is not about good or bad ways
to eat bananas or banana best practices BSIMM is about observations
BSIMM is descriptive, not prescriptive BSIMM describes and measures
multiple prescriptive approaches
Slide 12
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. A Software Security Framework Four domains Twelve
practices See informIT article on BSIMM website
http://bsimm.com
Slide 13
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. Example Activity [AA1.2] Perform design review for
high-risk applications. The organization learns about the benefits
of architecture analysis by seeing real results for a few
high-risk, high- profile applications. The reviewers must have some
experience performing architecture analysis and breaking the
architecture being considered. If the SSG is not yet equipped to
perform an in-depth architecture analysis, it uses consultants to
do this work. Ad hoc review paradigms that rely heavily on
expertise may be used here, though in the long run they do not
scale.
Slide 14
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. NEW BSIMM-V Activity [CMVM3.4] Operate a bug bounty
program. The organization solicits vulnerability reports from
external researchers and pays a bounty for each verified and
accepted vulnerability received. Payouts typically follow a sliding
scale linked to multiple factors, such as vulnerability type (e.g.,
remote code execution is worth $10,000 versus CSRF is worth $750),
exploitability (demonstrable exploits command much higher payouts),
or specific services and software versions (widely- deployed or
critical services warrant higher payouts). Ad hoc or short-duration
activities, such as capture-the-flag contests, do not count. [This
is a new activity that will be reported on in BSIMM6.]
Slide 15
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. BSIMM-V measurements
Slide 16
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. Real-world Data (67 firms) Initiative age o Average: 6
years o Newest: 0.4 o Oldest: 18.1 o Median: 5.3 SSG size o
Average: 14.78 o Smallest: 1 o Largest: 100 o Median: 7 Satellite
size o Average: 29.6 o Smallest: 0 o Largest: 400 o Median: 4 Dev
size o Average: 4190 o Smallest: 11 o Largest: 30,000 o Median:
1600 Average SSG size: 1.4% of dev group size
Slide 17
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. BSIMM-V Scorecard
Slide 18
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. Earth (67)
Slide 19
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. BSIMM-V as a measuring stick
Slide 20
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. BSIMM-V as a Measuring Stick Compare a firm with peers
using the high water mark view Compare business units Chart an SSI
over time
Slide 21
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. BSIMM-V Scorecard with FAKE Firm Data Top 12 activities o
purple = good? o red = bad? Blue shift practices to emphasize
Slide 22
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. comparing groups of firms
Slide 23
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. We Are a Special Snowflake (NOT) ISV (25) results are
similar to financial services (26)
Slide 24
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. BSIMM Longitudinal: Improvement over Time 21 firms
measured twice (an average of 24 months apart) Show how firms
improve o An average of 16% activity increase
Slide 25
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. BSIMM by the Numbers
Slide 26
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. The BSIMM Community BSIMM Conferences 2010: Annapolis, MD
2011: Stevenson, WA 2012: Galloway, NJ 2013: Dulles, VA BSIMM EU
Conferences 2012: Amsterdam 2013: London 2014: Ghent BSIMM RSA
Mixers 2010: RSA 2011: RSA 2012: RSA 2013: RSA 2014: RSA BSIMM
mailing list Moderated High S/N ratio BSIMM Community Conference
2014 November in San Diego
Slide 27
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. BSIMM-V to BSIMM6 BSIMM-V released October 2013 under
creative commons o http://bsimm.com http://bsimm.com o Italian,
German, and Spanish translations available BSIMM is a yardstick o
Use it to see where you stand o Use it to figure out what your
peers do BSIMM-VBSIMM6 o BSIMM is growing o Goal = 100 firms
Slide 28
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. where to learn more
Slide 29
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. SearchSecurity + Justice League www.searchsecurity.com
No-nonsense monthly security column by Gary McGraw
www.cigital.com/~gem/writing www.cigital.com/justiceleague In-depth
thought leadership blog from the Cigital Principals Gary McGraw
Sammy Migues John Steven Scott Matsumoto Paco Hope Jim
DelGrosso
Slide 30
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. Silver Bullet + IEEE Security & Privacy
www.cigital.com/silverbullet Building Security In Software Security
Best Practices column www.computer.org/security/bsisub /
Slide 31
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. The Book How to DO software security o Best practices o
Tools o Knowledge Cornerstone of the Addison- Wesley Software
Security Series www.swsec.com
Slide 32
Copyright 2013, Cigital and/or its affiliates. All rights
reserved. Build Security In WE NEED MORE BSIMM FIRMS Read the
Addison-Wesley Software Security series Send e-mail:
[email protected]