Transcript BSIMM-V

BSIMM-V
THE BUILDING SECURITY IN
MATURITY MODEL
GARY MCGRAW, PH.D.
CHIEF TECHNOLOGY OFFICER
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
Cigital
• Providing software security professional services since
1992
• World’s premiere software security consulting firm
o 270 employees
o Washington DC, New York, Santa Clara, Bloomington, Boston,
Chicago, Atlanta, Amsterdam, and London
• Recognized experts in software security
o Widely published in books, white papers, and articles
o Industry thought leaders
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
BSIMM basics
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
We Hold These Truths to be Self-evident
• Software security is more than a set of security functions
o Not magic crypto fairy dust
o Not silver-bullet security mechanisms
• Non-functional aspects of design are essential
• Bugs and flaws are 50/50
• Security is an emergent property of the entire system
(just like quality)
• To end up with secure software, deep integration with the
SDLC is necessary
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
2006: A Shift From Philosophy to HOW TO
• Integrating best practices into large organizations’ SDLC
(that is, an SSDL)
o Microsoft’s SDL
o Cigital’s Touchpoints
o OWASP CLASP
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
Prescriptive vs. Descriptive Models
Prescriptive Models
•
Prescriptive models describe
what you should do
•
•
•
•
•
•
SAFECode
SAMM
SDL
Touchpoints
Descriptive Models
•
•
Descriptive models describe
what is actually happening
The BSIMM is a descriptive
model that can be used to
measure any number of
prescriptive SSDLs
Every firm has a methodology
they follow (often a hybrid)
You need an SSDL
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
BSIMM: Software Security Measurement
• Real data from (67) real
initiatives
• 161 measurements
• 21 (4) over time
• McGraw, Migues, & West
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
67 Firms in the BSIMM-V Community
Intel
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
Building BSIMM (2009)
• Big idea: Build a maturity model from actual data
gathered from 9 well known large-scale software security
initiatives
o
o
o
o
o
Create a software security framework
Interview nine firms in-person
Discover 110 activities through observation
Organize the activities in 3 levels
Build scorecard
• The model has been validated with data from 67 firms
• There is no special snowflake
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
The Magic 30
• Since we have data from > 30 firms we can perform
statistical analysis (Laurie Williams from NCSU is doing
more of that now)
o How good is the model?
o What activities correlate with what other activities?
o Do high maturity firms look the same?
• We now have 67 firms with 161 distinct measurements
o
o
o
o
o
o
BSIMM (the nine)
BSIMM Europe (nine in EU)
BSIMM2 (30)
BSIMM3 (42)
BSIMM4 (51)
BSIMM-V (67) ← data freshness emphasized
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
Monkeys Eat Bananas
• BSIMM is not about good
or bad ways to eat
bananas or banana best
practices
• BSIMM is about
observations
• BSIMM is descriptive, not
prescriptive
• BSIMM describes and
measures multiple
prescriptive approaches
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
A Software Security Framework
• Four domains
• Twelve practices
• See informIT article on BSIMM website http://bsimm.com
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
Example Activity
[AA1.2] Perform design review for high-risk applications.
The organization learns about the benefits of architecture
analysis by seeing real results for a few high-risk, highprofile applications. The reviewers must have some
experience performing architecture analysis and breaking
the architecture being considered. If the SSG is not yet
equipped to perform an in-depth architecture analysis, it
uses consultants to do this work. Ad hoc review paradigms
that rely heavily on expertise may be used here, though in
the long run they do not scale.
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
NEW BSIMM-V Activity
[CMVM3.4] Operate a bug bounty program. The
organization solicits vulnerability reports from external
researchers and pays a bounty for each verified and
accepted vulnerability received. Payouts typically follow a
sliding scale linked to multiple factors, such as vulnerability
type (e.g., remote code execution is worth $10,000 versus
CSRF is worth $750), exploitability (demonstrable exploits
command much higher payouts), or specific services and
software versions (widely- deployed or critical services
warrant higher payouts). Ad hoc or short-duration activities,
such as capture-the-flag contests, do not count. [This is a
new activity that will be reported on in BSIMM6.]
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
BSIMM-V measurements
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
Real-world Data (67 firms)
• Initiative age
o
o
o
o
Average: 6 years
Newest: 0.4
Oldest: 18.1
Median: 5.3
• SSG size
o
o
o
o
Average: 14.78
Smallest: 1
Largest: 100
Median: 7
• Satellite size
o
o
o
o
Average: 29.6
Smallest: 0
Largest: 400
Median: 4
• Dev size
o
o
o
o
Average: 4190
Smallest: 11
Largest: 30,000
Median: 1600
Average SSG size: 1.4% of dev group size
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
BSIMM-V Scorecard
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
Earth (67)
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
BSIMM-V as a measuring stick
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
BSIMM-V as a Measuring Stick
• Compare a firm
with peers using
the high water
mark view
• Compare
business units
• Chart an SSI
over time
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
BSIMM-V Scorecard with FAKE Firm Data
• Top 12 activities
o purple = good?
o red = bad?
• “Blue shift”
practices to
emphasize
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
comparing groups of firms
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
We Are a Special Snowflake (NOT)
ISV (25) results are similar to financial services (26)
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
BSIMM Longitudinal: Improvement over Time
• 21 firms measured twice (an average of 24 months apart)
• Show how firms improve
o An average of 16% activity increase
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
BSIMM by the Numbers
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
The BSIMM Community
BSIMM Conferences
BSIMM RSA Mixers
•
•
•
•
•
•
•
•
•
2010: Annapolis, MD
2011: Stevenson, WA
2012: Galloway, NJ
2013: Dulles, VA
2010: RSA
2011: RSA
2012: RSA
2013: RSA
2014: RSA
BSIMM EU Conferences
•
•
•
2012: Amsterdam
2013: London
2014: Ghent
BSIMM mailing list
•
•
Moderated
High S/N ratio
BSIMM Community
Conference 2014
•
November in San Diego
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
BSIMM-V to BSIMM6
• BSIMM-V released October 2013 under creative
commons
o http://bsimm.com
o Italian, German, and Spanish translations available
• BSIMM is a yardstick
o Use it to see where you stand
o Use it to figure out what your peers do
• BSIMM-V→BSIMM6
o BSIMM is growing
o Goal = 100 firms
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
where to learn more
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
SearchSecurity + Justice League
www.cigital.com/justiceleague
In-depth thought leadership
blog from the Cigital Principals
www.searchsecurity.com
No-nonsense monthly security
column by Gary McGraw
•
•
•
•
•
•
Gary McGraw
Sammy Migues
John Steven
Scott Matsumoto
Paco Hope
Jim DelGrosso
www.cigital.com/~gem/writing
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
Silver Bullet + IEEE Security & Privacy
Building Security In
Software Security Best Practices
column
www.computer.org/security/bsisub/
www.cigital.com/silverbullet
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
The Book
How to DO software security
o Best practices
o Tools
o Knowledge
Cornerstone of the AddisonWesley Software Security
Series
www.swsec.com
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.
Build Security In
WE NEED MORE BSIMM FIRMS
Read the Addison-Wesley Software
Security series
Send e-mail: [email protected]
Copyright © 2013, Cigital and/or its affiliates. All rights reserved.