Understanding Mistakes Developers Make:
Qualitative Analysis
from Build It, Break It, Fix ItDaniel Votipka, Kelsey R. Fulton,
James Parker, Matthew Hou, Michelle L. Mazurek, and Michael
Hicks
University of Maryland, Maryland Cybersecurity Center
sec-professionals.cs.umd.edu
Research Questions
Build It, Break It, Fix It
Vulnerabilities Introduced
Recommendations
Problems
Exploitability
What types of vulnerabilities do developers introduce?
How severe and exploitable are they?
builditbreakit.org
Build It
Website:
Break It
Secure Log
Secure Comms
Multiuser Database
Event LogTime User Action Where
8:00 AM Bob Enter Gallery8:01 AM Alice Enter Office8:15 AM Alice
Exit Office
Misunderstanding
Unintuitive
Bad Choice
No Implementation
Mistake
Conceptual Error
Intuitive
Single step Few steps (deterministic) Many steps (deterministic)
Many steps (probabilistic)
Mis
take
No
Impl
.M
isun
d.
0 10 20 30 40
Deep InsightSource
Execution
Deep InsightSource
Execution
Deep InsightSourceExecution
# of vulnerabilities introduced
1. API improvement • Make unintuitive requirements
transparent • Document security assumptions and
risks of non-default use 2. Security education
• Cover all assumptions of security requirements
• Highlight importance of implementation decisions through
practice and feedback
3. Vulnerability analysis tools • Further work is needed to
address
design-level, conceptual issues
Easy to find and exploit
Easy to find
Hard to exploit
Examples
Associated with problem and design
complexity
Most related to misunderstanding security concepts Mistake
Conceptual Error
Disabled automatic integrity checks
Forgot to save nonce
mistakes Execution
mistakes Execution/source
misunderstood Execution
misunderstood Execution/source
misunderstood Execution/source/underlying concepts
noattempt Execution
noattempt Execution/source
noattempt Execution/source/underlying concepts
0 10 20 30 40value
com
bine
d
diff_to_exploit
Multi−staged/probabilistic
Mult−staged/deterministic/large
Multi−staged/deterministic/small
Single−staged
mistakes Execution
mistakes Execution/source
misunderstood Execution
misunderstood Execution/source
misunderstood Execution/source/underlying concepts
noattempt Execution
noattempt Execution/source
noattempt Execution/source/underlying concepts
0 10 20 30 40value
com
bine
d
diff_to_exploit
Multi−staged/probabilistic
Mult−staged/deterministic/large
Multi−staged/deterministic/small
Single−staged
mistakes Execution
mistakes Execution/source
misunderstood Execution
misunderstood Execution/source
misunderstood Execution/source/underlying concepts
noattempt Execution
noattempt Execution/source
noattempt Execution/source/underlying concepts
0 10 20 30 40value
com
bine
d
diff_to_exploit
Multi−staged/probabilistic
Mult−staged/deterministic/large
Multi−staged/deterministic/small
Single−staged
mistakes Execution
mistakes Execution/source
misunderstood Execution
misunderstood Execution/source
misunderstood Execution/source/underlying concepts
noattempt Execution
noattempt Execution/source
noattempt Execution/source/underlying concepts
0 10 20 30 40value
com
bine
d
diff_to_exploit
Multi−staged/probabilistic
Mult−staged/deterministic/large
Multi−staged/deterministic/small
Single−staged
mistakes Execution
mistakes Execution/source
misunderstood Execution
misunderstood Execution/source
misunderstood Execution/source/underlying concepts
noattempt Execution
noattempt Execution/source
noattempt Execution/source/underlying concepts
0 10 20 30 40value
com
bine
d
diff_to_exploit
Multi−staged/probabilistic
Mult−staged/deterministic/large
Multi−staged/deterministic/small
Single−staged
Hard to find and exploit
mistakes Execution
mistakes Execution/source
misunderstood Execution
misunderstood Execution/source
misunderstood Execution/source/underlying concepts
noattempt Execution
noattempt Execution/source
noattempt Execution/source/underlying concepts
0 10 20 30 40value
com
bine
d
diff_to_exploit
Multi−staged/probabilistic
Mult−staged/deterministic/large
Multi−staged/deterministic/small
Single−staged
mistakes Execution
mistakes Execution/source
misunderstood Execution
misunderstood Execution/source
misunderstood Execution/source/underlying concepts
noattempt Execution
noattempt Execution/source
noattempt Execution/source/underlying concepts
0 10 20 30 40value
com
bine
d
diff_to_exploit
Multi−staged/probabilistic
Mult−staged/deterministic/large
Multi−staged/deterministic/small
Single−staged
mistakes Execution
mistakes Execution/source
misunderstood Execution
misunderstood Execution/source
misunderstood Execution/source/underlying concepts
noattempt Execution
noattempt Execution/source
noattempt Execution/source/underlying concepts
0 10 20 30 40value
com
bine
d
diff_to_exploit
Multi−staged/probabilistic
Mult−staged/deterministic/large
Multi−staged/deterministic/small
Single−staged
http://builditbreakit.org
