SATE VI BackgroundBug Injection in SATE VI Aurelien Delaitre Lead, SATE VI Classic Track Prometheus...

Bug Injection in SATE VI

Aurelien DelaitreLead, SATE VI Classic TrackPrometheus Computing LLC

SATE VI Workshop - September 19, 2019 - MITRE, McLean VA

https://samate.nist.gov/SATE.html

Why Bug Injection?

Relevance

GroundTruth

StatisticalSignificance

2

Why Bug Injection?

Relevance

GroundTruth

StatisticalSignificance

ProductionSoftware

Common Vulnerabilities and Exposures (CVE)

SyntheticTest Suites

3

Why Bug Injection?

Relevance

GroundTruth

StatisticalSignificance

BugInjection

4

Ways to “Get” Bugs

● Bug Injectors● Manual & Semi-Automated Injection● Specifically Developed Test Suites● Existing Bugs

○ Discovered○ Undiscovered

5

Bug Types in SATE VI

C: Undefined Behavior

● Pointers● Buffers● Initialization

Java: Code Injection

● Cross-Site Scripting (XSS)● SQL Injection

▶ High-Impact▶ Easy to Prove

6

Proof of Vulnerability (PoV)

Why?

● Proves Bug Matters● Retrieve Bug Trace

How?

● Fuzzing● Bug Tracker● Manual

7

Bug Traces

● Based on PoVs○ C: GDB / Valgrind / ASAN○ Java: Flow

● Manual Analysis○ Doc Review○ Code Review

8

What Went Wrong?

9

Cheap but Hard Bugs

10

packet-arp.c

▶ Almost Never Found by Tools

Asymmetrical Bug/Fix Pairs

11

SimplePageBean.java

Buggy

Fixed

Buggy Bugs

▶ Implementation-dependent▶ Unknown Sink

▶ Tainted Data Questionable▶ Unintended Bug Type

fts3_write.c

global.c

12

fts3_write.c

Buggy Fixes

▶ Tainted Data Questionable▶ Condition Always False

global.c

13

pragma.c

Buggy Bugs

▶ Tainted Data Questionable▶ Condition Always True

global.c

14

pragma.c

Buggy Fixes

▶ Tainted Data Questionable▶ Condition Always False

global.c

15

fts3_tokenize_vtab.c

Buggy Fixes

▶ Tainted Data Questionable▶ Condition Always False Due to Programming Error

global.c

16

Sink Separationdate.c insert.c

17

Shadowing

18

Shadowing

19

Shadowing

20

Take Away

21

AutomatedBug Injection Curation Test Suites

StrongerBetterFaster