Upload
yury-chemerkin
View
216
Download
0
Embed Size (px)
Citation preview
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
1/13
You wont believe that blind can see:Benchmarking SQL injection
scannersAndrew Petukhov, Karim Valiev
Moscow State University
Thursday, May 24,
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
2/13
Results Our approach showed that theres no such thing as the
best scanner
Questions?
See you!!!
Thursday, May 24,
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
3/13
Motivation Pentesting and auditing
- p0wn vs fight for completeness
- good idea to collect the low hanging fruits first
- but what if a scanner finds nothing?
- are there really no vulns? this is a point when you want to know aboutlimitations of your tools
- and what about of a superposition of tools? I.e. skipfish then w3af thensqlmap
Look if we could do better than Larry Suto
Look if we could produce a sound testing
Thursday, May 24,
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
4/13
SQLi detection methods Error-based- a baseline method, should be good in every scanner
- should be no problems with implementation
Blind time-based- main problem: submit a payload which leaves a query syntactically correct
- SELECT id, date, text FROM news LIMIT ?, 1
Blind content-based
- main problem: compare true and false responses
- http://vulnapp/item?id=13 should be the same as http://vulnapp/item?id=13 and1=1 and differ from http://vulnapp/item?id=13 and 1=0
- what is the same for an automated tool?
Thursday, May 24,
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
5/13
What our research IS NOT It is not sponsored => Acunetix does not win :(((( It does not give an answer on stupid questions like what is
the best scanner?
- suppose you have two equal scanners implementing time-based technique- scanner A: if(substr(field,i,1) = a, sleep(5), 0) for all characters
- scanner B: if(substr(field,i,1) < n, sleep(5), 0) - dichotomy
- one minimizes scan time and the other - number of requests - which one is better?
It does not measure the crawling process- yes, we know that in point-and-shoot scenario it is crucial
Quantity metrics is not a final result
We do not expect tools to test for second order SQLi
Thursday, May 24,
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
6/13
Wait! Tell me more about metrics!
One could expect to measure scanners using FP/FN rates
Imagine: scanner A detects 80 vulns out of 100 and scannerB detects 20 out of 100
- Scanner A is better? You cannot tell!
- 80 test out of 100 were error-based and 20 out of 100 were blind
- If we generated 100 000 blind tests and 1000 error-based, FP/FN rates would change!
Thursday, May 24,
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
7/13
Wait! Tell me more about metrics! Now one could expect to measure scanners
using FP/FN rates on test case classes- Like (80%, 20%, 30%) for coverage and (0.1%, 0.9%, 0.2%) for FP
Scanner metrics become incomparable- consider (80%, 20%, 30%) vs (55%, 85%, 0%)
- you can hope that one result would inset the other {(80%, 20%,30%) could include (60%, 10%, 25%)}, but this does not happen
After all, who would establish these classes?- one needs good performance in injection after the LIMIT
statement and the other one - good performance in injection intoDML queries
Thursday, May 24,
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
8/13
Ideas behind SQLi bench
We wanted our test cases to be representative
We wanted our test cases to be as complete as possible
How would we do that?
Thursday, May 24,
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
9/13
Our approach
Thursday, May 24,
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
10/13
Classification! A model of a general workflow with DBMS interaction
- Get user input
- Validate user input
- Construct a query
- Perform a query and handle the result
- Construct and issue an HTTP response
Classification of each step
Test set is a permutation of all classes with each other(44 536 test cases)
Thursday, May 24,
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
11/13
NumbersVulnerable False positive
Vuln. witherror output
No err. outputand no sleep()
Total 28848 15688 23544 2652
arachni-0.3 15088 0 13876 0sqlmap-0.9 12246 1208 10771 447
sqlmap-r5059 10939 83 10152 102
wapiti-2.2.1 11280 0 10068 0
skipfish-2.03b 18110 1680 16038 1036
skipfish-2.06b 22142 3360 19062 1540
w3af-1.1 21972 450 21576 193
Burp Suite Pro 1.4.07 13459 294 13099 180
Thursday, May 24,
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
12/13
Some findings Error-based method - approximately equal
results
Blind method - all scanners perform bad Bugs: skipfish and sqlmap does not detect
sql-inj with output in HTTP header Fastest scanner: w3af
Thursday, May 24,
7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners
13/13
Contacts
Karim: [email protected] Andrew: [email protected]
Th d M 24
mailto:[email protected]:[email protected]