Evaluation of Security Scanners for Web Application

Presented By:Sunint Kaur Khalsa (100875000)Sarabjeet Kaur Saini(6235987)

Outline

Context Goal and Scope of Study Methodology Evaluation Criteria Evaluation of Candidate Tools Conclusion and Recommendation

News…

Harvard Website attacked by Syrian Protesters

77 US Law Enforcement Websites hit in mass attack by “LulzSec” hacking group.

The website of World’s most popular Martial Arts Organisation “Ultimate Fighting Championship” hacked

…

Solution…

Firewall ?

Blue Crystal Inc.

Web Application Development firm with a Work Force of 15 people

Develop web applications based on .Net Platform

Incepted the idea of giving security services to their clients after selecting a suitable tool

Wanted a tool with high functionality, low cost, low resource consumption and high vulnerability detection

Goal and Scope of Study

Goal Select the most suitable tool for Blue Crystal as per their

given requirements.

Scope To conduct the evaluation of selected tools on the basis of

High Impact and Low impact criteria.

Methodology Used

Test Cases for the Evaluation Test websites provided by the vendors are used

Score given to each tool on the scale of 0-10 for the corresponding evaluation criteria

Weights have been assigned to the evaluation criteria

Final score =Where i= Evaluation Criteria wi = Weight of ith evaluation criteriasi= Score of the tool corresponding to the ith evaluation criteria

High Impact and Low Impact Criteria

Evaluation Criteria High Impact Criteria

Crawling and Parsing 5

Vulnerability Identification 5

Performance 4

Cost and License 5

Evaluation Criteria Low Impact Criteria

Ease of Installation 3

Usability 3

Scan Control Capability 3

Reporting and Documentation 3

Weighing Scheme

Tools Selected Rational Appscan

A Product of IBM Originally developed by Sanctum Ltd. First released in 1998

HP WebInspect A Product of HP Originally developed by SPI Dynamics

Test Websites

Tool Host Web Pages

Operating System

Web Server

Application Language

AppScan http://demo.testfire.net 34 Win32 – Windows XP

IIS ASP.NET

WebInspect http://zero.webappsecurity.com

100 Win32 – Windows XP

IIS ASP.NET

Ease of Installation This criterion considered the ease of acquisition and

installation of the tool Rational Appscan had a file of size 497 MB and took 5 hours

for its installation HP WebInspect took 2 hours for the installation of 641 MB file

but we had to wait for 6 hours to get the key as that required domain verification.

WebInspect also required SQL server and there is no such requirement for Appscan

Appscan = 8 WebInspect = 6

Usability

Usability Criterion is a combination of Ease of use Efficiency

AppScan takes screenshots of the browser responses corresponding to the generated attacks

AppScan provides in depth description of the detected vulnerabilities including possible causes, technical description and fixing recommendation whereas WebInspect provides only recommendations

WebInspect creates macros to record testing steps during scan and automate repeated testing

Appscan = 9 WebInspect = 8

Usability…

Usability…

Usability…

Scan Control Capability

Evaluated the scan control capabilities of both the tools to find which tool is better for handling the scan.

Both tools provide operator with the ability to Pause a scan Restart the scan at a later time

Both tools provides the viewing the real-time status of running scans. This status could include information such as which tests are currently being run and the scan completion percentage.

Appscan = 9 WebInspect = 9

Reporting and Documentation

This criterion evaluates the tool on the basis of Generation of reports in different formats Comprehensiveness of the generated reports

Appscan can generate different types of reports Security Report Industry Standard Report Regulatory Compliance Report Delta Analysis Report Template Based Report

Reporting and Documentation

Features of Appscan’s Report Report was divided into different sections based on the URLs, where

vulnerabilities have been encountered. Reports consisted of tables, text and graphs and hence more readable

and understandable The reports by WebInspect comprised of a lot of text with

definitions and explanation and less of graphs, tables.

Appscan = 10 WebInspect = 8

Report Generation in AppScan

Report Generation in WebInspect

Crawling and Parsing Crawling is an activity by which the scanner browses

various web elements like cookies, forms, parameters, links etc looking for vulnerabilities

Parsing is defined as crawling for the various types of contents like HTML, ActiveX objects, Java Applets, Java Scripts, XML etc

Both the tools have automated crawling In manual configuration, user is given the option

Specifying a request delay, Maximum crawl depth Have concurrent sessions

Crawling and Parsing

WebInspect has a feature which shows the steps the scanner took to reach a specific vulnerability, pointing to the specific element.

It is good if we want to retest certain flaws and to see how the scanner is working on it

WebInspect gives the feature to specify the request delay which is of interest to Blue Crystal Inc. as it might help them to use the bandwidth wisely

Appscan = 9 WebInspect = 10

Vulnerability Assessment

This criterion evaluates the total vulnerabilities which have been found by the web scanners on their respective test cases.

In order to find the vulnerabilities on the test websites the number of attacks sent by AppScan 18,634 on 34 pages as compared to 19,968 on 100 pages of WebInspect.

With three times the size of the test website WebInspect generates less attacks and this results in exposing less vulnerabilities.

Vulnerability Identification

Appscan exposed 120 vulnerabilities as compared to 272 vulnerabilities exposed by WebInspect. Here it is worth mentioning that the size of WebInspect’s test case is thrice as that of Appscan’s test case.

The various types of attacks detected by both the tools are SQL Injection Cross Site Scripting Buffer Overflow File guessing

Etc…

Appscan = 9 WebInspect = 7

Performance

This criterion covers the time in which the tool completes the scan and the resources utilized during the scan

Appscan completed the scan of website with 34 pages in 31 minutes where as WebInspect completed the scan of 100 pages in 15 minutes showing the better performance of WebInspect

The minimum system requirements of Appscan are 2.4GHz processor 2GB RAM 30GB of free disk space

The minimum system requirements for WebInspect are 1.5GHz processor 2GB of available RAM 10GB of free disk space

Appscan = 7 WebInspect = 8

Cost and License

Cost = Training cost + License Cost

The Training cost is considered the same for both the tools as both of them have online tutorials and quick start up kits.

Appscan = 8 WebInspect = 7

WebInspect Annual Audit License:  This licence type allows

access to client’s partner portal (They have the ability to scan

unlimited customers on any IP in their environment) + Annual

maintenance + customer support + access to daily updated

vulnerability checks + Additional Overhead for each additional

user

$ 20,000

IBM Rational App Scan Standard Edition + SW Subscription & Support 12 Months

$19,700

Score Earned by each Tool

Ease

of I

nstallatio

n

Usabilit

y

Scan C

ontro

l Cap

abilit

y

Repor

ting a

nd Doc

umen

tatio

n

Crawlin

g and P

arsin

g

Vulner

abilit

y Id

entifi

catio

n

Perfo

rman

ce

Cost a

nd Lice

nse

0

2

4

6

8

10

12

Rational Appscan

Total Score of each ToolEvaluation Criteria(i) Weight (wi) AppScan(si) WebInspect(si)

Ease of Installation 3 8 6

Usability 3 9 8

Scan Control Capability 3 9 9

Reporting and Documentation

3 10 8

Crawling and Parsing 5 9 10

Vulnerability Identification 5 9 7

Performance 4 7 8

Cost and License 5 8 7

Total Score 266 245

Conclusion and Recommendation

Rational AppScan is a clear winner and hence a better tool to fulfill the requirements prescribed by Blue Crystal Inc.

Number of attacks sent by AppScan were more as compared to WebInspect for exposing the vulnerabilities in the test website.

AppScan provides in depth description of the detected vulnerabilities including possible causes, technical description and fixing recommendation whereas WebInspect provides only recommendations, required from development point of view.

References

http://www.ibm.com/software/awdtools/appscan/, http://welcome.hp.com/country/us/en/prodserv/

software.html http://en.wikipedia.org