Embed Size (px)
Citation preview
Effective Anomaly Detection with Scarce Training Data
William Robertson∗
[email protected] Berkeley
Federico Maggi∗
[email protected] di Milano
Christopher Kruegel Giovanni Vigna{chris,vigna}@cs.ucsb.eduComputer Security Group
UC Santa Barbara
Abstract
Learning-based anomaly detection has proven to bean effective black-box technique for detecting unknownattacks. However, the effectiveness of this techniquecrucially depends upon both the quality and the com-pleteness of the training data. Unfortunately, in mostcases, the traffic to the system (e.g., a web applica-tion or daemon process) protected by an anomaly de-tector is not uniformly distributed. Therefore, somecomponents (e.g., authentication, payments, or contentpublishing) might not be exercised enough to train ananomaly detection system in a reasonable time frame.This is of particular importance in real-world settings,where anomaly detection systems are deployed with lit-tle or no manual configuration, and they are expected toautomatically learn the normal behavior of a system todetect or block attacks.
In this work, we first demonstrate that the featuresutilized to train a learning-based detector can be se-mantically grouped, and that features of the same grouptend to induce similar models. Therefore, we proposeaddressing local training data deficiencies by exploitingclustering techniques to construct a knowledge base ofwell-trained models that can be utilized in case of un-dertraining. Our approach, which is independent of theparticular type of anomaly detector employed, is vali-dated using the realistic case of a learning-based systemprotecting a pool of web servers running several webapplications such as blogs, forums, or Web services. Werun our experiments on a real-world data set containingover 58 million HTTP requests to more than 36,000 dis-tinct web application components. The results show thatby using the proposed solution, it is possible to achieveeffective attack detection even with scarce training data.
Keywords: Anomaly detection, training data, web appli-cation.
1 Introduction
The Internet has evolved from its humble beginningsat CERN in 1991 into a massive network of ubiquitousservices that spans the globe and reaches an estimated1.4 billion people [27]. The World Wide Web containsmore than 100 million sites [46] and around 1 trillionunique URLs as indexed by Google [1]. Due to its per-vasive nature, the Internet – and in particular the Web– has become a predominant medium for disseminat-ing and collecting information. In fact, web applicationshave enjoyed immense popularity as an efficient meansfor providing services to users. For instance, Face-book has more than 250 million active users, upload-ing more than 1 billion photos each month, and Twit-ter distributed more than 3 million messages per day inMarch 2008 [34].
Unfortunately, applications have been found to con-tain many security vulnerabilities, due to a combina-tion of unsafe development tools and a historical lackof security awareness among developers. In addition,the risks are magnified when vulnerable software is de-ployed in the context of the Web, since applications aretypically widely accessible and often have access to sen-sitive information. These factors have naturally resultedin web-related vulnerabilities receiving substantial at-tention from the criminal underground [40]. As a con-sequence, the incidence of data breaches, online fraud,and other crimes resulting from the exploitation of webapplication vulnerabilities continues to rise [29,33], and,therefore, it is essential to protect applications and sys-tems connected to the Internet against such attacks.
Anomaly detection has received much attention fromthe research community as an approach to detecting andpreventing unknown attacks by monitoring a network’straffic [20,25,26,32,43,44,47] or a host’s operating sys-tem [3,14,21,24,28,31,38,42]. Recently, anomaly-basedtechniques have also been shown to be effective againstweb-based threats [5, 15, 19, 30]. Effective anomaly de-tection systems are attractive because they consider theprotected system as a black box. As a result, they can
be deployed in live environments without any a prioriknowledge about the application.
Anomaly detection systems contain specifications, ormodels, of the normal behavior of the protected system,and consider deviations from the specifications to be ev-idence of malicious behavior. In contrast to signature-based systems, anomaly detectors have the desirableproperty that previously unknown attacks can be iden-tified automatically. Though anomaly detection modelscan be manually specified by domain experts, this is atedious, labor-intensive, and error-prone process. There-fore, most research has instead focused on applying ma-chine learning techniques to automatically derive mod-els of normal behavior from unlabeled training data. Theterm normal behavior generally refers to a set of charac-teristics (e.g., the distribution of the symbols of strings,or the mean and standard deviation of the values of nu-merical variables) extracted from data observed duringa system’s normal operation. For instance, such datacould be the payloads of network packets, or HTTP re-quests and responses exchanged between a web serverand clients. Those characteristics are used to build mod-els of normal behavior. Learning-based anomaly detec-tors obviate the tedious and error-prone task of creat-ing specifications, and, additionally, are able to adapt tothe particular characteristics of the local environment.Therefore, anomaly detectors typically require only amodest initial configuration effort to provide effectiveattack detection.
In an ideal case, a learning-based anomaly detectionsystem is deployed in front of a system and, in a com-pletely automated fashion, learns the normal interactionbetween the system and its users. Once enough trainingdata has been analyzed and the profiles for the moni-tored systems have been established, the anomaly detec-tor switches to detection mode; it is then able to detectattacks that represent anomalies with respect to normalusage. These types of anomaly detection systems areextremely attractive to security officers and site admin-istrators, who have neither the resources nor the skills tomanually analyze applications composed of hundreds ofcomponents. Because of this, several commercial webapplication firewalls implement some form of machinelearning to support anomaly detection [4, 6, 11].
Learning-based anomaly detectors are not withouttheir drawbacks, however. In fact, these systems areknown for their tendency to produce a non-negligibleamount of false positives due to the difficulty of accu-rately modeling non-trivial domains. This is a limit-ing factor for the effectiveness of such systems [2, 13].An additional limitation is that anomaly detection sys-tems critically rely upon the quality of the training dataused to construct their models. In particular, trainingsets must be free from attacks. Otherwise, the result-
ing models will be prone to false negatives, as attackmanifestations will have been learned as normal behav-ior [8, 10, 17, 39].
Another limitation that is well-known in the researchcommunity is the difficulty of obtaining enough high-quality training data. Unfortunately, to our knowledge,no proposals exist that satisfactorily address the prob-lem. In particular, our experiments suggest that web ap-plication component invocations are non-uniformly dis-tributed. That is, relatively few components dominate,and the remaining components are accessed relativelyinfrequently. Thus, for those components, it is oftenimpossible to gather enough training data to accuratelymodel their normal behavior. We informally refer tothe components that receive insufficient accesses as the“long tail” of a web application. Note that, however, thisdoes not necessarily imply a power law distribution ofthese accesses. Nevertheless, components that are infre-quently accessed lead to poor detection capabilities dueto undertrained models (i.e., models possessing knowl-edge limited to the low number of samples they havebeen trained on).
This work addresses the problem of undertrainedmodels by exploiting natural similarities among themodeled features. We demonstrate this hypothesis usingthe web applications context as a real-world example.In particular, we show that the values of the parametersextracted from HTTP requests can generally be catego-rized according to their type, such as an integer, date, orstring. Moreover, our experiments demonstrate that pa-rameters of similar type induce similar models of normalbehavior. Taken together, these results can be leveragedto effectively supplement a lack of training data for oneweb application component with similar data from an-other component that has received more requests.
In this paper, we make the following contributions:
• We introduce the problem that arises from the factthat traffic is distributed in a non-uniform fashion,and we provide evidence that this occurs in the realworld in the case of web applications.
• We propose an approach to address the problem ofundertraining by using global knowledge built byexploiting similarities between web application pa-rameters of similar type.
• We evaluate our approach on a large data set ofreal-world traffic from many web applications, anddemonstrate how anomaly detectors can accuratelymodel those components that would otherwise beassociated with undertrained models.
The results of our experiments show that by usingour approach, it is possible to improve the detection rateof undertrained models. In particular, web application
Ri =
ri,1 = /article,
ri,2 = /comments,
ri,3 = /comments/edit,
ri,4 = /account,
ri,5 = /account/password
Figure 1: Example resources comprising a web application.
components that have received only a few dozen requestscan be protected almost as effectively as those that com-ponents have received thousands of requests.
2 Training data scarcity
To understand why the problem of undertrained mod-els exists, we first present a general description of ananomaly detection system designed to protect web ap-plications. While this description is based on the systemdescribed in [19], we believe it represents an accurateabstraction of web application anomaly detection. Wethen use these concepts to introduce the long-tail prob-lem and its ramifications on the feasibility of construct-ing accurate models.
It is important to note that although there is no estab-lished architecture for these systems, a large portion oflearning-based anomaly detectors are designed in a sim-ilar manner. In particular, they extract some significantfeatures from the captured traffic, estimate the parame-ters of a set of pre-existing models (learning phase), andthen use these models to analyze live traffic and recog-nize unexpected values of the selected features. Thus,the description that follows can easily be generalized.In addition, we remind the reader that a low detectionaccuracy due to undertraining may affect any type oflearning-based protection system, rather than those de-tectors specifically designed for the web domain.
2.1 Web application anomaly detection
Without loss of generality, a set of web applicationsA can be organized into a set of resource paths or com-ponents R, and named parameters P . For example,a web application ai = blog.example.com might becomposed of the resources shown in Figure 1.
In this example, resource path ri,5 might take a setof parameters as part of the HTTP request: Pi,5 ={pi,5,1 = id, pi,5,2 = oldpw, pi,5,3 = newpw}.
A generic learning-based application intrusion detec-tion system operates by observing a sequence of re-quests Q = {q1, q2, . . .} issued by clients to the set ofmonitored applications. Each request q ∈ Q is repre-sented by the tuple ⟨ai, ri,j , Pq⟩, where Pq is a set ofparameter name-value pairs such that Pq ⊆ Pi,j .
As highlighted in Figure 5, the initial training is per-formed offline. During this phase, the anomaly detec-tor learns the behavior of the monitored web applica-tions in terms of models. As new web application, re-source path, and parameter instances are observed, thesets A, R, and P are updated. For each unique pa-rameter pi,j,k observed in association with a particularapplication ai and path ri,j , a set of models that char-acterizes the normal behavior of various features of theparameter is constructed. The set of models associatedwith each unique parameter instance can be representedas a tuple c(.) = ⟨m1,m2, . . . ,mu, . . . ,mU ⟩, referredto as a profile or model composition. Therefore, foreach application ai and resource path ri,j , a set Ci,j ofmodel compositions is constructed, one for each param-eter pi,j,k ∈ Pi,j . The knowledge base of an anomalydetection system trained on web application ai is de-noted by Cai =
∪j Ci,j . A graphical representation of
how a knowledge base is modeled for multiple web ap-plications is depicted in Figure 2.
To introduce and address the problem of undertrain-ing, we leverage the set of models described in [19] asa real-world case. According to the system describedin [19], a profile for a given parameter pi,j,k is the tupleci,j,k = ⟨m(tok), m(int), m(len), m(char), m(struct)⟩. m(tok)
models parameter values as a set of legal tokens (e.g.,the set of of possible values for the parameter genderobserved during training). m(int) and m(len) describenormal intervals for integers and string lengths, respec-tively, in a distribution-independent fashion using theChebyshev inequality. m(char) models character stringsas a ranked frequency histogram, or Idealized Charac-ter Distribution (ICD), that are compared using the χ2
or G tests. m(struct) models sets of character strings byinducing a Hidden Markov Model (HMM). The HMMencodes a probabilistic grammar that can produce a su-perset of strings observed in a training set. Aside fromthe addition of m(int), which is a straightforward gen-eralization of m(len) to numerical ranges, the interestedreader may refer to [19] for further details.
After training, learning-based systems are typicallyswitched to detection mode, which is performed online.The models trained in the previous phase are queried todetermine whether or not the new parameters observedare anomalous. Without going into the details of a par-ticular implementation, each parameter is compared toall the applicable models (e.g., an integer parameter iscompared to m(int), while a string parameter is com-pared to m(len), m(char), and m(struct)) and an aggregatedanomaly score on the interval [0, 1] is calculated by com-posing the values returned by the individual models. Ifthe anomaly score is above a certain threshold, an alertis generated.
Ca1 · · · Cai · · · CaI
Cr1,1 · · · · · · Cr1,j · · · · · · Cri,J
cp1,1,1 · · · c(·) · · · c(·) · · · cpi,j,k · · · c(·) · · · c(·) · · · cpi,j,K
‖ ‖ ‖〈m1, . . . ,mU 〉 · · · 〈m1, . . . ,mU 〉 · · · 〈m1, . . . ,mU 〉
http : //blog.example.com,
...
http : //dav.example.com
/article,
/comments,
...
/account,
/account/password
id = 1
date = 10− 11− 2004
title = foo
Figure 2: Overview of web application model construction.
2.2 The problem of nonuniform training data
Because learning-based detectors dynamically buildspecifications of normal behavior from training data, itis clear that the quality of the detection critically reliesupon the quality of the training data. For instance, onerequirement typically imposed upon a training set is thatit should be attack-free – that is, it should not containtraces of malicious activity that would induce the result-ing models to consider malicious behavior as normal.One solution to this issue is described in [8]. Anotherrequirement that a training set should satisfy is that itshould accurately represent the normal behavior of themodeled features. In some ways, this requirement isthe dual of the previous one: training data should com-pletely cover all aspects of normal behavior.
The difficulty of obtaining sufficient training datato accurately model application behavior is intuitivelyclear. We are, however, not aware of any solutions thatcan address this issue when insufficient training datais available. In a sense, this issue is similar to thoseaddressed by statistical analysis methods with missingdata [22]. Although a training procedure would benefitfrom such mechanisms, they require a complete redesignof the training algorithm specific to each model. Instead,a non-obtrusive approach that can improve an existingsystem without modifying the undertrained models ismore desirable. Typically, anomaly-based detectors can-not assume the presence of a testing environment thatcan be leveraged to generate realistic training data thatexercises the web application in a safe, attack-free envi-ronment. Instead, the anomaly detection system is de-ployed in front of live web applications with no a prioriknowledge of the applications’ components and their be-havior. If anomaly-based detectors required manual orsemi-automatic testing to be effective, their maintenancewould be as tedious as that of misuse-based systems.
/article = 475, 000
/comments = 15, 000
/comments/edit = 9, 000
/account = 900
/account/password = 100
Figure 4: Example non-uniform web application request dis-tribution.
In the case of low-traffic applications, problems ariseif the rate of client requests is inadequate to allow mod-els to train in a timely manner. However, even in thecase of high-traffic applications, a large subset of re-source paths might fail to receive enough requests to ad-equately train1 the associated models. This phenomenonis a direct consequence of the fact that requests issuedby clients often follow a non-uniform distribution. Toillustrate this point, Figure 3 plots the normalized cumu-lative distribution function of web client resource pathinvocations for a variety of real-world, high-traffic webapplications (details on this data are provided in Sec-tion 4). Although several applications have an approxi-mately uniform client access distribution, a clear major-ity exhibit skewed distributions. Indeed, in many cases,a large percentage of resource paths receive a compar-atively minuscule number of requests. Returning to theexample resources shown in Figure 1, assuming an over-all request volume of 500,000 requests per day, the ex-ample resource path set might result in the client accessdistribution shown in Figure 4.
Clearly, profiles for parameters to resource paths suchas /article will likely receive sufficient training data.
1A more formal definition of the minimum amount of samples re-quired for a complete training is provided in Section 3.1.
0
0.2
0.4
0.6
0.8
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Acce
ss C
DF
Resources
Site 1Site 2Site 3Site 4Site 5Site 6Site 7Site 8Site 9
Site 10Site 11Site 12Site 13Site 14Site 15Site 16Site 17Site 18Site 19Site 20Site 21Site 22Site 23Site 24
Figure 3: Web client resource path invocation distributions from a selection of real-world web applications.
This is not true, however, for profiles associated withpaths such as /account/password. Further exacerbat-ing the situation is the fact that a client request does notnecessarily include all possible parameters.
The infeasibility for an anomaly detection system toaccurately model a large subset of a web application isproblematic in itself. We argue, however, that the impactof the problem is magnified by the fact that componentsof a web application that are infrequently exercised arealso likely to contain a disproportionately large share ofsecurity vulnerabilities. This is a consequence of the re-duced amount of testing that developers invariably per-form on less prominent components of a web applica-tion, resulting in a higher rate of software defects. In ad-dition, the relatively low request rate from users of theweb application results in a reduced exposure rate forthese defects. Finally, when flaws are exposed and re-ported, correcting the flaws may be given a lower prior-ity than those in higher traffic components of a web ap-plication. An example is the authentication manager ofa blog, which receives a low share of the traffic if com-pared with the component that handles the comments orthe archive.
Therefore, we conclude that a mechanism to addressthe problem of model undertraining caused by the non-uniform distribution of training data is necessary for aweb application anomaly detection system to provide anacceptable level of security.
3 Exploiting global knowledge
The lack of available training data is a fundamentalobstacle when constructing accurate profiles for manyparameters of a web application. Without a minimumnumber of requests to a given parameter, it is infeasi-ble to construct models that encode a reasonably preciseapproximation of normal behavior.
We observe, however, that parameters associatedwith the invocation of components belonging to differ-ent web applications often exhibit a marked similarityto each other. Referring again to the example shown inFigure 1, many web applications take an integer value asa unique identifier for a class of objects such as a blogarticle or comment, as in the case of the id parameter.Many web applications also accept date ranges similar tothe date parameter as identifiers or as constraints upona search request. Similarly, as in the case of the title
parameter, web applications often expect a short phraseof text as an input, or perhaps a longer block of text inthe form of a comment body. One can consider each ofthese groupings of similar parameters as distinct param-eter types, though this need not necessarily correspondto the concept of types as understood in the program-ming languages context.
The key insight behind our approach is that parame-ters of the same type tend to induce model compositionsthat are similar to each other in many respects. Conse-quently, if the lack of training data for a subset of the
components of a web application prevents an anomalydetection system from constructing accurate profiles forthe parameters of those components, it is possible tosubstitute profiles for similar parameters of the sametype that were learned when enough training data wasavailable. It must be underlined that the substitution op-erates at the granularity of parameters rather than re-quests (which may contain more than one parameter).This increases the likelihood of finding applicable pro-file similarities, and allows for the substitution of mod-els taken from radically different components. However,although the experiments we run on real-world data con-firm that the aforementioned insight is realistic, our hy-pothesis might not hold in some very specific settings.Thus, to minimize the risks brought by migrating globalknowledge across different deployments, we interpretedthis result only as an insight and developed a robust cri-terion able to find similar profiles independently fromthe actual types of the modeled parameters.
Our approach is composed of three phases. The firstphase, shown in Figures 5 and 6b, is an extension ofthe training procedure originally implemented in [19],where undertrained versions of profiles are recorded inaddition to their final states. In the second phase, aglobal knowledge base of profiles C =
∪aiCai is con-
structed offline, where Cai are knowledge bases contain-ing only well-trained, stable profiles (right side of Fig-ure 6a) from anomaly detection systems previously de-ployed on a set of web applications
∪i ai. The left side
of Figure 6a depicts the progressive construction of theknowledge base CI =
∪aiCIai
of undertrained profiles(i.e., an index into C, where CI
aiis a knowledge base of
undertrained profiles from the web application ai). Ad-ditionally, we define a mapping f :
{CI
}× Cai 7→ C
(shown as a dotted, directed arrow) between under-trained and well-trained profiles.
The third phase is performed online. For any newweb application where insufficient training data is avail-able for a component’s parameter, the anomaly detectorfirst extracts the undertrained profile c′. Then, the globalknowledge base C is queried to find a similar, previouslyconstructed profile f
(CI , c′
)= c. The well-trained pro-
file c is then substituted for the undertrained profile c′ inthe detection process.
3.1 Phase I: Enhanced training
As a first step, we extended the anomaly detectordescribed in [19] with a mechanism to generate under-trained profiles from a data set. These undertrained pro-files are generated using the following procedure. LetQ
(p)ai = {q(p)1 , q
(p)2 , . . .} denote a sequence of client re-
quests containing parameter p for a given web applica-tion. Over Q(p)
ai , profiles are deliberately undertrained on
randomly sampled κ-sequences, where κ can take val-ues in
∪8i=0 2
i (a discussion of appropriate values for κis deferred until Section 3.4). Each of the resulting pro-files is then added to a knowledge base CI
ai. Note that
the random sub-sampling is performed with the goal ofinducing undertraining to show that clustering is feasibleand leads to the desired grouping even – and especially– in the presence of undertraining.
In general, κ corresponds to a number of train-ing samples that is considered insufficient to accuratelycharacterize a feature. This, however, warrants a discus-sion of what is considered sufficient. An obvious choiceis to fix a large, constant training phase length (e.g.,1000 requests). Unfortunately, an appropriate trainingphase length is dependent upon the complexity of mod-eling a given set of features. Therefore, we have devel-oped an automated method that leverages the notion ofmodel stability to determine when a model has observedenough training samples to accurately approximate thenormal behavior of a parameter.
As new training samples are observed early in thetraining phase, the state of a model typically exhibitsfrequent and significant change as its approximation ofthe normal behavior of a parameter is updated. Infor-mally, in an information-theoretic sense, the average in-formation gain of each new training sample is high. Asa model’s state converges to a more precise approxima-tion of normal behavior, its state gradually exhibits in-frequent and incremental changes. In other words, theinformation gain of new training samples approacheszero, and the model stabilizes.
Each model monitors its stability during the trainingphase by maintaining a history of snapshots of its inter-nal state. Periodically, a model checks if the sequence ofdeltas between each successive historical state is mono-tonically decreasing and whether the degree of changedrops below a certain threshold. If both conditions aresatisfied, then the model is considered stable. Let κ(u)
stabledenote the number of training samples required for amodel to achieve stability. A profile is considered sta-ble when all of its constituent models are stable. Thus,the number of training samples required for a profile toachieve stability is given by
κstable = maxu∈U
κ(u)stable. (3.1)
At the end of this phase, the final state of each well-trained, or stable, profile is stored in a knowledge baseCai . Both Cai and CI
aiare collected from each web ap-
plication, and serve as input to the next phase.Instead of describing the internal stop criterion spe-
cific to each model, if any, we developed a model-agnostic minimization algorithm detailed in Section 3.4(and evaluated in Section 4) that allows one to trade off
Qa1
. . .
QaI
Training
. . .
Training
CIa1
Ca1
CIaI
CaI
Clustering
CI
C
Q Training
CI C
c! c
1st phase (o!ine) 2nd phase (o!ine) 3rd phase (online)
1
Figure 5: Overall procedure. Profiles, both undertrained and well-trained, are collected from a set of web applications. Theseprofiles are processed offline to generate the global knowledge base C and index CI . At another web application, givenan undertrained profile c′, C can then be queried to find a suitable replacement profile c.
detection accuracy against the number of training sam-ples available.
3.2 Phase II: Building profile knowledge bases
The second phase consists of processing the outputof the first phase, namely the sets of knowledge bases ofboth undertrained and well-trained profiles learned froma variety of web applications. The goal is to create Cand CI , global knowledge bases of well-trained and un-dertrained profiles, respectively, and a mapping betweenthe two, allowing CI to serve as an index to C.
3.2.1 Constructing global knowledge base indices
The construction of the undertrained profile databaseCI begins by merging a set of knowledge bases{CIa1, CI
a2, . . . , CI
aI
}that have previously been built by
a web application anomaly detector over a set of webapplications
∪i ai during the first phase. The profiles in
CI are then clustered to group profiles that are seman-tically similar to each other. Profile clustering is per-formed in order to time-optimize query execution whenusing CI as an index into C. The resulting clusters ofprofiles in CI are denoted by HI =
∪i h
Ii . In this work,
an agglomerative hierarchical clustering algorithm usinggroup average linkage was applied, although the cluster-ing stage is agnostic as to the specific algorithm. In prin-ciple, other mechanisms (e.g., Support Vector Machines,Locality-Sensitive Hashing) to find groups or classes ofsimilar models based on their features can be used. Weopted for a clustering algorithm as a proof-of-conceptimplementation of Phase II primarily due to its simplic-ity. For an in-depth discussion of clustering algorithmsand techniques, we refer the reader to [45].
Central to any clustering algorithm is the distancefunction, which defines how similarities between the ob-
jects to be clustered are calculated. A suitable distancefunction must reflect the semantics of the objects un-der consideration, and should satisfy two conditions: 1)the overall similarity between elements within the samecluster is maximized, and 2) the similarity between ele-ments within different clusters is minimized.
We define the distance between two profiles to bethe sum of the distances between the models compris-ing each profile. More formally, the distance betweenthe profiles ci and cj is defined as:
d (ci, cj) =1
|ci∩
cj |∑
m(u)i ,m
(u)j ∈ci
∩cj
δu
(m
(u)i ,m
(u)j
),
(3.2)where δu : Mu × Mu 7→ [0, 1] is the distancefunction defined between models of type u ∈ U ={tok, int, len, char, struct}.
The token model m(tok) is represented as a setof unique tokens observed during the training phase.Therefore, two token models m(tok)
i and m(tok)j are con-
sidered similar if they contain similar sets of tokens. Ac-cordingly, the distance function for token models is de-fined as the Jaccard distance [7]:
δtok
(m
(tok)i ,m
(tok)j
)= 1−
∣∣∣m(tok)i
∩m
(tok)j
∣∣∣∣∣∣m(tok)i
∪m
(tok)j
∣∣∣ . (3.3)
The integer model m(int) is parametrized by the sam-ple mean µ and variance σ2 of observed integers. Twointeger models m
(int)i and m
(int)j are similar if these pa-
rameters are also similar. Consequently, the distancefunction for integer models is defined as:
δint
(m
(int)i ,m
(int)j
)=
∥∥∥σ2i
µ2i− σ2
j
µ2j
∥∥∥σ2i
µ2i+
σ2j
µ2j
. (3.4)
cp1,1
cp2,1
cp3,1
cp4,1
cp5,1
cp6,1
cp7,1
cp8,1
cp9,1cp10,1
cp1,2
cp2,2
cp3,2
cp4,2
cp5,2
cp6,2
cp7,2
cp8,2
cp9,2cp10,2
cp1,4
cp2,4
cp3,4
cp4,4
cp5,4
cp6,4
cp7,4
cp8,4
cp9,4cp10,4
cp1,!
cp2,!
cp3,!
cp4,!
cp5,!
cp6,!
cp7,!
cp8,!
cp9,!cp10,!
cp1
cp2
cp3
cp4
cp5
cp6
cp7
cp8
cp9cp10
c
! = 0
! = 1
! = 2
! = 4
! = !min
(a) (b)
CI1
CI2
CI4
CI!min
Wel
l-tr
ain
ed
C
!stable ! !min
! = 1
! = 2
! = 4
! = 8
! = 16
! = 32
! = 64
Req
ues
tsin
Q(p
)
Figure 6: Global knowledge base indices (a) constructed by (b) sub-sampling the training set Q into small chunks of samples ofprogressively larger size (as detailed in Section 3.1). In (b), a dot indicates a sample for a certain parameter, and a fulltraining set for one parameter corresponds to one horizontal row of dots. For instance, at κ = 2, the profiles for thesame parameter are trained several times on 2-sized training subsets. This leads to the profiles indicated as small graydots in (a) CI
2 and that, in this example, can be grouped into three clusters according to their similarity. As explainedin Section 3.2.1, as κ increases, more accurate clusterings are achieved, and the undertrained knowledge base betterresembles the well-trained one shown on the top-right.
As the length model is internally identical to the inte-ger model, its distance function δlen is defined similarly.
Recall that the character distribution model m(char)
learns the frequencies of individual characters compris-ing strings observed during the training phase. Thesefrequencies are then ranked and coalesced into n bins tocreate an ICD. Two character distribution models m(char)
i
and m(char)j are considered similar if each model’s ICDs
are similar. Therefore, the distance function for charac-ter distribution models is defined as
δchar
(m
(char)i ,m
(char)j
)=
n∑l=0
∥bi (l)− bj (l)∥maxk=i,j bk (l)
, (3.5)
where bi (k) is the value of bin k for m(char)i .
The structural model m(struct) builds an HMM by ob-serving a sequence of character strings. The resultingHMM encodes a probabilistic grammar that can pro-duce a superset of the strings observed during the train-ing phase. The HMM is specified by the tuple ⟨S,O, MS×S,P (S,O), P (S)⟩. Several distance metricshave been proposed to evaluate the similarity betweenHMMs [16,23,36,37]. Their time complexity, however,is non-negligible. Therefore, we adopt a less precise,but considerably more efficient, distance metric betweentwo structural models m
(struct)i and m
(struct)j as the Jac-
card distance between their respective emission sets
δstruct
(m
(struct)i ,m
(struct)j
)= 1− |Oi
∩Oj |
|Oi
∪Oj |
. (3.6)
3.2.2 Constructing a global knowledge base
Once a knowledge base of undertrained models CI hasbeen built, the next step is to construct a global knowl-edge base C. This knowledge base is composed of theindividual, well-trained knowledge bases from each webapplication as recorded during the first phase – that is,C =
∪i Cai . Because undertrained profiles are built for
each well-trained profile in C, a well-defined mappingf ′ : CI 7→ C (i.e., independent from the particular un-dertrained model in Cai , as defined by f ) exists betweenCI and C. Therefore, when a web application parameteris identified as likely to be undertrained, the correspond-ing undertrained profile c′ can be compared to a similarundertrained profile in CI , that is then used to select acorresponding stable profile from C.2
3.3 Phase III: Mapping undertrained profilesto welltrained profiles
With the construction of a global knowledge base Cand an undertrained knowledge base CI , we can performonline querying of C. That is, given an undertrained pro-file from an anomaly detector deployed over a web ap-plication ai, the mapping f :
{CI
}× Cai
7→ C is im-plemented as follows. A nearest-neighbor match is per-formed between c′ ∈ Cai
and the previously constructedclusters HI from CI to discover the most similar clus-ter of undertrained profiles. This is done to avoid a fullscan of the entire knowledge base, which would be pro-hibitively expensive due to the cardinality of CI .
Then, using the same distance metric defined inEquation (3.2), a nearest-neighbor match is performedbetween c′ and the members of HI to discover the un-dertrained profile cI at minimum distance from c′. Ifmultiple nearest-neighbors are found, then one is cho-sen at random. Finally, the global, well-trained profilef ′ (cI) = c is substituted for c′ for the web applicationai.
To make explicit how global profiles can be usedto address a scarcity of training data, consider the ex-ample of Figure 4. Since the resource path /ac-
count/password has received only 100 requests, theprofiles for each of its parameters {id, oldpw, newpw}are undertrained. According to our confidence met-ric defined in Section 3.1, models that have receivedless than 1000 samples are likely undertrained (i.e.,with a confidence close to zero). In the absence ofa global knowledge base, the anomaly detector wouldprovide no protection against attacks manifesting them-selves in the values passed to any of these parameters.Also, the system may report too many false positivesdue to a lack of model generalization. If, however, a
2Note that f ′ is a generalization of f .
global knowledge base and index are available, the sit-uation is considerably improved. Given C and CI , theanomaly detector can simply apply f to each of the un-dertrained parameters to find a well-trained profile fromthe global knowledge base that accurately models a pa-rameter with similar semantics, even when the model isfor another web application. Then, these profiles canbe substituted for the undertrained profiles for each of{id, oldpw, newpw}. As will be demonstrated in the fol-lowing section, the substitution of global profiles pro-vides an acceptable detection accuracy for what wouldotherwise be a completely unprotected component (i.e.,without a profile, none of the attacks against that com-ponent would be detected). If no matching well-trainedprofiles can be found, then the undertrained profile isused until the knowledge base is updated at a later time.This ensures that our approach can be deployed trans-parently and is adopted only when applicable, incurringno additional risk or overhead to the existing detectionprocedure.
3.4 Mapping quality
The selection of an appropriate value for κ is centralto both the efficiency and the accuracy of querying C.Clearly, it is desirable to minimize κ in order to be ableto index into C as quickly as possible once a parame-ter has been identified to be subject to undertraining atrun-time. On the other hand, setting κ too low is prob-lematic. In fact, as Figure 6a indicates, for low values ofκ, profiles are distributed with relatively high uniformitywithin CI , such that clusters in CI are significantly dif-ferent than clusters of well-trained profiles in C. There-fore, slight differences in the state of the individual mod-els can cause profiles that are close in CI to map to rad-ically different profiles in C. As κ → κstable, however,profiles tend to form semantically-meaningful clusters,and tend to approximate those found in C. Therefore, asκ increases, profiles that are close in CI become closein C according to f – in other words, f becomes robustwith respect to model semantics.3
A principled criterion is required for balancing quickindexing against a robust profile mapping. To this end,we first construct a candidate knowledge base CI
κ for agiven κ. Additionally, we cluster the profiles in C as inthe case of the undertrained knowledge base. Then, wedefine a robustness metric as follows. Recall that HI =∪
i hIi is the set of clusters in CI , and let H =
∪i hi be
the set of clusters in C. Let g : HI 7→ Z+ be a mappingfrom an undertrained cluster to the maximum number of
3Our use of the term “robustness” is related, but not necessarilyequivalent, to the definition of robustness in statistics (i.e., the propertyof a model to perform well even in the presence of small changes inthe underlying assumptions.)
elements in that cluster that map to the same cluster inC. The robustness metric ρ is then defined as
ρ(CI
)=
1
|CI |∑i
g(hIi
). (3.7)
With this metric, an appropriate value for κ can nowbe chosen as
κmin = minκ
(ρ(CIκ
)≥ ρmin
), (3.8)
where ρmin is a minimum robustness threshold.
4 Evaluation
The goal of this evaluation is three-fold. First, we in-vestigate the effects of profile clustering, and support thenotion of parameter types by examining global knowl-edge base clusters. Then, we study how the qualityof the mapping between undertrained profiles and well-trained profiles improves as the training slice length κis increased. Finally, we present results regarding theaccuracy of a web application anomaly detection systemincorporating the application of a global knowledge baseto address training data scarcity.
The experiments that follow were conducted usinga data set drawn from real-world web applications de-ployed on both academic and industry web servers. Ex-amples of representative applications include payrollprocessors, client management, and online commercesites. For each application, the full content of eachHTTP connection observed over a period of approxi-mately three months was recorded. A portion of theresulting flows were then filtered using Snort to removeknown attacks. In total, the data set contains 823 distinctweb applications, 36,392 unique components, 16,671unique parameters, and 58,734,624 HTTP requests.4
4.1 Profile clustering quality
To evaluate the accuracy of the clustering phase, wefirst built a global knowledge base C from a collectionof well-trained profiles. The profiles were trained on asubset of the aforementioned data, containing traffic in-volving a wide variety of web applications. This subsetwas composed of 603 web applications, 27,990 uniqueresource paths, 9,023 unique parameters, and 3,444,092HTTP requests. The clustering algorithm described inSection 3.2.2 was then applied to group profiles. Sampleresults from this clustering are shown in Figure 7b. Eachleaf represents a profile. The name of the parameter and
4Unfortunately, due to contractual agreements, we are unable todisclose specific information identifying the web applications them-selves.
samples values observed during training are included togive an idea of the parameter’s “type.” As κ increases,profiles are clustered more accurately.
As the figure indicates, the resulting clusters in Care accurately grouped by parameter type. For instance,date parameters from different web applications belonga single hierarchy, while unstructured text strings aregrouped into a separate hierarchy.
4.2 Profile mapping robustness
Recall that in order to balance the robustness of themapping f between undertrained profiles and globalprofiles against the speed with which undertraining canbe addressed, it is necessary to select an appropri-ate value for κ. To this end, we generated under-trained knowledge bases for increasing values of κ =1, 2, 4, 8, 16, 32, 64 from the same data set used to gener-ate C, following the procedure outlined in Section 3.2.1.The resulting hierarchical clusters for various κ are pre-sented in Figure 7c, 7d, 7a.
At low values of κ (e.g., Figure 7c), the clusteringprocess exhibits non-negligible systemic errors. For in-stance, the parameter stat should be clustered as a to-ken set of states, but instead is grouped with unstruc-tured strings such as cities and addresses. A more ac-curate clustering would have dissociated the token andstring profiles into well-separated sub-hierarchies.
As shown in Figure 7d, larger values of κ lead tomore semantically meaningful groupings. Some inac-curacies are still noticeable, but the clustering is signif-icantly better that the one obtained at κ = 8. A furtherimprovement in the clusters is shown in Figure 7a. Atκ = 64, the separation between dates and unstructuredstrings is sharper; except for one outlier, the two typesare recognized as similar and grouped together in theearly stages of the clustering process.
Figure 8 plots the profile mapping robustness ρagainst κ for different cuts of the clustering hierarchy,indicated by Dmax. Dmax is a threshold representing themaximum distance between two clusters. For low Dmax,the “cut” will generate many clusters with a few ele-ments. On the other hand, for high values of Dmax, thealgorithm will tend to form less clusters, each having alarger number of elements. In general, Dmax can in-fluence the result of a clustering algorithm significantly.Figure 8, however, shows two important properties ofour technique. First, it demonstrates that the robust-ness is fairly insensitive to Dmax. Second, the robust-ness of the mapping increases with κ until saturation at32 ≤ κ ≤ 64. This not only confirms the soundnessof the mapping function, but it also provides insights onthe appropriate choice of κmin to minimize the delay toglobal profile lookup while maximizing the robustness
notes: {1/29/07 - email, 10/26/06 thru, spoke}notes: {1/29/07 - email, 10/26/06 thru, spoke} notes: {1/29/07 - email, 10/26/06 thru, spoke} notes: {1/29/07 - email, 10/26/06 thru, spoke}type: {health care, wholesale}notes: {1/29/07 - email, 10/26/06 thru, spoke}name: {Foo LLC, Bar Inc.}type: {General Auto, Painters, unknown}
(a) κ = 64
stdate: {01/01/1900, 04/01/2007, 05/01/2007}stdate: {01/01/1900, 04/01/2007, 05/01/2007}ref: {01/29/2007, 01/30/2007, 01/31/2007}stdate: {02/19/2004, 09/15/2005, 12/07/2005}stdate: {01/01/1900, 05/08/2006}stdate: {01/31/2006, 11/01/2006}stdate: {01/30/2007, 02/10/2007}indate: {01/29/2007, 12/29/2006}exp: {01/01/2008, 05/22/2007}exp: {02/09/2007, 09/30/2006}exp: {02/01/2007, 08/01/2006, 09/01/2006}date: {1/29/07, 12/31/2006}date: {1/29/07, 12/31/2006}note: {10-5, no ans, called and emailed, no client resp}note: {10-5, no ans, called and emailed, no client resp}
(b) κstable ≃ 103
city: {OUR CITY, OTHER CITY, San Diego}stat: {GA}code: {OD}w: {Ineligible, Old}cd: {XX}type: {Payment, Sales}code: {OD}w: {Eligible, New}w: {Ineligible, New}stat: {CA, TX}stat: {CA, TX}addr: {15 ROOF AVE, 373 W SMITH, 49 N Ave}
(c) κ = 8
thepage: {TKGGeneral, TKGGeneral, KZDA.pdf} updateTask: {TKGGeneral, KZDA.pdf, Chan.cfm?taskna} code: {CK-1006, NES}thepage: {TKGGeneral, TKGGeneral, KZDA.pdf}code: {CK-1006, NES} thepage: {TKGGeneral, TKGGeneral, KZDA.pdf}accode: {r94, xzy}code: {CK-1006, NZS}code: {CK-1006, NZS}code: {02-286, BE2}thepage: {TKGGeneral, TKGGeneral, KZDA.pdf}
(d) κ = 32
Figure 7: Graphical representation of the various steps of the hierarchical clustering of C, (a-b), and CI , (c-d), at various κ. Eachleaf represents a profile. The name of the parameter and samples values observed during training are included to give anidea of the parameter’s “type.” We note that these sample values are shown for visualization purposes only, and are nevertaken into account by the clustering algorithm. As κ increases, profiles are clustered more accurately.
of the mapping.
4.3 Detection accuracy
Having studied the effects of profile clustering andvarying κ upon the robustness of the profile mappingf , a separate experiment was conducted to evaluate thedetection accuracy of an anomaly detector incorporat-ing the knowledge bases C and CI constructed in theprevious experiments. In particular, the goal of thisexperiment is to demonstrate that an anomaly detectorequipped with a global knowledge base exhibits an im-proved detection accuracy in the presence of trainingdata scarcity. As detailed in the following, we measuredthe baseline accuracy, on the same system and under thesame conditions, with all the models undertrained.
The data used in this experiment was a subset of thefull data set described above, containing traffic fromone related set of web applications implementing on-line commerce sites. This data set was completely dis-joint from the one used to construct the global knowl-edge base and its indices, to prevent any potential for thesubstitution of profiles from the same application. Ad-ditionally, the use of a global knowledge base generatedfrom many types of web applications to address a lackof training data for a specific web application mirrors
the intended usage of the technique. In total, this dataset consisted of 220 unique real-world web applications,8,402 unique resource paths, 7,648 distinct parameters,and 55,290,532 HTTP requests.
The threat model we assume is that of an attacker at-tempting to compromise the confidentiality or integrityof data exposed by a web application by tampering withrequest parameters.5 Therefore, a set of 100,000 at-tacks was introduced into the data set. These attackswere real-world examples and variations upon cross-sitescripting (XSS), SQL injection, command execution ex-ploits, and other attacks that manifest themselves in re-quest parameter values.6 Examples of these attacks in-clude:
• malicious code inclusion: <script
src="http://example.com/malware.js"></script>;• bypassing login authentication: ’ OR ’x’=’x’--;
5Although the anomaly detector used in this study is capable ofdetecting more complex session-level anomalies, we restrict the threatmodel to request parameter manipulation because we do not addresssession profile clustering in this work.
6These attacks remain the most common attacks against web ap-plications. However, both the anomaly detector and the improvementwe designed apply to any malicious activity caused by modificationsto HTTP requests, and therefore we by no means limit our scope tothese classes of attacks.
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0 10 20 30 40 50 60 70
Map
ping
rob
ustn
ess
κ
Dmax = 0.10Dmax = 0.125Dmax = 0.15
Dmax = 0.175Dmax = 0.20
Dmax = 0.225Dmax = 0.25Dmax = 0.30Dmax = 0.35
Dmax = 0.375Dmax = 0.40
Dmax = 0.425Dmax = 0.45
Dmax = 0.475Dmax = 0.50
Figure 8: Plot of profile mapping robustness for varying κ.
• command injection: ; cat /etc/passwd | mail
To establish a worst-case bound on the detection ac-curacy of the system, profiles for each observed requestparameter were deliberately undertrained to artificiallyinduce a scarcity of training data for all parameters.That is, for each value of κ = 1, 2, 4, 8, 16, 32, 64, theanomaly detector prematurely terminated profile train-ing after κ samples, and then used the undertrained pro-files to query C. The resulting global profiles were thensubstituted for the undertrained profiles and evaluatedagainst the rest of the data set. The sensitivity of thesystem was varied over the interval [0, 1], and the result-ing ROC curves for each κ are plotted in Figure 9.
It must be noted that, as shown in Section 4.2, withκ = 1 our system cannot reliably find semantically sim-ilar well-trained models. As κ increases, so does thequality of the global profiles returned by the queryingprocess. In particular, this increase in quality closelyfollows the mapping robustness plot presented in Fig-ure 8. As predicted, setting κ = 32, 64 leads to fairlyaccurate global profile selection, with the resulting ROCcurves approaching that of fully-trained profiles. Thismeans that even if the component or, in general, a pa-rameter of a web application has received only a few re-quests (i.e., 64), by leveraging a global knowledge base,it is possible to achieve effective attack detection. As aconsequence, our approach can improve the effective-ness of real-world web application anomaly detection
systems. Clearly, the detection accuracy will improveas more training samples (e.g., 128, 256) become avail-able. However, the goal of this experiment was to eval-uate such an improvement with a very limited trainingset, rather than showing the detection maximum accu-racy achievable.
One concern regarding the substitution of global pro-files for local request parameters is that a global profilethat was trained on another web application might notdetect valid attacks against the undertrained parameter.Without this technique, however, recall that a learning-based web application anomaly detector would other-wise have no effective model whatsoever, and, there-fore, the undertrained parameter would be unprotectedby the detection system. Furthermore, the ROC curvesdemonstrate that while global profiles are in general notas precise as locally-trained models, they do provide asignificant level of detection accuracy.7 More precisely,with κ = 1, undertraining condition and system off, only67.5% of the attacks are detected, overall, with around5% of false positives. On the other hand, with κ = 64(undertraining and system on), more than 91% of the at-tacks are detected with less than 0.2% of false positives(vs., 0.1% of false positives in the case of no under-training and system off). Therefore, we conclude that,assuming no mistrust among the parties that share the
7Note that if global profiles were found to be as accurate as localprofiles, this would constitute an argument against site-specific learn-ing of models, since in that case, models could be trained for one webapplication and applied directly to other web applications.
0.5
0.6
0.7
0.8
0.9
1
0 0.01 0.02 0.03 0.04 0.05
True
pos
itive
rate
False positive rate
k=1k=2k=4k=8
k=16k=32k=64
k-stable
Figure 9: Global profile ROC curves for varying κ. In the presence of severe undertraining (κ ≪ κstable), the system is not ableto recognize most attacks and also reports several false positives. However, as κ increases, detection accuracy improves,and approaches that of the well-trained case (κ = κstable).
global knowledge base, our approach is a useful tech-nique to apply in the presence of undertrained modelsand, in general, in the case of training data scarcity.
5 Related work
Anomaly detection has a rich history that dates backto Denning’s seminal paper on intrusion detection [9].It has been extensively applied to modeling legal se-quences of system calls [12, 42] as well as other systemcall features [28]. Network-based anomaly detection hasalso been extensively studied [21, 41].
In [24], the authors further exploit the use of statis-tical learning procedures to build models of normal se-quences of system calls in the Linux kernel. Here, ad-hoc distances are proposed to perform clustering of sys-tem calls in order to identify types of similar calls. Thus,the system calls are “compacted” and the reduced sizeof the input makes it feasible to train Markov chains thatattempt to capture the behavior of each host application.Probabilistic thresholds are then used to detect deviatingbehaviors.
Anomaly detection techniques have also been appliedat the application level. In [44], Wang and Stolfo de-scribe PAYL, a service-agnostic anomaly detection sys-tem that models normal behavior by recording byte fre-quencies of network streams. An extension of the tech-
nique to higher-order n-grams is introduced in [43].This system is also notable for its inclusion of a stop cri-terion for ending model training by estimating the likeli-hood of observing new n-grams; this approach is relatedto the notion of model stability described in Section 3.1.
One of the first applications of anomaly detectiontechniques to the web domain is described in [19], inwhich a multi-model approach to characterizing the nor-mal behavior of web application parameters is shownto reliably detect web attacks while maintaining a lowfalse positive rate. In [18], the use of Bayesian networksis proposed to compose models and express inter-modeldependencies in order to further reduce false positiverates. A system for clustering anomalies and classify-ing clusters according to the type of attack is proposedin [30] to improve the explanatory power of web appli-cation anomaly detection as well as further reduce theirfalse positive rate.
A recent effort on addressing training set deficien-cies has been proposed in [8]. In this work, a sanitiza-tion phase is first performed to remove suspected attacksand other abnormalities from the data. Instead of creat-ing one model instance, a set of “micro-models” is trai-ned against disjoint subsets of the training data. Thesemicro-models are then subject to one of several votingschemes to recognize and cull outliers that may repre-sent attacks. While this work is somewhat related to
ours in that slices of training data are considered, theapplication of micro-models is intended to ensure thatattacks are not present in training data. In contrast, ourapproach is complementary in the sense that it is focusedon addressing the lack of training data.
In [13], the authors propose a cluster-based anomalydetection system as a means of reducing false positives.The system accomplishes this by clustering similar be-havioral profiles for individual hosts using the k-meansalgorithm, although the exact distance metric used wasnot explicitly given. Then, alerts are generated accord-ing to a voting scheme, where the causal event for analert is evaluated against behavior profiles from othermembers of that host’s cluster. If the event is deemedanomalous by all members of the cluster, an alert is gen-erated. Though this system shares the element of profileclustering with our work, the scope of the clustering islimited to end hosts connected to a single switch, whileour work clusters across a large population of web ap-plication profiles. Also, the models operate over coarsenetwork statistics such as the average number of hostscontacted per hour, the average number of packets ex-changed per hour, and the average length of packets ex-changed per hour. Our system, on the other hand, con-siders fine-grained features specific to web applications.Finally, our system does not require the use of a dis-tributed voting scheme, which incurs additional over-head.
A recent proposal for the anomaly-based detection ofweb-based attacks is presented in [35]. In this work,a mixture of Markov chains incorporating n-gram tran-sitions is used to model the normal behavior of HTTPrequest parameters. The resulting system attains a highdetection accuracy for a variety of web-based attacks.In contrast to our work, however, a single mixture islearned for an entire web application. Additionally,the proposed system utilizes a supervised learning algo-rithm (i.e., attacks must be labeled as such in the trainingset), whereas ours operates on unlabeled training data.
6 Conclusions
In this work, we have identified that non-uniform webclient access distributions cause model undertraining.This is an issue that must be addressed by web appli-cation anomaly detection systems in order to provide areasonable level of security. The impact of this issue isparticularly relevant for commercial web-based anomalydetection systems and web application firewall, whichhave to operate in real-world environments where suffi-cient training data might be unavailable within a reason-able time frame.
We propose the use of global knowledge bases ofwell-trained, stable profiles to remediate a local scarcity
of training data by exploiting global similarities in webapplication parameters. We have evaluated the efficacyof this approach over an extensive data set collectedfrom real-world web applications. We found that al-though using global profiles does result in a small reduc-tion in detection accuracy, the resulting system, whengiven appropriate parameters, does provide reasonablyprecise modeling of otherwise unprotected web applica-tion parameters.
As future work, we plan to investigate the extensionof global model clustering to other types of models, par-ticularly HTTP response and session models. An addi-tional line of future work is to investigate the applicationof different clustering methods in order to improve theefficiency of the querying procedure for high-cardinalityknowledge bases.
Acknowledgments
The authors wish to thank the anonymous reviewersand our shepherd, Angelos Stavrou, for their insightfulcomments. This work has been supported by the Na-tional Science Foundation, under grants CCR-0238492,CCR-0524853, and CCR-0716095, by the EuropeanUnion though the grant FP7-ICT-216026-WOMBAT,and by Secure Business Austria (SBA).
References
[1] J. Alpert and N. Hajaj. We knew the web wasbig... http://googleblog.blogspot.com/2008/
07/we-knew-web-was-big.html, July 2008.[2] S. Axelsson. The Base-Rate Fallacy and the Difficulty of
Intrusion Detection. ACM Transactions on Informationand System Security, 3(3):186–205, August 2000.
[3] S. Bhatkar, A. Chaturvedi, and R. Sekar. DataflowAnomaly Detection. In Proceedings of the IEEE Sym-posium on Security and Privacy (S&P 2006), Oakland,CA, USA, May 2006. IEEE Computer Society.
[4] Breach Security, Inc. Breach WebDefend. http://
www.breach.com/products/webdefend.html, Jan-uary 2009.
[5] S. Cho and S. Cha. SAD: Web Session Anomaly De-tection Based on Parameter Estimation. Computers &Security, 23(4):312–319, 2004.
[6] Citrix Systems, Inc. Citrix Application Fire-wall. http://www.citrix.com/English/PS2/
products/product.asp?contentID=25636, Jan-uary 2009.
[7] W. Cohen, P. Ravikumar, and S. Fienberg. A compari-son of string distance metrics for name-matching tasks.In Proceedings of the IJCAI-2003 Workshop on Informa-tion Integration on the Web (IIWeb-03), 2003.
[8] G. F. Cretu, A. Stavrou, M. E. Locasto, S. J. Stolfo,and A. D. Keromytis. Casting out Demons: Sanitiz-ing Training Data for Anomaly Sensors. In Proceedings
of the IEEE Symposium on Security and Privacy (S&P2008), pages 81–95, Oakland, CA, USA, May 2008.IEEE Computer Society.
[9] D. E. Denning. An Intrusion-Detection Model. IEEETransactions on Software Engineering, 13(2):222–232,1987.
[10] H. J. Escalante and O. Fuentes. Kernel Methods forAnomaly Detection and Noise Elimination. In Pro-ceedings of the International Conference on Computing(CORE 2006), pages 69–80, Mexico City, Mexico, 2006.
[11] F5 Networks, Inc. BIG-IP Application Se-curity Manager. http://www.f5.com/
products/big-ip/product-modules/
application-security-manager.html, January2009.
[12] S. Forrest, S. A. Hofmeyr, A. Somayaji, and T. A.Longstaff. A Sense of Self for Unix Processes. In Pro-ceedings of the IEEE Symposium on Security and Pri-vacy (S&P 1996), pages 120–128, Oakland, CA, USA,May 1996. IEEE Computer Society.
[13] V. Frias-Martinez, S. J. Stolfo, and A. D. Keromytis.Behavior-Profile Clustering for False Alert Reduction inAnomaly Detection Sensors. In Proceedings of the An-nual Computer Security Applications Conference (AC-SAC 2008), Anaheim, CA, USA, December 2008.
[14] J. T. Giffin, D. Dagon, S. Jha, W. Lee, and B. P. Miller.Environment-Sensitive Intrusion Detection. In Proceed-ings of the International Symposium on Recent Ad-vances in Intrusion Detection (RAID), pages 185–206.Springer-Verlag, 2005.
[15] L. Guangmin. Modeling Unknown Web Attacks in Net-work Anomaly Detection. In Proceedings of the 3rd
International Conference on Convergence and HybridInformation Technology (ICCIT 2008), pages 112–116,Washington, DC, USA, 2008. IEEE Computer Society.
[16] B. Juang and L. Rabiner. A probabilistic distance mea-sure for hidden Markov models. AT&T Bell Laborato-ries Technical Journal, 64(2):391–408, 1985.
[17] S.-i. Kim and N. Nwanze. Noise-Resistant PayloadAnomaly Detection for Network Intrusion DetectionSystems. In Proceedings of the Performance, Com-puting and Communications Conference (IPCCC 2008),pages 517–523, Austin, TX, USA, December 2008.IEEE Computer Society.
[18] C. Kruegel, D. Mutz, W. Robertson, and F. Valeur.Bayesian Event Classification for Intrusion Detection. InProceedings of the Annual Computer Security Applica-tions Conference (ACSAC 2003), Las Vegas, NV, USA,December 2003.
[19] C. Kruegel, W. Robertson, and G. Vigna. A Multi-modelApproach to the Detection of Web-based Attacks. Jour-nal of Computer Networks, 48(5):717–738, July 2005.
[20] C. Kruegel, T. Toth, and E. Kirda. Service-SpecificAnomaly Detection for Network Intrusion Detection. InProceedings of the Symposium on Applied Computing(SAC 2002), Spain, March 2002.
[21] W. Lee and S. J. Stolfo. A Framework for ConstructingFeatures and Models for Intrusion Detection Systems.ACM Transactions on Information and System Security,3(4):227–261, 2000.
[22] R. Little and D. Rubin. Statistical analysis with missingdata. Wiley New York, 1987.
[23] R. B. Lyngsø, C. N. S. Pedersen, and H. Nielsen. Metricsand similarity measures for hidden markov models. InProceedings of the Seventh International Conference onIntelligent Systems for Molecular Biology, pages 178–186. AAAI Press, 1999.
[24] F. Maggi, M. Matteucci, and S. Zanero. Detecting intru-sions through system call sequence and argument anal-ysis (preprint). IEEE Transactions on Dependable andSecure Computing, 99(1), 2009.
[25] M. Mahoney and P. Chan. Detecting novel attacks byidentifying anomalous network packet headers. Techni-cal Report CS-2001-2, Florida Institute of Technology,2001.
[26] M. V. Mahoney and P. K. Chan. Learning Rules forAnomaly Detection of Hostile Network Traffic. In Pro-ceedings of the IEEE International Conference on DataMining, 2003.
[27] Miniwatts Marketing Group. World Internet UsageStatistics. http://www.internetworldstats.com/
stats.htm, January 2009.[28] D. Mutz, F. Valeur, C. Kruegel, and G. Vigna. Anoma-
lous System Call Detection. ACM Transactions on Infor-mation and System Security, 9(1):61–93, February 2006.
[29] Open Security Foundation. DLDOS: Data LossDatabase – Open Source. http://datalossdb.org/,January 2009.
[30] W. Robertson, G. Vigna, C. Kruegel, and R. A. Kem-merer. Using Generalization and Characterization Tech-niques in the Anomaly-based Detection of Web Attacks.In Proceedings of the Network and Distributed Sys-tem Security Symposium (NDSS 2006), San Diego, CA,USA, February 2006.
[31] R. Sekar, M. Bendre, D. Dhurjati, and P. Bollineni. AFast Automaton-Based Method for Detecting Anoma-lous Program Behaviors. In Proceedings of the IEEESymposium on Security and Privacy (S&P 2001), pages144–155, Oakland, CA, USA, May 2001. IEEE Com-puter Society.
[32] R. Sekar, A. Gupta, J. Frullo, T. Shanbhag, A. Tiwari,H. Yang, and S. Zhou. Specification-based Anomaly De-tection: A New Approach for Detecting Network Intru-sions. In Proceedings of the ACM Conference on Com-puter and Communications Security (CCS 2002), pages265–274, New York, NY, USA, 2002. ACM Press.
[33] O. Shezaf, J. Grossman, and R. Auger. Web Hack-ing Incidents Database. http://www.xiom.com/
whid-about, January 2009.[34] A. Singer. Social Media, Web 2.0 And Internet
Stats. http://thefuturebuzz.com/2009/01/12/
social-media-web-20-internet-numbers-stats/,January 2009.
[35] Y. Song, A. D. Keromytis, and S. J. Stolfo. Spectrogram:A Mixture-of-Markov-Chains Model for Anomaly De-tection in Web Traffic. In Proceedings of the Networkand Distributed System Security Symposium (NDSS2009), San Diego, CA, USA, February 2009.
[36] A. Stolcke and S. Omohundro. Hidden Markov ModelInduction by Bayesian Model Merging. Advances inNeural Information Processing Systems, pages 11–11,1993.
[37] A. Stolcke and S. Omohundro. Inducing ProbabilisticGrammars by Bayesian Model Merging. Lecture Notesin Computer Science, pages 106–106, 1994.
[38] G. Tandon and P. Chan. Learning Rules from SystemCall Arguments and Sequences for Anomaly Detection.In ICDM Workshop on Data Mining for Computer Secu-rity (DMSEC), pages 20–29, 2003.
[39] G. Tandon, P. Chan, and D. Mitra. MORPHEUS: Mo-tif Oriented Representations to Purge Hostile Eventsfrom Unlabeled Sequences. In Proceedings of the 2004ACM Workshop on Visualization and Data Mining forComputer Security, pages 16–25, Washington DC, USA,2004. ACM.
[40] D. Turner, M. Fossi, E. Johnson, T. Mark, J. Blackbird,S. Entwise, M. K. Low, D. McKinney, and C. Wueest.Symantec Global Internet Security Threat Report –Trends for 2008. Technical Report XIV, Symantec Cor-poration, April 2009.
[41] A. Valdes and K. Skinner. Adaptive, Model-based Mon-itoring for Cyber Attack Detection. In H. Debar, L. Me,and F. Wu, editors, Proceedings of the InternationalSymposium on Recent Advances in Intrusion Detection(RAID 2000), number 1907 in Lecture Notes in Com-puter Science, pages 80–92, Toulouse, France, October2000. Springer-Verlag.
[42] D. Wagner and D. Dean. Intrusion Detection via StaticAnalysis. In Proceedings of the IEEE Symposium onSecurity and Privacy (S&P 2001), pages 156–168, Oak-land, CA, USA, 2001. IEEE Computer Society.
[43] K. Wang, J. J. Parekh, and S. J. Stolfo. Anagram: A Con-tent Anomaly Detector Resistant to Mimicry Attack. InProceedings of the International Symposium on RecentAdvances in Intrusion Detection (RAID 2006), Ham-burg, GR, September 2006. Springer-Verlag.
[44] K. Wang and S. J. Stolfo. Anomalous Payload-basedNetwork Intrusion Detection. In Proceedings of the In-ternational Symposium on Recent Advances in Intru-sion Detection (RAID 2004). Springer-Verlag, Septem-ber 2004.
[45] R. Xu and D. Wunsch. Survey of clustering algorithms.IEEE Transactions on Neural Networks, 16(3):645–678,2005.
[46] R. H. Zakon. Hobbes’ Internet Timeline v8.2. http:
//www.zakon.org/robert/internet/timeline/,November 2006.
[47] S. Zanero and S. M. Savaresi. Unsupervised learningtechniques for an intrusion detection system. In Pro-ceedings of the 2004 ACM Symposium on Applied Com-puting, pages 412–419. ACM Press, 2004.
