Network Defense: Approaches, Methods and Techniques

Rup Kumar Dekaa, Kausthav Pratim Kalitaa, D. K. Bhattacharyaa, Jugal K. Kalitab

aDepartment of Computer Science and Engineering, Tezpur University, Napaam, Assam, IndiabDepartment of Computer Science, College of Engineering and Applied Science, University of Colorado, Boulder, CO, United States

Abstract

To defend a network from intrusion is a generic problem of all time. It is important to develop a defense mechanism to secure thenetwork from anomalous activities. This paper presents a comprehensive survey of methods and systems introduced by researchersin the past two decades to protect network resources from intrusion. A detailed pros and cons analysis of these methods andsystems is also reported in this paper. Further, this paper also provides a list of issues and research challenges in this evolving fieldof research. We believe that, this knowledge will help to create a defense system.

Keywords: DoS, Intrusion, Defense, Response, Tolerance.

1. Introduction

Computerization and internetization of the world is happen-ing at an astonishing speed. In spite of growth at breakneckpace, service providers are doing their best to provide the high-est quality of service. At every step, an aspect that stands outis security, which is indeed a very serious topic of concern. Anintrusion or attack may be fast or slow. When an attack useslarge size packets or extremely high volume tra�c within a veryshort time, say a fraction of a minute, to disrupt service, it canbe termed a fast attack. On the other hand, some attacks takeminutes or hours to complete the process, and are referred to asslow attacks.

Frequently, network or system activities are carried out withmalicious intentions or other network policy violations takeplace. This type of attempt or activity can be termed intrusionand its creator is known as an intruder. The goal of intrusiondetection is to make the whole network secure by thwarting at-tempts to compromise confidentiality, integrity or availabilityof resources.

1.1. MotivationThere are several published surveys on approaches to in-

trusion detection and/or prevention such as Patel et al. (2010),Bhuyan et al. (2014), Hoque et al. (2013), Kumar (2007), Rich-hariya and Srivastava (2013), Patel et al. (2013). These au-thors usually provide details of a few approaches although somecover a larger number of defense systems. Bhuyan et al. (2014)present a comprehensive survey of DDoS attacks, detectionmethods and tools used in wired networks. Hoque et al. (2013)provide a taxonomy of attack tools and also present a compre-hensive and structured survey of existing tools and systems that

Email addresses: rup.deka@gmail.com (Rup Kumar Deka),koztov.project@gmail.com (Kausthav Pratim Kalita),dkb@tezu.ernet.in (D. K. Bhattacharya), jkalita@uccs.edu (JugalK. Kalita)

can support both attackers and network defenders. An exhaus-tive survey of intrusion defense systems is presented by Patel etal. (2013), where the authors discuss approaches against intru-sion by creating a layered taxonomy in addition to discussingcloud-based intrusion defense systems. Neither of the surveysby Patel et al. (2010) and Richhariya and Srivastava (2013) in-clude issues of defense, challenges and solutions. In this pa-per we present a structured and comprehensive survey of de-fensive approaches, in terms of general overview, modules ofa defense architecture, infrastructure and a taxonomy. We alsoattempt to present challenges in developing e↵ective defensiveapproaches.

This paper provides a structured and comprehensive surveyof approaches to counter intrusions. The major contributions ofthis survey are the following.

• Our presentation is more streamlined. First, we describea defense system, in particular whether it detects or pre-vents intrusions considering the modules it contains. Thenwe focus on various detection techniques. Infrastructureneeds, location and control of defense systems are also dis-cussed.

• Most existing surveys do not fully cover the large numberof issues, related to intrusion defense systems, but we do.

• We present a taxonomy to ensure that we cover a large areawithin the intrusion defense process.

• We also identify challenges encountered by approaches toprevent intrusions.

1.2. Prior Surveys

Richhariya and Srivastava (2013) address issues of infor-mation security and describe the security needs of an organi-zation to protect its critical information from attacks. A welltrained sta↵ of analysts is required to continuously monitor

Preprint submitted to Journal of Network and Computer Applications May 26, 2015

*ManuscriptClick here to view linked References

Table 1: Comparison with Existing Surveys

References IDS IPS Di↵erent DefenseApproaches challenges

Bai and Kobayashi (2003) Yes No Yes NoMurali (2005) Yes No Yes NoKabiri and Ghorbani (2005) Yes No Yes NoKumar (2007) Yes No Yes NoPatel et al. (2010) Yes Yes No NoSandhu et al. (2011) Yes Yes No NoRathore (2012) Yes Yes Yes NoPatel et al. (2013) Yes Yes Yes YesRichhariya and Srivastava (2013) Yes No Yes NoBhuyan et al. (2014) Yes No Yes YesThis Paper Yes Yes Yes Yes

the system. In such an environment, a huge amount of ef-fort is required to construct new security strategies. Patel etal. (2010) review current trends in intrusion detection togetherwith a study of implemented technologies. Kabiri and Ghor-bani (2005) identify main categories of intrusion detection andprevention systems. They also provide a comparison of variousapproaches. Rathore (2012) also provides a survey of di↵erentapproaches to intrusion detection. Sandhu et al. (2011) reviewsmethods for building Intrusion Detection and Prevention Sys-tems (IDPS) and uses a cost-e↵ective intrusion detection andprevention method based on the concept of intelligent mobileagents to design an e↵ective Agent based Intrusion PreventionSystem (AIPS). AIPS works well in a distributed environmentdue to the use of software agents.

Murali (2005) surveys recent IDPSs and alarm managementtechniques by providing a comprehensive taxonomy and inves-tigating possible solutions to detect and prevent intrusions incloud computing systems. Considering the desired characteris-tics of IDPSs and cloud computing systems, a list of require-ments is identified and four concepts of autonomic computing,viz., self-management, ontology, risk management, and fuzzytheory are leveraged to satisfy these requirements.

A survey of technologies for defense against intrusion isgiven in Patel et al. (2013). This paper discusses aspects ofintrusion defense systems and data collection techniques. Datamining-based and data fusion-based IDSs are discussed to em-phasize the need for large-scale data collection. Current defensetechnologies face powerful challenges and these are also de-scribed here, along with some suggested methods to overcomethem.

Bai and Kobayashi (2003) describe detailed designs of bothsignature and anomaly-based NIDS (Network based IntrusionDetection System). Requirements of such systems are thor-oughly discussed. Kumar (2007) presents a nomenclature ofIDSs that he uses for his survey. This paper also identifiesstrengths as well as the limitations of several IDSs.

Our survey di↵ers from these previous surveys in the follow-ing ways.

• In all the papers mentioned in this section, there is little

information regarding where to deploy IDSs and other de-tails of issues in deployment of IDSs.

• Most papers, which are mentioned in this section, do notprovide any discussion of challenges faced when an intru-sion defense system is deployed.

• We describe modules of an intrusion defense model in thispaper. A thorough understanding of these modules is nec-essary to develop successful defense systems. Such dis-cussions are not usually found in other survey papers.

1.3. OrganizationThe rest of the paper is organized as follows. Concepts re-

lated to approaches and intrusion defense systems are discussedin Section 2. This section also presents a classification of intru-sion detection, prevention, response and tolerance systems. Aselection of such systems is presented in Section 3. Section 4 isdedicated to issues and challenges in building a defense system.Finally, we present conclusions in Section 5.

2. Intrusion Defense Solutions

We can visualize three di↵erent types of attack which arementioned below.

In Scanning attack, an attacker tries to gather informationsuch as network topology, types of network tra�c allowed bythe firewalls, versions of operating system and kernel hosted ona network and identities and versions of server software run-ning. The whole process may be initiated and accomplished bysending a stealth SYN packet. It is stealthy because the attackerjust tries to create a half open TCP connection. When the serverreceives a SYN packet, server responds with a SYN/ACK andgoes to a listening state, allowing the attacker to gather the re-quired information. If no SYN/ACK is received by the attacker,it is assumed that the server is in a closed state.

In a penetration attack, an attacker tries to access a systemand its resources without authorization. The attacker seeks toacquire the privileges of root to execute code easily and exploitsystem resources. After compromising the system, the attacker

2

can use the machine as a launchpad for di↵erent types of at-tacks.

A Denial of Service (DoS) attack tries to exhaust the re-sources of a network or a system. An attack can be carried outby a few malformed packets that exploit vulnerabilities in thehost or by a vast number of legitimate packets that exhaust thevictim’s network bandwidth or resources (Bhattacharya et al.,2013). As a precursor, a distributed DoS attacker may accessmany machines to lunch a coordinated distributed DoS attack.A DoS attack causes frequent congestion, hindering legitimatecommunication.

With the rapid emergence of external and internal threatsto networks and resources, we must think about security all thetime. As a result, researchers and practitioners have looked ata variety of approaches such as Intrusion Detection Systems(IDS), Intrusion Prevention Systems (IPS), Intrusion ResponseSystem (IRS) and Intrusion Tolerance System (ITS). IPS andIDS are important components of a layered security infrastruc-ture.

Four main steps (Bhattacharya et al., 2013) taken by an at-tacker prior to executing an intrusion into a network or systemare as follows.

(a) Prepare: In this first step, the attacker attempts to col-lect network configuration information using port scannersto identify vulnerabilities in the network (Bhuyan et al.,2011). Port scanning gathers information such as com-puter IP addresses, operating systems, open ports withidentities and version of listening software.

(b) Exploit: Once vulnerabilities are identified, in the secondstep, the attacker attempts to exploit these vulnerabilities.The attacker may execute multiple attempts during thisstep.

(c) Leave Behind: If the lunching of an attack is successful,the attacker installs additional software to create contin-ued access to the network. This process, termed as leavebehind, includes installation of network sni↵ers or addi-tional back-door network services.

(d) Clean Up: At last, the attacker tries to clean up any evi-dence left due to the actions in the previous steps. This stepmay include restarting daemons crashed during the secondstep, clearing logs and other information, and installingmodified system software designed to override the pres-ence of other software from normal system commands.

2.1. Based on Approach Used

Based on the approach used to counter intrusions the fourmain intrusion defense systems such as: intrusion detection sys-tems, intrusion prevention systems, intrusion response systemsand intrusion tolerance systems are found to work as follows.

2.1.1. Intrusion Detection SystemAn intrusion detection system (IDS) (Ertoz et al., 2004)

monitors a network or system for malicious activities or policy

violations. Some systems or approaches may try to stop an in-trusion attempt, but this is neither required nor expected of amonitoring system. If an IDS detects any threat, it alerts thesystem or network administrator. The objective of an IDS is todetect and inform active defenders about intrusions. An IDSalso uses techniques that can detect abnormalities both at thenetwork and host levels. Figure 1 shows a generic view of anIDS. The components are: a managing system, a monitoringcomponent and a detection component.

• The managing system oversees tra�c flow in the network.It provides tra�c information to the monitoring compo-nent for analysis.

• The monitoring component monitors tra�c and analyzesthe behavior of the network.

• The detection component detects any suspicious behaviorwith respect to the normal working nature of the network.If any abnormal behavior is detected, it is communicatedto the reaction component.

• The reaction component reacts to the situation. After de-tection of abnormality, it raises an alarm so that the intru-sion can be handled appropriately.

Figure 1: Intrusion Detection System: A Generic View

Figure 2: Intrusion Prevention System: A Generic View

2.1.2. Intrusion Prevention SystemAn IPS is considered an upgraded version of an intrusion de-

tection system (Desai, 2009). They both monitor network traf-fic and/or system activities for malicious activity, but the maindi↵erence is that intrusion prevention systems are able to ac-tively prevent intrusions that are detected. An IPS executes

3

steps such as sending an alarm, dropping malicious packets, re-setting the connection and/or blocking tra�c from the o↵endingIP addresses. Figure 2 presents a generic view of an intrusionprevention system. The managing system, monitoring compo-nent and detection component are similar to those in an IDS, butin the reaction component prevention procedures are applied bythe prevention engine.

• In the reaction component, the prevention engine appliesprocedures according to the pattern of behavior of the sus-picious tra�c by working closely with the Managing sys-tem.

• The managing system manages the tra�c flow and appliesthe procedures provided by prevention engine.

• The monitoring system and the detection component worksimilarly to those in an IDS.

2.1.3. Intrusion Response SystemAn intrusion response system (IRS) (Stakhanova et al.,

1991) continuously monitors system health based on IDS alerts,so that malicious or unauthorized activities can be handled ef-fectively by applying appropriate actions to prevent problemsfrom worsening the situation and to return the system to ahealthy mode. A notification system generates alerts when anattack is detected. An alert can contain information like attackdescription, time of attack, source IP and user accounts usedto attack. An IRS automatically executes a preconfigured setof response actions based on the type of attack. An automatedapproach requires no human intervention, unlike an IDS wherethere is a delay between intrusion detection and response. Fig-ure 3 shows the generic structure of an intrusion response sys-tem. It is comprised of a reaction component, a detection com-ponent, a monitoring component and a managing system. Inparticular,

• The reaction component has a response system, and

• It responds to the intrusion using a predefined approach inan automated manner.

Figure 3: Intrusion Response System: A generic View

2.1.4. Intrusion Tolerance SystemAn Intrusion tolerance system (Deswarte et al., 1991) takes

a fault-tolerant design approach to defend information systems

against malicious attacks. In lieu of the general aim of pre-venting all intrusions, intrusion tolerance uses mechanisms thatprevent intrusions from leading to system security failure. As amatter of fact, intrusion tolerance is not a new concept. Clas-sical fault tolerance techniques are useful for tolerating intru-sion and error detection and recovery. Error hiding techniquescan also be applied to provide data integrity or service avail-ability despite intrusions. However, such fault-tolerance tech-niques are usually considered harmful for data confidentialitydue to the redundancy that they imply. Figure 4 provides ageneric view of an intrusion tolerance system. The managingsystem, the monitoring component and the detection compo-nent are similar to those in an IDS, but the reaction componentuses tolerance techniques.

• In the reaction component, intrusion tolerance techniquestry to prevent intrusions from causing system failure.

• Classical techniques may be useful and e�cient.

• A tolerance approach di↵ers from the conventional way ofpreventing the attacks.

Figure 4: Intrusion Tolerance System: A Generic View

2.2. Modules of A Defense SystemIn this section, we discuss the components of a generic de-

fense system.

2.2.1. MonitoringNetwork monitoring collects data on the state of the network

(Conorich, 2004). Tra�c analysis requires inspection of ser-vices being used on a network or system and comparing themagainst activities that are expected. This allows one to identifysuspicious services within a network. To perform basic net-work monitoring, one needs to collect tra�c characteristics atvarious points within the network. Although it is necessary tolook carefully at network borders, if there are internal hosts pro-viding unauthorized services for other internal hosts, one willmiss this tra�c if one only looks at the borders. There are fourdi↵erent types of TCP activity that should be considered.

• Are three-way handshakes being completely executed ornot?

• Are three-way handshakes being initiated but never suc-cessfully completed?

4

• Is a client getting any response to a connection attempt?The client often does two or three retries with slight delaysamong them in case of failure.

• Is a client getting any negative response to a connectionattempt, for example, a TCP RST packet or an ICMP hostunreachable or port unreachable packet?

2.2.2. DetectionA detection module provides reports (Mukherjee et al., 1994)

to a management section . Some detection modules may try tostop an attack but this is neither required nor expected. Theintrusion detection module is primarily focused on identifyingpossible incidents, logging information about them and report-ing intrusion attempts. A detection module can be used forvarious purposes such as identifying problems with securitypolicies, documenting existing threats and deterring individualsfrom violating security policies. A detection module acquiresand analyzes information from various areas within a computeror a network to identify possible security breaches, vulnerabil-ities, which include both intrusions and misuse. This modulemay use scanning, which is a technology developed to assesssecurity vulnerabilities of a computer system or network.

2.2.3. ReactionTypically, a defense system reacts using a two-step process.

The first set of procedures constituting the passive component,involves inspection of the system’s configuration files to detectinadvisable settings, inspection of the password files to detectinadvisable passwords, and inspection of other system areas todetect policy violations. The second set of procedures consti-tute the active component. Here mechanisms are set in placeto react to known methods of attack and to generate systemresponses. IDSs can respond to suspicious events in severalways, which include displaying an alert, logging the event oreven paging an administrator. Alarm management can be cate-gorized into two (Kluft and Staaf, 2012; Pietraszek and Tanner,2005).

• Alert/alarm quality improvement: This approach tries toimprove alert quality by using information such as vulner-ability reports or alert contexts. One can prioritize alertswith respect to the vulnerabilities of the victims.

• Alarm correlation: This approach creates a more ambi-tious goal. It tries to reconstruct higher-level incidentsfrom lower-level alerts. Sometimes, a defense system maygenerate more alarms than normal within a short period. Ifa set of alerts are triggered, and knowing this without anyadditional background knowledge, one cannot determinewhether these are coordinated/distributed attacks or inde-pendent attacks that happen to be interleaved. If it is a sin-gle multistage attack, alarms would have to be generatedin a single incident. In the case of multiple attacks, thealerts should be divided into multiple incidents, namely,one incident per attack. Grouping alerts that constitute a

single attack into a single meta-alert is aggregation. Thetask of clustering alerts into incidents is called correlation(Julisch, 2003).

To prevent an attack before damaging the network system,it is need to adopt preventive measure like, creating a databaseof detected signatures of abnormalities to filter out threateningpackets, analyze pattern of network behavior, reconfigure othersecurity controls etc. Trace-backing (Xiang and Zhou, 2004)the source of attack is a good way to prevent the attack in fu-ture. But, it also important to look for low collateral damagewhile trace backing a huge botnet attack. So, reaction proce-dure in terms of prevention is suitable aspect. Passive systemscan attempt to terminate the connection before an attack cansucceed, for example, by ending an existing TCP session.

Researcher have demonstrated that no system can assure todetect and prevent from any kind of anomalous activities in aexposed and live network in a generic way. Thus, the solutionto react, lead in two di↵erent track. One option is that, respondthe attack or intrusion in an e�cient way to provide the usualservice to general user and stop the service to the non-legitimateuser. Second option is to provide fault tolerance approach to-wards attack.

2.3. Based on Nature of Control

In this section, we discuss types of defense systems basedon the control structure used to counter attack tra�c. Thereare three basic ways used to control detection and preventionprocesses, viz., centralized, hierarchical and distributed (Patelet al., 2013).

2.3.1. CentralizedIn this type of defense, each detection element produces

alerts locally. The generated alerts are sent to a central serverthat plays the role of a correlation handler and analyzes them.Using centralized control, an accurate detection decision can bemade based on all available alert information. The main draw-back of this approach is that the central unit is crucially vulner-able; any failure in the central server leads to the collapse ofthe whole process of correlation. In addition, the central unitshould be able to handle the high volume of data which it mayreceive from the local detection elements in a short amount oftime.

2.3.2. HierarchicalThe whole system is divided into several small groups based

on features such as geography, administrative control, and soft-ware platforms. The IDPSs at the lowest level work as detectionelements while an IDPS at a higher level is furnished with botha detection element and a correlation handler, and it correlatesalerts from both its own level and lower levels. The correlatedalerts are then passed to a higher level for further analysis. Thisapproach is more scalable than the centralized approach, butstill su↵ers from the vulnerability of the central unit. Besides,the higher level nodes have a higher level abstraction of the in-put, which may limit their detection coverage.

5

Table 2: Comparison of Control Mechanisms for Defense Against Intrusion

ControlMechanism

Advantages Disadvantages

Central - Each IDPS acts as a detection element. - The central unit is crucially vulnerable.- Every detection element produces alerts locally. - Any failure in the central server leads to deacti-

vation of the whole process of correlation.- Accurate detection decisions can be made basedon all available alert information.

- The central unit handles high volume of data re-ceived from local detection elements.

Distributed - Need not have complete information of networktopology.

- Information about all alerts is not available dur-ing decision making about detection.

- Possible to have a more scalable design sincethere is no central entity.

- Alert information may be too narrow to detectlarge scale attacks.

- Local alarm correlation is simpler in this struc-ture.

Hierarchical - This approach is more scalable than the central-ized approach.

- Su↵ers from the vulnerability of a central unit.

- Higher level nodes have higher level abstractionsof the input.

- At each level, the detection coverage may be lim-ited.

Figure 5: Central Management Structure

Figure 6: Hierarchical Management Structure

2.3.3. DistributedIn this approach, there is no centralized coordinator to pro-

cess the information, and is comprised of fully autonomous sys-

tems with distributed management control. All participatingIDPSs have their own components communicating with eachother. The advantages of the fully distributed IDPS (Leitneret al., 2007) are that although the network entities do not havecomplete information about the network topology, it is possi-ble to have a scalable design since there is no central entityresponsible for doing all the correlation work, and local alarmcorrelation is simpler in this structure. Meanwhile, the fully dis-tributed approach has its own drawbacks (Zhou et al., 2010): (a)Information on all alerts is not available during decision mak-ing, so the accuracy may be reduced; (b) The alert informationusually has a single feature (like an IP address), which may betoo narrow to detect large scale attacks.

Figure 7: Distributed Management Structure

Looking at the di↵erent approaches to control we observethe following.

• Each of the three ways of control has advantages and dis-advantages. We summarize them in Table 2.

6

Table 3: Comparison Between Host-based and Network-based Defense

Infrastructure Strengths LimitationsHost-based - It is good in detecting and verifying inside at-

tacks as they reside on host.- They are vulnerable to both direct attacks andattacks against host operating system.

- It is able to decrypt encrypted packets in incom-ing tra�c.

- They are vulnerable to denial of service attacks.

- There is no need for additional hardware. - Performance overhead may increase.Network-based

- It is designed to work in large networks. - Due to large network size, there is chance thatthe system fails to recognize attacks.

- It is usually passive and can be easily deployedon an existing network with no disruption to thenormal network operation.

- They cannot analyze packets which are en-crypted.

- It is less susceptible to direct attack. - Whether the attack is successful or not may notbe reliably detected.

• Depending on the network under consideration we may beable to choose any one among them for defense.

• In hierarchical and distributed defense, every level can de-tect attacks and react accordingly in its own neighborhood.Each level or each unit usually handles a low volume ofdata.

• In centralized defense, only the central server takes part inthe decision making process. The server covers the entirenetwork with no redundancy.

2.4. Based on Defense Infrastructure

In this section, we discuss various defense systems based onthe infrastructure used. These are two basic types, viz., host-based and network-based.

2.4.1. Host-basedIn this architecture, data is analyzed by individual computers

that serve as hosts. The network architecture used is agent-based, which means that a software agent resides on each of thehosts in the system. Thus, a host-based Intrusion Detection andPrevention System (HIDPS) processes data that originates onthe computers themselves, such as event and kernel logs. AnHIDPS (Yeung and Ding, 2003) can also monitor which pro-gram accesses which resources and may flag anomalous usage.An HIDPS also monitors the state of the system and makes surethat everything makes sense, which is necessary for the use ofanomaly filters.

2.4.2. Network-basedA network based detection system examines data exchanged

among computers in the network. A Network-based IntrusionDetection and Prevention System (NIDPS) (Vigna et al., 2004)captures network tra�c from the wire as it travels to a host.This can be analyzed for a particular signature or for unusual orabnormal behaviors. Several sensors are used to sni↵ packetson the network; these are computer systems designed to mon-itor network tra�c. If any suspicious or anomalous behavior

occurs, it triggers an alarm and passes the message to the cen-tral computer system or an administrator, and generates an au-tomatic response.

2.5. Defense LocationAn intrusion defense system can be deployed in three possi-

ble locations: victim-end, intermediate and source-end. Eachhas its advantages and disadvantages.

2.5.1. Victim-end defense mechanismVictim-end detection (Douligeris and Mitrokotsa, 2004) ap-

proaches are conventionally employed in the routers of the vic-tim side network. The detection software stores informationabout known intrusion signatures or profiles of normal behav-ior. This information is updated by the processing elements asnew knowledge becomes available. The processing element ina detection engine frequently stores intermediate results in whatis called configuration data. Detecting attacks at the victim endis relatively easy, but requires higher resource consumption. Animportant drawback is that these approaches detect the attackonly after it reaches the victim and thus legitimate clients havealready been a↵ected.

2.5.2. Intermediate network defense mechanismThe intermediate network defense scheme (Wang et al.,

2001) balances detection accuracy and attack bandwidth con-sumption, which are the main issues in source-end and victim-end detection approaches. The main di�culty with this ap-proach is deployability. To achieve full detection, all routerson the Internet will have to use this detection scheme, becausenon-availability of this scheme on only a few routers may causedetection failure. Thus, full practical implementation of thisscheme is extremely di�cult.

2.5.3. Source-end defense mechanismThis type is somewhat similar to victim-end detection. It is

the best option if we want to reliably detect or stop intrusion(Mirkovic and Reiher, 2005). It prevents congestion not onlyon the victim side, but also in the whole intermediate network.The main di�culty is that during attacks, sources are widely

7

Table 4: Comparison of Defense Solutions Based on Locations

DefenseLocation

Advantages Disadvantages

Victim-end - Stored known intrusion signatures are applied forbetter performance.

- Till the time an attack is detected and stopped,legitimate clients are a↵ected.

- To detect attack at this end is easy because a highamount of resources is available.

Intermediate - At this location, we can balance the resources - The main problem is deployability.used for detection and bandwidth consumed by anattack

- All routers in the network must employ the de-tection scheme.

- Su↵ering of legitimate clients is lower than in thecase of victim-end.

- Failure at a few routers can cause damage towhole detection process.

Source-end - It is the best option that avoids congestion andcollateral damage to the whole network.

- The resources needed to deploy are lower than atany other network locations.

- Legitimate clients su↵er less. - It is di�cult to distinguish between normal andabnormal tra�c because they behave almost simi-larly.

distributed and a single source may behave almost similarlyas in normal tra�c. Another crucial problem is the practicalimpossibility of deploying at the source-end when the sourcesaren’t known or are too many. We note the following.

• Depending on the techniques used for intrusion defense,resource requirements vary at these three locations.

• Each mechanism has advantages and disadvantages, whichare listed clearly in Table 4.

• Legitimate clients su↵er the most in the victim-end ap-proach compared to source-end and intermediate networkapproaches.

• To balance resources needed to handle an attack and toallow normal tra�c for legitimate clients during an attackis tougher in source-end and intermediate locations.

2.6. Based on Technique Used

Many techniques have been developed to prevent intrusion.We cover a variety of them below under various categories andsubcategories.

2.6.1. Misuse DetectionIn misuse detection (Kumar and Spa↵ord, 1994), one defines

abnormal system behavior first, and then define any other be-havior as normal. This is in contrast to anomaly detection,which uses the reverse approach, defining normal system be-havior first and defining any other behavior as abnormal. Inother words, anything one doesn’t know as bad is normal inmisuse detection. Using attack signatures in IDSs is an exam-ple of this approach. Such misuse detection IDSs attempt todetect only known attacks based on predefined attack character-istics. The accuracy of such IDSs solely depends on how wellthe knowledge of known attacks has been preprocessed and fedto the IDS’s detection engine. Well-crafted expert knowledge

of known attacks can enable misuse detection IDSs to performaccurately with a low number of false positives.

A. Signature-based approach: A signature detection or mis-use detection scheme (Kruegel and Toth, 2003) stores se-quences of patterns and signatures of attacks in a database.When an attacker tries to attack, the IDS matches the signatureof the particular attack type with predefined signatures that arealready stored in the database. If there is successful match, thesystem generates alarm. In this approach, the semantic charac-teristics of an attack are analyzed and details are used to formattack signatures. Attack signatures are constructed in such away that they can be searched using information in audit datalogs produced by computer systems. A database of attack sig-natures is built using well-described known attacks and the de-tection engine of the IDS compares string log data or audit dataagainst the database to detect an attack. Each time a new attackis discovered, the attack signature database must be quickly up-dated to obtain up-to-date performance. Many di↵erent sig-nature matching algorithms are used in signature based attackdetection systems.

B. Rule-based approach: (Porras and Kemmerer, 1992) Rulebased systems are built using a number of if-then rules. Rulesare developed by analyzing attacks or misuses by experts.These are later used by inference modules of IDSs to compareagainst monitored data to detect any misuse.

C. State Transition Approach: (Ilgun et al., 1995) In this ap-proach, attacks or misuses are represented as a series of activi-ties and a single activity or a combination of activities can causetransition from one state of a monitored sensor to another state,eventually reaching the security alert state of a monitored sys-tem.

2.6.2. Anomaly Detection TechniqueTheses techniques first establish the normal behavior of a

subject, e.g., a user or a system. Any action that significantlydeviates from normal behavior is considered intrusive. If wecan establish normal activity profile for a system, we can flag all

8

Table 5: Comparison of Di↵erent Misuse Detection Techniques

Signature-Based Approach Rule-based Approach State Transition Approach- During intrusion, it matches the pat-tern of attack with stored patterns.

- Rules are developed by analyzing at-tacks.

- One needs to identify the statesof the system and transitions amongthem.

- When a new attack is discovered,database must be quickly updated.

- The rules are used as conditions todetect an attack on the monitored net-work or system.

- Single or multiple changes can causetransition among states of the moni-tored system. Such information maybe useful in finding a path to a safestate.

system states that vary significantly from an established profile.Thus, anomaly based techniques try to detect the compliment ofbad behavior. Two types of errors in detection may occur: (1)False positives: Anomalous activities that are not intrusive butare flagged intrusive. (2) False Negatives: Anomalous activi-ties that are intrusive but are flagged non-intrusive. The mainadvantage of anomaly detection is that it can detect unknownattacks.

A. Statistical: Statistical methods have long been used foranomaly detection (Stibor et al., 2005). In this approach, nor-mal user behavior is first defined based on what is acceptablewithin system usage policies. Using statistical modeling tech-niques, user behavior is monitored and if there is a large devia-tion from predefined normal behavior thresholds, such anoma-lous activity is considered attack.

A.1. Markov process: Intrusion detection in this model isperformed by investigating the system at fixed intervals andkeeping track of probability for each state at a given time in-terval. A change of state of the system occurs when an eventhappens and the behavior is detected as anomalous if the prob-ability of occurrence of that event is low. Transitions betweencertain commands may lead to anomaly detection, especiallywhere a sequence of commands is necessary to perform a task(Ye, 2000).

A.2. Operational: The count of certain events that occur overa period of time may determine the alarm to be raised. Forexample, alarm may be raised if an event occurs fewer than m ormore than n times. This can be visualized in Win2k lock, wherea user is blocked after n unsuccessful login attempts. Here thelower limit is ‘0’ and the upper limit is n. The size of executablefiles that can be download is restricted in some organizations toabout 4 MB. The di�culty in this sub-model is determiningappropriate values for m and n.

A.3 Statistical moments: In statistics, mean and standard de-viation are known as moments. An event that falls outside a setinterval, above or below the moment, is said to be anomalous.The system is updated considering data over time by makingchanges to the statistical database. There are two major advan-tages over an operational model (Jyothsna et al., 2011). First,prior knowledge is not required to determine normal activity inorder to set limits. Second, determining the confidence inter-vals depends on observed data, as it may vary from user to user.A threshold model lacks this flexibility. A major variation onthe mean and standard deviation model gives higher weights to

recent activities.A.4. Time series: A time series model needs interval timers

(Viinikka et al., 2009) together with event counters or resourcemeasures. The system observes and stores values of order andinter-arrival times. If the probability of occurrence of a newobservation is too low, it is considered an anomaly.

B. Data mining or Machine learning: In this approach, his-torical usage data obtained from a monitored system is nor-mally categorized as acceptable or unacceptable and labeled ac-cordingly. By using a learning algorithm, the system is trainedto learn either what is acceptable or what is unacceptable con-sidering usage in a network or system. If any deviation occurs,the system triggers an intrusion alert. Many machine learning(Lee and Stolfo, 1998; Tsai et al., 2009) techniques can be usedfor data classification. We present the general concepts anda brief description of some popular supervised learning tech-niques below.

B.1. Decision Trees: Decision Trees represent a well-knownand e�cient classification algorithm (Peddabachigari et al.,2004). A decision tree consists of non-terminal nodes (a rootand internal nodes) and terminal nodes (leaves). The root nodeis the first attribute with test conditions to move an input datarecord down the tree toward a leaf node depending on charac-teristics of the data record. Testing and splitting are repeatedat every internal node. First, a decision tree is trained withknown labeled data before it can classify new or untrained data.The training process builds the decision tree by identifying at-tributes and values that would be used to test the input data ateach internal node. After training, the tree can classify new databy starting from a root node and traversing down internal nodesbased test conditions until it arrives at a leaf (terminal) nodeconsisting of an answer class.

B.2. Artificial Neural Networks: In this method, a datasetof input vectors and corresponding target vectors are used intraining the network to associate input with output (Mukkamalaand Sung, 2003; Hagan et al., 1996). A neural network can becreated from audit data that specify a string of events as abnor-mal or normal. Ghosh and Schwartzbaxd (1999) used ArtificialNeural Networks (ANNs) to detect both known and novel at-tacks on a computer system using supervised learning. Theyused Sun Microsystem’s Solaris Basic Security Module (BSM)generated host audit data to train a multi-layered back propa-gation feed forward neural network to learn normal system be-havior. In order to verify complete session activities, which

9

Table 6: Summary of Di↵erent Anomaly-based Defense Techniques

Anomaly Detec-tion Technique

Description Type Description

Statistical - Any activity is consideredattack if there is any devia-tion from predefined normalbehavior threshold.

- Markov process - Keeps track of the di↵erent states the sys-tem is in at a specific time interval. The oc-currence of a state is low if it correspondsto abnormality.

- A normal user behavior isdefined using statistical mod-eling techniques by comput-ing mean and standard devia-tion

- Operational - An alarm detecting abnormality is raisedif count of events that occur is lower than athreshold value or greater than a thresholdvalue

- Statistical Moments - An event that falls outside the set inter-val, above or below the moment can betreated as anomalous. Standard deviationand other summaries are known as mo-ments.

- Time Series - If the probability of occurrence of a newobservation is too low, it is considered ananomaly.

Data Mining orMachine Learning

- Data mining or machinelearning algorithms are goodat finding similarities or pat-

- Decision Trees - A decision tree is trained with known databefore it can classify new or untrained data.After training, it can classify new data.

terns in data. Knowledgegained by training on data isused to classify new data.

- Artificial Neural Net-works

- A dataset of input vectors and corre-sponding target vectors are used in trainingthe network to associate input with output.- A neural network is created from auditdata to determine a string of events as ab-normal or normal.

- Bayesian Networks - In this graphical technique, nodes in thegraph represent random variables. TheBayesian network learns causal or depen-dency relations among attributes in thetraining data set before classifying newdata.

- Fuzzy Logic - Such a system learns the characteristicsof network tra�c by applying fuzzy logic.Signatures are developed by analyzing net-work protocols.

- Outlier Detection - If a data point is very di↵erent from therest of the data, it is an outlier. Abnormaltra�c that is very di↵erent from acceptabletra�c based on metrics, is identified as out-lier.

Cognition-basedor Expert System

- Detection of intrusion is in-fluenced by a set of predefinedparameterized rules that clas-sify training data.

- Finite State Ma-chines

- A state contains information about thepast, and any changes in the input are notedto identify the abnormality.

10

consist of multiple BSM events, they applied the leaky bucketalgorithm (Lee and Un, 1993) to capture recent activities fromneural network generated outputs.

B.3. Bayesian Networks: A Bayesian Network (Burroughset al., 2002) involves a probabilistic model representing ran-dom variables and conditional independence using a directedacyclic graph. Nodes in the graph represent random variableswhereas edges represent conditional dependencies. Thus nodesthat are not connected represent variables that are condition-ally independent of each other. The Bayesian Network learnscausal relations among attributes and class labels from the train-ing dataset before it can classify unknown data.

B.4. Fuzzy logic: Such a system learns the characteristicsand behavior of network tra�c by applying fuzzy logic (Klirand Yuan, 1995). Signatures are developed by learning or an-alyzing network protocols. Abouzakhar and Manson (2003)proposed a neuro-fuzzy technique for detecting distributed net-work attacks such as denial of service (DoS). The proposedsystem learned characteristics of network tra�c by applyingfuzzy logic. Chavan et al. (2004) proposed a neuro-fuzzy adap-tive IDS for IP networks where a database of pattern signatureswas built to complement the SNORT signature database (Snort,1999). These signatures were developed by analyzing networkprotocols and adaptive learning based on combination of artifi-cial neural networks and fuzzy inference techniques.

B.5. Outlier Detection: If a data point is very di↵erent fromthe rest of the data, it is an outlier. According to statistical dis-tribution of data points based on a given mean and a standarddeviation, data points that do not fall under a specific range areconsidered outlier. In network intrusion detection, anomalousnetwork tra�c data or abnormal network behavior is di↵erentfrom acceptable tra�c data or normal network behavior basedon appropriate metrics and thus may be identified as outlier.There are various methodologies used to detect outliers (Gogoiet al., 2011).

C. Cognition-based Expert system: Cognition-Based (alsocalled knowledge-based or expert systems) (Jyothsna et al.,2011) detection techniques classify the audit data with the helpof a set of predefined rules. These rules may be created fromtraining data or may be built by expert or a combination of thetwo.

C.1. Finite state machines: A Finite State Machine (FSM)(Hershey et al., 1995) or finite automation is a model of behav-ior captured in states, transitions and actions. A state denotes asituation. Any changes in the input may cause transitions. Anaction is a description of an activity to be performed. Thereare several action types: entry action, exit action and transitionaction.

2.6.3. Applications and protocolsAn application protocol-based intrusion detection system

(APIDS) (Boukerche and Notare, 2000; Dreger et al., 2006)monitors dynamic behavior and states of the protocol and typi-cally consists of a system or agent that sits between a process,or a group of servers, monitoring and analyzing the applicationprotocol between any two connected devices. A typical place

for an APIDS is between a web server and the database man-agement system, monitoring the SQL protocol specific to themiddleware/business logic as it interacts with the database. Aprotocol-based intrusion detection system (PIDS) is typicallyinstalled on a web server, and is used in monitoring and anal-ysis of the protocol in use by the computing system. A PIDSmonitors the dynamic behavior and state of the protocol andtypically consists of a system or agent that sits at the front endof a server, monitoring and analyzing the communication be-tween a connected device and the system it is protecting. Atypical use for a PIDS is at the front end of a web server mon-itoring the HTTP (or HTTPS) stream. Because it understandsthe HTTP data stream relative to the web server/system it is try-ing to protect, it can o↵er greater protection than less in-depthtechniques such as filtering by IP address or port number alone.

3. Defense Systems

In this section we discuss several defense systems and ana-lyze their pros and cons.

(a) STAT: State Transition Analysis Technique (STAT) (Por-ras, 1992) is a suite of tools for misuse detection. It usesthe state transition mechanism to identify intrusive activities incomputer systems. This suite includes the STATL (Eckmann etal., 2002) language to define attack scenarios with the help ofdomain independent attack attributes in a high level language atan abstract level. These definitions must be included by securitysystem developers to meet the needs of a specific environment.The basic concept behind detecting an attack or misuse is thatbefore penetration by an attacker, a computer system is in aninitial secure state. A series of activities by an attacker causes asystem to transition to various intermediate states before reach-ing a final state where the system has been successfully pene-trated by an intruder.

(b) USTAT: UNIX State Transition Analysis Tool (USTAT)(Ilgun, 1992) is the first STAT based tool to analyze audit logsgenerated by UNIX-based systems for misuse identification. Itcan act in real time.

(c) ARMD: ARMD (Adaptable Real-time Misuse DetectionSystem) is an abstraction based misuse detection system, de-signed by Lin et al. (1998). In a UNIX environment, this sys-tem uses its own high level language called MuSigs for misusesignature abstraction from audit log. Using MuSigs, misusesare described in easily understandable abstract forms or signa-tures. ARMD uses a monitoring algorithm that searches a givenevent history looking for matching signatures and if any match-ing signatures are found, they are reported. Using ARMD as apart of the monitoring system, real time misuses could be de-tected. Other parts of the system are monitored o↵-line.

(d) NIDES: The continuation of research work on the IDESsystem led to the Next Generation Intrusion Detection System(NIDES) (Anderson, 1995). NIDES is built using client-serverarchitecture. Log data from various hosts on a network are gath-ered at a specific host. On that specific host, rule based anomalydetection is performed. The P-BEST expert system (Lindqvistand Porras, 1999) is used for misuse detection.

11

Figure 8: A Taxonomy of Intrusion Defense Solutions

(e) NSTAT: USTAT was originally capable of analyzing auditlog of a single UNIX host. Its extension to analyze audit logsof multiple UNIX systems changed it into NSTAT (Kemmerer,1997). It gathers audit records of a set of distributed hosts. Allthese records are processed in a central system for misuse de-tection.

(f) EMERALD: The EMERALD (Event Monitoring En-abling Responses to Anomalous Live Disturbances) system(Porras and Neumann, 1997) also uses the P-BEST expert sys-tem. EMERALD is an environment for misuse and attack de-tection in a large scale network, employing both anomaly andmisuse detection techniques.

(g) NetSTAT: NetSTAT (Vigna and Kemmerer, 1998), a real-time network misuse detection system, is also based upon theSTAT framework. NetSTAT represents the network topologyas a hypergraph model and uses STAT definitions of networkbased attacks to map specific misuses to specific network con-figurations. It preprocesses network tra�c log given as inputand filters network packets for relevant network events and gen-erates abstract events.

(h) Bro: Bro is an open source UNIX based network intrusiondetection system (Paxon, 1999). It uses a signature based ap-proach and detects known attacks and events using predefinedattack signatures. Bro tries to detect uncommon activities suchas failed connection attempts.

(i) Snort: Snort is a popular signature based lightweight net-work IDS (Snort, 1999). It uses a database consisting of user

defined attack signature rules and uses the Boyer-Moore pat-tern matching algorithm (Boyer and Moore, 1977; Knuth etal., 1977) against the database for each network tra�c packet.Snort has three modes. The first one is the packet sni�ng mode,which enables it to monitor and display tra�c packets. The sec-ond one is the network tra�c logger mode where Snort writesnetwork tra�c log into a database. The last one is the IDS modein which it has intrusion detection and prevention capabilities,both in real time.

(j) MADAM ID: Lee et al. (1999) apply various data min-ing techniques such as association rules and classification tech-niques to develop an automated misuse detection model usingdata from audit logs and system calls. They develop an intru-sion detection framework called MADAM ID (Mining AuditData for Automated Models for Intrusion Detection).

(k) Ghosh and Schwartzbaxd (1999): The authors use Arti-ficial Neural Networks (ANNs). Both known and unknown at-tacks against a computer system are detected using supervisedlearning. It uses audit data to train a multi-layered back propa-gation feed forward neural network to learn normal system be-havior.

(l) Chavan et al. (2004): They propose a Neuro-Fuzzy adap-tive IDS for IP networks where a database composed of pat-terns of signatures is built to complement the SNORT signaturedatabase. These signatures are developed by analyzing networkprotocols and using adaptive learning based on combination ofartificial neural networks and fuzzy inference techniques.

12

Table 7: A Survey Table of Di↵erent Systems

Year System References Approach Technique Infrastructure Real time1992 STAT Porras (1992) Detection Misuse detection, uses

state transition mecha-nism

Host-based Yes

1992 USTAT Ilgun (1992) Detection Misuse detection Host-based Yes

1995 ARMD Lin et al. (1998) Detection Abstraction based mis-use detection

Host-based Yes

1995 NIDES Anderson (1995) Detection Rule-based expert sys-tem, anomaly detection

Host-based Yes

1997 EMERALD Porras and Neumann(1997)

Detection Rule-based signaturedetection using P-BESTexpert system

1998 NetSTAT Vigna and Kemmerer(1998)

Detection Misuse detection, usesnetwork topology as hy-pergraph model

Network-based Yes

1999 Bro Paxon (1999) Detection Signature Network-based Yes1999 Snort Snort (1999) Detection,

preventionSignature based Network-Based Yes

1999 MADAM ID Lee et al. (1999) Detection Data mining Host-based No

1999 Ghosh andSchwartzbaxd

Ghosh and Schwartzbaxd(1999)

Detection Artificial Neural Net-works based

Host-based

2000 NFIDS Mohajerani et al. (2000) Detection Fuzzy logic and neuralnetwork combined

Network-based

2001 PHAD Mahoney and Chan (2001) Detection Data mining techniques Network-based No2002 Qiao et al. Qiao et al. (2002) Detection Statistical modeling

techniques, HiddenMarkov Model (HMM)

Host-based

2002 MINDS Ertoz et al. (2004) Detection Outlier detection tech-niques

Network-based Yes

2003 Abouzakhar andManson

Abouzakhar and Manson(2003)

Detection Neuro-fuzzy technique Network-based

2003 Kruegel and Toth Kruegel and Toth (2003) Detection Misuse and anomalybased

Network-based Yes

2003 WHIPS Battistoni et al. (2004) Prevention Monitoring criticalWindows system calls

Host-based Yes

2004 Chavan et al. Chavan et al. (2004) Detection Neuro-Fuzzy Network-based Yes2005 Yao et al Yao et al. (2005) Detection Fuzzy logic and Sup-

port Vector Machines(SVM)

Network-based

2005 FLIPS Locasto et al. (2005) Prevention Signature based,anomaly detection

Host-based Yes

2006 Weinsberg et al. Weinsberg et al. (2006) Prevention String matching algo-rithm

Network-based Yes

2008 NFR Ranum (2008) Detection Signature based Network-based Yes2008 Zhang et al. Zhang et al. (2004) Prevention Rule-based Network-based Yes2009 Luo et al. Luo et al. (2009) Detection Game-theory based,

Multi-stage attackdefense

Host-based Yes

2010 Huang et al. Huang et al. (2010) Detection Distributed multi agentbased

Host-based Yes

2011 Computer wormdefense system

Aziz (2011) Detection,Prevention

Sensor based Network-based Yes

13

(m) NFIDS: Mohajerani et al. (2000) use fuzzy logic andneural networks to build the Neuro-Fuzzy Intrusion DetectionSystem (NFIDS). They use neural networks to learn fuzzy rulesfor each type of attack listed by the system administrator o✏ine.After learning the fuzzy rules, the neural network performs afuzzy inference process to detect intrusions.

(n) Abouzakhar and Manson (2003): They also propose aneuro-fuzzy technique for detecting distributed network attackssuch as denial of service (DoS) attacks. The proposed systemlearns the characteristics of network tra�c by applying fuzzylogic.

(o) PHAD: The Packet Header Anomaly Detector (PHAD)system detects anomalous field values using a data mining tech-nique from the information contained in the data link, andnetwork and transport layers protocol headers (Mahoney andChan, 2001). The algorithm is trained with normal networktra�c data so that it can learn the normal range of allowablevalues for each field. The algorithm calculates the number ofpreviously unseen values and frequencies of each value for eachfield and assigns an estimated probability to a given field valuebeing abnormal. An abnormality field score is calculated usingthe time since the last abnormality was observed in that field.Finally a packet score is calculated by summing up all the ab-normality field scores in the packet.

(p) Qiao et al. (2002): The authors use a Hidden MarkovModel (HMM) built using system calls to detect intrusions. Todetermine various state transitions that a special UNIX basedprocess goes through from the start to the end, they collect allthe system calls specific to that process and train an HMM onsystem calls associated with such processes.

(q) MINDS: The Minnesota Intrusion Detection System(MINDS), described in (Ertoz et al., 2004), successfully appliesoutlier detection techniques on network tra�c data to detect at-tacks on a computer network. MINDS uses a suite of data min-ing algorithms in its various detection modules. Each moduledetects a specific type of computer attacks and intrusive activi-ties in a networked environment.

(r) Kruegel and Toth (2003): They apply the decision treeapproach to match attack signatures instead of traditional sig-nature matching techniques such as in Snort and achieved im-proved detection speed.

(s) WHIPS: Battistoni et al. (2004) propose WHIPS, a host-based IPS for Windows operating systems. The system solelymonitors critical Windows system calls in the kernel mode. Theproposed system identifies privileged processes and identifiesharmful processes by examining access tokens.

(t) Yao et al. (2005): The authors develope a hybrid intru-sion detection system using both fuzzy logic (Klir and Yuan,1995) and Support Vector Machines (SVM) (Mukkamala andSung, 2003). They apply SVMs on network tra�c data multi-ple times by changing values of parameters and obtain sets ofsupport vectors during the training phase of the SVMs. Thenthey apply fuzzy logic to develop fuzzy rules to make decisionfrom various sets of support vectors obtained from SVM train-ing.

(u) FLIPS: Locasto et al. (2005) introduce a hybrid adap-tive intrusion prevention system called FLIPS (Feedback Learn-

ing IPS) . Host-based FLIPS uses both signature matching andanomaly based classification. Its goal is to detect and preventcode injection attacks. It uses an intermediate emulator to de-tect injected malicious attack code and does not generate attacksignatures.

(v) Weinsberg et al. (2006): They propose a high perfor-mance string matching algorithm and use it to develop. Itworkes as a network-based IPS. The hardware-based algorithmhas the ability to match multiple patterns at a time, making itfaster.

(w) NFR: Network Flight Recorder (NFR) (Ranum, 2008)is a commercially used powerful network intrusion detectionand analysis tool. It uses signatures of known attacks to raisealarm if any attack is detected. NFR uses a scripting languagecalled n-code for generating signatures and for network packetanalysis.

(x) Zhang et al. (2004): They describe a network-based IPSwhich is distributed in action. The rule-based IPS uses a net-work management module with a number intrusion detectionmodules. Multiple IDSs are placed in various locations on thenetwork. The IPS was created using application-specific in-tegrated circuits (ASICs)and therefore, it had fast processingability.

(y) Luo et al. (2009): This system presents a game theorymethod to monitor and analyze the risk and impact of multi-stage attacks in intrusion detection system. They developed analgorithm to defend attack, termed as multi-stage attacker de-fender(MAD) and to provide help to administrator in defending.The believes of the attacker and the administrator are updatedbased on the analysis of the life-cycle for the multi-stage attacksto reduce the horizon e↵ect.

(z) Huang et al. (2010): It is a distributed defense systemwhich detects intrusion and perform as supplement of firewallfor more e↵ective protection means. It used a Multi-Agent-Based Distributed Intrusion Detection System. It assures gooddetection accuracy and detection speed and enhance the systemsecurity.

(aa) Computer worm defense system: This system consistsof multiple containment systems bind together by a manage-ment section (Aziz, 2011). Each containment system can bedeployed on a separate communication network and contains aworm sensor and a blocking system. Worm sensor generatescomputer worm identifiers from every containment system andit can provide distributed blocking system.

4. Defense Issues and Challenges

Flaws in the architecture of a network system allows intru-sions to take place. When intrusions happen, regular benigntra�c to the victim’s network su↵ers and the victim loses accessto other networks. The victim can try to prevent these attacks orintrusions by installing firewalls or intrusion detection systems.However, developments on the Internet take place rapidly andthe quality and intensity of attacks also keep on improving andescalating, respectively. As a result, security at the victim-endoften begins to deteriorate. This causes legitimate connections

14

to su↵er during attacks. Inter-dependencies on the Internet arevery high and as a result, the probability of getting attacked byan attacker depends on the rest of the global Internet.

• E↵ective prevention mechanisms should be able to per-form well in real time. Good intrusion detection and pre-vention systems must perform well by two metrics: ac-curacy and timely performance. Performance in time hasreceived relatively lower attention. If not addressed prop-erly, lack of e�cient performance can be the roadblock tolarge-scale adoption of real-time intrusion prevention so-lutions. Specifically, the overhead associated with mon-itoring (e.g., data collection), analysis (e.g., signature-matching), and response, in terms of their impact on fore-ground tasks is not well understood.

• A successful prevention mechanism must be dynamic. Dy-namism is a strategy to provide system components withsecurity awareness and adaptability to address runtimepolicy changes. In addition, if a system can be constructedto be protocol independent, it is even more desirable.

• There are no common characteristics among tra�cstreams comprising various attacks that can facilitate earlydetection and filtering. Additionally, if the attack is dis-tributed, participating machines usually observe no higheroutgoing load than usual.

• When faced with an intrusion, if all resources of a sys-tem must be engaged to rectify the situation, legitimateusers su↵er unnecessarily. So, it is necessary not only todetect and restrain attacks, but also provide good serviceto legitimate tra�c between the network where the solu-tion mechanism is deployed and the victim. The solutionshould have low false positives and have a low deploy-ment cost. This requires that source-end response mustbe selective. Source-end response must also be flexibleto compensate for poor detection and to o↵er deploymentincentive.

• It is ideally necessary to have a distributed and coordinatedresponse system. It is also crucial that the response bemounted at many points on the Internet to cover the net-work. Since the Internet is administered in a distributedmanner, wide deployment of any defense system (or evenvarious systems that cooperate) cannot be enforced orguaranteed. This discourages many researchers and prac-titioner from even designing distributed solutions.

• Internet resources are limited. Most intelligence and re-sources needed to service clients but fight intrusion are lo-cated at the end hosts. However, the high bandwidth path-ways needed for large throughput reside in the intermedi-ate network. As a result, resources present in unwittingparts of the network are widely exploited by attackers tolaunch successful attacks.

• Many vendors make bold claims that their solution han-dles the intrusion problem completely. However, there is

currently no standardized approach for testing intrusiondefense systems that would enable true comparison andcharacterization.

• The defense should have a fast response time and shouldbe able to respond quickly to any change in the tra�c pat-tern. It is necessary to execute the defense functions onlyat a time of attack and at the other times, it should act as anormal system and not expend much system resources.

5. Conclusion

Many defense techniques are available in the research com-munity and many techniques have been proposed to providebetter defense against intrusion. In this paper, we have covereda general overview of di↵erent approaches and also discussedcurrent defense issues and challenges. We started this paperwith a brief description of defense structures, control mecha-nisms and infrastructure layouts to mount a good defense. Wealso have provided discussions of some popular IDSs and IPSs.

References

Abouzakhar, N. S., & Manson, G. A. Networks Security Mea-sures Using Neuro-Fuzzyagents. In: Journal of InformationManagement and Computer Security 2003;11(1):33–38.

Anderson, D., Frivold, T., & Valdes, A. Next-Generation Intru-sion Detection Expert System (NIDES). In:Technical Report,SRI International, Computer Science Lab, 1995.

Aziz, A. Computer worm defense system. U.S. Patent No.8,006,305. Washington, DC: U.S. Patent and Trademark Of-fice, 2011.

Bai, Y., & Kobayashi, H. Intrusion Detection Systems: Tech-nology and Development. In: International Conferenceon Advanced Information Networking and Applications2003:710–715.

Battistoni, R., Gabrielli, E., & Mancini, L. V. A Host Intru-sion Prevention System for Windows Operating Systems. In:Proceedings of the 9th European Symposium on ResearchComputer Security (ESORICS ), France 2004;352–368.

Bhattacharya, D. K., & Kalita, J. K. Network Anomaly De-tection: A Machine Learning Perspective. CRC Press, BocaRaton, FL, 2013.

Bhuyan, M. H., Bhattacharyya, D. K., & Kalita, J. K. Networkanomaly detection: methods, systems and tools. In: Commu-nications Surveys & Tutorials, IEEE 2014, 16(1):303–336.

Bhuyan, M. H., Bhattacharya, D. K., & Kalita, J. K. Survey-ing Port Scans and Their Detection Methodologies. In: TheComputer Journals 2011;54(10):1565–1581.

Bhuyan, M. H., Kashyap, H. J., Bhattacharya, D. K., &Kalita, J. K. Detecting Distributed Denial of Services At-tacks:Methods, Tools and Future Directions. In: The Com-puter Journals 2014;57(4):537–556.

Boukerche, A., & Notare, M. S. M. A. Neural Fraud Detectionin Mobile Phone Operations. In: Springer, Parallel and Dis-tributed Processing, 2000:636–644.

15

Boyer, R. S. & Moore, J. S. A Fast String Searching Algorithm.Communications of the ACM, 1977;20(10):762-772.

Burroughs, D. J., Wilson, L. F., & Cybenko, G. V. Analysisof Distributed Intrusion Detection Systems Using BayesianMethods. In: 21st IEEE International Conference on Perfor-mance, Computing, and Communications, 2002;329–334.

Chavan, S., Shah, K., Dave, N., Mukherjee, K., Abraham, A.,& Sanyal, S. A Study on Fuzzy Intrusion Adaptive Neuro-Fuzzy Intrusion Detection Systems. In: Proceedings of In-ternational Conference on Information Technology: Codingand Computing, Las Vegas, USA 2004;1:70–74.

Conorich, D. G. Monitoring Intrusion Detection Systems:From Data to Knowledge. Information Systems Security,2004;13(2):19–30.

Desai, N. Intrusion Prevention Systems: The Next Step in TheEvolution of IDS [Retrieved 10.10.2009].

Deswarte, Y., Blain, L., & Fabre, J. C. Intrusion Tolerance inDistributed Computing Systems. In: Research in Securityand Privacy Proceedings, IEEE Computer Society Sympo-sium, 1991;110–121.

Douligeris, C., & Mitrokotsa, A. DDoS Attacks and DefenseMechanisms: Classification and State-of-the-Art. In: Com-puter Networks 2004;44(5):643–666.

Dreger, H., Feldmann, A., Mai, M., Paxson, V., & Sommer, R.Dynamic Application-Layer Protocol Analysis for NetworkIntrusion Detection. In USENIX Security.

Eckmann, S., Vigna, G., & Kemmerer, R. STATL: An AttackLanguage for State-based Intrusion Detection. In: JournalComputer Security 2002;10(1–2):71–103.

Ertoz, L., Eilertson, E., Lazarevic, A., Tan, P. N., Kumar, V.,Srivastava, J., & Dokas, P. MINDS : Minnesota Intrusion De-tection System. In: Next Generation Data Mining, Chicago2004;199–218.

Ghosh, A. K., & Schwartzbaxd, A. A Study in Using NeuralNetworks for Anomaly and Misuse Detection. In: Proceed-ings of The 8th USENIX Security Symposium, Washington,D.C. 1999;141–152.

Gogoi, P., Bhattacharyya, D. K., Borah, B., & Kalita, J. K. ASurvey of Outlier Detection Methods in Network AnomalyIdentication. In: The Computer Journal 2011;54:570–588.

Hagan, M. T., Demuth, H. B., & Beale, M. H. Neural networkdesign. Boston, Pws Pub:2–14.

Hoque, N., Bhuyan, M. H., Baishya, R., Bhattacharya, D. K.,& Kalita, J. K. Network Attacks: Taxonomy, Tools and Sys-tems. In: Journal of Network and Computer Application2013;40(1):307–324.

Hershey, P. C., Johnson, D. B., Le, A. V., Matyas, S. M., Wa-clawsky,J. G., & Wilkins, J. D. Network Security System andMethod Using a Parallel Finite State Machine Adaptive Ac-tive Monitor and Responder. U.S. Patent No. 5,414,833. 9May 1995.

Huang, W., An, Y., & Du, W. A Multi-Agent-Based DistributedIntrusion Detection System. In: 3rd International Conferenceon Advanced Computer Theory and Engineering (ICACTE),2010;3:V3–141.

Ilgun, K. USTAT: A Real-Time Intrusion Detection System forUNIX. Masters thesis, Computer Science Department, Uni-

versity of California, Santa Barbara, 1992.Ilgun, K., Kemmerer, R. A., & Porras, P. A. State Tran-

sition Analysis: A Rule-based Intrusion Detection Ap-proach. In:l IEEE Transactions on Software Engineering,1995;21(3):181–199.

Julisch, K. Clustering Intrusion Detection Alarms to SupportRoot Cause Analysis. ACM Transactions on Information andSystem Security (TISSEC), 2003;6(4):443–471.

Jyothsna, V., Prasad, V. R., & Prasad, K. M. A Review ofAnomaly Based Intrusion Detection Systems. InternationalJournal of Computer Applications 2011;28(7):26–35.

Kabiri, P., & Ghorbani, A. A. Research on Intrusion Detectionand Response: A Survey. In: International Journal of Net-work Security 2005;1(2):84–102.

Kemmerer, R. A. NSTAT: A Model-Based Real-Time NetworkIntrusion Detection System. In: Technical Report, Depart-ment of Computer Science, UC SantaBarbara, 1997.

Klir, G., & Yuan, B. Fuzzy sets and fuzzy logic. New Jersey:Prentice Hall, Vol. 4, 1995.

Kluft, S., & Staaf, E. L. Alarm Management for IntrusionDetection Systems – Prioritizing and Presenting Alarmsfrom Intrusion Detection Systems. Master of Science The-sis, Computer Science Programme, University of Gothen-burg, Sweden, 2012.

Knuth, D. E., Morris, Jr, J. H., & Pratt, V. R. Fast Pattern Match-ing in Strings. In: SIAM Journal on Computing 1977;6:323–350.

Kruegel, C., & Toth, T. Using Decision Tree to Improve Signa-ture Based Intrusion Detection. In: 6th Symposium on Re-cent Advances in Intrusion Detection 2003;173–191.

Kumar, S. Survey of Current Network Intrusion DetectionTechniques. 2007:1–18.

Kumar, S., & Spa↵ord, E. H. A Pattern Matching Model forMisuse Intrusion Detection. 1994.

Lee, J. Y, & Un, C. K. Performance of DynamicRate Leaky Bucket Algorithm. In: Electronics Letters1993;29(17):1560–1561.

Lee, W., Stolfo, S. J., & Mok, K. W. A Data Mining Frameworkfor Building Intrusion Detection Models. In: Proceedings ofThe 1999 IEEE Symposium on Security and Privacy, Oak-land, California 1999;120–132.

Lee, W., & Stolfo, S. J. . Data Mining Approaches for IntrusionDetection. In Usenix Security, 1998.

Leitner, M., Leitner, P., Zach, M., Collins, S., & Fahy, C.Fault Management Based on Peer-to-Peer Paradigms; A CaseStudy Report from The Celtic Project Madeira. In: 10thIFIP/IEEEInternational Symposium on Integrated NetworkManagement, Munich 2007;697–700.

Lin, J., Wang, X., & Jajodia, S.. Abstraction-Based Misuse De-tection, High-Level Specifications and Adaptable Strategies.In: 11th IEEE Computer Security Foundations Workshop,IEEE; Rockport, MA 1998;190–201.

Lindqvist, U., & Porras, P. A. Detecting Computer and Net-work Misuse through The Production-based Expert SystemToolset (P-BEST). In: IEEE Symposium on Security and Pri-vacy, 1999:146–161.

Locasto, M., Wang, K., Keromytis, A., & Stolfo, S. FLIPS:

16

Hybrid Adaptive Intrusion Prevention. In: 8th InternationalSymposium, RAID(Recent Advances in Intrusion Detec-tion), Seattle, USA 2005;82–101.

Luo, Y., Szidarovszky, F., Al-Nashif, Y., & Hariri, S. A GameTheory Based Risk and Impact Analysis Method for Intru-sion Defense Systems. In: IEEE/ACS International Confer-ence on Computer Systems and Applications, 2009:975–982.

Mahoney, M., & Chan, P. Detecting Novel Attacks by Iden-tifying Anomalous Network Packet Headers. In: Technicalreport, Florida Institute of Technology, Melbourne, 2001.

Mirkovic, J., & Reiher, P. D-WARD: A Source-end De-fense Against Flooding Denial-of-Service Attacks. In:IEEE Transactions on Dependable and Secure Computing,2005;2(3):216–232.

Mohajerani, M., Moeini, A., & Kianie, M. NFIDS: A Neuro-fuzzy Intrusion Detection System In: Proceedings of the 10thIEEE International Conference on Electronics, Circuits andSystems, Sharjah, United Arab Emirates 2000;1:348–351.

Mukherjee, B., Heberlein, L. T., & Levitt, K. N. Network Intru-sion Detection. Network, IEEE, 1994;8(3):26–41.

Mukkamala, S., & Sung, A. H. Feature selection for intrusiondetection with neural networks and support vector machines.Transportation Research Record: Journal of the Transporta-tion Research Board, 2003;1822(1):33–39.

Murali, A. A Survey on Intrusion Detection Approaches. In:First International Conference on Information and Commu-nication Technologies, ICICT; 2005:233–240.

Patel, A., Qassim, Q., & Wills, C. A Survey of Intrusion Detec-tion and Prevention Systems. In: Information Managementand Computer Security 2010;18(4):277–290.

Patel, A., Taghavi, M., Bakhtiyari, K., & Junior, J. C. An Intru-sion Detection and Prevention System in Cloud Computing:A Systematic Review. In: Journal of Network and ComputerApplications 2013;36:25–41.

Paxson, V. B. A System for Detecting Network Intrudersin Real-Time. In: Computer Networks: The InternationalJournal of Computer and Telecommunications Networking1999;31(23–24):2435–2463.

Peddabachigari, S., Abraham, A., & Thomas, J. Intrusion De-tection Systems Using Decision Trees and Support Vec-tor Machines. International Journal of Applied Science andComputations, USA, 2004;11(3):118–134.

Pietraszek, T., & Tanner, A. Data Mining and Machine Learn-ingTowards Reducing False Positives in Intrusion Detec-tion. In: Information Security Technical Report, Elsevier2005;10(3):169–183.

Porras, P. STAT A State Transition Analysis Tool for Intru-sion Detection. Masters thesis, Computer Science Depart-ment, University of California, Santa Barbara, 1992.

Porras, P. A. & Neumann, P. G. Emerald: Event Monitoring En-abling Responses to Anomalous Live Disturbances. In: Pro-ceedings of the 20th National Information Systems SecurityConference, Baltimore, MD 1997;353–365.

Porras, P. A., & Kemmerer, R. A. Penetration state transi-tion analysis: A rule-based intrusion detection approach. In:Proceedings on Computer Security Applications Conference,1992;Eighth Annual:220–229.

Qiao, Y., Xin, X. W., Bin, Y., & Ge, S. Anomaly Intrusion De-tection Method Based on HMM. In: IEEE Electronic Letters2002;38(13):663–664.

Ranum, M. Network Flight Recorder. Inc. Intrusion Detection:Challenges and Myths, 2008.

Richhariya, V., & Srivastava, R. Survey of Current NetworkIntrusion Detection Techniques. In: Journal of InformationEngineering and Applications 2013;3(6):27–33.

Rathore, J. S. Survey on Intrusion Detection and PreventionSystem and Proposed Cost E↵ective Solution. In: Interna-tional Journal of Advanced Research in Computer Scienceand Electronics Engineering 2012;1(3):1–9.

Sandhu, U. A., Haider, S., Naseer, S., & Ateeb, O. U. Survey ofIntrusion Detection and Prevention Techniques. In: Interna-tional Conference on Information Communication and Man-agement, Islamabad 2011;16.

Snort, R. M. Lightweight Intrusion Detection for Networks. In:Proceedings of The 13th USENIX Conference on SystemAdministration, Seattle, Washington 1999;99:229–238.

Stakhanova, N., Basu, S., & Wong, J. A taxonomy of IntrusionResponse Systems. International Journal of Information andComputer Security 2007;1(1):169–184.

Stibor, T., Timmis, J., & Eckert, C. A Comparative Studyof Real-Valued Negative Selection to Statistical AnomalyDetection Techniques. In: Artificial Immune Systems,Springer:262–275.

Tsai, C. F., Hsu, Y. F., Lin, C. Y., & Lin, W. Y. Intrusion Detec-tion by Machine Learning: A Review. Expert Systems withApplications, 2009;36(10):11994-12000.

Vigna, G., & Kemmerer, R. A. NetSTAT: A Network-BasedIntrusion Detection Approach. In: Proceedings of the 14thAnnual Computer Security Applications Conference, IEEE1998;25–34.

Vigna, G., Robertson, W., & Balzarotti, D. Testing Network-based Intrusion Detection Signatures Using Mutant Exploits.In: Proceedings of the 11th ACM conference on Computerand communications security, 2004;21–30.

Viinikka, J., Debar, H., Me, L., Lehikoinen, A., & Tarvainen,M. Processing Intrusion Detection Alert Aggregates WithTime Series Modeling. Information Fusion 2009;10(4):312–324.

Weinsberg, Y., Tzur-David, S., Anker, T., & Dolev, D. HighPerformance String Matching Algorithm for a Network In-trusion Prevention System (NIPS). IN: High PerformanceSwitching and Routing (HPSR06) Poznan, 2006.

Wang, X., Reeves, D. S, Wu, S. F., & Yuill, J. Sleepy Water-mark Tracing: An Active Network-based Intrusion ResponseFramework. In: Trusted Information, Springer, 2001;369–384.

Xiang, Y., & Zhou, W. Trace IP Packets by Flexible Determin-istic Packet Marking (FDPM). In: Proceedings IEEE Work-shop on IP Operations and Management, 2004;246–252.

Yao, J. T, Zhao, S. L., & Saxton, L. V. A Study on Fuzzy In-trusion Detection. In: Proceedings of SPIE Vol. 5812, DataMining, Intrusion Detection, Information Assurance, AndData Networks Security, Orlando, Florida 2005;23–30.

Ye, N. A Markov Chain Model of Temporal Behavior for

17

Anomaly Detection. In: Proceedings of the IEEE Systems,Man, and Cybernetics Information Assurance and SecurityWorkshop, 2000;166:169.

Yeung, D. Y., & Ding, Y. Host-based Intrusion Detection UsingDynamic and Static Behavioral Models. Pattern recognition,2003;36(1):229–243.

Zhang, X., Li, C., & Zheng, W. Intrusion Prevention SystemDesign. In: The Fourth International Conference on Com-puter and Information Technology, Wuhan, China 2004;386–390.

Zhou, C. V., Leckie, C., & Karunasekera, S. A Survey of Co-ordinated Attacks and Collaborative Intrusion Detection. In:Computer and Security Journal, Elsevier 2010;29:124–140.

18