17

Click here to load reader

Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

Embed Size (px)

Citation preview

Page 1: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

School of Computer Science and Information TechnologyUniversity of Nottingham

Jubilee CampusNOTTINGHAM NG8 1BB, UK

Computer Science Technical Report No. NOTTCS-TR-2005-1

Firewalls, Intrusion Detection Systems and Anti-Virus Scanners

Julie Greensmith and Uwe Aickelin

First released: February 2005

© Copyright 2005 Julie Greensmith and Uwe Aickelin

In an attempt to ensure good-quality printouts of our technical reports, from the supplied PDF files, we

process to PDF using Acrobat Distiller. We encourage our authors to use outline fonts coupled with

embedding of the used subset of all fonts (in either Truetype or Type 1 formats) except for the standard

Acrobat typeface families of Times, Helvetica (Arial), Courier and Symbol. In the case of papers prepared

using TEX or LATEX we endeavour to use subsetted Type 1 fonts, supplied by Y&Y Inc., for the

Computer Modern, Lucida Bright and Mathtime families, rather than the public-domain Computer Modern

bitmapped fonts. Note that the Y&Y font subsets are embedded under a site license issued by Y&Y Inc.

For further details of site licensing and purchase of these fonts visit http://www.yandy.com

Page 2: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

Firewalls, Intrusion Detection and Anti-virusScanners

Julie GreensmithASAP Group, University Of Nottingham, UK

email: [email protected]

June 21, 2004

1 Introduction

While the sharing of resources and information in an interconnected communicationnetwork is essential, it is necessary to impose access restrictions. As a consequence,systems can be vulnerable to misuse by other users through access violation attempts.A number of tools have been developed to prevent this vulnerability including firewalls,intrusion detection systems and anti-virus software. However, the differences betweenthese tools are not immediately obvious, but do exist and play a core role in securingsystems. This article will examine the process involved in using each of the toolsand will highlight the differences between the tools themselves and their subsequentdeployment throughout a network of computers.

2 Securing Networks

Security is needed throughout distributed systems (interconnected components forminga network) in order to build dependable and trusted computing platforms. During thedesign phase of a distributed system, security policies are developed which account forthe measures taken to ensure both the confidentiality and integrity of the system, whenit is necessary. Confidentiality in this context refers to access constraints on users, andis there to protect the data. The integrity refers to the correct running of the systemand the data contained on the system. Additionally, the usability of the system mustbe preserved, which is tied in with preserving the integrity of the system so that it isstill functioning at the use level. There are several ways in which a system can becompromised, as stated in [7].

� Interception can occur when an unauthorised user gains access to a service or toa resource, such as the illegal copying of data after breaking into a restricted filesystem.

� Interruption can occur when files are corrupted or erased, occurring as the resultof denial of service attacks or from the action of a computer virus.

1

Page 3: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

� Modification involves an unauthorised user or program making changes to dataor system configuration, and can also include the modification of transmitteddata, leading to a breakdown of trust between parties.

� Fabrication is where data or activities are generated which would not normallyoccur. An example of this would be the addition of information to a passwordfile in order to compromise a system. To prevent such events from taking placewithin a system, a security policy must be put into place, and the necessarymeasures taken. Such measures can include the encryption of data, correct au-thentication and authorisation of users with respect to data access and commandexecution, and the conscientious audit of log files monitoring system activity.

From these descriptions it is evident that potential abusers of these systems can beboth external and internal to the system.

Many tools and techniques exist with the purpose of ensuring the confidentialityand integrity of a system. The use and deployment of the tools (in this particularinstance, firewalls, intrusion detection systems and anti-virus scanners) is dependentupon where in the system they are placed, and indeed, the architecture of the systemitself. Therefore, I will briefly digress and discuss what is meant by ‘systems’ withinthis context.

The system in question is a network of interconnected computers and servers, form-ing a local area network. This network could be used for example, in the inland rev-enue. A diagram of the connected components is represented in figure 1.

This local network additionally needs to be connected to the external world i.e. theInternet. There are several security challenges that need to be addressed for this net-work. The data within the system must be protected: not all users within that localnetwork need to have access to all files on the network or the external Internet en-vironment. Similarly, external entities may need to access the web server within thenetwork, for instance, to access a particular forum held on the web server. These func-tions must be available without compromising the integrity of confidentiality of thesystem, data or users. The level of security and methods of ensuring this are defined ina security policy. The type of tool used and the way in which it is implemented is de-pendent on the contents of the policy. For example, the policy would be used to defineif incoming telnet connections were permitted. If so, there are various constraints andconfigurations that should be applied to the system to enforce this.

3 Security Measures

3.1 Firewalls

Firewall systems are commonly implemented throughout computer networks. They actas a measure of control, enforcing the relevant components of the security policy. Afirewall can be a number of different components such as a router or a collection of

2

Page 4: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

Figure 1: A simple network structure

3

Page 5: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

host machines. However, the basic function of a firewall is to protect the integrity ofthe network which is firewall controlled. There are different types of firewall that canbe implemented, with the choice of firewall being dependent upon the security policyand the level of deployment in the system.

3.1.1 Packet Filtering Firewalls

Packet filtering firewalls work at the transport layer of the seven layer model[8]. Thismeans that they are commonly deployed on routers and act as a bottleneck betweenthe local network and the external Internet. As the name suggests, a packet filteringfirewall examines a packet passing through it, comparing it against a set of criteria forwhat is permissible either in or out of the network. The criteria for this is defined bythe security policy. There are two ways in which packet filters operate; either acceptall packets except those which are specified; or deny all packets except those whichare specified. The advantage of the ‘accept’ method is that it gives legitimate usersof the network greater flexibility. For example, a remote user of the system from apreviously unseen IP address (e.g. in an Internet cafe) could log in to an office machinefrom a remotely connected laptop while working out of the office. However, it alsoincreases the vulnerability of the network because not all attacks could come fromrules which are already known: this is why the ‘denial’ paradigm is more likely tobe used. This method should be deployed frequently but often isn’t due to a lack ofunderstanding from persons responsible for the configuration of the firewall. Denyingall that is unknown can give greater security, however, it can cause inconvenience tolegitimate users. Packet filters can examine the following attributes of a packet:

� Source IP address

� Destination IP address

� TCP/UDP source port

� TCP/UDP destination port

If in the example network an external user was trying to connect to port 23 ofa machine on the local network, then it is likely that the external user is trying toTELNET into that machine. This operation is likely to not be authorised and thereforethe firewall on the router would not permit the transmission of the packets into thenetwork.

3.1.2 Circuit Level Gateways

The situation could arise when an external user (not from the local area network) wishesto access information on a file server, behind at least one firewall. The security policyfor the network would not permit a direct connection between the external user and thefile server (as shown as part of the LAN in Figure 1) as this could leave the networkvulnerable to attack. The solution to this is for the two parties to create a ‘tunnel’between the two components, employing a method of encryption in the connection.The initial connection request is filtered (and is subject acceptance based on the security

4

Page 6: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

policy) but all packets following are not, as it acts as a relay between the two entities.In this case is important in this case to explicitly state the use of circuit level gatewaysin order to avoid the exploitation of the network.

3.1.3 Application Gateways

Application gateways, also known as proxies, are a commonly used firewall mecha-nism. It is feasible to want a particular component of a network, such as a publicallyavailable interface e.g. an online enquires form, to be available to entities outside ofthe local network. While remote access to other components of the network may notbe allowed, the inclusion of components in a demilitarized zone ( in between the twofirewalls) would allow access to components which were needed by the external net-work. Restricting the access to components via a DMZ, and through the use of a proxyserver allows external users to perform functions on, for example a web server, butwould not disclose the architectural details of the LAN. As with circuit level gateways,this proxy can act as a mediator between an external entity and a component behindthe packet filtering firewall on the main router. However, unlike circuit gateways, ap-plication gateways can filter IP traffic. This is an advantage because it would not allowcertain actions to be taken once a connection to the proxy has been made e.g. it canprevent anonymous FTP log-in to the system. Proxies can also act as caches for thelocal users accessing the Internet. This can be useful in the event of restricting accessto certain blacklisted web-sites. For example, in corporate LANs, where common mailproviders such as hotmail and Yahoo, cannot be accessed. Allowing employees to surfsuch sites is seen as a waste of resources, not to mention a breeding ground for viruses1.

Again, the pre-defined security policy, if adequately prepared, would define theaccess permitted to each individual user of the network. Additionally, application gate-ways can perform packet logging for a post hoc inspection of the traffic going both inand out of the network. The disadvantage with using an application gateway is that itrequires a multi-stage handshake for the initialisation of a connection which could slowdown the performance of that application considerably as opposed to making a directconnection. Due to the optional requirement for restricted commands execution, as inthe case of FTP through a proxy, then modified clients may need to be installed, whichis extra work for both the system administrators and the users. Hence, the transparencyof the service to the users is affected.

3.1.4 Other Points to Note

One feature of firewalls is that they should provide a high level of user transparency,meaning that the end user should be unaware of the action of the firewall, so quality ofservice is maintained as a result. Transparency is high for packet filtering firewalls asthe user is not always aware of the firewall until a transmission is denied. Applicationgateways have a lower transparency as it often requires the users to use modified soft-ware clients in order to use the proxy’s service which could result in the user attemptingto bypass the system entirely.

1More about this in a little while

5

Page 7: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

Recently, Stateful Multilayer Inspection Firewalls have been deployed, at the ap-plication layer, transport layer and network layer, which combines the packet filterproperty with the packet sniffing capabilities of gateways. Stateful inspection can beused to prevent attacks such as the Loki or Smurf denial of service attack, as the fire-wall would be aware that the original packet was not sent as a broadcast message froma machine on the network [6]. However, experience has shown that these systems aredifficult to manage due to the complexity of the rules and the processes involved, ren-dering them less secure than their separate counterparts.

With respect to the actual hardware required in order to implement firewalls, thereare two types, namely bridging firewalls and firewall routers. Bridging firewalls aresoftware firewalls that can be run on a standard machine, using a firewall such as IPta-bles. Firewall routers are a specific piece of hardware designed to perform as a routerand a firewall, and have been implemented as the first line of defence in many net-works. Bridging firewalls are becoming prominent due to their ease of configuration,ease of initial installation, good performance (little computational overhead) and theirability to be stealthy and so are less likely to be attacked [15] .

3.2 Intrusion detection systems

As previously stated, the majority of traffic on the network is not malicious, and userswithin a system do not set out to gain unauthorised access to information. However,the use of an intrusion detection system is becoming increasingly commonplace due toboth the increase in complexity of attack and of the computer systems themselves. Aswith any complex system, emergent properties can arise unexpectedly. In the case ofsuch systems, unexpected interactions between the various components can give rise tovulnerabilities which can be exploited. Additionally, the use of a firewall may not pre-vent internal abuse from an otherwise legitimate user of the system (either for breachesof confidentiality or for system integrity).

When defining what intrusion detection systems are, it perhaps makes more senseto describe what they are not. IDS are not a preventive measure. They will not stopintruders breaking into a system. Neither will they prevent internal damage to a system.As the name clearly states, they are a detection system, thus implying that abuse of asystem is reported as and when it happens. In essence, they are analogous to a burglaralarm in a house. Such an alarm can trigger an immediate response e.g. call the police,can be used to alert the owner that unauthorised behaviour is taking place, or simply tocause annoyance to the neighbours.

As with firewalls, different types of intrusion detection system exist. There are twodifferent ways of classifying an IDS. The first way is to classify based on the method ofdetection, in the form of either misuse detection or anomaly detection. An alternativeway is to classify based on the position of deployment within a network. IDS can beeither network based, host based or application based, depending on where they aredeployed [9].

6

Page 8: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

Irrespective of the specifics regarding implementation and deployment, IDS func-tion in a generic way. Input data from a system is collected and processed into amanageable format. The data items are classified as a threat or harmless. If a threatis detected, then a response is produced, usually in the form of an alert to the systemadministrator. A more detailed explanation of the process is as follows:

1. Data has to be captured, often in the form of IP packets.

2. The data are decoded and transformed into a uniform format, through the processof feature extraction.

3. The data are then analysed in a manner which is specific to the individual IDS,and classified as threatening or not.

4. Alerts are generated if and when a threatening pattern is encountered.

However, precautions must be taken to stealth this part of the system, so that anintruder can not spoof alerts (potentially leading to a denial of service attack). Varioustechniques are employed to produce correlations of the results; this can be done usingan automated system, or manually.

3.2.1 IDS Classification based on style of detection

Misuse Detection :

This type of IDS can also be called a signature recognition system. Misuse de-tection systems rely on the accurate matching of system or network activity [19].This method of detection is accurate for matching behaviour against a list of al-ready documented patterns, known as signatures. An example of this type of IDSis a system known as Snort [4]. The means by which snort functions involvesthe use of software component processing information regarding network con-nections. Snort examines the network traffic at its position on the network in apassive manner: it sniffs the network. Examination of the headers and content ofTCP packets is performed and matched against patterns contained in a signaturedatabase. If certain patterns of traffic are captured, then an alert is generated2.The use of only already known signatures means that the system will produceonly a few false positives, or false alarms where an alert is generated yet there isnot actual attack. There is a relatively high maintenance cost in that the signaturebase has to be kept up to date, else potential attacks could go unnoticed. Addi-tionally, this type of system can miss highly novel attacks to which a signaturedoes not yet exist, giving a higher rate of false negatives (where a real attack isnot detected) than would be desired. Missing an actual attack is probably worsethan being inundated with false alarms, though this is debatable.

2Let me pose a question: is this really an intrusion detection system, or is it a TCP pattern detectionsystem? This depends immensely on how you define an intrusion

7

Page 9: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

Snort is an open source IDS which implements a range of pattern matching algo-rithms of the input data and produces alerts based on the matching of the input toa signature base. For example it is likely that multiple port-scans on a particularcomponent would raise some sort of alarm. The advantage of the system beingopen source is that if a vulnerability is found, then it is likely to be posted ona user forum. The idea being that 1000 pairs of eyes are more likely to noticea vulnerability in the software than a select few hired ‘experts’. A recent ex-ample of this is a vulnerability found in the snort program itself, in which aninteger overflow was discovered in one of the stream processors responsible forthe calculation of the segment size for re-assembly. This could lead to a bufferoverflow which could turn into a denial of service attack on the system itself, oreven remote command execution of the host running the program (for examples,see [18]).

Anomaly Detection :

The goal of anomaly detection systems is to successfully classify user or networkbehaviour as normal or abnormal, based on a profile of information gathered dur-ing a training period. This is performed by taking into account the amount ofbackground noise or user variation which is intrinsic to the system. The charac-terisation of what constitutes ‘normal’ behaviour is certainly a non-trivial issue.There have been many approaches used in order to perform this classification, in-cluding statistical models, Markov chains, neural nets and ideas based on othermodern AI techniques (inclusive of artificial immune systems[3]). Normal be-haviour is profiled either from an individual user or from the network, variantsfrom this are defined as anomalies and alerts are generated. For example, a userof the example network ordinarily runs word processing applications and Inter-net browsers. If this user suddenly gains super-user privileges, starts changingfile permissions and sending broadcast SYN packets, then it is likely that theintegrity of the system is being compromised. A corresponding alert would begenerated and some form of action would be taken by the system administrator.

An example of this type of IDS is the experimental artificial immune system de-veloped by Somayaji et al[5]. This IDS resides on a host machine and examinesnumerous Unix system calls to construct a profile of normal behaviour over atraining period through examining the IP traffic in and out of the host machine.Once this period had ends (approximately two weeks was used for the trainingperiod), an insight into normal behaviour was used as the basis of the classifi-cation: if the observed behaviour deviates from the normal, then an anomaly isdetected. This causes the generation of a warning message which is sent to theuser. While anomaly detection is a relatively effective way of predicting novelattacks, they do not as yet feature in many commercially produced systems par-tially due to the high rate of false positives. Still, it remains a promising area ofresearch[].

One of the potential drawbacks with anomaly detection systems is the genera-

8

Page 10: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

tion of false positives. This could occur if the user behaviour suddenly changed;perhaps the user went on holiday! However, the change of behaviour caused bythis would be sufficiently different to the normal profile that an excessive amountof alerts could be generated. Additionally, user behaviour is dynamic, changingover time as the user needs change. As a consequence of this increased amount ofalerts, not only does it become irritating to the administrator, but it also becomesmore difficult to detect an actual attack. The amount of false positives can be re-duced using various methods, specific to the technique involved in the anomalydetection process. In the case of the system in Hofmeyer and Forrest [3], theamount of false positives were reduced through using a richer representation ofthe network traffic and through the finer tuning of several system parameters[2].

3.2.2 Classification through deployment

There are several places throughout a system where an IDS could be placed, includingon switches, router, even within programs themselves. Here are some specific detailsregarding where IDS are placed and how this affects their functions.

Network Based :

This type of IDS sniffs the traffic on the network by capturing packets of data(often IP data) and using them in the analysis. Data capture is performed at thenetwork switch level, so providing detection for traffic going in and out of multi-ple hosts. This method of deployment is popular for commercially available IDS[1] as they are relatively scalable so can be used for large scale networks: onlyone system is used to detect attacks covering many hosts. Additionally, the pres-ence of the ‘sniffing’ device on the network should operate in a stealthy mannermaking it difficult for malicious users to launch an attack on the IDS itself. TheIDS should not interfere with the end-users of the system thus providing a highdegree of transparency. As this type of IDS is passive i.e. does not have a di-rect effect on the system, it is relatively easy to apply to pre-existing networkswithout causing too much disruption. The methods used in these products canprovide a large amount of audit data so attack patterns can be studied retrospec-tively, and the security vulnerabilities of a system can be explored in a post-hocmanner.

However, it should be taken into account that if the network is subject to par-ticularly large amounts of traffic, then it would be difficult to detect an attackwith large amounts of ‘background noise’. The use of a token bucket filter [] inthis case would be preferable, but this could potentially slow down the network,eliminating such a degree of user transparency. It is also difficult to analysethe content of an IP packet if a method of encryption is used. This could be aproblem especially if virtual private networks form part of the system, as once aconnection has been established, the level of encryption used makes it difficult todetect suspicious behaviour. Additionally, the problem of packet fragmentation

9

Page 11: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

is often not overcome in this type of system, as it is difficult to piece together thefragmented packets in a way in which to both capture the necessary information,without increasing greatly the computational overheads. All of the above arenon-trivial issues, which may have to be resolved before such systems can reachthe effectiveness which they promise.

Host Based :

There are examples of systems that use a bottom up system of decentralised de-ployment based on a per host distribution. There are several advantages of usingone of these types of detection systems. Prominently, the analysis of the trafficand the impact of any disruption can be analysed with greater accuracy, with theinformation of exactly what is happening within the system becoming integral tothe alert generating process. Additionally, logs based on the host machine recordthe outcome of an attack, which can assist in the development of various counter-measures. The operation of such systems rely on the availability of system logswhich are used as an audit trail, often generated at the kernel level of the sys-tem. Compressing the data contained within these logs is difficult as it requiressignificant feature extraction of relevant information from a potentially data richsource. In addition to the wealth of data provided, a further advantage is thathost based systems can view encrypted malicious traffic that a network basedsystem would not be able to examine in detail [16]. Such systems often use userprofiling in a manner similar to anomaly detectors, for example the statisticalprofiling method as described in [19]. This vantage point can also be used to de-tect processes which should not be running in this manner, namely it can detectTrojan horse programs (programs which perform malicious operations, but poseas something non-threatening) based on the detection of unexpected behaviour.

However, the major disadvantage with the deployment of this type of system liesin the distributed nature of such a system. Scalability issues become a consider-ation. If a signature based system is implemented, then the signature data-basemust be kept up to date on every machine in the network. The high maintenancecost of this means that the situation of the database becoming obsolete is likely,and the machines would become increasingly vulnerable. An adaptive systemwhere each machine would adapt to the perils of the dynamic network environ-ment would reduce the maintenance of the network, and avoid the users of thesystem being directly involved in the protection of the system as a whole. But,combining transparency and autonomy into a system is difficult.

The computational resources for the host based systems are provided by the hostmachine. If the intrusion system consumes too many system resources and slowsthe system down to an unacceptable level, then the user may be inclined to switchoff the system. This also applies for excessive amounts of alerts caused by a sys-tem with too many false positives; the system would be rendered useless if theuser turned it off. In theory it is possible to disable the system by using a denial

10

Page 12: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

of service attack in the form of alert flooding, either causing the system to crashor the deactivation of the system because of the annoyance to the user. Finally,another major disadvantage is that the information from host based systems can-not be used in order to detect attacks on the network itself, so systemic port scanscould go undetected.

Application Based :

Application based IDS are a subset of host based systems. These systems analysethe behaviour of applications running on a host machine. They are specificallyused to detect unauthorised usage of an application within a system, and usethe information generated from the application logs in order to detect unusualbehaviour. They can also monitor systems using encryption as it runs on thehost machine. Unfortunately, such systems are relatively easy to attack throughprogram exploits or denial of service, as they run within applications themselves,or even embedded into an operating system. However, they are more effectivewhen used in combination with other types of IDS.

3.3 Anti-virus Scanners

Anti-virus (AV) scanners used in an attempt to directly protect systems from damage.AV scanners detect a specific type of unauthorised activity in the form of maliciousmobile code, collectively known as malware. The behaviour of these malware agentsvaries considerably, as does the resultant effect on the system. A relatively benignbut annoying virus could change a small feature of a program or system[11]; on theother hand, a maliciously designed Internet worm can bring the world of interconnectedcomputers to a standstill within a matter of hours. In order to really appreciate the roleof AV scanners in context, it is necessary to explore and explain the basic principles asto what scanners have to protect against3.

3.3.1 Malware in a minute

Malicious code is essentially a computer program that modifies a system call or thefunctioning of a program without the consent of the user of the system. Due to thesheer amount of unexpected ‘bugs’ that can occur due to program interactions, thereare a number of holes that can be, and indeed have been exploited. For example, theNetwork Associates virus glossary gives a definition of malware as “programs that areintentionally designed to perform some unauthorised act”[12]. As there are a plethoraof ways in which such malicious code can be written and deployed, various classifica-tion systems exist.

Computer viruses first emerged during the 1980’s and their main transmission vec-tor was shared floppy disks. The virus would often modify critical code within theboot sector of a machine rendering it useless, or would cause programs to crash. The

3Bearing in mind that examining computer viruses and attacks would be an essay worth in itself!

11

Page 13: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

term ‘virus’ was coined due to the similarity to biological viruses; both do not have thecapability to replicate on their own, and rely on using other cells/files on a host in orderto spread. The spread of viruses before computers were connected was relatively slow,due to the fact that the transmission was not over a scale free network, as the spreadhad to be via a physical floppy disk. Computer viruses in the conventional sense of theword are now less prevalent, due to two facts; people rarely use actual mediums (floppydisks, CD’s) to exchange information and booting from floppy disks is less common,with this option frequently disabled as a default.

However, the increasing interconnection of computers spawned a new transportvector for malware, and thus, many of the most disruptive pieces of malware are in theforms of computer worms. A worm can be defined as malicious code which is eitherfile infecting or not, which may or may not require user intervention, but propagatesthrough a network. Worms are a serious problem for organisations large and small,and cost billions of pounds in wasted time and resources. There are several aspectsto worm design and propagation. As with malware, loose classification schemes existfor worms, based on how they install themselves on a host machine and also how theypropagate through a network.

In general worms use Internet connectivity in the form of either e-mail, windowsfile sharing systems or through direct TCP/IP connections. However, such definitionsare not mutually exclusive; indeed, the Nimda worm utilised all three methods of dis-ruption and propagation. In the last five years, proliferation of worms through e-mailbased transmission has become a real problem. To illustrate this, the Virus Library [17]recorded one e-mail worm in 1998, 44 in 2000, but by the first half of 2003, 192 hadbeen reported. Such worms propagate through the network masquerading as an e-mailattachment, which the user downloads. The worm then replicates by sending itself toall addresses in the host machine’s address book. This often leads to a mass duplica-tion of the virus, slowing up both the host machine and mail servers In the case of suchworms as Nimda and SobigF, brought the Internet to a total standstill. The recoveryafter such events costs billions in both financial terms and in people’s time. This typeof worm required an element of social engineering as the actual e-mail message oftencontains a generic message, enticing the user to open the virus containing attachment,with frequently disastrous consequences.

Despite all the publicity and hype surrounding the dangers and damage caused byInternet worms, another persistent viral offender is a Trojan horse program. This typeof malware does not focus on using networking to spread, but is a malicious programdesigned to cause damage to computer systems, which masquerade as benign pro-grams. It is defined by the SANS institute [14]as “a computer program that appearsto have a useful function, but also has a hidden and potentially malicious function”.For example, someone within an organisation receives an e-mail attachment whichis a screen-saver, and this user decides to download this attachment and install it ontheir machine. However, this screen saver on execution infects the computer causingmalfunctions in a multitude of system processes. Unfortunately, the screen saver isamusing to the user, and so they forward this e-mail complete with its virus infected

12

Page 14: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

attachment to a series of their colleagues. It is only once the symptoms of this virusare noted that a system administrator is alerted to any danger, and by this time it couldbe too late. Trojan horse programs often cause minor malfunctions in applications:decimal point errors in spreadsheets or formatting issues in word processing software.A more serious security hole created by Trojan horse programs is known as a back-door. The Trojan program can convert a host machine into a ‘zombie’ machine for thepurposes of launching a distributed denial of service attack. If the program is run on amachine, it can create remote access to the machine for the creator of the virus. Thiscan greatly compromise data confidentiality and the integrity of a system.

3.3.2 Scanners: The Remedy

For an organisation to completely stop the proliferation of computer viruses, e-mailservices could be restricted to management staff only, and by prohibiting downloadingof attachments. However, this provides a severe impediment to modern business prac-tises and is obviously no solution. The most favourable method of protecting againstmalware is the installation of an anti-virus scanner. This software examines processesat the application layer of the network, and can also be run at both the level of theserver (to detect viruses that could infect servers) and the individual host machines.Anti-virus scanners are popular in the commercial sector, in a multitude of compa-nies, including F-secure[10] and Norton[13] who provide several products availablefor home and commercial use.

Such scanners acting on the user machines contain ( as with misuse detectors) asignature base containing pre-defined virus behaviour patterns which can include in-formation about what anomalies to examine in terms of system calls or the presence offiles with certain extensions. There are two points at which an anti-virus scanner is runon a host machine: on the commencement of downloading an attachment, and whenthe computer is booted. Efficient pattern matching in terms of computational resourcesis required in order to provide any protection, as if the virus scanner was to slow downthe system processes sufficiently, then the user could be tempted to turn off the soft-ware. On discovery of a virus, the user or administrator is informed and the anti-virusvendors often provide a virtual antidote in the form of a patch to aid in fixing any dam-age caused by the virus. It is worth bearing in mind that the more publicised virusesof late have not caused unsurmountable damage to the individual machines itself, buthave often been used to create distributed denial of service attacks on large corpora-tions including the Microsoft web site. However, as with misuse IDS, the protectionfrom such viruses is only effective providing that the signature base is kept up to datewith the latest update. Updates can become so voluminous that individual users do nothave the time, patience (or in the case of those still running dial-up connections) theresources to keep constantly updating virus definitions.

There are several ways in which the signature bases can be updated across thenetwork, but there are various ethical and practical considerations. Firstly, the mostobvious method involves the user downloading updates and patches. The problem with

13

Page 15: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

this is that it is not scalable in large organisations, and it is difficult to get everyone to beresponsible for this. This is more suitable for very small networks, and would ensurea higher level of protection. An alternative method would include the installation ofnew signatures and patches via a network administrator. Again, the AV-scanner wouldonly be effective if this was done on a regular basis, but is likely to be more reliable asupdates would be performed as a matter of due course, and any potential problems thatcould be caused via software interaction are likely to be noticed.

The most effective form of updates would come directly and automatically fromthe vendor. However, there are obvious privacy issues that can arise because of this: the vendor would need some form of access to the network. There are two foresee-able problems: certain individuals at the vendor end abusing the trust and using theaddition of a patch to open up a back-door, compromising the security of the networkwithout the administrator being aware; and the addition of a patch or a new signaturecausing an unexpected error. On the face of it this does not seem like too much of aproblem, however, if this was on a critical system such as a air traffic control system ormedical system, then the consequences could be disastrous. It is true that if the vendorwas responsible for providing and making the updates, virus incidents would be lesscommon, however, it is still seen to be the responsibility of the individual user or theirorganisation. This is similar to the problem faced by the administrators of host basedintrusion detection systems.

4 Comparatively Speaking

Superficially, IDS, firewalls and anti-virus scanners perform similar roles, though oncloser inspection differences become apparent. When viewed in terms of deployment,the systems appear to have similarities. Firewalls can be implemented on a network atmultiple layers, or in the case of personal firewalls, examining network connections ona host machine. This type of deployment can be seen in both IDS - the network IDSSnort and the host based Tripwire; and anti-virus products - for use on mail servers asfilters and for use on host machines checking system calls. Additionally, in all cases,a database of already known patterns of misbehaviour, represented as a rule set can beused to specify what is permitted and what is not, as stated in the security policy. Thisis seen in many different types of firewall, misuse detecting IDS and most anti-virusscanners. It really does beg the question then, “why are these systems classified as dif-ferent types of security measure when they essentially perform similar, if not the same,function?”

The differences lie in two main aspects: what the component is looking for as aviolation and how the component responds to the detection of a violation.

Vigilance :- Firewalls are implemented to prevent connections being mad and packetsbeing transmitted on violation of the rules laid out by the security policy. In-trusion detection systems are looking for anomalous system/network behaviour,through the examination of communication mediums and system calls, be it us-ing a pre-defined pattern base or profiling mechanism. Anti-virus scanners are

14

Page 16: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

looking for the presence of pre-defined files or the execution of system com-mands which are known to cause problems.

Response :- The common response of a firewall is to deny a connection or to dropa packet which would be seen to be against the security policy, without excep-tion. The action of an anti-virus product would involve quarantining ‘infected’ ormodified files and producing an notification message to the user, informing themof the problem. The same applies for anti-virus systems applied at a mail server:if a mail message contains a suspected virus, then the recipient is warned and themessage is quarantined. More passively still, the common response from an in-trusion detection system is to notify the administrator of a system of a suspectedintrusion.

Of course there are exceptions to all of the above. The distinctions between thedifferent components is not entirely clear cut: a well implemented intrusion detectionsystem should, in theory, be able to detect the action of a computer virus too. The threecomponents, when used in conjunction, form a spectrum of overlapping function. Aseach of the components is developed with different constraints in mind, the use of allthree in combination, with due care and attention, provide a higher level of securitythan treating only one component as the security solution.

5 Summary

This article has concentrated on exploring and explaining three countermeasures whichare used to improve the security of networked computers. The basic concept of a net-work and the need for an effective security policy, was introduced. The different typesof firewall, intrusion detection system and anti-virus scanner, where they are deployedand their functions and respective behaviour was discussed, along with some exam-ples of intrusion and attack to which they are used to protect against. The differencesbetween these systems is not as clear cut as was first thought: indeed there is someoverlap in the functioning of all these systems. Yet, they are sufficiently different intheir mechanism of action and their response, to warrant being treated as separate com-ponents. The correct implementation, deployment and configuration of all of thesesystems form some of the most effective measures that are available in the battle forthe defence of computer systems.

References

[1] Rebecca Bace and Peter Mell. Intrusion detection systems. NIST Special Publi-cation on Intrusion Detection System.

[2] J Balthrop, F Esponda, S Forrest, and M Glickman. Coverage and generaliszationin an artificial immune system. Proceedings of GECCO, pages 3–10, 2002.

[3] S Hofmeyr and S Forrest. Immunity by design. Proceedings of GECCO, pages1289–1296, 1999.

15

Page 17: Firewalls, Intrusion Detection Systems and Anti-Virus … · Firewalls, Intrusion Detection and Anti ... The integrity refers to the correct running of the ... In this case is important

[4] Martin Roesch. Snort: Lightweight intrusion detection for networks. In Proceed-ings of the 13th Conference on Systems Administration, pages 229–238. USENIXAssociation, 1999.

[5] A Somayaji, S Forrest, S Hofmeyr, and T Longstaff. A sense of self for unixprocesses. IEEE Symposium on Security and Privacy, pages 120–128, 1996.

[6] Judy Novak Stephen Northcutt. Network Intrusion Detection. New Riders, 3rdedition edition, 2003.

[7] Andrew S Tanenbaum. Distributed Systems: Principles and Paradigms. PrenticeHall, 2002.

[8] Andrew S Tanenbaum. Computer Networks. Prentice Hall, 4th edition edition,2003.

[9] H Venter and J Eloff. A taxonomy for information security technologies. Com-puters and Security, 22(4):299–307, 2003.

[10] www.fsecure.com/.

[11] www.fsecure.com/v descs/nuclear.shtml.

[12] www.nai.com/.

[13] www.norton.com/.

[14] www.sans.org.

[15] www.securityfocus.com/infocus/1737.

[16] www.tripwire.com.

[17] www.viruslibrary.com/virusinfo.

[18] www.whitehats.com.

[19] Nong Ye, Xiangyang Li, Qiang Chen, Syed Masum Emran, and Mingming Xu.Probabilistic techniques for intrusion detection based on computer audit data. InIEEE Transactions on systems, man and cybernetics- part A, systems and humans,volume 31:4, pages 266–274, 2001.

16