Honours Project Report

Collection and Aggregation ofSecurity Events

Author:Henk Joubert

Supervisors:Dr. Andrew Hutchison

Dr. Michelle Kuttel

Category Min Max Chosen

1 Requirements Analysis and Design 0 20 15

2 Theoretical Analysis 0 25 0

3 Experiment Design and Execution 0 20 5

4 System Development and Implementation 0 15 10

5 Results Findings and Conclusion 10 20 15

6 Aim Formulation and Background Work 10 15 15

7 Quality of Report Writing and Presentation 10 10

8 Adherence to Project Proposal and Qualityof Deliverables

10 10

9 Overall General Project Evaluation 0 10 0

Total 80 80

University of Cape Town

Department of Computer Science

November 3, 2012

The financial assistance of the National Research Foundation (NRF)towards this research is hereby acknowledged. Opinions expressed and

conclusions arrived at, are those of the author and are not necessarily tobe attributed to the NRF.

Abstract

Monitoring systems are required for managing computer networksof any nontrivial size. It is difficult to provide real time reporting andvisualisation that scales to large networks, as a result there are fewsolutions in this area.

This project presents anomalous, a Security Information and EventManagement system. The component of that system described here isthe Collection component, which collects, aggregates and stores datafor use by the other Intrusion Detection and Visualisation components.

i

Acknowledgements

Thank you to my supervisors, Dr. Michelle Kuttel and Dr. Andrew Hutchi-son, for advice that made it possible not only to carry out the research andthe writing of this report.

Thank you to Amazon.com for sponsoring credit on Amazon Web Services.Thank you to GitHub for sponsoring a student account.

Thank you to 2go Interactive, for providing a quiet place to work, and inparticular, Alan Wolff for sharing insight into designing/configuring systemsfor scalability.

Thank you to Blythe Watson for the support without which this paperwould not exist.

ii

Contents

1 Introduction 1

2 Background 22.1 IDS Technology . . . . . . . . . . . . . . . . . . . . . . . . . . 22.2 Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3 Design 53.1 Design Aims . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53.2 Design Approach . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.2.1 Collection . . . . . . . . . . . . . . . . . . . . . . . . . 73.2.2 Normalisation . . . . . . . . . . . . . . . . . . . . . . . 73.2.3 Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2.4 Database Storage . . . . . . . . . . . . . . . . . . . . . 8

4 Implementation 104.1 Description of the syslog service . . . . . . . . . . . . . . . . . 104.2 Challenges in Implementation . . . . . . . . . . . . . . . . . . 10

4.2.1 Network Intrusion Detection . . . . . . . . . . . . . . 104.3 Collecting Log Data . . . . . . . . . . . . . . . . . . . . . . . 11

4.3.1 Firewalls . . . . . . . . . . . . . . . . . . . . . . . . . 12

5 Evaluation and Results 145.1 Event Frequency . . . . . . . . . . . . . . . . . . . . . . . . . 145.2 Testing Methodology . . . . . . . . . . . . . . . . . . . . . . . 14

5.2.1 Test Hardware . . . . . . . . . . . . . . . . . . . . . . 145.3 Processing Time Per Event . . . . . . . . . . . . . . . . . . . 155.4 Storage Requirements . . . . . . . . . . . . . . . . . . . . . . 15

5.4.1 Database storage requirements . . . . . . . . . . . . . 15

6 Conclusion 186.1 Possible Disk Latency . . . . . . . . . . . . . . . . . . . . . . 186.2 Threading and Database Locking . . . . . . . . . . . . . . . . 186.3 Component Integration . . . . . . . . . . . . . . . . . . . . . 18

7 Future Work 207.1 Component Integration . . . . . . . . . . . . . . . . . . . . . 20

A Appendix 23A.1 Intrusion Detection Using Neural Networks and Support Vec-

tor Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

iii

List of Figures

1 A high level overview of component relations . . . . . . . . . 12 Overview of the collection system . . . . . . . . . . . . . . . . 63 MongoDB architechure . . . . . . . . . . . . . . . . . . . . . . 94 Database Benchmarks . . . . . . . . . . . . . . . . . . . . . . 16

List of Tables

1 Comparison of IDS types . . . . . . . . . . . . . . . . . . . . 32 Syslog protocol severity levels . . . . . . . . . . . . . . . . . 73 Syslog facilities . . . . . . . . . . . . . . . . . . . . . . . . . . 134 The features table . . . . . . . . . . . . . . . . . . . . . . . . 23

iv

1 Introduction

Computer networks in organisations are growing larger in size, to the pointwhere it is infeasible for system administrators to manage them effectivelywith current tools. Monitoring systems allow administrators to identifyanomalies and investigate potential security issues. These monitoring sys-tems involve collecting data such as system metrics or log files, storing thedata for later retrieval, applying some transformation or processing to detectanomalies, and ultimately representing the results in some way.

Figure 1: A high level overview of component relations

This report describes anomalous, an approach to a scalable monitoringsystem. The design was divided into 3 components, Collection, IntrusionDetection and Visualisation shown in figure 1. The Collection componentdescribed in this report, is responsible for gathering data from an installa-tion on a network and making it accessible to the other components. TheIntrusion Detection component uses a genetic algorithm to detect anomaliesin the input data. The Visualisation component represents information fromboth components providing an overview and details on demand.

We designed a monitoring system and implemented a proof of concept.The implementation was then tested to make recommendations for futureattempts in this area.

1

2 Background

As the number of distributed systems continues to increase, better tools arerequired to secure these systems. There is a lot of recent work in resourcemonitoring of distributed systems, and it is natural that this work can beextended to securing large distributed systems.

Scaling a monitoring system to thousands of nodes presents the challengeof effectively aggregating data from each node. Adding significant trafficto the monitored network would likely result in degraded performance ofother applications. There have been sevaral recent attempts at buildingmonitoring and response consoles for distributed environments, but thesesolutions are still maturing[8].

2.1 IDS Technology

There are 2 main classification schemes for intrusion detection systems. Thefirst scheme by the NIST[15] lists 4 main types:

Network based, which monitors network, transport and applicationprotocols for suspicious activity;

Network behaviour analysis, which examines network traffic statisticsto identify unusual traffic patterns;

Wireless, which monitors and analyzes wireless protocols;

Host based, which monitors characteristics of and events occurringwithin a single host.

The alternative classifies IDS according to how they detect incidents:

signature based;

anomaly based;

self learning[1];

stateful protocol analysis[16].

There are open source Host-based (OSSEC) and Network-based (snort)Intrusion Detection Systems as well as frameworks such as PRELUDE thatenable security applications to to report to a central system. Snort hasbeen downloaded millions of times and is used by large corporations andgovernments to protect their networks[16].

It should be possible to link various intrusion detection systems together,using the intrusion detection message exchange format (IDMEF)[4, 5], whichis xml based, for communication. Simple Object Access Protocol (SOAP)is another way to communicate between nodes but some overheads must beaccepted for the sake of interoperability [14].

2

IDS type Malicious activity detected Strength

Network-Based

Network, transport, and ap-plication layer activity

Widest range of applicationprotocols

Wireless Unauthorized wireless localarea network use

Only IDS that can monitorwireless activity

NetworkBehaviourAnalysis

Network, transport, and ap-plication TCP/IP layer activ-ity that causes anomalous net-work flows

More effective at identifyingreconnaissance scans and DoSattacks than others

Host-Based Host application and OS ac-tivity; network, transport,and application layer activity

Can analyse traffic that wastransmitted over end-to-endencrypted links

Table 1: Comparison of IDS types

2.2 Scaling

Distributed Intrusion Detection Systems may be applied to detect and pos-sibly respond to intrusions based on collected data. For such a system torespond in real time depends on the efficiency of the aggregation and analysisof data. A hierarchical approach is taken by papers in distributed intrusiondetection [6], network monitoring [7] as well as general monitoring systems[11, 17]. Grzech [6] recommends implementing a three layered hierarchywith the bottom layer collecting local monitoring data for analysis by themiddle layer nodes, and correlation at the top level. Work by Debar andWespi[5] suggests probes at the bottom of a tree structure with as manylevels of filtering as required for scalability.

Grid resource monitoring faces similar problems at scale. Cai et al.[3]propose a peer to peer (P2P) approach to avoid having a single point of fail-ure, however Roschke et al.[13] state that a centralized approach is requiredfor deploying intrusion detection in the cloud.

Lee et al.[10] proposes a data mining framework to improve upon currentIDS extensibility and adaptability. Data mining algorithms can be appliedto the audit data to more efficiently analyze system behaviour from multiplesources.

Intrusion detection management systems need to be highly scalable todeal with the vast amounts of data that can be collected from a large numberof nodes. It is most effective to use a combination of different IDS types, inparticular network based is the most versatile, but wireless and host basedIDSs are still required for comprehensive monitoring of threats.

Security Event and Information Management (SIEM)[9] systems are de-signed to extract information from various security related logs, normalizeand then correlate data. SIEM has the advantage over IDS that it can corre-

3

late data between different systems and attain higher accuracy in reportedevents, however processing log files incurs additional latency that can beavoided with IDS integration. A SIEM system can be incorporated in ahybrid approach, where IDSs report to a representative node on the localnetwork that then creates logs for analysis by the SIEM system.

4

3 Design

The log collection and aggregation system is designed as a core component ofour SIEM system, Anomalous. Log files can provide a wealth of informationabout system activity to an administrator attempting to diagnose problemswith the system. When a system is compromised, if someone manages togain unauthorized access, the attacker can delete the system log directoryto prevent or impede identification of the breach. Using a remote server tocollect and store log data means that another machine must be compromisedfor an attacker to hide his actions in the network. A log server can be moreeasily secured than other servers, since it does not need to be accessibleoutside of the local network or on ports other than those explicitly used forlogging or remote access.

3.1 Design Aims

The main objective is to create a platform that will allow monitoring ofvarious services in an extensible manner. This platform can be used to in-vestigate whether the benefit of centralising analysis of a large set of securityevent data can offset the cost in terms of network overhead, or how muchdata can be logged without adversely affecting network performance.

It should be easy to manage which events are collected from a centrallocation, without having to connect to each individual machine and adjustthe configuration. It will also be useful to query which log files exist onendpoints that are not currently being reported.

There are existing solutions for centralizing log collection in a network,but these are not easy to scale to large networks. It is relatively easy toconfigure a syslog daemon to forward logs to a server over UDP. However,after deployment, there is no central way to change which events are sentover the network. This is problematic when trying to scale the log collectionof a network that is reaching capacity by disabling more verbose logging.Any process can utilize the syslog daemon, which can create a large amountof log data that would need to be transmitted over the network. Exactlywhich files will be logged remotely must be decided each machine is initiallyconfigured, and will seldom change during the life as a result.

It is good practice to use a system with as few (user) permissions aspossible, this can prevent accidental damage to the system because mostdangerous actions will require additional permissions. A mistyped com-mand can be as dangerous as executing malicious code, but will usually failunless executed as a superuser or even a user with the correct additionalpermissions. It is almost always important to log privilege escalations, thatis when normal users attempt to run commands as another user which hasadditional permissions, often that user will be root which has unrestrictedaccess to system. Access to the root account should be restricted as much

5

as possible for this reason, additionally users should never be allowed to login directly as root since that scenario does not provide any accountability.

3.2 Design Approach

The proposed design is separated into 3 layers, which are the event layer,aggregation layer, and reporting layer. The event layer comprises of clientswith the event monitoring system installed. Each client communicates withan aggregation server, sending events and possibly receiving changes to it’sconfiguration. The ability to remotely modify the configuration of one ormore clients at the aggregation layer will allow administrators to more ef-fectively tune the level at which events are reported. When investigating anissue with a particular service – a kernel related issue for example – thenmore verbose messages can be reported, providing more information on theaffected service.

Raw events originating from the event layer will be normalised at the ag-gregation layer for use by the Visualisation and Anomaly Detection systemsat the reporting layer.

This project deals with the aggregation layer, focussing on the collectionand storage of event information.

Figure 2: Overview of the collection system

6

3.2.1 Collection

The collection subsystem will listen for events from clients configured to re-port to the server. These events will then be placed on a queue for processingby the normalisation and filtering subsystem.

3.2.2 Normalisation

The events reported by syslog have a common prefix format, explained in4.1, but the messages reported by processes are not required to conform toany standard. Firewall event logs will usually contain source or destinationIP addresses, but it cannot be expected that different firewalls will reportevents in the same structured format. This problem does not only existbetween different programs providing the same functionality, but also whendifferent operating systems are present on the same monitored network.

3.2.3 Filtering

The syslog protocol1 defines 8 severity levels shown in Table 2. The severitylevel of an event is decided by the application reporting the event. Thesyslog configuration file specifies which security levels are recorded for eachprocess (referred to as a facility in the RFC) and which file they are writtento.

Debug and information level messages are the least urgent messages thatcan be reported and should not need to be collected under normal circum-stances. The number of such messages can however be a useful metric forthe purposes of anomaly detection. If clients store a short history of theseunreported messages, and only report the number of debug messages perhour for example, it will allow trends in the usage pattern of clients to beobserved with minimal communication overhead.

Code Severity Interpretation

0 Emergency system is unusable1 Alert action must be taken immediately2 Critical critical conditions3 Error error conditions4 Warning warning conditions5 Notice normal but significant condition6 Informational informational messages7 Debug debug-level messages

Table 2: Syslog protocol severity levels

1tools.ietf.org/html/rfc5424

7

3.2.4 Database Storage

MongoDB2 is a NoSQL database well suited for log storage or archival.There are several differences between MongoDB and a relational database:

Tables are referred to as collections.

Data is stored in documents as opposed to rows. A document is aBSON (Binary JavaScript Object Notation) encoded key-value store.

MongoDB is schemaless, the only required field in a document is id,and documents in a collection do not need to have the same fieldsdefined. This is ideal because while the events have a similar structure,there are variations in the message fields. With mongodb space is notwasted to store a NULL value in that column as with SQL.

MongoDB does not support joins on collections so achieving the sameresult requires additional program logic.

MongoDB capped collections have a maximum size set, which is usefulto maintain a reasonable amount of log history without the need to worryabout disk usage. Capped collections can also be used to mitigate denial ofservice attacks – attempts to flood the log server with events – against thelog collection system.

When log traffic on network grows beyond what a single log server canhandle, the database can be sharded – by specifying keys used to deter-mine which mongod server a document will be stored known as shard keys–however the current version does not support sharding capped collections.When a database is sharded, requests received by the mongos process arerouted to a mongod instance based on the shard key.

Figure 3 shows the MongoDB architecture. The mongos process handlesrequests by routing them to the appropriate data mongod (also called ashard) server where queries are executed. The config mongod instance storesthe meta data that that mongos uses to determine which data resides onwhich mongod shard. The mongos process caches config information, forperformance reasons, and also because a failure to read the configurationdata would cause all requests to fail.

2http://www.mongodb.org/

8

Figure 3: MongoDB architechure

9

4 Implementation

4.1 Description of the syslog service

There are 3 main implementations of the syslog service:

syslog-ng, version licenced under GPL but premium edition is propri-etary, maintained by BalaBit IT Security,

sysklogd, licenced under GPL, last updated in 2007,

rsyslog, fork of sysklogd licenced under GPL, maintained by RainerGerhards.

It was decided to test using rsyslog since it is the default syslog installationon Ubuntu 12.04 (the latest long term support release) and is backwardscompatible with sysklogd.

The configuration file can also specify a remote server using the followingsyntax:

facility.severity_level @log.server.domain

The client and server require the following lines appearing uncommented inthe /etc/rsyslog.conf file:

$modload imudp

$UDPServerRun port_number

Note that port_number is 514 by default but it can be changed to any validport provided that it is consistent between the server and client sides.

Syslog generates messages in the following format and writes them toone or more files depending on rules specified in the configuration file. If norule is defined for that facility or if the severity level is set to none, then nofiles are written to. The messages are written to file in the following format:

month day time hostname service[PID]: message_text

4.2 Challenges in Implementation

4.2.1 Network Intrusion Detection

The initial design proposed collecting network traffic data for the visuali-sation and anomaly detection components. The network data would allowthe visualisation system to draw links between machines, for the purpose oftesting that aspect of its design.

Feasibility testing used snort, an open source network intrusion detec-tion and prevention system, to gather network data. The program can run

10

as a daemon in packet logger mode or NIDS mode. In packet logger mode,it logs every network packet – headers only or payload as well – on a speci-fied network interface for later analysis. In NIDS mode a configuration file,specifying dynamically loaded preprocessor modules and rule sets to analysenetwork traffic for potential malicious activity, is required.

However, during testing, snort could not be started as a system dae-mon (the system ensures that it is running) in NIDS mode. The programproduced errors indicating that it expected fields that were defined in theconfiguration file to be defined despite trying various program and operatingsystem versions. This implied that the configuration file was not being cor-rectly parsed. Attempting to use the default configuration that is suppliedwith the snort installation did not work correctly.

Although snort works to detect malicious activity which is what theanomaly detection system is being designed for, it could be useful in demon-strating aspects of the visualisation component that require network data.There is also no reason why the two systems cannot operate together withanomalous performing a high level evaluation of the events reported bysnort.

4.3 Collecting Log Data

Due to time constraints and modifications to the design the client setup doesnot support modification by the aggregation server. Clients are configuredto report to a rsyslog server running on Amazon EC2. The server has aPython script running that reads the created log files, parses the messagesto extract structured data and uses the pymongo driver to perform databaseinsertions.

The database scheme used only a single collection to store events, thiscould easily be extended to a collection for every monitored facility. Theavailable facilities vary between operating systems and syslog implementa-tions and can usually not be changed without modifying the syslog daemonsource code and compiling a custom version, which is not ideal for keep-ing software up to date on operating systems that distribute pre compiledbinaries such as Ubuntu. However it is worth noting that Apple’s implemen-tation of syslog3 distributed with their operating systems allows applicationsto specify the facility they wish to log to.

A more general solution will have to rely on the facilities specified in theRFC document, reproduced in table 3. Notice that 4 (keyword auth) and10 (keyword authpriv) are both used for authorization messages with littledistinction between the two. The local facilities are meant for administratoror application use. Cisco routers configured for syslog use either local7 orlocal4 by default.

3https://developer.apple.com/library/mac/#documentation/Darwin/Reference/

ManPages/man3/asl.3.html#

11

https://developer.apple.com/ library/mac/#documentation/ Darwin/Reference/ManPages/ man3/asl.3.html#https://developer.apple.com/ library/mac/#documentation/ Darwin/Reference/ManPages/ man3/asl.3.html#

4.3.1 Firewalls

There is also no facility defined for firewalls. UFW (Uncomplicated Fire-wall) is a front end for iptables to configure the kernel firewall, and can beconfigured to log messages using syslog. These messages are usually writtento the /var/log/syslog file and contain the string "[UFW" in the messagetext field. This seems to be common strategy used by applications thatare not assigned to a facility. These tags can then be used by the programreading the log files after syslog has processed them to filter messages todifferent files.

The rsyslog implementation has support for creating filters in the con-figuration file based on name of the program logging each message as in thefollowing example:

if $programname == 'firewall' and $syslogseverity

Numerical Code Facility

0 kernel messages1 user-level messages2 mail system3 system daemons4 security/authorization messages5 messages generated internally by syslogd6 line printer subsystem7 network news subsystem8 UUCP subsystem9 clock daemon

10 security/authorization messages11 FTP daemon12 NTP subsystem13 log audit14 log alert15 clock daemon (also used by cron)16 local use 0 (local0)17 local use 1 (local1)18 local use 2 (local2)19 local use 3 (local3)20 local use 4 (local4)21 local use 5 (local5)22 local use 6 (local6)23 local use 7 (local7)

Table 3: Syslog facilities

13

5 Evaluation and Results

5.1 Event Frequency

Using data aquired from the ADEWaS project, funded by Deutsche TelekomAG, over a 3 month period in 2011 from 215 different workstations. Overa 24 hour period 74000 events were collected by Tivoli Security OperationsManager which equates to 0.004 events per second (EPS) per workstation.A benchmark by Butler [2] finds an average of 0.05 EPS assuming a 750employee organisation. However average EPS is only one metric that needsto be considered because system load is never uniformly distributed. Butlerfinds the 750 workstations and their domain controllers producing 404 EPSat peak which is roughly 0.5 EPS per workstation. This metric is usefulin determining how many workstations a log server can support based onserver benchmarks.

5.2 Testing Methodology

Obtaining hundreds of machines for testing is infeasible for this project, butas far as collection is concerned, the content of each event does not signifi-cantly affect the time taken to process each event. A set of events recordedby syslog was taken from a real machine and used to create a sample logfile that can be used to test the collection system. While then collection sys-tem should ideally be receiving these events over a network socket, readingthem from file allowed a much simpler parser to be implemented. Readingevents from a file in a simulated environment also allows different databasesto benchmarked with identical data. The script used to generate the test filecan also serve to simulate several systems sending events to a syslog serverand writing them to a file in real time.

Events are processed extremely fast so 1000000 fake events were gener-ated and written the test log file. The parsing script was executed in batchmode instead of continuously reading the file.

5.2.1 Test Hardware

Tests were conducted on a desktop computer with an Intel Core i7 950 CPUclocked at 3.20GHz. The file system being used was ext4 on a 2TB 7200rpmSATA2 hard disk. Memory was 6GB of DDR2 RAM clocked at 1066MHzrunning in triple channel. This is a fairly high end desktop machine, com-parable to a low end server. Servers would have faster hard disks configuredin RAID for better performance, however this did not seem to influence theresults as noted in section 6.1.

14

5.3 Processing Time Per Event

The initial design proposed a document oriented NoSQL datastore such asMongoDB, due to the advantages over relational databases. However it wasnecessary to investigate whether these advantages were worth pursuing. Animplementation with MongoDB replaced by MySQL was created. In termsof implementation complexity, connecting to a database and performing in-sertions was easier using MongoDb and the pymongo driver than mysql andthe MySQLdb driver. InnoDB was used as the storage engine in mysqlwith the configuration option flush_log_at_trx_commit set to 2, sacrific-ing ACID compliance for speed in order to achieve a closer comparison withMongoDB that does not attempt to guarantee ACID compliance.

The script was executed 10 times with each database implementation,clearing the table or collection in between executions, and recording tim-ing information. By performing a million insertions, the total run time inseconds is also the average time taken to insert one document or row inmicroseconds. Figure 4 shows a plot of the execution times, with the aver-age for MongoDB being 79.568 and mysql being 277.578 microseconds perinsertion. The data is represented in separate graphs due to the differenceWhile performing the MongoDB test the system consistently ran at 99%CPU usage while the mysql test had CPU usage in the range of 44% to55%. This includes time taken to parse the string into a structured for-mat, extracting the timestamp, host name, process and message fields ofthe event. This does mean that events are processed twice, however this isunavoidable without reworking the syslog implementation and creating yetanother fork of the project.

5.4 Storage Requirements

logrotate is a unix tool designed to ease administration of systems thatgenerate large numbers of log files allowing automatic rotation, compression,removal, and mailing of log files. This is the standard unix tool for dealingwith log files.

Uncompressed events are typically 100 to 200 bytes and the ones usedin the benchmark test averaged 158 bytes. A one terabyte disk should beable to store up to 10 000 million such uncompressed events.

5.4.1 Database storage requirements

The log file used for testing was 88 MiB in size containing 1000000 lines.Running the stats() command in the mongo shell revealed that storing theevent data used 149.39 MiB of storage space. Inspecting the mysql tablewith phpMyAdmin revealed that only 120.2 MiB was being used. Thisincludes space used to index the automatically generated primary key fieldsin both databases.

15

Figure 4: Database Benchmarks

16

It is interesting to note that MongoDB used the most space to store theevent data. One reason for this is that the full field names are repeated foreach document in the collection. This means that every event contains thestrings “ id”, “timestamp”, “host”, “service”, “pid” and “message” whenshorter versions would suffice. There is a closed ticket on GitHub4 adressingthe issue, but the documentation still recommends using short field names5

hence it is not clear if this has been resolved. This additional space usageis the trade-off for a schemaless design.

4 https://github.com/hmarr/mongoengine/issues/4835 http://www.mongodb.org/display/DOCS/Optimizing+Storage+of+Small+Objects

17

https://github.com/hmarr/mongoengine/issues/483http://www.mongodb.org/display/DOCS/Optimizing+Storage+of+Small+Objects

6 Conclusion

6.1 Possible Disk Latency

While MongoDB was expected to be faster than mysql, the difference waslarger than expected. MongoDB attempts to keep as much of the working setin memory as possible, so writes to MongoDB could succeed before they werestored on disk. The lower CPU usage relative to MongoDB indicated thatdisk access was a possible bottleneck for the mysql implementation. In anattempt to verify that this was the case, the mysql experiment was repeated,but with the database stored on a tmpfs mount, that is temporary file systemmounted in RAM. If disk access was the bottleneck then execution timewould be expected to be lower and CPU usage increased. This was notthe case and both parameters were within statistical variance of the originalexperiment.

This led to consultation with more experienced mysql database adminis-trators, subsequently revealing that apparmor, which is installed by defaulton Ubuntu, could be interfering when mysqld attempts to write outside of/var/lib/mysql. A final test was performed where the storage engine waschanged from InnoDB to Blackhole. The Blackhole engine processes requestsand then discards them, but still writes to the binary log. This resulted ina slightly lower execution time, but still above 240 seconds and significantlyslower than MongoDB.

6.2 Threading and Database Locking

The benchmark tests were executed with a single thread for the sake of sim-plicity. The rsyslog pipeline supports multiple input and output threads,but with some serial processing occurring in between. Under the reason-able assumption that there will be concurrent writes, the throughput to thedatabase will be affected by the storage engine’s locking implementation.Mysql’s InnoDB already provides row-level locking as opposed to table lock-ing which is the lowest level the MyISAM engine can achieve. MongoDB hasrecently, in release version 2.26, removed global (process level) locking andimplemented database level locking, with the intention of eventually sup-porting collection level locking in a future release. The release supportingcollection locking should theoretically support a higher throughput in ouruse case.

6.3 Component Integration

During the implementation phase, failed experimentation with Network-based IDS led to the development of a system for collecting Host-based

6 http://docs.mongodb.org/manual/release-notes/2.2/

18

http://docs.mongodb.org/manual/release-notes/2.2/

event data. The visualisation component of anomalous which requires dataon network features such as IP addresses.

The intrusion detection component required collecting features that wereonly defined towards the end of the project based on work by Mukkamala[12].These features are reproduced in table 4, and include parameters such asnumber of failed logins, open connections, open shells, and outbound com-mands for example. The creation of a system to monitor these features isnot a trivial task.

The combination of the aforementioned feature monitoring component,integrated with the collection system described in this report, backed by theanomaly detection and visualisation components, has potential as an indus-try viable SIEM solution. The entire design is platform agnostic whereasthe implementation requires rsyslog which is available free on linux but ata cost on Windows.

19

7 Future Work

It would be worth investigating the possibility of adding modules to rsyslogto allow updates to the configuration remotely and create filters that willallow events to be written to MongoDB collections.

A user interface for configuring syslog remotely still needs to be devel-oped. The current solution requires configuration files to be modified on eachhost. If syslog could be modified to query a database to retrieve configura-tion settings at startup, it will be possible simplify the process of updatingconfigurations for entire networks. The configuration files can become toocomplex to modify by a short database query without some abstraction builtinto the interface.

7.1 Component Integration

The feature collection component referred to in section 6.3 still needs to bedeveloped before it is possible to combine all the components of anomalousinto a complete SIEM solution. Once the integration of all components iscomplete, anomalous can be tested with real world data, provided that theethical and legal issues are addressed. This would likely require deploymentin a simulated environment running on EC2 for further analysis.

Once anomalous is running the database can be profiled in order optimizethe system for scalability. The profiling data should reveal which keys arequeried most frequently and thus which secondary indexes should be created.This information will also allow effective shard keys to be chosen so thatqueries will be distributed evenly between mongod instances.

20

References

[1] Axelsson, S. Intrusion detection systems: A survey and taxonomy.Tech. rep., Chalmers University of Technology, 2000.

[2] Butler, J. Benchmarking security information event management.Tech. rep., SANS, 2 2009.

[3] Cai, M., and Hwang, K. Distributed aggregation algorithms withload-balancing for scalable grid resource monitoring. In Parallel andDistributed Processing Symposium, 2007. IPDPS 2007. IEEE Interna-tional (2007), IEEE, pp. 1–10.

[4] Debar, H., Curry, D., and Feinstein, B. The intrusion detectionmessage exchange format. Tech. rep., IETF, 3 2007.

[5] Debar, H., and Wespi, A. Aggregation and correlation of intrusion-detection alerts. In Proceedings of the 4th International Symposiumon Recent Advances in Intrusion Detection (London, UK, UK, 2001),RAID ’00, Springer-Verlag, pp. 85–103.

[6] Grzech, A. P. Optimal monitoring system for a distributed intrusiondetection system.(report). Artificial Life and Robotics 14, 3 (2009),453.

[7] Habib, A., Khan, M., and Bhargava, B. Edge-to-edgemeasurement-based distributed network monitoring. Computer Net-works 44, 2 (2/5 2004), 211–233.

[8] Hulme, G. V. Data deluge; security systems generate an overloadof information. new tools help manage it all more effectively, 08/17;2012/4 2002.

[9] Kent, K., and Souppaya, M. Guide to computer security log man-agement. NIST Special Publication (2006), 800–92.

[10] Lee, W., Stolfo, S., and Mok, K. A data mining framework forbuilding intrusion detection models. In Security and Privacy, 1999.Proceedings of the 1999 IEEE Symposium on (1999), IEEE, pp. 120–132.

[11] Lin, Y.-J., and Chan, M. C. A scalable monitoring approach basedon aggregation and refinement. Selected Areas in Communications,IEEE Journal on 20, 4 (2002), 677–690. ID: 1.

[12] Mukkamala, S., Janoski, G., and Sung, A. Intrusion detectionusing neural networks and support vector machines. In Neural Net-works, 2002. IJCNN’02. Proceedings of the 2002 International JointConference on (2002), vol. 2, IEEE, pp. 1702–1707.

21

[13] Roschke, S., Cheng, F., and Meinel, C. Intrusion detection inthe cloud. In Dependable, Autonomic and Secure Computing, 2009.DASC’09. Eighth IEEE International Conference on (2009), IEEE,pp. 729–734.

[14] Sallay, H. Towards an Integrated Intrusion Detection Monitoring inHigh Speed Networks Hassen Sallay Department of Computer Sciences, College of Computer and information Sciences Imam Mohamad ibnSaud University , Riyadh , Saudi Arabia. Journal of Computer Science7, 7 (2011), 1094–1104.

[15] Scarfone, K., and Mell, P. Guide to intrusion detection and pre-vention systems (idps). NIST Special Publication 800, 2007 (2007), 94.

[16] Timofte, J. Intrusion detection using open source tools. InformaticaEconomica Journal XII, 2 (2008), 75–80.

[17] Wang, X., Wang, H., and Wang, Y. A unified monitoring frame-work for distributed environment. Intelligent Information Management2, 7 (07 2010), 398–405.

22

A Appendix

A.1 Intrusion Detection Using Neural Networks and Sup-port Vector Machines

Table 4: The features table from work by Mukkamala[12].

Feature Description Type

Duration Length (number of seconds) of theconnection

Continuous

Protocal type Type of the protocal e.g. tcp, udp,etc.

Discrete

Service Network service on the destination,e.g., http, telnet, etc.

Discrete

Src bytes Number of data bytes from sourceto destination

Continuous

Dst bytes number of data bytes from destina-tion to source

Continuous

Flag Normal or error status of the con-nection

Discrete

Land 1 if connection is from/to the samehost/port; 0 otherwise

Discrete

Wrong fragment Number of “wrong” fragments Continuous

Urgent Number of urgent packets Continuous

Hot Number of “hot” indicators Continuous

Num failed logins Number of failed login attempts Continuous

Logged in 1 if successfully logged in; 0 other-wise

Discrete

Num compromised Number of “compromised” condi-tions

Continuous

Root shell 1 if root shell is obtained; 0 other-wise

Discrete

SU attempted 1 if “su root” command attempted;0 otherwise

Discrete

Num root Number of “root” accesses Continuous

Num file creations Number of file creation operations Continuous

Num shells Number of shell prompts Continuous

Num access files Number of operations on access con-trol files

Continuous

Num outbound cmds Number of outbound commands inan ftp session

Continuous

Continued on next page

23

Table 4 – continued from previous page

Feature Description Type

Is hot login 1 if the login belongs to the “hot”list; 0 otherwise

Discrete

Is guest login 1 if login is a “guest” login; 0 other-wise

Discrete

Count Number of connections to the samehost as the current connection in thepast two seconds

Continuous

Serror rate % Of connections that have “SYN”errors

Continuous

Rerror rate % Of connections that have “REJ”errors

Continuous

Same srv rate % Of connections to the same ser-vice

Continuous

Diff srv rate % Of connections to different ser-vices

Continuous

Srv count Number of connections to the sameservice as the current connection inthe past two seconds

Continuous

Srv serror rate % Of connections that have “SYN”errors

Continuous

Srv rerror rate % Of the connections that have“REJ” errors

Continuous

Srv diff host rate % Of connections to different hosts Continuous

24

IntroductionBackgroundIDS TechnologyScaling

DesignDesign AimsDesign ApproachCollectionNormalisationFilteringDatabase Storage

ImplementationDescription of the syslog serviceChallenges in ImplementationNetwork Intrusion Detection

Collecting Log DataFirewalls

Evaluation and ResultsEvent FrequencyTesting MethodologyTest Hardware

Processing Time Per EventStorage RequirementsDatabase storage requirements

ConclusionPossible Disk LatencyThreading and Database LockingComponent Integration

Future WorkComponent Integration

AppendixIntrusion Detection Using Neural Networks and Support Vector Machines