Snort Architectural DocumentBased on snort-2.9.4.6 open source code

Contents

1Introduction42Snort Components42.1Snort Executable52.2DAQ52.3Dynamic Preprocessors52.4Detection Engine52.5Snort Text Rules52.6Dynamic rules52.7Snort Configuration53Snort Architectural Overview64Snort In Execution75Snort Example Topology86Snort-Initialization96.1InitSignal106.2InitGlobal106.3ParseCmdLine106.4RegisterOutputPlugins106.5FileApiInit106.6ParseSnortConf106.7Load Plugins116.8Initialize Plugins116.9Configure Plugins116.10Parse Rules116.11Doing Optimizations for faster packet detection116.12Control Socket Initialization117DAQ127.1DAQ initialization127.2DAQ Start137.3Data Acquisition148Decode158.1Decoding process169Preprocess179.1Dynamic Preprocessors179.2Initialization of Dynamic Preprocessor179.3Sequence Diagram of Initialization of Dynamic Preprocessor189.4Non Dynamic Preprocessors1910Detection2010.1Rule Tree2210.2How Detection works2311Output Module2412Snort Maintenance Thread25

IntroductionThis document describes the snort architecture based on the open source code Snort-2.9.4.6.Snort Components Output PluginDAQDynamic PreprocessorsDynamic RulesDetection EngineSnort ExecutableSnort Text Rules

Snort Configuration

Fig: 1

Snort ExecutableThe main part of snort system which loads and controls other components. Snort Executable communicates with other components to perform various tasks like data acquisition, decoding protocols, normalizing network traffic, detecting and preventing attacks and create alerts and logs.DAQThird party library used by snort executable to acquire data from network.Dynamic PreprocessorsShared library loaded by snort executable to normalize network traffic.Detection EngineDetection engine is responsible for controlling detection process. Detection engine evaluates network packet against a known attack signatures. These signatures are contained in snort rules.Snort Text RulesSnort rules in the form of text files.Dynamic rulesSnort rules in the form of shared library. These libraries are loaded by snort dynamically.Snort ConfigurationConfiguration file used to configure different components in snort system.

Snort Architectural OverviewThe flow of snort working is as shown in Fig 2. Start

Snort Initialization

Maintenance thread Main threadPreprocess

Detect

Decode

ExitPeriodically checkup configuration to inform Main thread configuration changesDAQ

Exit

Yes yes Yes No No

Fig: 2

Snort started with snort initialization. Under normal IPS mode, snort works under two threads. One is the main-thread and the other is the maintenance thread. All the major snort operations are done under the main-thread. The maintenance thread is for performing periodic checkup for the proper functioning of snort and also for reloading the snort configuration data and its preprocessing on error conditions and also on snort rule updates. A set of checkup functions are executed during periodic checkups.Snort In Execution

Snort ExecutableDynamic PreprocessorDynamic RulesText RulesDAQUSER SPACEKERNEL SPACE System CallSocket

Fig: 3

Snort Example Topology

Remote Monitoring WorkstationSnort IPS 172.20.1.186 I/F 1I/F 3 Internet

I/F 2

192.168.50.1192.168.50.2

Client Machine 2

Client Machine 1 Client Machine 2

Fig: 4

Snort-InitializationSnort performs a series of operation while initializing. Fig: 2 show the order of operations. InitSignals

InitGlobals

ParseCmdLine

RegisterOutputPlugins

FileApiInit

ParseSnortConf

LoadPlugins

Initializes Plugins

Configure plugins

Parse Rules

Do optimizations for fast Packet Detection

Control Socket Initialization

Fig: 5

InitSignalInitialize the snort specific signals with the corresponding signal handler.InitGlobalInitialize the netmask array and protocol-names array. Netmask [] is an array of 32 elements and protocol_names[] array contains 256 entries. Netmask[] contains the hexa decimal mask values. Protocol_names[] contains different protocol names which will be accessed from different part of the program by giving its corresponding ID.ParseCmdLineSnort command line arguments are parsed and assigned to the concerned variables.RegisterOutputPluginsThere are many initialization functions in the preproc-list. Here the preprocessor keywords are linked to the corresponding initialization function in the preproc-list.FileApiInitThe FileAPI struct will be initialized from here. ParseSnortConfSnort config file is parsed and the information is filled in the SnortConf structure. The details of the command line arguments are also updated in the SnortConf structure.Load PluginsA link list of _DynamicEnginePlugin structure is there to store the informations of all dynamically located plugins. The plugins handle, meta data, initialization function and its version informations are stored in this structure. The loaded plugins includes the snort rules plugins and preprocessor plugins.Initialize PluginsInitialization involves mapping snort specific keywords with the corresponding initialization function pointers in the respective plugins.Configure PluginsAll the plugins are configured and their information is stored in a linked list. The respective Config structure is selected from the linked list based on the keyword associated to it. Each plugin is associated with a keyword.Parse RulesUsing the default parser policy, the snort config file is parsed at first and information are stored in SnortConfig structure. It then process individual snort rule and add it to the snort rule list. It extensively uses PCRE library functions while parsing the snort rules.Doing Optimizations for faster packet detectionSome optimization for the creation of faster Packet inspection engine is done here. Creates two groups to categorize snort rules. The rules contain content option are grouped into one group and rules do not contain content option form the second group. The rules with same destination ports are again form subgroups within these two main groups.Control Socket InitializationThe entire socket related initialization is done here.

DAQDAQ module does data acquisition from network During snort initialization, it loads all the libraries needed for data acquisition. For DAQ activities, snort communicates with DAQ libraries through a mediator library( shim layer ). Mediator library ( shim layer ) communicates with specific DAQ library which is specified in the snort configuration file.DAQ initializationDuring DAQ initialization, it opens PF_PACKET raw socket to receive all network traffic which is completely unmodified. Then it creates a bridge between two network interfaces. It then allocates the desired memory for both the interfaces. Fig 3 shows a typical DAQ initialization process.

Fig: 6

DAQ StartDuring DAQ start, each interface is bind to specific socket. It then Turns on the promiscuous mode for the Network Interface controller. DAQ module then asks the kernel to create ring buffer for holding the packet data. When the packet data is obtained through PF_SOCKET, it gets added to ring buffer. Fig: 7

Data AcquisitionDuring data acquisition, DAQ module fetches data associated with one of the socket corresponds to a network interface and send data to the snort application through callback function. Then it passes that data to other interface using associated sockets. Fig: 5 show the complete data acquisition flow.

Fig: 8

DecodeIn this module the data link layer packets get decoded and the information are stored in respective structures in snortDecoding process

Fig: 9

The packets arrived from the network card have been passed to the snort decode engine by DAQ to decode the raw Data Link Layer packets. The DAQ enabled by snort is in an endless loop watching for the arrival of packets. When it receives the packets from the network card driver, it then runs the SetPktProcessor() function to set the root decoder based on the datalink layer into the Grinder .Then it calls ProcessPacket() function.In the ProcessPacket () function, it calls the Data link layer Decoding function for decoding Data Link Layer packet frame. During the Data Link Layer decoding, it calls the Network Layer decode function for finding the network layer protocol type. The network layer decoding function calls the next higher (transport) layer decode function for finding type of packet. Finally the ProcessPacket () function link all the information decoded to appropriate data structures. The packets are now ready for detectionPreprocessPreprocessors reassemble packets, do protocol specific decoding and normalization. This enables detection engine to easily perform attack detection. If packet payload is not normalized, the probability of getting false positives will be more. So preprocessor improve the accuracy of detection process.Dynamic PreprocessorsThe dynamic preprocessors are plugins loaded dynamically by snort executable. These preprocessors are configurable and initialized during snort initialization.Initialization of Dynamic Preprocessor The interface between snort executable and a dynamic preprocessor is a data structure named _DynamicPreprocessorData or DPD. DPD contains addresses of many functions as function pointers. The snort executable first fills this structure with its function addresses and pass this structure to dynamic preprocessors through preprocessors exposed function named InitializePreprocessor. InitializePreprocessor calls preprocessors setup function using a macro DYNAMIC_PREPROC_SETUP(). Setup function gets address of function named RegisterPreprocessor from DPD and calls this function with a call back function and a unique string as arguments. This unique string is used to identify the preprocessor, which is an entry in snort configuration file. Snort executable uses these parameters using the above mentioned unique string and invokes the callback function.

Sequence Diagram of Initialization of Dynamic PreprocessorFig 3 shows the sequence diagram showing the Initialization of Dynamic Preprocessor. Fig: 10

The following example code shows how to register a preprocessor._dpd.registerPreproc(ftp_telnet, FTPTelenetInit);The registerPreproc() function takes two parameters. The first parameter is a unique string to identify preprocessor. This string would be entered in the snort.conf file when configuring snort to use the preprocessor. The line added to snort configuration file is shown below.Preprocessor ftp_telnet: global inspection_type stateful encrypted_traffic no check_encryptedThe second parameter is pointer to an initialization function. The initialization function is used to process options in the configuration file that affect how preprocessor analyzes packet. The function receives a pointer to a character string as function argument. Snort passes global inspection_type stateful encrypted_traffic no check_encrypted as an argument to FTPTelenetInit. In addition to initialization, register a preprocessor function. The preprocessor function is the function that analyze packet and normalize traffic. The following is an example code shows adding preprocessor function._dpd.addPreproc(FTPTelenetChecks,PRIORITY_APPLICATION, PP_FTPTELNET, PROTO_BIT__TCP);Non Dynamic PreprocessorsRule-based pattern matching can often fail on protocols for which data can be represented in many different ways. For example, Web servers accept many different ways of writing a URL. IIS, for example, will accept backslash (\) characters in place of forward slash (/) characters in URLs. In such cases, wecan define a single rightway to write the data that we are matching.Thus Snort has relied on preprocessors that are compiled into Snort. There are three non-dynamicpreprocessors enabled in snort.conf include HTTP Inspect, RPC Decode and BO (Back Orifice). The http_inspect preprocessor is a generic HTTP decoder that examines HTTP requests and replies. http_inspect detects previously unknown Web servers or proxies, allowing a better understanding of what HTTP activity are taking place on the network.Step to activate this preprocessor, add the following in Snort configuration file:

preprocessor http_inspect: global \iis_unicode_map unicode.map 1252These values tell HTTP Inspect where to find the unicode.map file and tells Snort to use the United States Unicode codemapp (1252) when decoding Unicode characters.preprocessor http_inspect_server: server default profile all ports { 80 8080 8180 } oversize_dir_length 500

This set of values establishes a default profile for all Web servers answering requests on ports 80, 8080 and 8180 TCP. The only other option tells Snort to generate alerts when a URL directory length exceeds 500 characters.The RPC Decode preprocessor normalizes fragmented RPC into a single record. The default values watch for traffic involving the portmapper (port 111) and 32711, which is only one of many possible RPC server ports.

preprocessor rpc_decode: 111 32771

The final default preprocessor included in the stock snort.conf is the Back Orifice preprocessor, indicated by:

preprocessor bo

BO has not been a popular back door communications channel for many years, this preprocessor should be left out of your new configuration file.DetectionThe main components of detection module of snort are detection engine, snort rule set, snort dynamic rules and PCRE library. Snort loads detection engine, rule set, snort dynamic rules and PCRE library. Snort rule set is in the form of text files and other three are shared libraries. Using the snort configuration file we can configure the rule set and dynamic rules. That means, user can specify the required rules in snort configuration file. During detection, each incoming packet will be processed using the set of rules. Fig: 8 show the general code flow during initialization of detection modules.

Fig: 11

Detection engine consists of one or more dynamic plugins which process the Ethernet packets against a set of snort rules. Each plugin will do its own specific task. Snort by itself contain a set of rules by default, and also snort can have dynamic rules which are loaded dynamically as per the configuration in snort configuration file. For the efficient use of snort rules, these rules are stored in the form of a tree structure. The header data of rule is used to build a rule tree node (RTN) and option data of rule is used to create option tree node (OTN). All of the OTNs with a matching header are grouped together under a single RTN. The node structure will also contains a list of function pointers that need to be executed for processing the Ethernet packet for the corresponding rule. PCRE (Perl Compatible Regular Expression) library will be used by the detection engine for regular expression pattern matching of packet data content.Rule Tree

Fig: 12

Rule tree is data structure used to hold all the snort rules. Each rule tree node(RTN) corresponds to a rule header. Each option tree node(OTN) corresponds to an option header. Each RTN is associated with different OTNs. RTNs are branched into other RTNs and thus form a rule tree. OTNs contain address of packet evaluation functions in addition to rule option values. During detection, snort detection engine walks through rule tree to find appropriate rule set to apply those on incoming packet.

How Detection worksAfter normalizing the packet, preprocessor pass packet to detection module to evaluate packet against a known attack pattern. The detection module groups rules according to destination ports. Each group is associated with specific attack pattern. When a packet comes, pattern match algorithm is used to find a match against the packet data and corresponding rule set is selected for packet evaluation. The alerts found during packet evaluation are added to event queue. These entries in event queue are later used by output modules to generate alerts.

Fig: 13Output Module

Snort output plug-ins (snort output modules) provides the administrators the ability to configure logs and alerts in a manner that is easy to understand, read and use in their organization's environment.The Output plug-ins are initialized, registered and configured at the time of snort initialization itself. These output modules are executed whenever snort's alert or logging subsystems are called following the execution of preprocessors and the packet capture engine. The packet capture engine in snort receives packets off the wire and sends to the analysis module. If the packet or packets trigger an alert or log event, the data is passed to the corresponding output module. The parsecmdLine () function finds whether we need to log or alert the packet when parsing snort config file. Depending upon the snort config run mode, the parsecmdLine () function determines whether to log packet or print packet. If printpacket (alert) is selected as log_func, the corresponding data will be printed on console. ie alert. If LogPacket is selected as log_func, CallLogPlugins() function is called when data is passed to the output module. In this function corresponding log function is handled depending upon the type. it can be in the form of default logging, SNMP traps, Xml logging, syslog, snortdb, unified logs, barnyard etc.

Snort Maintenance Thread

Snort supports reloading a configuration in lieu of restarting Snort in so as to provide seamless traffic inspection during a configuration change. A separate thread will parse and create a swappable configuration object while the main Snort packet processing thread continues inspecting traffic under the current configuration. When a swappable configuration object is ready for use, the main Snort packet processing thread will swap in the new configuration to use and will continue processing under the new configuration. The SnortStartThreads function calls ReloadConfigThread for creating reload maintenance thread for snort. The function ReloadConfigThread checks for new snort configurations by calling reloadConfig() function. This function will set the new Configurations if available and thread will wait till snort swap or exiting and when it happens it will delete old snort configuration files if newer configuration is set, by calling Snortconffree() function. If the configurations reload is success it will load the new preprocessor configurations and free the old ones, so any preprocessor cleanup that requires a configuration will be using the new one unless it relies on old configurations that are still attached to the existing sessions. Then it calls the function updatePeriodicCheck(). Maintenance thread Continuous this process till the snort exits.