An Introduction of Botnet Detection – Part 2 Guofei Gu, Wenke Lee (Georiga Tech)

An Introduction of Botnet Detection – Part 2

Guofei Gu, Wenke Lee (Georiga Tech)

2009/5/26 Speaker: Li-Ming Chen 2

Reference

Guofei Gu, Wenke Lee, et al. BotHunter: Detecting Malware Infection through IDS-driven Dial

og Correlation USENIX Security 2007

BotSniffer: Detecting Botnet Command and Control Channels in Network Traffic ACM NDSS 2008

BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-independent Botnet Detection USENIX Security 2008

Moheeb Abu Rajab, et al. A Multifaceted Approach to Understanding the Botnet Phenomen

on ACM IMC 2006


Lifecycle of a Typical Botnet Infection

Why Botnet is hard to detect?• involving multiple steps• flexible design of C&C

channels

6. Malicious activities (e.g., DDoS)(borrow infection strategies from traditional malicious attacks)

(optional)

authentication


C&C (Command and Control) Channels

Centralized C&C channel

P2P C&C channel

Message Response Crowd

Activity Response Crowd


Comparison of the 3 ApproachesBotHunter BotSniffer BotMiner

Detection Target

Bot Botnet Botnet

Description Detect the lifecycle of a bot, including infection and command execution

Detect group of hosts with spatial-temporal similarity in C&C communication

BotSniffer extension.

Support various C&C comm. framework.

Assumptions Predefine bot infection lifecycle

Focus on centralized C&C communication

Bots will perform tasks and response

Insight Vertical correlation of IDS alerts

Horizontal correlation of similar behaviors

Cluster hosts with similar traffic patterns

Approach detect individual events identify parts of the lifecycle

group hosts connect to the same C&C server detect similar activity or message response behaviors

cluster similar C&C comm. cluster similar malicious traffic. cross clustering


BotHunter

Utilize Snort to detect sign of local infection

Signs matchthe predefinedevidences (dialogtransitions)

A Bot could be:• E2 AND E3-E5• At least two distinct

signs of E3-E5

Predefined Lifecycle


BotHunter (cont’d)

• Current bots are multi-vector• Design two modules (inbound/outbound) for scan detection• Assign high weight to ports often used by malware (predefined)• Observe outbound scan rate, outbound connection failure rate, and address dispersion

• Anomaly-based payload exploit detection• Learn normal profile (using 2-gram PAYL)• Check deviation distance of a test payload from the normal profile

• Use bot-specific heuristics to build signatures (rules)


BotHunter:Evaluation Results (1/2) Experiments in a virtual network

To test FN rate (by examining 10 different bots)# of generated dialog warnings

# involving the victim


BotHunter:Evaluation Results (2/2) Honeynet-based experiments

Use SRI honeynet to capture real-world bot infection Use BotHunter to analysis these traces 95.1% TP rate (1920/2019 in 3 weeks) FN is due to:

Infection failure, honeynet setup and policy failure, data corruption failure.

Experiments in a campus network 98 profiles were generated in 4 months (no FP)

Experiments in SRI laboratory network Generate 1 bot profile and it is FP (a 1.6 GB multifile FTP transfer

matchs “E2 & E3”)


BotHunter:Pros and Cons Pros:

Real-time detection of bot infections Evidence trail gathering for investigation of putative inf

ections Cons:

Use heuristic (2 conditions) to decide a bot infection Less flexible


BotSniffer

Response crowd:• Density check• Homogeneity check

(data reduction)

Port-independent,payload inspection


BotSniffer:Evaluation Methodology Use normal traffic traces to test the FP rate and

use botnet traces (mix normal traffic) to test the detection performance

Normal traces: Capture 8 IRC traces (port 6667) and 5 complete trace

s from campus network Botnet traces:

Collect 3 real-world IRC-based botnet traces Generate 3 botnet traffic by modifying source codes of

3 common botnets Implement 2 http-based botnet


BotSniffer:Evaluation Results (1/2)All FP are generated due to

single client incoming messageresponse analysis.

(Apply both activity response and message response group analysis)


BotSniffer:Evaluation Results (2/2)

honeynet

IRC logs

(both messageand activity)

(periodically connect to server)

(random delay)

(the randomization of connection periods did not cause a problem, becausethere were still several clients performing activity responses at the time window)


BotSniffer:Pros and Cons Pros

Successfully detect all botnets (low FP rate) Efficient alert reduction More robust than other botnet detection system

Cons Focus on centralized C&C communication Configure time window for group analysis Possible evasions (e.g., misusing whitelist, encryption,

protocol matcher, long response delay, obfuscation)


BotMiner (similar to BotSniffer)

Focus on flow statistics, not message response!

log

log

• Combine results and make final decision

(more straightforward)

(more complex)


BotMiner: Evaluation Methodology (same) use normal traffic traces to test the FP ra

te and use botnet traces (mix normal traffic) to test the detection performance

Normal traces: Capture 10 days traffic record at the campus network

Botnet traces: 4 IRC, 2 HTTP and 2 P2P botnets

2 IRC and 2 HTTP are also used for BotSniffer P2P: 2 real-world traces (Nugache and Storm)

TCP, encrypted UDP


BotMiner: Evaluation Results (1/3) (C-plan data reduction)

Most useful,Only record internal toexternal flows. Remove

helf-openTCP flows Whitelist


BotMiner: Evaluation Results (2/3)

4 features:• temporal – fph, bps• spatial – ppf, bpp

Cluster by using themean and varianceof the features

Further cluster by separatingeach feature as a vector of13 elements according to their distribution

Ignore clusters only contain 1 host

Most FP clusters containonly 2 hosts


BotMiner: Evaluation Results (3/3)

FN


BotMiner:Pros and Cons Pros:

Anomaly-based botnet detection system (independent of the protocol and structure used by botnets)

Low FN and FP rate Cons:

Stealthy: botmaster can commond the bots to perform extremely delayed task (evade cross clustering)


Summary

Bothunter: Vertical Correlation Correlation on the behaviors of single host

Botsniffer: Horizontal Correlation Focus on centralized C&C botnets

Botminer: Extension on Botsniffer No limitations on the C&C types.

Documents

An Introduction of Botnet Detection – Part 2 Guofei Gu, Wenke Lee (Georiga Tech)