Embed Size (px)
Citation preview
2
The Big Question:What Is Under The Radar?
First Goal (1)
To create an automated method of detectingunusual connections and/or anomalies with netflows, thereby finding compromised hosts
3
Top Down Method Works Well With IDS
4
N=100
P2P BitTorrent transfer „Deviation“ (total traffic)
Netflow Anomaly DetectionTop Down Strategy
Analyzing the total traffic down to individual hostsby detecting behavioral deviations.
Been there and done that with netflows (2008).
The problem:Even though a malicious traffic event is usuallyan anomaly, an anomaly is not always a malicious event
5
With an automated method, finding the correlating netflows of incidents regardless of source of information (IDS, Switch, AV, User, Admin, Netflows)
New Goal (2)
6
Netflow Anomaly Detection Bottom up Strategy
Viewing what an individual host is doing compared to the general population
7
Common Problems Regardless of Source of Information
• Vague Indications• Detected Anomalies• Recurring Compromise• False Negatives• False Positives
8
youtube
InternetETH
Usual Connections
9
youtube
???????
InternetETH
Unusual Connections
???????Causing IDS anomalies
10
youtube
???????
InternetETH
Correlations/Validations
???????Causing IDS anomalies
???????
??????? 11
Pros and Cons With This MethodPro: It is automatic and indeed sometimes successfullCon: It may take a long time to run
Again:All anomalies are not malicious.
The potential problem is when the individual host is generating harmless but very diverse unusal traffic.
Both pros and cons:It is possible to automatically sort the connections based on how usual/unusal they are
12
youtube
?
InternetETH
Unusal Connections
?
Causing IDS anomalies ?
?
?
?
?
?
?
?
13
A Real Example (Switch)[SWITCH-CERT #22814x ]Most likely compromised system[129.132.208.10x][Botnet]
Based on received information about a ‘malicious IRC command master at 183.203.15.205
2013-08-14 17:40:08.07014
./analyzerdynamic2.sh 129.132.208.10x 20130814 1800 Enter Comment. End it with ^DSubject: [SWITCH-CERT #228144] Most likely compromised system [129.132.208.10x ...] [Botnet]Done. Comment stored in -rw-r--r-- 1 hall nsg 93 Aug 22 18:05 commentDEBUG: starthour:201308141700 endhour:201308141800 startday:201308131800port:-1First DYNAMIC STAGE *************************************nfdump -M /nfsen -R nfcapd.201308141700:nfcapd.201308141800 '( host129.132.208.10x )' | grep ^2013 | awk '{ print $7 }' | sed s/:/\ /g | awk '{ print $1 }' | grep -v "129.132.208.10x:"\ | sort -u > ./analyzer_129.132.208.10x.outp Debug dnumber:41 Second DYNAMIC STAGE *************************************107.21.234.205112 107.21.234.205 (40)108.160.162.531938 108.160.162.53 (39)108.160.163.46229 108.160.163.46 (38)12.130.131.80120 12.130.131.80 (37)... 15
…120 121 12.130.131.80 10 5.0000000 .8264462112 113 107.21.234.205 11 4.5454545 .884955795 96 77.67.22.188 2 .7142857 1.041666645 46 62.146.92.202 12 1.090909 2.1739130434 35 137.205.124.72 3 .2497918 2.857142829 30 54.224.40.139 10 4.8309178 3.333333327 28 134.60.1.5 13 .0336482 3.571428523 24 88.156.222.90 4 1.8691588 4.166666617 18 178.63.20.18 5 2.6041666 5.55555557 8 193.36.36.16 91 5.6627255 12.50000003 3 183.203.8.238 1 .0959692 33.33333332 3 183.203.15.205 34 3.333333 33.3333333
16
List ResultN
umbe
r of i
ps
Num
ber o
f ips
+
inve
stig
ated
des
t ip In
vest
igat
ed
dest
ip
Num
ber o
f flo
ws
Flow
s in
%
(1/N
umbe
r of
ips
+ in
vest
igat
ed
dest
ip)*
100
Blacklist Check Routine77.67.22.18862.146.92.202137.205.124.7254.224.40.139134.60.1.588.156.222.90178.63.20.18193.36.36.16183.203.8.238183.203.15.205botcc:183.203.15.205
17
183.203.8.238
InternetETH
Further Correlations/Validations
Can Now be Done
183.203.15.205
18
129.132.208.10x
The Concept in a Nutshell
1. Find all connections around the investigatedIP over a 60 minute period
2. Take those connections and rate how usual (or unusal) these are in the general population over a 24hr period
19
Find All Connections Around the InvestigatedIP Over a 60 Minute Period
20
Time point of interestfor the investigated ip
InternetInvestigated IP at ETH network
Take Those Connections and Rate How Usual (or Unusal) These are in the General
Population Over a 24hr Period
InternetETH network
Another Real Example (IDS)
IDS Event with destination google:
EVENT: ET TROJAN Zeus Bot Get to Google checking Internet connectivity Date: 08/24-13:13:01.713666 SOURCE: 129.132.211.21x:50086 DEST: 173.194.112.210.80
22
./analyzerdynamic2.sh 129.132.211.21x 20130824 1320 Enter Comment. End it with ^DEVENT: ET TROJAN Zeus Bot Get to Google checking Internet connectivity DATE: 08/24-13:13:01.713666 SOURCE: 129.132.211.21x:50086 DEST: 173.194.112.210:80 Done. Comment stored in -rw-r--r-- 1 hall nsg 124 Aug 28 15:41 commentDEBUG: starthour:201308241220 endhour:201308241320 startday:201308231320port:-1First DYNAMIC STAGE *************************************nfdump -M /nfsen -R nfcapd.201308241220:nfcapd.201308241320 '( host129.132.211.215 )' | grep ^2013 | awk '{ print $7 }' | sed s/:/\ /g | awk '{ print $1 }' | grep -v "129.132.211.21x:"\ | sort -u > ./analyzer_129.132.211.21x.outp Debug dnumber:90 Second DYNAMIC STAGE *************************************108.160.162.111133 108.160.162.111 (89)108.160.162.99118 108.160.162.99 (88)111.111.111.11119 111.111.111.111 (87)12.161.242.20...
23
List Result
24
…19 20 66.70.34.97 3 4.531722054 5.000000000019 19 111.111.111.111 148 .267413497 5.263157894714 15 77.72.169.160 24 4.633204633 6.666666666614 15 62.146.42.159 15 15.463917525 6.666666666614 15 98.139.205.30 9 20.930232558 6.66666666669 10 50.17.207.124 3 23.076923076 10.00000000006 7 98.139.114.30 6 31.578947368 14.28571428575 6 204.13.161.111 154 15.247524752 16.66666666665 6 54.225.249.200 14 28.000000000 16.66666666662 3 77.72.174.136 15 93.750000000 33.33333333331 2 140.116.72.75 2 100.000000000 50.00000000001 2 173.194.112.210 4 100.000000000 50.00000000001 2 66.196.120.57 2 100.000000000 50.00000000001 2 66.196.121.20 24 100.000000000 50.0000000000
Num
ber o
f ips
Num
ber o
f ips
+
inve
stig
ated
des
t ip In
vest
igat
ed
dest
ip
Num
ber o
f flo
ws
Flow
s in
%
(1/N
umbe
r of
ips
incl
udin
g de
st ip
)*10
0
Blacklist Check Routine…66.70.34.97111.111.111.11177.72.169.16062.146.42.15998.139.205.3050.17.207.124pbl:!50.17.207.12498.139.114.30204.13.161.11154.225.249.20077.72.174.136140.116.72.75sbl hit: 140.116.72.75173.194.112.21066.196.120.5766.196.121.20 25
Black List SBL Referencehttp://www.spamhaus.org/sbl/query/SBL193024 Ref: SBL193024140.116.72.75/32 is listed on the Spamhaus Block List - SBL140.116.72.75/32 is listed on the Spamhaus Botnet C&C List - BGPCC2013-08-26 15:56:50 GMT | edu.twCitadel botnet controller @140.116.72.75 Update Aug 26, 2013 Problem still exists, Citadel botnet controller located here:http://dashuxmaecrme.com/wel/file.phphttp://dashuxmaecrme.com/wel/qwrt.phphttp://frontrunnings.com/fdet/file.phphttp://joyrideengend.net/wel/file.phphttp://spottingculde.com/wel/file.phphttp://eenyellowredpf.su/wel/file.phphttp://stabilitymess.net/wel/file.phphttp://systemlevelge.com/wel/file.php…
26
Possible to do’s
• Include (dest) Port in the analysis• Automatically track compromised Ips• Automatically analyse compromised Ips• Automatically build and update CC lists• Automatically correlation check between CC-
clusters and malware
27
Q&A
28
![PHY-layer Spoofing Detection with Reinforcement Learning in ... · in wireless networks. The event detection algorithm proposed in [24] accelerates the event detection in wireless](https://img.pdfslide.us/doc/110x75/5fd4452aa4993062414857eb/phy-layer-spooing-detection-with-reinforcement-learning-in-in-wireless-networks.jpg)
