Upload
raffael-marty
View
1.468
Download
0
Embed Size (px)
Citation preview
Creating Your Own Threat Intel Through Hunting & Visualization
RaffaelMartyVPSecurityAnalytics
May11,2016
HoneynetWorkshop2016– SanAntonio,TX
©RaffaelMarty 2
"This presentation was prepared solely by RaffaelMarty in his personal capacity. The material, views,and opinions expressed in this presentation are theauthor's own and do not reflect the views of SophosLtd. or its affiliates."
Disclaimer
Overview
HUNTINGAKAINTERNALTHREATINTELLIGENCE
THREAT INTELLIGENCEAPROCESSANDINFRASTRUCTUREVIEW
1
2 VISUALIZATIONATHREATINTELLIGENCEGOLDMINE
3
©RaffaelMarty 4
ThreatIntelligence
©RaffaelMarty 5
©RaffaelMarty 6
• Products / Tools
• Firewall - Blocks traffic based on pre-defined rules• Web Application Firewall - Monitors for signs of known malicious activity in Web traffic• Intrusion Prevention System - Looks for ‘signs’ of known attacks in traffic and protocol violations• Anti Virus - Looks for ‘signs’ of known attacks on the end system• Malware Sandbox - Runs new binaries and monitors their behavior for malicious signs• Security Information Management - Uses pre-defined rules to correlate signs from different data
streams to augment intelligence• Vulnerability Scanning - Searches for known vulnerabilities and vulnerable software
• Rely on pattern matching and signatures based knowledge from the past• Reactive -> always behind• Unknown and new threats -> won’t be detected• ‘Imperfect’ patterns and rules -> cause a lot of false positives
WeAreMonitoring– WithPastKnowledge
Verizon 2015 DBIR
70–90%OF MALWARE SAMPLES ARE UNIQUE TO
AN ORGANIZATION.
7
ANewArchitecture– TheSecurityDataLake
anydata BigDataLakeRules
contextIOCs
8
ExploringYOUREnvironment- Hunting
anydata Rules
IOCs
Hunting• Interactivevisualization•Analystdriven•Machineassisted
context
BigDataLake
9
HuntingCreatesInternalThreatIntelligence
anydata Rules
IOCs
Novel,AdvancedAttacks
internal TI
behavioralmonitoringscoringanomalydetectionmachine learningartificial intelligence
“models”
data science
x
new rules
context
BigDataLake
10
HowDoWeGoHunting?Inthefollowingwe’llexplorehowthisallmatters…
…butfirst,let’sseehowvisualization playsakeyrole.
Visualization
©RaffaelMarty 11
S e c u r i t y . A n a l y t i c s . I n s i g h t . 12
“HowCanWeSee,NotToConfirm- ButToLearn”
- EdwardTufte
13
WhyVisualization?dp
ort
time
14
• SELECT count(distinct protocol) FROM flows;
• SELECT count(distinct port) FROM flows;
• SELECT count(distinct src_network) FROM flows;
• SELECT count(distinct dest_network) FROM flows;
• SELECT port, count(*) FROM flows GROUP BY port;
• SELECT protocol,
count(CASE WHEN flows < 200 THEN 1 END) AS [<200],
count(CASE WHEN flows>= 201 AND flows < 300 THEN 1 END) AS [201 - 300],
count(CASE WHEN flows>= 301 AND flows < 350 THEN 1 END) AS [301 - 350],
count(CASE WHEN flows>= 351 THEN 1 END) AS [>351]
FROM flows GROUP BY protocol;
• SELECT port, count(distinct src_network) FROM flows GROUP BY port;
• SELECT src_network, count(distinct dest_network) FROM flows GROUP BY port;
• SELECT src_network, count(distinct dest_network) AS dn, sum(flows) FROM flows GROUP BY port, dn;
• SELECT port, protocol, count(*) FROM flows GROUP BY port, protocol;
• SELECT sum(flows), dest_network FROM flows GROUP BY dest_network;
• etc.
OneGraphSummarizesDozensofQueriesport dest_network
protocol src_network flows
Hunting
©RaffaelMarty 15
16
Technical
• Visualization
• Context
• DataScience
Non-Technical
• Analysts areyourbestandmostexpensiveresource
• Theyneedtherighttoolsanddata• Speed(seethedatalake)• Interaction(visual!)• Machine-assisted insight(datascience)
CoreComponents ToEnableHunting
17
UsersaccessingSharepointservers
User
SharepointServer
This graph of users accessingsharepoint servers, does notimmediately reveal any interestingpatterns.
data processing visualization
18
UsingHRdataascontext
RemoteUser
SanFranciscoOfficeUser
SharepointServer
data processing visualization
HRdata
Using color to add context to thegraph helps immediately identifyoutliers and potential problems.
19
• Simpleapproachesworks!• dc(dest),dc(d_port)
• Whatisnormal?
• Use data science / data mining to prepare data. Then visualize the output for human analyst.
DataScienceinSecurity- WordsofCaution
20
ChallengesWithClusteringNetworkTrafficThegraphshowsanabstractspacewithcolorsbeingmachineidentifiedclusters.
HardQuestions:• What aretheseclusters?• DoWebserverscluster?• Whataregood clusters?• What’sanomalous?
21
HBIMetricAnalysis
Visually learn, Test, Automate
24
• Wehavetriedmanything:o SocialNetworkAnalysiso Seasonalitydetectiono Entropyovertimeo Frequentpatternminingo Clustering
• Allkindsofchallenges• Simpleworks!
Let’sGetMathematical
25
Simple- DataAbstraction
26
LateralMovement- CrossNetworkCommunications
Challenges• Scale• Youwillfindoneofeverything• Definingwhite-listsandkeepingthemuptodate(i.e.,networkandassethygiene)
VPN
DMZ
Office
GIA
UnknownInternet
AWS
VisualAnalyticsDelivering Actionable Security
Intelligence
July 30,31 & August 1,2 - Las Vegas, USA
big data | analytics | visualization
http://secviz.org
WantToDiveDeeper?
©RaffaelMarty 27
[email protected]@raffaelmarty
©RaffaelMarty 28