View
218
Download
0
Category
Preview:
Citation preview
Understanding Encrypted Traffic Using "Joy" for Monitoring and Forensics
Bill Hudson, bhudson@cisco.comSecurity and Trust Organization
DEVNET-1218
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco Spark
Questions? Use Cisco Spark to communicate with the speaker after the session
1. Find this session in the Cisco Live Mobile App
2. Click “Join the Discussion”
3. Install Spark or go directly to the space
4. Enter messages/questions in the space
How
cs.co/ciscolivebot#DEVNET-1218
• Introduction
• Encrypted Traffic
• What Data is Available
• Enhanced Telemetry
• What is “Joy”
• Differentiating Traffic with TLS
• Using Machine Learning Classifiers
• Conclusion
Agenda
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Network encryption is increasingly important
• Distributed security architectures
• Public Cloud
• Private Cloud
• Zero Trust
• Virtualization
• Data privacy and security
• Government Regulations
• Healthcare, Banking, etc.
• Sophisticated attackers
Gartner predicts that by 2019, 80% of all traffic on the network will be encrypted!
DEVNET-1218 5
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Trusted Man-In-The-Middle Inspection
InternetPremises
MITM
Detects
Malicious
Behavior
DEVNET-1218 6
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Trusted Man-In-The-Middle Inspection
InternetPremises
MITM
Detects
Malicious
Behavior
Certificates
Computational CostSecurity &
Privacy
DEVNET-1218 7
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Where do we need to go?
• Know about crypto vulnerabilities, attacks, threats
• Know about malicious communication
• Minimal use of MITMs
DEVNET-1218 8
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Where do we need to go?
• Know about crypto vulnerabilities, attacks, threats
• Know about malicious communication
• Minimal use of MITMs
DEVNET-1218 9
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco PublicDEVNET-1218 10
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Server
Destination Address
Destination Port
Session
# Bytes
# Packets
Client
Source Address
Source PortTCP/IP
DEVNET-1218 11
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Server
Destination Address
Destination Port
Session
# Bytes
# Packets
Packet Lengths
Packet Arrival Times
Client
Source Address
Source PortTCP/IP
Intraflow
DEVNET-1218 12
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
TLSv1.2 metadata
DEVNET-1218 13
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Server
Destination Address
Destination Port
Certificate Chain
Selected Ciphersuite
Session
# Bytes
# Packets
Packet Lengths
Packet Arrival Times
Client
Source Address
Source Port
Ciphersuite Offer Vector
Extensions Offer
Supported Elliptic Curves
SNI
TCP/IP
Intraflow
TLS
Record Length
Record Times
Record Types
DEVNET-1218 14
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Server
Destination Address
Destination Port
Certificate Chain
Selected Ciphersuite
Response Code
TTL
Headers
Session
# Bytes
# Packets
Packet Lengths
Packet Arrival Times
Client
Source Address
Source Port
Ciphersuite Offer Vector
Extensions Offer
Supported Elliptic Curves
SNI
Name
Headers
TCP/IP
Intraflow
TLS
DNS
HTTP
Co
nte
xtu
al
Flo
ws
Record Length
Record Times
Record Types
Headers
File Magic
DEVNET-1218 15
Enhanced Telemetry
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
src dst
Enhanced Telemetry Data Types
• SPLT – Sequence of Packet Lengths and Arrival Times
• Byte Distribution
• Byte Entropy
• TLS unencrypted header data
• Certificates, SNI, Ciphersuites, Extensions
• DNS linked flows
• HTTP linked flows
17DEVNET-1218
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18DEVNET-1218
Sequence of Packet Lengths and Times
src dst
Time
Clie
nt p
acke
tsS
erv
er
pa
cke
ts
"packets": [
{ "b": 22, "ipt": 33, "dir": ">" } ,
{ "b": 1432, "ipt": 4, "dir": "<" } ,
{ "b": 30, "ipt": 1, "dir": ">" } ,
{ "b“: 4, "ipt": 145, "dir": "<" },
...
]
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19DEVNET-1218
Byte Distribution and entropy
“entropy": 7.165,
"bd": [
23, 7, 4, 8, 4, 12, 7, 4,
12, 5, 98, 6, 5, 101, 14, 8,
9, 9, 6, 8, 10, 6, 10, 6,
16, 8, 3, 16, 7, 7, 3, 11,
189, 6, 24, 9, 10, 10, 5, 7,
19, 8, 16, 8, 34, 79, 61, 90,
102, 91, 56, 47, 35, 47, 30, 25,
...
]
What is “Joy”?
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21DEVNET-1218
”Joy”
joypcap
joy
Offline
Collector
exporter collector
joyq.py
sleuth
Model.py
json
json
joyAvailable at https://github.com/cisco/joy
joy jsonOnline
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Download, Build, Run, Install
• Download• git clone https://github.com/cisco/joy
• Build * (for windows there is a Visual Studio project file)• cd joy
• ./config
• make
• Run• ./bin/joy [options]
• (Optional) Install• Make install
DEVNET-1218 22
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
joy [OPTIONS] file1 [file2 ... ]
23DEVNET-1218
General options
-x F read configuration commands from file F
interface=I read packets live from interface I
promisc=1 put interface into promiscuous mode
output=F write output to file F (otherwise stdout is used)
logfile=F write secondary output to F (otherwise stderr used)
count=C rotate output files so each has about C records
upload=user@server:path upload to user@server:path with scp after rotation
keyfile=F use SSH identity (private key) in file F for upload
anon=F anonymize addrs matching the subnets listed in file F
retain=1 retain a local copy of file after upload
nfv9_port=N enable Netflow V9 capture on port N
verbosity=L verbosity level: 0=quiet, 1=pkt metadata, 2=payloads
https://github.com/cisco/joy/doc/using-joy-05.pdf
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
joy [OPTIONS] file1 [file2 ... ]
24DEVNET-1218
Data feature options
bpf="expression" only process packets matching BPF "expression”
zeros=1 include zero-length data (e.g. ACKs) in packet list
bidir=1 merge unidirectional flows into bidirectional ones
dist=1 include byte distribution array
entropy=1 include byte entropy
tls=1 include TLS data (ciphersuites, record lengths, ...)
exe=1 include information about host process assoc w/flow
classify=1 include results of post-collection classification
num_pkts=N report on at most N packets per flow (0 <= N < 200)
idp=N report N bytes of the init data packet of each flow
label=L:F add label L to addrs that match the subnets in file F
model=F1:F2 change classifier parameters, SPLT=F1, SPLT+BD=F2
URLmodel=URL specify URL to update classifier data
URLlabel=URL specify URL to update label data
dns=1 include dns names
hd=1 include header description
wht=1 include walsh-hadamard transform
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Using joy to process PCAP files
25DEVNET-1218
sh$ bin/joy bidir=1 http=1 dns=1 tls=1 dist=1 output=test.gz test.pcap
sh$ ./sleuth --pretty test.gz | less
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
> bin/joy bidir=1 tls=1 dns=1 http=1 output=sample.gz tls12_handshake.pcap
> ./sleuth sample.gz
26DEVNET-1218
{"pr": 6, "tls": {"tls_crandom":
"0a9aab7b80199b7965aab25ee11ac9d9d562d541e2a7e0015010c0385d744ba2", "tls_ov": 5, "SNI":
["www.facebook.com"], "tls_ext": [{"data": "00130000107777772e66616365626f6f6b2e636f6d",
"length": 21, "type": "0000"}, {"data": "", "length": 0, "type": "0017"}, {"data": "00",
"length": 1, "type": "ff01"}, {"data": "0008001d001700180019", "length": 10, "type":
"000a"}, {"data": "0100", "length": 2, "type": "000b"}, {"data": "", "length": 0, "type":
"0023"}, {"data": "000c02683208687474702f312e31", "length": 14, "type": "0010"}, {"data":
"0100000000", "length": 5, "type": "0005"}, {"data": "", "length": 0, "type": "0012"},
{"data": "", "length": 0, "type": "ff03"}, {"data":
"001604030503060308040805080604010501060102030201", "length": 24, "type": "000d"}],
"srlt": [{"b": 196, "tp": "22:1", "ipt": 0, "dir": "<"}], "cs": ["c02b", "c02f", "cca9",
"cca8", "c02c", "c030", "c00a", "c009", "c013", "c014", "0033", "0039", "002f", "0035",
"000a"]}, "ts": 1491509125.654878, "sp": 38388, "packets": [{"b": 201, "ipt": 0, "dir":
"<"}], "ob": 201, "da": "31.13.69.228", "ottl": 64, "sa": "10.0.2.15", "te":
1491509125.654878, "dp": 443, "op": 1}
{"pr": 6, "tls": {"srlt": [{"b": 74, "tp": "22:2", "ipt": 0, "dir": "<"}], "tls_srandom":
"b1ae9dda9138839f8d338138727c931587b2248712bef8fbbca710b110b10245", "s_tls_ext": [{"data":
"", "length": 0, "type": "0000"}, {"data": "00", "length": 1, "type": "ff01"}, {"data":
"03000102", "length": 4, "type": "000b"}, {"data": "", "length": 0, "type": "0023"},
{"data": "0003026832", "length": 5, "type": "0010"}], "scs": "c02b", "tls_ov": 5}, "ts":
1491509125.663982, "sp": 443, "packets": [{"b": 2760, "ipt": 0, "dir": "<"}, {"b": 729,
"ipt": 0, "dir": "<"}], "ob": 3489, "da": "10.0.2.15", "ottl": 64, "sa": "31.13.69.228",
"te": 1491509125.664122, "dp": 38386, "op": 2}
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
> bin/joy bidir=1 tls=1 http=1 output=sample.gz rc4_sample.pcap
> ./sleuth --pretty --select sa,da --where “tls{scs}=0004” sample.gz
27DEVNET-1218
{
"sa": "192.168.56.117",
"da": "192.168.56.202”
}
https://github.com/cisco/joy/doc/using-joy-05.pdf
Differentiating Traffic with TLS
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29DEVNET-1218
Passive Network Crypto Audit
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
520 2048 776 1024 4096 768 512 1016 3072
Key Size (bits)
Public Key Lengths
0
0.05
0.1
0.15
0.2
0.25
0.3
c02f c028 0035 c02b c014 c030 0004 c013 002f c027
hex code
Selected Ciphersuites
FIPS and PCI Compliance
RC4
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30DEVNET-1218
SSL implementation detection
• What devices and applications use unpatched software?
0.9.8
1.0.0
1.0.1
1.0.2
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31DEVNET-1218
Vulnerability Detection - Heartbleed
0.9.8
1.0.0
1.0.1
1.0.2
TLS pad extension to fix
TLS hang bug
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 32DEVNET-1218
Improved Threat Detection
• Independent source of weak convictions
• Reduces overall false positive rate
Using Machine Learning Classifiers
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 34DEVNET-1218
Flow classification
Classifier
Flow
Records
sh$ joy bidir=1 dist=1 classify=1 capture.pcap > capture.gz
sh$ joyq.py capture.gz --where "p_malware > 0.01"
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 35DEVNET-1218
Training architecture
Malware
Detonation
Training
Benign
Records
Malware
RecordsClassifier
analysis/model.py
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 36DEVNET-1218
Results
• L1-logistic regression
• SPLT + 7-tuple + BD
• 172.2 non-zero parameters
• 0.01 FDR: 0.1%
• Total Accuracy: 96.1%
• L1-logistic regression
• SPLT + 7-tuple + BD + TLS
• 137.2 non-zero parameters
• 0.01 FDR: 90.4%
• Total Accuracy: 99.6%
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 37DEVNET-1218
Combining views of data features reduces false positives
(v: 1.0.1r)
(v: 52.0)
+
+
+
+
Firefox C2
Bestafera
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Conclusions
• Machine learning and rules applied to passively obtained network data features can
• Detect malware communication
• Detect misused or unpatched cryptography
• SPLT, Byte Distribution, and TLS header data are valuable
• Training classifiers is key!
• Better than MITM with respect to security, privacy and cost
• “Joy” open source package implements these features
• Support: best effort mail alias joy-users@cisco.com
38DEVNET-1218
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco Spark
Questions? Use Cisco Spark to communicate with the speaker after the session
1. Find this session in the Cisco Live Mobile App
2. Click “Join the Discussion”
3. Install Spark or go directly to the space
4. Enter messages/questions in the space
How
cs.co/ciscolivebot#DEVNET-1218
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Please complete your Online Session Evaluations after each session
• Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt
• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations
Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/.
Complete Your Online Session Evaluation
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Continue Your Education
• Demos in the World of Solutions!
• Enterprise Networking Area• Encrypted Traffic Analysis (ETA) with the new C9300 switch!
• Security Section• Encrypted Traffic Analysis (ETA) integrated in the Stealthwatch!
• Walk-in Self-Paced Labs
• Tech Circle
• Meet the Engineer 1:1 meetings
• Self Paced Workbook
• https://github.com/cisco/joy/doc/workbench.pdf
• ETA Overview Documentation
• http://cisco.com/go/eta
41DEVNET-1218
Thank you
Recommended