ICmyNet.FlowNetwork Traffic Analysis System

If You Want to See Your Netwww.icmynet.com

2

What is IP flow?

• IP flow is a unidirectional series of IP packets of a given protocol traveling between a source and destination IP address/port pair within a certain period of time

• IP flow parameters:• Src & Dst IP address• Src & Dst TCP/UDP port• Protocol• ToS field

3

What is IP flow accounting?

• IP flow accounting is a collection of statistical data for every single IP flow crossing a network device:

• Number of packets• Number of bytes• Timestamps

4

What is NetFlow?

NetFlow is a network protocol developed by Cisco Systems for export of collected IP flow statistics

5

NetFlow Statistics Collection

6

NetFlow Statistics Collection

7

ICmyNet.Flow system architecture

Binary raw data files

Flows_2009-10-21-09.20.00 Flows_2009-10-21-09.25.00 Flows_2009-10-21-09.30.00

ICmyNet.FlowCollector

ICmyNet.FlowAggregator

Database

ICmyNet.FlowWeb

Raw Data Files

Archive

8

ICmyNet.Flow/Collector

• ICmyNet.Flow/Collector is a part of the system that collects flow records exported over Netflow protocol.

• Exported flow records have statistics about every data flow transported over network device:

• Src & Dst IP address• Src & Dst TCP/UDP port• Protocol• ToS field• In & Out Interfaces of the network device• Statistics information contains timestamps and number of packets and bytes

carried over the data flow

• Supported NetFlow protocol versions:• Version 5 (supported on most of the network devices)• Version 9 (flexible format with support for IPv6, MPLS, Multicast and MAC

addresses)

• System can be easily extended to support different vendor protocols:

• J-Flow – Juniper protocol for statistics export• NetStream – Huawei protocol for statistics export• IPFIX – currently standardized protocol based on NetFlow v9

9

ICmyNet.Flow/Aggregator

• ICmyNet.Flow/Aggregator is performing analysis and aggregation over collected raw NetFlow records.

• This analysis is done according to the user configuration of the “Traffic Patterns” which is the basic element of the analysis

• Analyzed information is stored in the database and it is used for further search and view from the user interface

• System supports fast PostgreSQL database• The level of aggregated data can be configured according to

the user needs and the available server capabilities• Different grains for keeping the data. For example:

• High grain – 5 min aggregation sample, 7 days keeping• Medium grain – 60 min aggregation sample, 30 days keeping• Low grain – 360 min aggregation sample, 356 days keeping

10

Traffic Pattern – basic element of analysis

Local Network

External Network

• The basic element of the analysis is configurable Traffic Pattern defined by “Local” and “External” network which intercommunication is analysed.

• “Local” and “External” networks can be defined as IP address ranges or as single host IP address.

11

Traffic Pattern – basic element of analysis

Local Network10.0.0.0/8

Application Servers172.16.0.0/24

• The basic element of the analysis is configurable Traffic Pattern defined by “Local” and “External” network which intercommunication is analysed.

• “Local” and “External” networks can be defined as IP address ranges or as single host IP address.

12

Traffic Pattern – basic element of analysis

Local Network10.0.0.0/8

InternetExclude 10.0.0.0/8

• The basic element of the analysis is configurable Traffic Pattern defined by “Local” and “External” network which intercommunication is analysed.

• “Local” and “External” networks can be defined as IP address ranges or as single host IP address.

13

Traffic Pattern – basic element of analysis

Local Network10.0.0.0/8

Internet0.0.0.0/0

• The basic element of the analysis is configurable Traffic Pattern defined by “Local” and “External” network which intercommunication is analysed.

• “Local” and “External” networks can be defined as IP address ranges or as single host IP address.

14

Traffic Pattern – basic element of analysis

• The basic element of the analysis is configurable Traffic Pattern defined by “Local” and “External” network which intercommunication is analysed.

• “Local” and “External” networks can be defined as IP address ranges or as single host IP address.

Local Network10.0.0.0/8

15

10.3.0.0/16

10.2.0.0/16

10.1.0.0/16

Traffic analysis based on Subnets

Local Network10.0.0.0/8

External Network

• IP address space is usually divided in hierarchical manner to represent a logical or sometimes physical topology of the network. Example:

• Universities have /16 address range• Campuses have /21 address range• Faculties have /24 address range

16

10.3.0.0/16

10.2.0.0/16

10.1.0.0/16

Traffic analysis for Hosts

Local Network10.0.0.0/8

External Network

• Within the scope of the Subnet, system is accounting network traffic of single hosts.

• Cut-off value can be configured for minimum traffic • Universities have /16 address range• Campuses have /21 address range• Faculties have /24 address range

17

Parameters for traffic analysis• Traffic analysis gives a detail information about following

parameters at the Traffic Pattern level: • IP subnets traffic• Hosts traffic• Network Services and applications based on TCP/UDP ports• Network Protocols (TCP, UDP, ICMP, GRE...)• QoS markers (ToS, IP precedence or DSCP) • Autonomous System Numbers

• For every parameter of analysis there are following counters:• Traffic Bandwidth (in bits/s, kbps, Mbps..)• Traffic Volume (in MBytes, GB, TB...)• Number of Packets, volume and time based diagrams (pps)• Number of Flows, volume and time based diagrams (fps)

• Configurable cut-off percentage or data amount for negligible consumers

18

ICmyNet.Flow/Web

• Web application is chosen for the user interface

• De-facto standard for network management applications

• Accessibility, permanent development, flexibility

• Java application working under Tomcat

• JSF technologies

19

Settings Tab – Traffic Patterns

• Configuration of the NetFlow analysis is done from the Settings Tab

• User can configure following elements of analysis:• Traffic Patterns• Subnets• Subnet Sets• Services• Protocols• QoS markers• AS Numbers• Exporters

• Control Panel• General• Users• Update

• My Account

20

Settings Tab – Traffic Patterns

• Advanced Traffic Patterns can be configured with flexible matching of any supported NetFlow field

• Examples:• Local Network -> Facebook

• Local address 172.16.0.0/16, Src or Dst AS 32934 (Facebook)• Router X

• Local & External address: 0.0.0.0/0, Exporter 10.1.1.1• Potential attacks:

• Src or Dst port: 22, 135-139, 445, 1434,…• “Weird” Protocols:

• Protocols: Exclude 6 (TCP) or 17 (UDP)• Blocked Traffic:

• Out Interface: 0 (Null)

21

22

Subnets

• Each Subnet is defined with its Name and IP address range

• View tab, Address Space button:below Traffic Pattern element gives an IP address hierarchy in a tree structure

23

Subnet Set

• Subnet Set is user defined grouping of Subnets and other Subnets Sets.

• View tab, Custom Space button:below Traffic Pattern element gives user defined hierarchy of Subnet Sets and belonging Subnets

• Subnet Set can be any logical grouping of Subnets:

• Customer• Institution• Faculty• University

24

25

26

27

28

29

30

31

Viewing the analyzed NetFlow data

• ICmyNet.Flow system tends to give a user the best insight into the network traffic structure

• Therefore, every parameter of the network traffic analysis is presented to the user in various useful ways:

• Top – Visual representation of the distribution of the “Top N Talkers” in the form of the pie chart. Gives a data for the network traffic volume.

• Chart – Time based diagram with a Top N consuming parameters presented in different colors.

• List – Tabular form for reviewing of all parameters and data with advanced options for sorting according to different criteria.

• For every view user can select arbitrary time scale for convenient view

• Number of Top Talkers is user configurable parameter

32

View Tab – Top N

33

View Tab – Chart

34

View Tab – List

35

Archived raw data review

• Raw NetFlow records collected from network devices are archived in the files created every 5 minutes.

• When Collector closes a current file and Aggregator finish with analysis, file is compressed and archived in separate folder.

• Every single flow is saved in these files and no data is wasted• User can access, review and explore these files, searching for

a single flow or event that traversed the network.• Review of the raw data is done over User Interface and search is

available for every supported NetFlow field.

36

Archived raw data review

37

38

Searching and grouping raw data

39

Whois and DNS functions

40

Monitoring system performance

• At the View mode System Tab user can access to relevant graphs monitoring system performance

• Processed flows - number of flows in a single raw data file (created on 5 minutes)

• Matched flows – number of flows that match criteria of any Traffic Pattern

41

Monitoring system performance

• At the View mode System Tab user can access to relevant graphs monitoring system performance

• Processing time for a single raw data file (created every 5 minutes)• Required time to store aggregated data into database• Required time for aggregation between grains and deleting data

ICmyNet.FlowNetwork Traffic Analysis System

If You Want to See Your Netwww.icmynet.com