© 2002, Cisco Systems, Inc. All rights reserved. NetFlow Overview, 2/03

© 2002, Cisco Systems, Inc. All rights reserved.

NetFlow Overview, 2/03

2© 2003, Cisco Systems, Inc. All rights reserved.

NetFlow Overview, 2/03

NetFlow Overview

Technical Marketing

Internet Technologies Division

February 2003

333© 2003, Cisco Systems, Inc. All rights reserved. 3NetFlow Overview, 2/03

Agenda

• NetFlow Overview• Versions• Partners• Customer Applications• Solutions by Technology• Features and Uses• Platform Specifics• Performance• Roadmap and Future Direction• Summary

NetFlow Overview


NetFlow Origination & Innovation

• Developed by Darren Kerr and Barry Bruins at Cisco Systems in 1996

• The value of information in the cache was a secondary discovery

Initially designed as a switching path

• NetFlow is now the primary network accounting technology in the industry

• Sampled NetFlow a Cisco innovation

• NetFlow version 9 an IETF standard

• Answers questions regarding IP traffic: who, what, where, when, and how


What is a flow?

Exported Data

Defined by seven unique keys:

• Source IP address

• Destination IP address

• Source port

• Destination port

• Layer 3 protocol type

• TOS byte (DSCP)

• Input logical interface (ifIndex)


NetFlow SequenceRouter

1. Create and update flows in NetFlow Cache

• Inactive timer expired (15 sec is default)• Active timer expired (30 min (1800 sec) is default)•NetFlow cache is full (oldest flows are expired)• RST or FIN TCP Flag

He

ad

er

ExportPacket

Payload(flows)

2. Expiration

3. Aggregation?

Protocol Pkts SrcPort DstPort Bytes/Pkt

11 11000 00A2 00A2 1528

SrcIf SrcIPadd DstIf DstIPadd Protocol TOS Flgs Pkts SrcPort SrcMsk SrcAS DstPort DstMsk DstAS NextHop Bytes/Pkt Active Idle

Fa1/0 173.100.21.2 Fa0/0 10.0.227.12 11 80 10 11000 00A2 /24 5 00A2 /24 15 10.0.23.2 1528 1800 4

e.g. Protocol-Port Aggregation Scheme becomes

4. Export Version

SrcIf SrcIPadd DstIf DstIPadd Protocol TOS Flgs Pkts SrcPort SrcMsk SrcAS DstPort DstMsk DstAS NextHop Bytes/Pkt Active Idle

Fa1/0 173.100.21.2 Fa0/0 10.0.227.12 11 80 10 11000 00A2 /24 5 00A2 /24 15 10.0.23.2 1528 1745 4

Fa1/0 173.100.3.2 Fa0/0 10.0.227.12 6 40 0 2491 15 /26 196 15 /24 15 10.0.23.2 740 41.5 1

Fa1/0 173.100.20.2 Fa0/0 10.0.227.12 11 80 10 10000 00A1 /24 180 00A1 /24 15 10.0.23.2 1428 1145.5 3

Fa1/0 173.100.6.2 Fa0/0 10.0.227.12 6 40 0 2210 19 /30 180 19 /24 15 10.0.23.2 1040 24.5 14

YesNo

Aggregated Flows – export Version 8 or 9Non-Aggregated Flows – export Version 5 or 9

5. Transport Protocol


Core Network

Creating Export Packets

Enable NetFlow

Traffic

Collector(Solaris, HP-UX, or Linux)

UDP NetFlowExport

Packets

Application GUI

PE

Export Packets• Approximately 1500 bytes• Typically contain 20-50 flow

records• Sent more frequently if traffic

increases on NetFlow-enabled interfaces


NetFlow Principles

• Inbound traffic only

• Unidirectional flow

• Accounts for both transit traffic and traffic destined for the router

• Works with Cisco Express Forwarding (CEF) or fast switching

Not a switching path

• Supported on all interfaces and Cisco IOS Software platforms

• Returns the sub-interface information in the flow records


SiSiSiSi

Comprehensive Platform Support

GSR 12000GSR 12000

Catalyst 4500

Catalyst 4500

7200/7500/7200/7500/

37003700

2500/

2600

2500/

2600

36003600

AS5300/5800

AS5300/5800

4500/47004500/4700

1400/1600/1700

1400/1600/1700

Catalyst 5000/6500/

7600

Catalyst 5000/6500/

7600

ESR10000ESR

10000


Agenda


Versions


NetFlow Versions

NetFlow Version

Comments

1 Original

5 Standard and most common

7 Specific to Cisco Catalyst 6500 and 7600 Series Switches Similar to Version 5, but does not include AS, interface, TCP Flag & TOS information

8 Choice of eleven aggregation schemesReduces resource usage

9 Flexible, extensible file export format to enable easier support of additional fields & technologies; coming out now MPLS, Multicast, & BGP Next Hop


Agenda

Version 5Version 8Version 7Version 9

Version 5

NetFlow OverviewVersions


Version 5 - Flow Format

• Source IP Address• Destination IP Address

• Packet Count• Byte Count

Usage

QoS

Timeof Day

Application

PortUtilization

From/To

Routing and

Peering

• Input ifIndex• Output ifIndex

• Type of Service• TCP Flags• Protocol

• Start sysUpTime• End sysUpTime

• Source TCP/UDP Port• Destination TCP/UDP Port

• Next Hop Address• Source AS Number• Dest. AS Number• Source Prefix Mask• Dest. Prefix Mask



Agenda

Version 5Version 7Version 8Version 9

Version 7



Version 7

• Adds NetFlow switching support for:Cisco Catalyst 5000 Series Switches with an RSM

Cisco Catalyst 5000 Series Switches with an MSFC

• Uses MultiLayer Switching (MLS) or CEF with Cisco Catalyst 6000 Series Switches with SUP2

• IP unicast onlyNo multicast or IPX, even if MLS can do all three

• MLS cache is the equivalent of the NetFlow cache




Usage

QoS

Timeof Day

Application

PortUtilization

From/To

Routing and

Peering


• Input ifIndex• Output ifIndex

• Type of Service• TCP Flags• Protocol

• Packet Count• Byte Count

• Start sysUpTime• End sysUpTime

• Source TCP/UDP Port• Destination TCP/UDP Port

• Next Hop Address• Source AS Number• Dest. AS Number• Source Subnet Mask• Dest. Subnet Mask• RouterSc (router shortcut)*

* Added from version 5

Note that the ToS and TCP Flags fields are not populated


Agenda

Version 5Version 7Version 8Version 9Version 8



Version 8

• Router-based aggregation• Enables router to summarize NetFlow data• Reduces NetFlow Export data volume• Decreases NetFlow Export bandwidth requirements• Currently 11 aggregation schemes

Five original schemesSix new schemes with the TOS byte field

• Several aggregations can be enabled simultaneously



AS

Protocol-Port Source-Prefix Destination-Prefix Prefix

Source Prefix

Source Prefix Mask

Destination Prefix

Destination Prefix Mask

Source App Port

Destination App Port

Input Interface

Output Interface

IP Protocol

Source AS

Destination AS

First Timestamp

Last Timestamp

# of Flows

# of Packets

# of Bytes


AS-TOS

Protocol-Port-TOS

Source-Prefix-TOS

Destination-Prefix-TOS

Prefix-TOS Prefix-Port

Source Prefix

Source Prefix Mask

Destination Prefix

Destination Prefix Mask

Source App Port

Destination App Port

Input Interface

Output Interface

IP Protocol

Source AS

Destination AS

TOS

First Timestamp

Last Timestamp

# of Flows

# of Packets # of Bytes



Version 8 - Configuration

3600-4(config)# ip flow-aggregation cache ?

as AS aggregation

as-tos AS-TOS aggregation

destination-prefix Destination Prefix aggregation

destination-prefix-tos Destination Prefix TOS aggregation

prefix Prefix aggregation

prefix-port Prefix-port aggregation

prefix-tos Prefix-TOS aggregation

protocol-port Protocol and port aggregation

protocol-port-tos Protocol, port and TOS aggregation

source-prefix Source Prefix aggregation

source-prefix-tos Source Prefix TOS aggregation

Note – do not export version 5 at the same time “ip flow-export version 5”


Agenda

Version 5Version 8Version 7Version 9Version 9



Why a New Version?

• Fixed formats (versions 1, 5, 7, and 8) are not flexible and adaptable

Cisco needed to build a new version each time a customer wanted to export new fields

• When new versions are created, partners need to reengineer to support the new export format

Solution: Build a flexible and extensible export format!


Netflow v9 Principles

• Version 9 is an export format

• Still a push model

• Sent the template regularly (configurable)

• Independent of the underlying protocol, it is ready for any reliable protocol (ie: TCP, SCTP)


NetFlow v9 Export Packet

Data FlowSetTemplate FlowSet Option

Template

FlowSet

HeaderFlowSet ID #1

Data FlowSetFlowSet ID #2

Template ID

(specific

Field types

and lengths)

(version,

# packets,

sequence #,

Source ID)

• Matching ID #s is the way to associate Template to the Data Records

• The Header follows the same format as prior NetFlow versions so Collectors will be backward compatible

• Each Data Record represents one flow

• If exported flows have the same fields then they can be contained in the same Template Record e.g. unicast traffic can be combined with multicast records

• If exported flows have different fields then they can’t be contained in the same Template Record e.g. BGP next-hop can’t be combined with MPLS Aware NetFlow records

Flows from

Interface A

Flows from

Interface B

To support technologies such as

MPLS or Multicast, this export format can

be leveraged to easily insert new fields

Option Data

FlowSetFlowSet ID

Option Data

Record

(Field values)

Option Data

Record

(Field values)

Template Record

Template ID #2

(specific Field types and lengths)

Template Record

Template ID #1


Data Record

(Field values)

Data Record

(Field values)

Data Record

(Field values)


NetFlow v9 Flexible Format

Template FlowSet

Data FlowSetFlowSet ID

Data FlowSetFlowSet ID

Example of Export Packet right after router boot or NetFlow configuration

Example of Export Packets containing mostly flow information

Option Data

FlowSetFlowSet ID

Header

Header

Option Data

Record

(Field values)

Option Data

Record

(Field values)

Data Record

(Field values)

Data Record

(Field values)

Data Record

(Field values)

Data Record

(Field values)

Data Record

(Field values)

Data Record

(Field values)

Data Record

(Field values)

(version, # packets,

sequence #, Source ID)

(version, # packets,

sequence #, Source ID)

Template Record

Template ID


Template Record

Template ID


Template Record

Template ID


Template Record

Template ID


Option TemplateFlowSetTemplate

ID

(specific Field types

and lengths)


NetFlow v9 Export Packet IETF Specification

Headerversion (1, 5, 7, 8, 9)# records in Export PacketExport Packet sequence #source ID (identifies router)

Template FlowSetFlowSet ID (0)Length (bytes)

Template Record

Template ID (>255)Field Count (# fields)Field 1 TypeField 1 LengthField 2 TypeField 2 Length…Field N TypeField N Length

Template Record

Template ID (>255)Field Count (# fields)Field 1 TypeField 1 LengthField 2 TypeField 2 Length…Field N TypeField N Length

Data FlowSetFlowSet IDLength (bytes)

Data Record

Field 1 ValueField 2 Value…Field N Value

Data Record


Data FlowSetFlowSet IDLength (bytes)

Data Record


Data Record


Option Template FlowSetFlowSet ID (1)Length (bytes)Template ID (>255)Option Scope Length (bytes)Option Length (bytes)Scope 1 Field TypeScope 1 Field Length (bytes)Option 1 Field TypeOption 1 Field Length (bytes)…Option N Field TypeOption N Field Length

Option Data FlowSetFlowSet IDLength (bytes)

Option Data Record

Scope 1 ValueOption 1 Field Value…Option N Field Value

Option Data Record

Scope 1 ValueOption 1 Field Value…Option N Field Value

Option Flowsets send data associated with:

• System

• Interface

• Line Card

• Cache

• Template

Example:

The sampling rate associated with a particular interface


NetFlow v9 Export

pamela(config)# ip flow-export version ?

1

5

9

pamela(config)# ip flow-export version 9 .

Configuring Version 9 export

pamela(config)# ip flow-aggregation cache as

pamela(config-flow-cache)# enabled

pamela(config-flow-cache)# export ?

destination Specify the Destination IP address

version configure aggregation cache export version

pamela(config-flow-cache)# export version ?

8 Version 8 export format

9 Version 9 export format

pamela(config-flow-cache)# export version 9

Configuring Version 9 export for an aggregation scheme

Export versions available for standard NetFlow flows

Export versions available for aggregated NetFlow flows


NetFlow V9 and IETF

• Internet Protocol Flow Information eXport (IPFIX) is an IETF Working Group

http://ipfix.doit.wisc.edu/

• Netflow version 9 has been presented in the last IETF

• Informational RFC on NetFlow version 9 http://www.ietf.org/internet-drafts/draft-bclaise-netflow-9-00.txt

• Cisco is working on drafts for version 9




http://www.ietf.org/internet-drafts/draft-bclaise-netflow-9-00.txt

http://www.ietf.org/internet-drafts/draft-bclaise-netflow-9-00.txt


Agenda


Partners


NetFlow Infrastructure

Applications:Applications:

Router:• Cache Creation

• Data Export

• Aggregation

Router:• Cache Creation

• Data Export

• Aggregation

Collector:• Collection

• Filtering

• Aggregation

• Storage

• File System Management

Collector:• Collection

• Filtering

• Aggregation

• Storage

• File System Management

Accounting/Billing

Network Planning

Data Presentation

PartnersPartnersCisco & PartnersCisco & PartnersCiscoCiscoCisco


NetFlow Partners

CollectionCollectionTraffic AnalysisTraffic Analysis



Agenda

Customer ApplicationsGeneralEnterpriseService Provider

General


NetFlow Uses

• Attack Mitigation• User (IP)

monitoring• Application

monitoring



monitoring

• Billing• Chargeback• AS Peer

Monitoring


Monitoring

• Traffic Engineering

• Traffic Analysis


• Traffic Analysis

Ap

pli

cati

on

s • Attack Mitigation• User (IP)


monitoring



monitoring


Monitoring


Monitoring

Net

wo

rk L

ayer

AccessAccess DistributionDistribution DistributionDistribution AccessAccessCoreCore

Net

Flo

wF

eatu

res

• Aggregation Schemes (v8)

• “show ip cache flow” command

• Arbor Networks



• Arbor Networks

• NetFlow MPLS Egress Accounting

• BGP Next-hop (v9)

• Multicast NetFlow (v9)




• MPLS Aware NetFlow (v9)


• Sampled NetFlow

• MPLS Aware NetFlow (v9)


• Sampled NetFlow









• Arbor Networks



• Arbor Networks


Billing

• Flat-rate billing does not necessarily scaleCompetitive pricing models can be created with usage-based billing

• Usage-based billing considerationsTime of dayWithin or outside of the network ApplicationDistance-basedQuality of Service (QoS) / Class of Service (CoS)Bandwidth usageTransit or peerData transferredTraffic class


Tracking Users

• Who are my top N talkers, and what percentage of traffic do they represent?

• How many users are on the network at a given time?When will upgrades affect the least number of users?

• How long do users spend connected to the network?

• Where Internet sites do they use?

• What is a typical pattern of usage between sites?

• Are users staying within an acceptable usage policy (AUP)?

• Alarm DOS attacks like smurf, fraggle, and SYN floodWill watch for these attack, regardless of source / destination


Principle Netflow Benefits

Service ProviderService Provider EnterpriseEnterprise

• Internet access monitoring (protocol distribution, where traffic is going/coming)

• User Monitoring

• Application Monitoring

• Charge Back billing for departments

• Security Monitoring

• Internet access monitoring (protocol distribution, where traffic is going/coming)

• User Monitoring

• Application Monitoring

• Charge Back billing for departments


• Peering arrangements

• Network Planning


• Accounting and billing


• Peering arrangements

• Network Planning


• Accounting and billing



Current Market

• Current economic situation has sparked interest in the Service Provider and Enterprise markets

• Key areas of application

Traffic Engineering – 50%

Usaged Based Billing/Chargeback – 30%

DoS – rapidly emerging

Feature acceleration

Improved ACL performance


GeneralEnterpriseService Provider

• NetFlow Overview• Versions• Partners• Customer Applications

Enterprise

Agenda


NetFlow – Charge Back Billing

R&DHR

Finance

Account per network (rather that per IP addresses)

Internet

Example: charge the department for the cost of the Internet link


GeneralEnterpriseService Provider

• NetFlow Overview• Versions• Partners• Customer Applications

Service Provider

Agenda


NetFlow – Peering Agreement

Account per BGP AS, to Review Peering Agreements

ISP


UunetDigexErolsBBNAT&T

AMUC&WJHUPACBell Internet ServiceRCNOARnetSURAnetCompuserve

OLABSNETWebTVWEC

Public Routers 1, 2, 3 Month of September—Outbound Traffic

NetFlow – Peering Agreement

20%

32%

4%6%

8%

8%

10%

1% 1%1%

1%1%

1%

2%1%

1%1%


Agenda

MPLSAutonomous SystemMulticastBGP Next-hopAttack Mitigation – Denial of ServiceLayer 2 TechnologiesQuality of Service

MPLS

• NetFlow Overview• Versions• Partners• Customer Applications• Solutions by Technology


MPLS Aware NetFlow (v9)

IP Fields

Source and destination IP address

Input and output sub-interfaces

Transport layer protocol

Source and destination application port numbers

8 bit IP Type of Service (ToS)

TCP Flags (accumulation from all packets in the flow)

MPLS Fields

Up to three incoming MPLS labels with experimental (EXP) bits and end-of-stack (S) bit

Position of each of the three labels

Type of the top label

IP address associated with the top label

Traditional NetFlow Fields

Number of packets

Number of bytes (count either IP or MPLS header / payload)

Time-stamps of first and last packets in the flow


MPLS

Traditional NetFlow for IP to MPLS traffic

PEPE PP PEPE

Egress MPLS NetFlow Accounting• IP information only• Ideal for billing• Current availability: Cisco IOS Software Releases 12.0(10)ST and 12.1(5)T

MPLS Aware NetFlow (version 9)• Exports up to three MPLS labels, and IP packet information• Ideal for Traffic Engineering• Will be available in Cisco IOS Software Releases 12.0(24)S, 12.2S, and 12.3

Traffic Flow

IP

IP

Egress MPLS NetFlow Accountingfor MPLS to IP traffic

MPLS Aware NetFlow (version 9)

MPLS


Agenda



Autonomous System


Autonomous System

3600-4(config)# ip flow-export version 5 ?

origin-as record origin AS

peer-as record peer AS

<cr>

3600-4(config)#

• Origin-ASSpecifies that export statistics include the origin autonomous system (AS) for the source and destination

• Peer-ASSpecifies that export statistics include the peer AS for the source and destination

Note – this configuration command is optional


Autonomous System

AS 101

Configuring Peer-AS•Source AS = AS 103•Destination AS = AS 105

NetFlow enabled

AS 103 AS 104

AS 105

AS 106Configuring Origin-AS

• Source AS = AS 101• Destination AS = AS 106

AS 102


Agenda



Multicast


Multicast NetFlow

Three types of NetFlow implementations for Multicast traffic:

1. Traditional NetFlow

2. Multicast NetFlow Ingress

3. Multicast NetFlow Egress


Multicast – Traditional NetFlow

Eth 0

Eth 3Eth 1

Eth 2

Interface Ethernet 0

ip route-cache flow

ip flow-export version 9

ip flow-export destination 127.0.0.1 9995

127.0.0.1

NetFlowCollector

server

Traditional NetFlow configuration

10.0.0.2

(S, G) - (10.0.0.2, 224.10.10.100)

Flow Record Created in NetFlow Cache

• There is only one flow per NetFlow configured input interface• The 7 Key fields that define a unique flow are marked in red • Destination interface is marked as “Null”• Bytes and Packets are the incoming values

SrcIf SrcIPadd DstIf DstIPadd Protocol TOS Flgs SrcPort SrcMsk DstPort DstMsk NextHop Bytes Packets Active Idle

Eth 0 10.0.0.2 Null 224.10.10.100 11 80 10 00A2 /24 00A2 /24 23100 21 1745 4


Multicast NetFlow Ingress


ip multicast netflow ingress



Multicast NetFlow Ingress configuration

Flow Record Created in NetFlow Cache

• There is only one flow per NetFlow configured input interface• The 7 Key fields that define a unique flow are marked in red • Destination interface is marked as “Null”• Bytes and Packets are the outgoing values


Eth 0 10.0.0.2 Null 224.10.10.100 11 80 10 00A2 /24 00A2 /24 69300 63 1745 4

Eth 0

Eth 3Eth 1

Eth 2

127.0.0.1

NetFlowCollector

server

10.0.0.2

(S, G) - (10.0.0.2, 224.10.10.100)


Multicast NetFlow Egress


ip multicast netflow egress







Multicast NetFlow Egress configuration

Flow Records Created in NetFlow Cache

• There is one flow per Multicast NetFlow Egress configured output interface• One of the 7 Key fields that define a unique flow has changed from Source Interface to Destination Interface • Bytes and Packets are the outgoing values


Eth 0 10.0.0.2 Eth 1 224.10.10.100 11 80 10 00A2 /24 00A2 /24 23100 21 1745 4

Eth 0 10.0.0.2 Eth 2 224.10.10.100 11 80 10 00A2 /24 00A2 /24 23100 21 1745 4

Eth 0 10.0.0.2 Eth 3 224.10.10.100 11 80 10 00A2 /24 00A2 /24 23100 21 1745 4

Eth 0

Eth 3Eth 1

Eth 2

127.0.0.1

NetFlowCollector

server

10.0.0.2

(S, G) - (10.0.0.2, 224.10.10.100)


Multicast NetFlow – RPF Failures

• Flow is blocked because it has the same key fields as another flow; however, it is coming from the wrong physical interface

• Can be counted using Multicast NetFlow Egress if configured “ip multicast netflow rpf-failure” globally

• Once configured, there will be a new field in the NetFlow cache called “RPF Fail” to count flows that fail and how many times


Multicast NetFlow – Summary

• Supported via NetFlow version 9 export format• Availability

Cisco IOS Software Releases 12.0(27)S, 12.2S, and 12.3 Cisco 2500, 2600, 3600, 7200, and 7500 Series RoutersCisco 12000 Series Internet Router

• Performance: Ingress vs. EgressMulticast NetFlow Ingress and traditional NetFlow will have similar performance numbers Multicast NetFlow Egress will have performance impact that is proportional to the number of interfaces on which it is enabled (include input interface)

• Cisco Catalyst 6000 and 7600 Series SwitchesDo not currently support the tracking of multicast traffic via NetFlow due to current ASIC limitationWill have this support in a future Supervisor


Agenda



BGP Next-hop


BGP next-hop

• Supported only in version 9 export

• For traffic engineering/analysis and possible billing applications

• Fields that are exported include all those found in version 5 export

• Will be supported in Cisco IOS Software Releases 12.0(26)S, 12.2S, and 12.3


BGP next-hop

pamela(config)# ip flow-export version ?

1

5

9

pamela(config)# ip flow-export version 9 .

Configuring Version 9 export

pamela(config)# ip flow-export version 9 ?

bgp-nexthop record BGP NextHop

origin-as record origin AS

peer-as record peer AS

<cr>

pamela(config)# ip flow-export version 9 bgp-nexthop

Configuring Version 9 export with BGP next-hop


Agenda



Attack Mitigation – Denial of Service


NetFlow – Mitigating Attacks

1. Cost Saver• “sh ip cache flow” command to find top volume flows

• Identify source of attack

• Write access-list to block

• Monitor via “show ip cache flow” & “Null” entry in DestIf field to show that it is blocked

• Prefix-port aggregation can be configured, while “sh ip cache flow aggregation prefix-port” is used

2. Most Effective• Arbor Networks leverages NetFlow to provide a quicker

response and more sophisticated solution


Agenda


MPLSAutonomous SystemMulticastBGP Next-hopAttack Mitigation – Denial of ServiceLayer 2 TechnologiesQuality of ServiceLayer 2 Technologies


Sub and Virtual Interface Tracking

The following interfaces are tracked• Frame Relay sub-interfaces • ATM sub-interfaces • Inter-Switch Link (ISL) sub-interfaces • 802.1q sub-interfaces • Multilink PPP interfaces • Generic Routing Encapsulation (GRE) tunnel interfaces • Layer 2 Tunneling Protocol (L2TP) VPDN-group interfaces • MPLS-VPN interfaces • Tunnel Hopping

Packet arrived on one tunnel interface of a router, and was routed to a different tunnel interface on the same router


Agenda


MPLSAutonomous SystemMulticastBGP Next-hopAttack Mitigation – Denial of ServiceLayer 2 TechnologiesQuality of ServiceQuality of Service


Quality of Service Example

DiffServ fieldAKA

IP DSCP markings

Early Congestion Notification (ECN) bits

DS5 DS4 DS3 DS2 DS1 DS0 ECN ECN

128 64 32 16 8 4 2 1

Precedence bits

ToS bits


Quality of Service ExampleTOS byte

DS5 DS4 DS3 DS2 DS1 DS0 ECN ECN128 64 32 16 8 4 2 1

Precedence bits Decimal Precedence Function1 1 1 x x x x x 224 7 Network Control (link layer keepalives)1 1 0 x x x x x 192 6 Internetwork Control (Routing Protocols)1 0 1 x x x x x 160 5 CRITIC/ECP (Express Forwarding)1 0 0 x x x x x 128 4 Flash Override (Class 4)0 1 1 x x x x x 96 3 Flash (Class 3)0 1 0 x x x x x 64 2 Immediate (Class 2)0 0 1 x x x x x 32 1 Priority (Class 1)0 0 0 x x x x x 0 0 Routine (Best effort)

Delay, Throughput, and Reliability bitsDelay bit

x x x 0 x x x x 0 Delay - normalx x x 1 x x x x 16 Delay - low

Throughput bitx x x x 0 x x x 0 Throughput - normalx x x x 1 x x x 8 Throughtput - high

Reliability bitx x x x x 0 x x 0 Reliability - normalx x x x x 1 x x 4 Reliability - high

Early Congestion Notification (ECN) bitsECN-capable Transport (ECT) bit

Congestion Experienced (CE) bitx x x x x x 0 0 0 Not ECN-capablex x x x x x 0 1 1 Endpoints of transport protocol ECN-capablex x x x x x 1 0 2 Endpoints of transport protocol ECN-capablex x x x x x 1 1 3 Congestion experienced


Tracking TOS with NetFlow

7200-3-netflow# show ip cache verbose flowSrcIf SrcIPaddress DstIf DstIPaddress Pr TOS Flgs PktsPort Msk AS Port Msk AS NextHop B/Pk ActiveSR6/0 210.210.210.2 PO1/0 200.200.200.2 FF 00 10 21K0000 /0 0 0000 /0 0 0.0.0.0 1496 665.4SR6/0 210.210.210.2 PO1/0 200.200.200.2 06 C0 00 21K0000 /0 0 0000 /0 0 0.0.0.0 1496 666.0

7200-3-netflow# show ip cache verbose flow SrcIf SrcIPaddress DstIf DstIPaddress Pr TOS Flgs PktsPort Msk AS Port Msk AS NextHop B/Pk ActiveEt1/1 52.52.52.1 Fd4/0 42.42.42.1 01 55 10 37480000 /8 50 0000 /8 40 202.120.130.2 28 17.8Et1/2 52.52.52.1 Fd4/0 42.42.42.1 01 CC 10 35680000 /8 50 0000 /8 40 202.120.130.2 28 17.8Et1/2 10.1.3.2 Fd4/0 42.42.42.1 01 C0 10 11240000 /0 0 0000 /8 40 202.120.130.2 28 17.8

Hex Decimal Binary

55 85 0101 0101 Precedence 2 - Immediate (Class 2), Delay - low, Reliability - high, Endpoints of transport protocol ECN-capable

C0 192 1100 0000 Precedence 6 - Internetwork Control (Routing Protocols)

CC 204 1100 1100 Precedence 6 - Internetwork Control (Routing Protocols), Throughput - high, Reliability - high


Agenda


Features and Uses


Sampled NetFlow

• Deterministic Original typeCisco 12000 Series Internet RoutersCisco Catalyst 6500 Series Switches – Release 12.1(13)E

• Random (recommended per statistical principles)Cisco IOS Software Releases 12.0(26)S, 12.2S, and 12.3 Cisco 2500, 2600, 3600, 7200, and 7500 Series RoutersCisco 12000 Series Internet Routers

• Time-based Cisco Catalyst 6500 Series Switches – Release 12.1(13)E

• Trajectory (Hash-based)in development



Agenda

Platform SpecificsCisco Catalyst 6500 and 7600 Cisco 12000 Series Internet RouterNetflow Collector

Cisco Catalyst 6500 and 7600


Cisco Catalyst 6500 and 7600 Series Switches

• NetFlow is supported on both Hybrid and Native Cisco IOS Software

Hybrid: Cisco Catalyst OS on Supervisor and Cisco IOS Software on MSFC

Native Cisco IOS Software: Supervisor and the MSFC both run a single bundled Cisco IOS Software Image

• NetFlow support is more extensive via Native Cisco IOS Software


Cisco Catalyst 6500 and 7600 Series Switches (Cont)

• NetFlow record creation is supported in ASICs; exporting NetFlow is supported in software

• MSFC is for routing, PFC with Supervisor for switching

• Supervisor 1 & PFC – version 7

• Supervisor 2 & PFC2 – version 5, 7, or 8


• NetFlow Overview• Versions• Partners• Customer Applications• Solutions by Technology• Features and Uses• Platform Specifics

Agenda

Cisco Catalyst 6500 and 7600 Cisco 12000 Series Internet RouterNetflow CollectorCisco 12000 Series Internet Router


Cisco 12000 Series Internet Routers – NetFlow

• Engine 0 – software support

• Engine 1 – software support

• Engine 2 – supported in ASICs, but lower priority so beware if running many other features

• Engine 3 – version 5 support in software, version 8 support in ASIC

• Engine 4 – not supported

• Engine 4+ – supported in ASICs


Cisco 12000 Series Internet Routers Sampled NetFlow

• Engine 0 – both “full” and Sampled NetFlow

• Engine 1 - both “full” and Sampled NetFlow

• Engine 2 – Sampled NetFlow only

• Engine 3 – Sampled NetFlow only

• Engine 4 – not supported

• Engine 4+ - Sampled NetFlow only


Cisco 12000 Series Internet Routers Sampled NetFlow

Engine Full NetFlow Sampled NetFlow

0

1

2

3

4

4+

Not supportedSupported


• NetFlow Overview• Versions• Partners• Customer Applications• Solutions by Technology• Features and Uses• Platform Specifics

Agenda

Cisco Catalyst 6500 and 7600 Cisco 12000 Series Internet RouterNetflow CollectorNetflow Collector


NetFlow Collector 4.0

• Version 5, 7, 8, and 9

• XML interface

• Cafeteria aggregation

• Performance improvement

• .csv export e.g. export to MS Excel

• Solaris 2.7/2.8, Linux 7.1

• Appliance



NetFlow Performance Testing PaperNetflow Feature and AccelerationReducing Performance Impact

Agenda

PerformanceNetFlow Performance Testing Paper


NetFlow Performance Testing PaperPlatforms

• Cisco 2600 Series Routers

• Cisco 3600 Series Routers

• Cisco 7200 Series Routers NPE-400

NSE-1

• Cisco 7500 Series Routers RSP8 VIP4-80 with CEF and dCEF

• Cisco 12000 Series Internet Routers Engine 1 Linecard dCEF with “full” NetFlow versus 1:100 sampled NetFlow


NetFlow Performance Testing PaperTests

• Access lists (ACLs) 200 and 500 lines

• 0, 1, and 2 NetFlow Data Export destinations

• Initial performance after enabling

• V8 Aggregation vs. v5

• Configuring AS origin or peer

• Policy Based-Routing (PBR)


Performance Testing Conclusions

• NetFlow Data Export (single/dual)No significant impact

• NetFlow v5 versus v8: little or not impact• NetFlow Feature Acceleration:

>200 lines of ACLs and/or Policy Based-Routing (PBR)

• NetFlow versus Sampled NetFlow on the Cisco 12000 Series Internet Routers

23% versus 3% (65,000 flows, 1:100)

Number of Active Flows Additional CPU Utilization

10,000 <4%

45,000 <12%

65,000 <16%

• Additional CPU utilization


Performance TestingNetFlow Version 9

• Similar CPU and throughput numbers result from configuration of both NetFlow version 5 and 9

• No change in NetFlow performance after the addition of version 9

Cisco IOS Software Releases 12.0(24)S, 12.2S, and 12.3

• CPU is slightly higher immediately following initial boot up or configuration

Caused by sending Template Flowsets to Collector


Agenda

NetFlow OverviewVersionsPartnersCustomer ApplicationsSolutions by TechnologyFeatures and UsesPlatform SpecificsPerformance

NetFlow Performance Testing PaperNetflow Feature and AccelerationReducing Performance ImpactReducing Performance Impact


Reducing Performance Impact

Reduce CPU and memory impact on the router, collector, or network:

• Aging timers (router)

• Sampled NetFlow (router)

• Enable NetFlow Feature Acceleration (router)

• Flow Masks (only Cat6000/7600)

• Enable on specific sub-interface (upcoming router feature)

• Aggregation schemes (v8 on router or on collector)

• Filters (router or collector)

• Data Compression (collector)

• Increase collection bucket sizes (collector)

• Collector and router can be placed on the same LAN segment (network)


Agenda

• NetFlow Overview• Versions• Partners• Customer Applications• Solutions by Technology• Features and Uses• Platform Specifics• Performance• Roadmap and Future Direction• SummarySummary


NetFlow Deployment Managerial Advice

• Current economic environment drives the need to justify the cost of premium service(s) Accounting

• IT and management need to agree on which fields to track

• Where in the network (access, distribution, or core)?

• Crucial to set appropriate expectations for management with regards to frequency of NetFlow reports

• Cisco recommends a trial deployment in one department/area before network-wide implementation


NetFlow Deployment Technical Advice

• IT and management need to agree on which fields to trackCurrently and in the future

• Do not export versions 5,7, and 9 simultaneously with version 8

• Plan NetFlow deployment in the network topology to avoid a design that creates duplicate flows for billing

• Use a dedicated interface / VLAN for data export

• Monitor lost packet counter in NFC

• Check the export link bandwidthEstimated export of 1% to 1.5% of the interface throughput


NetFlow Summary

• Current economic environment drives need to cost-justify, and charge for IT network rollout / Service Provider premium services

• NetFlow is the primary Cisco accounting technology

• Cisco has IETF / industry leadership

• NetFlow is stable and proven code Only three bugs as of February 2003

• Cisco continues to invest in the technology

• Version 9 eases the exporting of additional fieldsContact your AM with requests for additional fields

90Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved.

Documents

© 2002, Cisco Systems, Inc. All rights reserved. NetFlow Overview, 2/03