PLNOG 13: Krzysztof Mazepa: BGP FlowSpec

BGP FlowSpec

Krzysztof Mazepa

Service Provider Networks Service Provider Networks Service Provider Networks Service Provider Networks ArchitectArchitectArchitectArchitect, Cisco Systems Poland, Cisco Systems Poland, Cisco Systems Poland, Cisco Systems Poland

September 2014

2© 2013-2014 Cisco and/or its affiliates. All rights reserved.

RFC 5575 „Dissemination of Flow Specification Rules”

„ ... a new Border Gateway Protocol Network Layer Reachability Information (BGP NLRI) encoding format that can be used to distribute traffic flow specifications. This allows the routing system to propagate information regarding more specific components of the traffic aggregate defined by an IP destination prefix”

ang. disseminate – rozpowszechniać

Flowspec


Powrót do przeszłości – „Wpływ wdrożenia rozwiązańAnti-DDoS na architekturę sieci operatora” – marzec 2014, PLNOG

BGP FlowSpec – remedium na ataki DDoS czy kolejne narzędzie w „koszyku” operatora ?

Parę szczegółów technicznych ...

Zobacz jak to działa na twoim laptopie czyli XRv w akcji ...

Agenda


TheTheTheThe Internet

• Denial Denial Denial Denial of Service of Service of Service of Service attacks attacks attacks attacks are of different natures:

• Application-layer attacks• Detected and handled by Firewalls, IDS or at the Server level

• Volumetric attacks (including Protocols attacks)• Can NOT be mitigated in datacenter or server farm (too late)• Should be handled in the backbone or at the border� Focus of this presentation

Next 4 slides comes from March PLNOG 2014 edition ...

IPS/IDSIPS/IDSIPS/IDSIPS/IDS FirewallFirewallFirewallFirewall

DPIDPIDPIDPI

WebWebWebWebServerServerServerServer

WebWebWebWebCacheCacheCacheCache

DatabaseDatabaseDatabaseDatabase

EdgeEdgeEdgeEdge

PeeringPeeringPeeringPeeringTransitTransitTransitTransit

CoreCoreCoreCoreDCDCDCDC


BlackHolingBlackHolingBlackHolingBlackHoling

� RTBH� BGP dummy route advertised� Route to null or route to a forensic probe

� Based on source or destination address

� Future better granularity with FlowSpec

� All traffic (good and bad) dropped

� Limits collateral damages but attackers’ main objective attained

How to React?In-House Solutions: BlackHoling vs Mitigating


MitigationMitigationMitigationMitigation

• SinkHoling to scrubbing device(s)

• Differentiation of legitimate and malicious traffic

• Victim’s services maintained

• Collateral damages avoided

How to React?In-House Solutions: BlackHoling vs Mitigating


• Addressing a DDoS attack is a process performed in several steps at different locations of your network and by different products

DDoS Mitigation Process

– DetectionDetectionDetectionDetection: action of identifying an abnormal behavior in the network and trigger an alert for the operator

– offRampoffRampoffRampoffRamp: action of diverting traffic targeted to the victim to the scrubbing device

– MitigationMitigationMitigationMitigation: action performed by the cleaning device to differentiate legit traffic from attack traffic and block the second

– onRamponRamponRamponRamp: action of re-injecting the legit traffic into the network and guarantee it will be able to reach the destination

Diversion

“offRamp”

Router

Re-injection

“onRamp”

Router

Mitigation

Appliance

Detection

Router

Collector


New NLRI defined (AFI=1, SAFI=133)

Notice from the RFC: “Flow specification components must follow strict type ordering. A given component type may or may not be present in the specification, but if present, it MUST precede any component of higher numeric type value.”

1. Destination IP Address (1 component)

2. Source IP Address (1 component)

3. IP Protocol (+1 component)

4. Port (+1 component)

5. Destination port (+1 component)

6. Source Port (+1 component)

7. ICMP Type

8. ICMP Code

9. TCP Flags

10. Packet length

11. DSCP

12. Fragment

Dissemination of Flow Specification Rules (RFC5575)

The The The The MP_REACH_NLRIMP_REACH_NLRIMP_REACH_NLRIMP_REACH_NLRI –––– RFC 4760RFC 4760RFC 4760RFC 4760

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9

• Type 1: Destination prefix component

• Type 3: IP Protocol component

The option byte is defined as following:

E bit: end of option (Must be set to 1 for the last Option)

A bit: AND bit, if set the operation between several [option/value] is AND, if unset the operation is a logical OR. Never set for the first Option

Len: If 0 the following value is encoded in 1 byte ; if 1 the following value is encoded in 2 bytes

Lt bit: less than comparison between the Data and the given value

Gt bit: greater than comparison between the Data and the given value

Eq bit: equal comparison between the Data and the given value

• Type 2: Source prefix component


• Type 4: Port number component

• Type 6: Source port number component

• Type 8: ICMP Code component

• Type 5: Destination port number component

• Type 7: ICMP Type component


• Type 9: TCP Flags component

The option byte is defined as following:

E bit: end of option (Must be set to 1 for the last Option)

A bit: AND bit, if set the operation between several [option/value] is AND, if unset the operation is a logical OR. Never set for the first Option

Len: If 0 the following value is encoded in 1 byte ; if 1 the following value is encoded in 2 bytes

NOT bit: logical negation operation between Data and the given value

m bit: match operation between the Data and the given value

• Type 10: Packet Length component

• Type 11: DSCP Value component

• Type 12: Fragment component


BGP Flowspec Traffic Actions

� Flowspec Traffic Actions

Extended Community – RFC 4360

� RFC5575 Flowspec available actions


• Traffic-rate action

Used for discard or rate-limit a specific flow. Discard action is actually a rate equal to zero. The remaining 4 octets carry the rate (in Bytes/sec) information in IEEE floating point [IEEE.754.1985] format.

• Traffic-action action

Used to trigger specific processing the corresponding flow. Only the last 2 Bits of the 6 bytes are currently defined

Terminal Action (bit 47): When this bit is set, the traffic filtering engine will apply any subsequent filtering rules (as defined by the ordering procedure). If not set, the evaluation of the traffic filter stops when this rule is applied.

Sample (bit 46): Enables traffic sampling and logging for this flow specification.


• Redirect action

Traffic redirection allows to specify a “route-target” community which will be handled by the router to redirect a Flow to a specific VRF.

• Traffic-marking action

Used to force a flow to be re-writted with a specific DSCP value when it leaves the routers.


RFC5575 Architecture

Customer InfraCustomer Infra Provider InfraProvider Infra

WebsiteWebsite

IP=1.2.3.4

CECECECE InternetPEPEPEPE

Transit1Transit1Transit1Transit1


UDP DDoS Traffic

UDP DDoS Traffic

BGP : 1.2.3.0/24

IP Destination: 1.2.3.4/32IP Protocol 17 (UDP)PacketSize <=28Rate-limit 10M


RFC5575 Architecture


WebsiteWebsite

IP=1.2.3.4




UDP DDoS Traffic

BGP : 1.2.3.0/24

IP Destination: 1.2.3.4/32IP Protocol 17 (UDP)PacketSize <=28Rate-limit 10M


Real life architecture

� In reality RFC 5575 „native” architecture is not deployed� Service Provider DO NOT trust the Customer

� It requires new BGP AFI/SAFI combination to be deployed between Customer and Service provider

� Both these result in Flowspec not being deployed between Customer and service provider

� What is done instead?� SP utilize a central Flowspec speaker(s)

� Have it BGP meshed within the Service Provider routers

� Only the central Flowspec speaker is allowed to distribute Flowspec rules

� Central Flowspec speaker is considered “trusted” by the network (no-validate)

� Central Flowspec speaker is managed by the service provider


„real life” RFC5575 Architecture


WebsiteWebsite

IP=1.2.3.4




UDP DDoS Traffic

BGP : 1.2.3.0/24

FlowspecFlowspecFlowspecFlowspec

Rules inserted by:Rules inserted by:Rules inserted by:Rules inserted by:CLICLICLICLICustomer WebpageCustomer WebpageCustomer WebpageCustomer WebpageOtherOtherOtherOther

Legitimate TCP Legitimate TCP Legitimate TCP Legitimate TCP TrafficTrafficTrafficTraffic


BGP FlowSpec RFC/drafts supported (phase 1)

� Flowspec support comprises of the following• Base RFC: RFC 5575

• IPv6 extensions: draft-ietf-idr-flow-spec-v6-03

• Redirect IP extension: draft-simpson-idr-flowspec-redirect-02

• IBGP extension: draft-ietf-idr-bgp-flowspec-oid-01

• Persistence Support: draft-uttaro-idr-bgp-persistence-02 *

• HA/NSR Support.


flowspec[local-install interface-all]address-family ipv4[local-install interface-all]service-policy type pbr <policy-name> [{local | remote}]service-policy type pbr <policy-name> [{local | remote}]

address-family ipv6[local-install interface-all]service-policy type pbr <policy-name> [{local | remote}]service-policy type pbr <policy-name> [{local | remote}]!

! vrf <vrf-name>address-family ipv4[local-install interface-all]service-policy type pbr <policy-name> [{local | remote}]service-policy type pbr <policy-name> [{local | remote}]

address-family ipv6[local-install interface-all]service-policy type pbr <policy-name> [{local | remote}]service-policy type pbr <policy-name> [{local | remote}] !

!!

!Interface <interface-name>{ipv4|ipv6} flowspec disable!!#local-install interface-all configuration is a knob to turn on for local installation of flowspec policy on all interfaces

VRFVRFVRFVRF and and and and flowspecflowspecflowspecflowspec policy mapping for distribution of policy mapping for distribution of policy mapping for distribution of policy mapping for distribution of flowspecflowspecflowspecflowspec rulesrulesrulesrules


FlowFlowFlowFlow----spec Policy and classspec Policy and classspec Policy and classspec Policy and class----map definitionmap definitionmap definitionmap definition

class-map type traffic match-all <class-name><match statement> #Any combination of tuples 1-13 match statements goes here.end-class-map!

!

policy-map type pbr <policy-name>class type traffic <class-name>

<action> #Any one of the extend community action listed belowclass class-defaultend-policy-map!

!


Match filters corresponding to BGP flowspec NLRI

#Type 1: match destination-address {ipv4| ipv6} <IPv4/v6 address>/<mask-length>

#Type 2:match source-address {ipv4| ipv6} <IPv4/v6 address>/<mask-length>

#Type 3:match protocol { <value> | <min_value> - <max_value> }<!In case of IPv6, it will map to last next-header >

#Type 4:#create two class-maps one with source-port and another with destionation-portmatch source-port { <value> | <min_value> - <max_value> }match destination-port { <value> | <min_value> - <max_value> }<! Applicable only for TCP and UDP protocol !>

#Type 5:match destination-port { <value> | <min_value> - <max_value> }

#Type 6:match source-port { <value> | <min_value> - <max_value> }

#Type 7:match {ipv4 | ipv6} icmp-type { <value> | <min_value> - <max_value>}


Match filters corresponding to BGP flowspec NLRI

#Type 8:match {ipv4 | Ipv6} icmp-code { <value> | <min_value> - <max_value>}

#Type 9:match tcp-flag <value> bit-mask <mask-value>

#Type 10:match packet length { <value> | <min_value> - <max_value> }

#Type 11:match dscp { <value> | <min_value> - <max_value> }match ipv6 traffic-class { <value> | <min_value> - <max_value> }

[for providing 8 bit traffic class value]

#Type 12:match fragment-type {dont-fragment, is-fragment, first-fragment, last-fragment}

#Type 13:match ipv6 flow-label { <value> | <min_value> - <max_value> }


Mapping between Extended community value and policy action

##Traffic rate:police rate < > |

drop

#Traffic action:sample-log

#Traffic marking:set dscp <6 bit value> |

set ipv6 traffic-class <8 bit value>

#VRF redirect based on route-target:redirect {ipv6} extcommunity rt <route_target_string>

# Redirect IP nexthop supportredirect {ipv6} next-hop <ipv4/v6 address> {ipv4/v6 address}


Example C3PL flowspec rule configuration

• Two flowspec rules are created for two different VRFs as follows.

• Goal

• all packets to 10.0.1/24 from 192/8 and destination-port {range [137, 139] or 8080, rate limit to 500 bps in blue vrf and drop it in vrf-default. Also disable flowspec getting enabled on gig 0/0/0/0.

class-map type traffic match-all fs_tuplematch destination-address ipv4 10.0.1.0/24match source-address ipv4 192.0.0.0/8match destination-port 137-139 8080end-class-map!

!policy-map type pbr fs_table_blueclass type traffic fs_tuplepolice rate 500 bps!

!class class-default!end-policy-map

!

!policy-map type pbr fs_table_defaultclass type traffic fs_tupledrop!!class class-default!end-policy-map!


Example flowspec config for vrf distribution

flowspeclocal-install interface-alladdress-family ipv4service-policy type pbr fs_table_default!

! vrf blueaddress-family ipv4service-policy type pbr fs_table_blue local!

!!

!

Interface GigabitEthernet 0/0/0/0vrf blueipv4 flowspec disable!

!


Implementation Considerations

� Flowspec rules can be injected in the Cisco box through C3PL CLI. It will not be regular expression as mentioned in the RFC and needs to be converted into equivalent C3PL CLI format. In later phase, Flowspec rules can be configured through ONEPK/SDN where we can allow flexible Flowspec rule specification as a part of new ONEPK application.

� BGP Flowspec allows very flexible way of providing match filters on L3 and L4 packet fields in the form of regular expressions. Internally flowspec rule is converted into C3PL policy and represented as hardware TCAM entries in respective platforms. This means complex flowspec rule with multi-value range going to consume more number of TCAM entries. It will cause TCAM space exhaustion in scaled complex flowspec rules.

� Scale of other TCAM based features like ACL, QOS, etc., in the LC will decrease the TCAM space available for BGP flowspec.

� In phase-1, Coexistence of other PBR based features like PBTS, etc., won’t be allowed with flowspec as only one PBR feature can be associated with given an interface in the LC.


� BGP Functional Structure

� Support for four new SAFIs will be defined in BGP to carry Flowspec routes

� Internet Flowspec in context of ipv4/ipv6 Flowspec SAFIs

� VPN Flowspec in context of vpnv4/vpnv6 Flowspec SAFIs to carry VRF specific flow routes (disjoint RTs can be used)

XR BGP Flowspec Implementation

Sig. Length

Internet Internet Internet Internet FlowSpecFlowSpecFlowSpecFlowSpec

router bgp 100address-family ipv4 flowspec!address-family ipv6 flowspec!neighbor 1.1.1.1remote-as 100address-family ipv4 flowspec!address-family ipv6 flowspec!

VPN VPN VPN VPN FlowspecFlowspecFlowspecFlowspec

router bgp 100address-family vpnv4 flowspec!address-family vpnv6 flowspec!neighbor 1.1.1.1remote-as 100address-family vpnv4

flowspec!address-family vpnv6

flowspec!

!vrf fooaddress-family ipv4

flowspec!address-family ipv6

flowspec!neighbor 10.10.10.10remote-as 100address-family ipv4

flowspec!address-family ipv6

flowspec!


BGP FlowSpec Validation

� BGP Flowspec validation would comprise of the following

� BGP will perform base validation of Flowspec routes as per RFC 5575 per default

� Ensure that the originator of the flow specification matches the originator of the best-match unicast route for the destination

� No more-specific routes from a different AS than that of the originator.

� The validation procedure will be enhanced to accommodate iBGP flowspec route sourcing on a centralized route-controller as per draft-ietf-idr-bgp-flowspec-oid-01 to allow for empty as-paths

� Effectively relax the originator check for Flowspec routes sourced on the iBGP controller

� Flowspec validation can be disabled as a whole for eBGP sessions by means of configuring an explicit knob.

Disabling ValidationDisabling ValidationDisabling ValidationDisabling Validationrouter bgp 100neighbor x.x.x.xaddress-family ipv4/6 flowspecvalidation disable


BGP FlowSpec IP Redirect support

• Support for draft-simpson-idr-flowspec-redirect-02 (Ability to source a IP redirect route)

� The redirect nexthop can be explicitly configured as part of the flowspec route specification

� Redirect nexthop is encoded as the MP_REACH nexthop in the BGP flowspec NLRI along with associated extended community.

� Recipient of such a flowspec route redirects traffic as per FIB lookup for the redirect nexthop, the nexthop can possibly resolve over IP/MPLS/tunnel.

� Since the MP_REACH nexthop can be overwritten at a ebgp boundary, for cases where the nexthop connectivity spans multiple ASes, the nexthop can be preserved through the use of the unchanged knob.

Preserving Redirect Preserving Redirect Preserving Redirect Preserving Redirect NexthopNexthopNexthopNexthop

router bgp 100neighbor x.x.x.xaddress-family ipv4/6 flowspecnext-hop-unchanged


The Show CLI output will include NLRI/flowspec rule detail and packet counters for each of them.

Summaryshow flowspec summary [vrf {<vrf> | all}] {ipv4 | ipv6 | afi-all} summary

Brief – 1-line per NLRI/flowspec ruleshow flowspec [vrf {<vrf> | all}] {ipv4 | ipv6 | afi-all}

Detail – Expanded info for each NLRI/flowspec ruleshow flowspec [vrf {<vrf> | all}] {ipv4 | ipv6 | afi-all} detail

Following filters will be provided to sort show output in the first phase• Source and destination IPV4/V6 address with mask• Protocol number• Source and destination port• [All the 12 tuples will be targeted as a best effort for first phase]

Flowspec Show CLI

Cisco Confidential 32© 2013 Cisco and/or its affiliates. All rights reserved.

RP/0/0/CPU0:ios# show ru class-map type traffic c2Thu Oct 10 11:05:49.459 PDTclass-map type traffic match-all c2match destination-address ipv6 2001:2::3/128match source-address ipv6 2002:2::3/128match destination-port 1-5 7-11 13-18 20-25 27-31 match source-port 33-37 39-43 45-50 53-58 60-65 match protocol 67-71 73-80 85-90 95-105 110-115 match ipv6 icmp-type 35 match ipv6 icmp-code 55 match packet length 120-130 135-140 145-160 165-200 205-225 match dscp 1-10 11-20 22-30 32-40 52-60 match fragment-type is-fragment dont-fragment first-fragment last-fragmentmatch tcp-flag 240 anyend-class-map!RP/0/0/CPU0:ios#show ru policy-map type pbr p1policy-map type pbr p1class type traffic c1 police rate 200 bps !redirect nexthop 223.255.254.254 set dscp 45!class type traffic class-default !end-policy-map!RP/0/0/CPU0:ios# show flowspec vrf all ipv4 nlriThu Oct 10 23:22:38.889 PDTVRF: default AFI: IPv4NLRI (Hex dump) : 0x011802020202180202020303434547034945500355455a035f4569036ec57305030145050307450b030d451203144519031bc51f06032145250327452b032d45320335453a033cc541078123088137099000f00a037845820387458c039145a003a545c803cdc5e10b0301450a030b45140316451e032045280334c53c0c810fActions :Traffic-rate: 200 bps DSCP: 45 Nexthop: 223.255.254.254 (policy.1)

Sample show flowspec CLI


- XRv – wersja 5.2

- platforma wirtualizacyjna (wg. preferencji)

- jeżeli używasz oprogramowania Windows oraz np. VMWare Workstation poszukaj dodatkowo oprogramowania „Named Pipe TCP proxy”

DEMO

Co potrzebujesz mieć by zobaczyć FlowSpec „control plane” ?

Thank you.

