BGP Peering Strategy & Data - 2017. · PDF file28/02/2017 BGP Traffic Engineering, Andy...

BGPPeeringStrategy&Data

_________________________________AndyDavidsonandy@asteroidhq.com

CTO@AsteroidInternationalAPRICOT2017,HCMC,VietnamDirector@LONAP,IXLeeds,Euro-IX March2017

BGPTrafficEngineering– version2.0(2016)

28/02/2017 BGPTrafficEngineering,AndyDavidson 2

WhydoTrafficEngineering?

Manageyourcapacitydemands

Ensureservicequality

RecoverfromFailures

Manageservice/circuitcosts

HandletrafficgrowthJamesCridland http://www.flickr.com/photos/jamescridland/

Complexity

Lifestartsoutverysimply,“sendtraffictopeersifpossible,thentransitproviders”

Butwhataboutwhenyournetworkgrows?

Whataboutwhenyourtrafficgrows?

Whatifyouaddmorecities/POPs/exchanges?Caisey Hussain Bisson -http://www.flickr.com/people/maisonbisson/

Realexamples

• Circuitswithcostdifference>$100/Mbit• Regionalnetworks- poorlocalpeering• Circuitfailurecausingcongestion• Changingcustomerdemand/behaviour– Increasedqualityexpectation– Newhighbandwidthservicessuchasvideo

InternalnetworkTE

• Simple comparedwithInterdomain TE

• Youadministratebothsides– Youknowtheprice ofallpaths– TheIGPknowsthecapacity ofallpaths– IGPprotocolsletyoumapprice,capacitytoshaperoutingusingcost.

Inter-domainTE

• YoudoNOTcontrolbothsides– Pathvectorprotocolshidemetric,capacity,cost– SimplicityofBGPprotocolimposeslimitations– Volumeoftrafficmatters,not#ofroutes

• However,largevolumeoftrafficisusuallywithasmallnumberofotherASNs

Youneeddata

AS-StatsManuelKasperhttps://neon1.net/as-stats/

..Butmoreonthislater

Netflow

• Export informationaboutpacketsroutedthroughyournetwork

• Normallysampled• SenttoacollectoroverUDP• Avarietyofcommercialandopen-sourcetoolssortanddisplaytheseflowrecords.

DifferentFlowprotocols

• Netflow – DesignedbyCiscoin‘90s,publishedasastandard(v9isRFC3954andsupportsIPv6)

• IPFIX(RFC5101)BasedonNetflow 9,2008• sFlow – NiceprotocolbutincompatiblewithNetflow,typicallyimplementedonL2switch.

• Jflow – EssentiallyNetflow onJunipers

Otherwaystogetdata

• Logfileanalysis– Usefulbeforeyouhaveanetwork,forworkingoutthebenefitofbuildinganetwork/peering.

– Bestfor‘singleservice’networks• DNSprovidershaveDNSlogswithtime&IP• Webprovidershaveweblogswithtime&IP• Hostedemailprovidershavemaillogs…

IPAddress Timeanddate AmountofTraffic

Datatellsyou

• Yourtrafficdirection–Mainlyinbound–Mainlyoutbound– Balanced

• Yourtoptrafficoriginatorsordestinations

As-stats

• https://neon1.net/as-stats/

• Opensource• Quicktosetup,simpletouse• Resourceintensive

Whoaremykeypeers?

(orpotentialpeers)

Toporiginatorsoftraffictome,topconsumersofmycontent

Chartcolour relatestoaninterfaceontheedgeofmynetwork

Historicaldata

Newpeeringadded,trafficgrowth!=$$$$$formynetwork!

Bonusplease!

Footballstarts

Footballends

Videocdn

SocialNetwork

Inmon sflow-rt

• http://www.sflow-rt.com/download.php

• Producesrealtime trafficgraphs• “Whatisthesituationrightnow”• Usefultocheckpeeringconfig hastakeneffect• Alsousefulinabusemitigation

Realtime aggregatedata

Realtime transitanalysis

ExportfromSflow-RT

• RESTful exportintologstash/influxdb/grafanaforhistoricaldata

• http://www.slideshare.net/pphaal/network-visibility-and-control-using-industry-standard-sflow-telemetry

pmacctd

• http://www.pmacct.net

• OpenSource• Highperformance,highscale,powerful• Mostflexible,mostconfigurationrequired

• Collectorà ownreports

1)Configureacollectorsfacctd_port: 2100sfacctd_as: sflowsfacctd_renormalize: true!plugins: print[testprint]!aggregate[testprint]: in_iface, out_iface, proto, peer_src_ip, peer_dst_ip, peer_dst_as, peer_src_as, src_as, dst_as!print_output_file[testprint]: /path/to/spool/blabla-$peer_src_ip-%Y%m%d-%H%M.txtprint_output[testprint]: csvprint_output_separator[testprint]: ;print_refresh_time[testprint]: 60print_history[testprint]: 1mprint_history_roundoff[testprint]: m

UsetheASNdatafromtherouterifitexists,noneedtosetupBGPflowexport

2)GetareportSRC_AS;DST_AS;PEER_SRC_AS;PEER_DST_AS;PEER_SRC_IP;PEER_DST_IP;IN_IFACE;OUT_IFACE;PROTOCOL;PACKETS;BYTES41230;224;41230;2603;x.x.x.253;x.x.x.246;3;4;tcp;2048;15155241230;15169;41230;15169;x.x.x.253;x.x.x.246;3;4;tcp;10240;88064041230;50247;41230;24724;x.x.x.253;x.x.x.246;3;4;tcp;2048;16793641230;9269;41230;1273;x.x.x.253;x.x.x.237;3;3;tcp;2048;13516841230;3356;41230;1273;x.x.x.253;x.x.x.237;3;3;tcp;32768;237568041230;209;41230;1273;x.x.x.253;x.x.x.237;3;3;udp;2048;294092820940;0;20940;0;x.x.x.253;x.x.x.246;3;4;tcp;43008;65458176

RedlinerepresentsasingleflowwithGoogleonbehalfofauser

Keepinghistoricaldataplugins: mysql[5mins], mysql[hourly]

sql_optimize_clauses: truesql_dont_try_update: truesql_multi_values: 1024000

sql_history_roundoff[5mins]: msql_history[5mins]: 5msql_refresh_time[5mins]: 300sql_table[5mins]: acct_bgp_5mins

sql_history_roundoff[hourly]: hsql_history[hourly]: 1hsql_refresh_time[hourly]: 3600sql_table[hourly]: acct_bgp_1hr

plugin_buffer_size: 10240plugin_pipe_size: 1024000aggregate: tag, src_as, dst_as, peer_src_as, peer_dst_as, peer_src_ip,

peer_dst_ip, local_pref, as_path

Examplebypmacct authorPaoloLucente

• Exportintoreports,webinterface,spreadsheet• Multiplebackendssupported,includingtimeseriesdatabases• Veryflexibleapproachbutneedsmoresetuptime

Kentik

• https://www.kentik.com

• Hostedsolution• Zeroconfiguration,zeroequipmentneeded• PointNetflow attheircollectorandreportsfollow

TrafficbySourceASN

Breakdownbyregion(US)

Topflowsperinterface

Outboundvs Inbound• Outboundheavynetworks– Somewhateasierlife

• Inboundheavynetworks– Youmusttrick theBestPathSelectionmethodsofnetworks sendingyoutraffic.

– Their config changewillmoveyour traffic.

OutboundHeavyTrafficEngineering

• BGPBestPathSelectionAlgorithm• Trafficengineeringisabout‘tricking’thisprocess• Affectstrafficinoutbounddirection– LocalPreference– ASPATHlength– LowestOriginType– LowestMED– PrefereBGP paths– LowestIGPMetric– Oldestroute

Mainlyoutbound,singlePOP

• Localpref– Ahammer – blunttool,inflexible..Butitisatool.– “Generally”prefertosendtraffictocustomers,thenpeers,thentransits.

–Managetop‘n’networks,sothatthereisapreferredpath,andafailurepath,withcapacityonbothcircuits.

Localpref – blunthammer

10.0.0.0/8Localpref 100via10012310.0.0.0/8Localpref 500via300200200200200123

Whichlinkwillyouprefer?AS123hereistryingtoshapeinboundtrafficviaAS100.Why?

Highercapacitylink?Morereliable?

Whatshouldyoudo?

Answer:Itdependsonthevolume oftraffic,cost ofcapacity,value oftraffic

Mainlyoutbound– ManyPOPs

• Usehotpotatoroutingtobesteffect– Nearestexitrouting– UnderstandwhoyourtoptrafficsinksareandpeeratallPOPs

– IgnoreMEDsfromothers– unlessyouwanttocarrythetrafficonyourbackbone

AS_PATHprepending(outbound)• MakeBGPprefixpaths”appear”longervialesspreferred

circuits

• “BGPpathfor10.0.0.0/8654_789iscongested”

• 123456789• 654789

• 100100100654789

• Willnotvaryinboundpacketroute(thoughthisroutemightnotbecongested!)

Inboundtrafficengineering

• Muchharder– Trickothers’BestPathcalculations– Youdonotadministrateoriginpartyrouter

• Butremember…– Largestflowscomefromasmallnumberofnetworks– Contentnetworkswanttodelivertraffictoyouaswellaspossible!

SelectiveAnnouncements

– Shortestprefix– LocalPreference– ASPATHlength– LowestOriginType– LowestMED– PrefereBGP paths– LowestIGPMetric– Oldestroute

PrefixlengthconsideredbeforeBGP.

10.0.0.0/16vs 10.0.0.0/17&10.128.0.0/17

ProblemofSelectiveAnnouncements

• Oftenfiltered• Consideredrude – mightleadtodepeering• Neverannounce‘globally’

…Butcanbeusedtogreateffect

• Tothesamepeerortransitprovider,announceaggregateandregionalpfx

Berlin Frankfurt

Transit

10.0.0.0/810.0.0.0/9 10.0.0.0/8

10.128.0.0/9

AddNO_EXPORTcommunity

Usewithpermission

AS_PATHprepending(inbound)

• SignalpreferredpathbygrowingAS_PATHonlesspreferredpaths

• Marginaleffectwhichdegradesquickly• SignalbackuplinktoasingleAS,butload-balancingcapacityismuchharder

• Maynotbeheardat‘distant’ASNs• Another‘blunt’tool,butcanmovesometraffic.

• Lowest MEDwins.– OppositeofNearestExitrouting,“carrytraffictome”– Onlyworkstothesamepeerinmultipleregions– CopyIGPmetrictoMED– Normallysubjecttonegotiation

• Sometimeshonoured,oftenwhennetworktrafficislatencyorlosssensitive.

MEDsareoftenfiltered

• ManynetworkssetMEDto0whentheylearnprefixes,sothathotpotatoroutingwilloverride MED.

route-map peers-in permit 10set local-preference 200set metric 0

Originchanging

• IGP• EGP• Incomplete

route-map PEERS permit 10set origin igp

route-route-map TRANSIT permit 10set origin incomplete

Oftenpeerssetto‘igp’or‘egp’staticallyonroutersto nullify effectsofOriginchanging.

Highestpriority

Inbound– whatdoesworkwell?• Overprovisioning• Peerwithtopnetworkswidely (buyoptions!)– Failureofsinglelinkwillnotbreakadjacency– Failurescanbehandledinpredictableways

• Buildrelationships• Constantlymonitorandmanage• Ifyoucareaboutyourtraffic,letitgo.J

– Playinggameswithpeeringhurtsyourcustomers’traffic

• AffectingdistantASNsisveryhard– aregionmayonlyseeasinglenext-hopASN.

Deterministicrouting

– LocalPreference– ASPATHlength– LowestOriginType– LowestMED– PrefereBGP paths– LowestIGPMetric– Oldestroute

Topflowsshouldleaveyournetworkviadeterministicmeans,andnotlefttoBGPBestPathselection(ortochance).

Ifyouarerelyingonoldestroutetomakethedecision,yourisktraffictakingunpredictableroutes.

However,oldestroutesdobreakthe‘flappingsessions’problem.Youneedtomonitorandmanageyourtopflowsconstantly.

Whatdoes“managerelationships”mean?

• Gobacktoyourdata– Collectandshareinformationwithpeeringco-ordinatorsatforumslikethis

– Youwillstandoutifyouknowexactlyhowmuchtrafficyouwillexchangeatpeakwithapeer

– Protectyourpeer’sinterests• Discussmutualpointsofinterconnectionthatsuitboth• Respondtoabusecomplaintspromptly• Usecontactstoreachotherpeeringco-ordinators• RespondpromptlytoBGPsessiondown/flapping• ListyournetworkonPeeringDB!

PublishinIRRDB

• PublishyourcircuitandadjacencyinformationinIRRDB

• Makeitpossibleforothernetworkstofigureouthowyouareintending yourtraffictoberouted

RT1 RT2

EX2EX1

PP2 PP3

Transit

4Gbit 4Gbit

AS2isyourlargestflow- viaPP2- maybeneedsasecondprivatepeerbackuponRT2?

4Gbit 4Gbit

AS12345AS2

Manypeers Manypeers

RT1 RT2

EX2EX1

PP2 PP3

Transit

4Gbit 4Gbit

AS2isyourlargestflow- viaPP2- maybeneedsasecondprivatepeerbackuponRT2?AS1viaPP1,configureabackupoverEX1orEX2fordeterministicrouting?

4Gbit 4Gbit

AS12345AS2

Manypeers Manypeers

RT1 RT2

EX2EX1

PP2 PP3

Transit

4Gbit 4Gbit

CanyoumovelargerpeersbehindEX1andEX2ontoprivatepeering?

4Gbit 4Gbit

AS12345AS2

Manypeers Manypeers

RT1 RT2

EX2EX1

PP2 PP3

Transit

4Gbit 4Gbit

CanyoumovelargerpeersbehindEX1andEX2ontoprivatepeering?Ifthereisanexchangefailure,wherewillthetrafficgo?Howbigaflowshouldyoucareabout?

4Gbit 4Gbit

AS12345AS2

Manypeers Manypeers

RT1 RT2

EX2EX1

PP2 PP3

Transit

4Gbit 4Gbit

CanyoumovelargerpeersbehindEX1andEX2ontoprivatepeering?Ifthereisanexchangefailure,wherewillthetrafficgo?Howbigaflowshouldyoucareabout?

IfyouloseRT2,howwilltraffictoPP3andtrafficvolumeviaEX2bedelivered?

4Gbit 4Gbit

AS12345AS2

Manypeers Manypeers

Buyingtransitinasmartway

• Buyingfromawellpeeredtransitprovider:– Canimprovequalityforthereasonsdiscussed– Hidescapacityproblemsfromyouautomatically

• Buyingfromyourtoptrafficdestination– Ifyourbusinessreliesonthetrafficquality,itmaymakesensetopay

– Data mayhelpyounegotiategoodterms

Dealingwitha“no”topeering

• Paidpeeringisoneoption– OftenmoreexpensivethanfullIPtransit– “Onceacustomer,neverapeer”

• Payforotherservicesinreturnforpeering– Transportforexample

• Peeraroundtheproblem– Trytopeerdirectlywithdownstreamcustomers– Trytoselldirectlytodownstreamcustomers• Ifyouarebetterpeered,youcansellbasedonquality

Constantlymanage

• PeeringontheInternetchanges everyday.• CapacityontheInternetgrows everyday.• Smallnetworksbecomelarge.• Largenetworksbecomelarger(consolidation)• A“bad”pathmightbecomegoodovernight

Questions?______________________________

AndyDavidsonandy@nosignal.org

Emailmetorequestacopyofthispresentation!

Feedbackandintroductiontopeeringco-ordinators welcome

Twitter:@andyd

BGP Peering Strategy & Data - 2017. · PDF file28/02/2017 BGP Traffic Engineering, Andy...

Documents

BGP Anomaly Detection and Robustness-NANOG-43 New...traffic back during a prefix hijack.” Josh Karlin, Stephanie Forrest, and Jennifer Rexford, “Pretty Good BGP: Improving BGP

Impactful Routing Research with Pᴇᴇʀɪɴɢ · BGP routes and traffic at locations around the world Pᴇᴇʀɪɴɢ: The BGP Testbed 2. ... propagated directly to peer Expected

BGP/2004.5.1 November 2004 © O. Bonaventure, 2004 · Organization of the global Internet BGP basics BGP in large networks Interdomain traffic engineering with BGP BGP-based Virtual

IPv4/IPv6 BGP Routing Workshop - wiki.apnictraining.net · IPv4/IPv6 BGP Routing Workshop November 6 –10, Colombo, ... Two Upstream –Load Sharing (Inbound Traffic) router bgp

Traffic Diversion Techniques for DDoS Mitigation using BGP Flowspec

40 - IDNOG03 - Bob Lau (Akamai) - BGP and Traffic Engineering

Bringing SDN to the Internet, one exchange point at the timeSDN controller also a Route Server BGP sessions. ... Inbound Traffic Engineering Traffic offloading forwarding optimization

Introduction to BGP - PacNOG•Transit – carrying traffic across a network, usually for a fee •Peering – exchanging routing information and traffic •Default – where to send

BGP and Traffic Eng. copy - slides.lacnic.netslides.lacnic.net/wp-content/themes/slides/docs/lacnic26/jueves/lacn… · DO and DON’T’s of Traffic Engineering. Typical BGP-based

1 Border Gateway Protocol (BGP). 2 Contents Internet connectivity and BGP connectivity services, AS relationships BGP Basics BGP sessions, BGP

More Specific Announcements in BGP - APNIC · Whyadvertise a more specific? II: To redirect incoming traffic to different network paths: “ traffic engineering prefixes” 10.0.0.0/8

CS 268: Computer Networking L-3 BGP. 2 Outline BGP ASes, Policies BGP Attributes BGP Path Selection iBGP

Traffic Engineering With BGP and Level3

BGP Tutorial Tutorial BGP

BGP MPLS-Based Ethernet VPN · Load Balancing of Unicast Packets ... (MP-BGP). Control-plane learning enables load balancing of traffic to and from CEs that are multihomed to multiple

BGP and Traffic Engineering with Akamai

Hands-on Cisco Service Provider Training · Configuring BGP on Cisco Routers (BGP) 6 Implementing Cisco Multicast (MCAST) 6 Implementing Cisco MPLS (MPLS) 7 Implementing MPLS Traffic

Border Gateway Protocol (BGP) Traffic Engineering Server ...njrusmc.net/pub/bts_leaf_spine.pdf · fabric using Border Gateway Protocol (BGP). BGP runs only on the leaves and is only

BGP Traffic Engineering Using RSVP-TE? · Traffic Engineering For Dummies • Today, operators have great software mechanisms for doing Traffic Engineering (TE) within our network

Border Gateway Protocol BGP Network delay … Gateway Protocol, BGP Network delay simulator, BGP Router emulator, BGP latency, BGP network emulator, network impairments, OSPFv2, OSPFv3,