Copyright © 2006 Juniper Networks, Inc. www.juniper.net 1
SEEREN2 Summer SchoolHeraklion, Sept 25th
Routing Issues: QoS/CoS
Jean-Marc UzéLiaison Research & Education, EMEA
2Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Agenda: QoS/CoS Workshop
Module 1: Overview of QoS/CoSModule 2: JUNOS CoS implementation (J/M/T-Series)Module 3: Introduction to JUNOS CLIModule 4: GEANT2 QoS services Implementation
The content of this module is courtesy of Dante (http://www.dante.net)http://www.dante.net/nep/geantqos/
http://www.dante.net/tf-ngn/activities.html
3Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Module 4:GEANT2 QoS services implementation
GÉANT Network and ServicesPremium IPLess than Best EffortQueuing on GÉANT and statusRouter ConfigurationPremium IP Management
4Copyright © 2006 Juniper Networks, Inc. www.juniper.net
10 Gb/s IP/MPLS backbone with Juniper T640s, M160s, M40s
4 x 10 Gb/s to North America
Dark fiber and WDM optical technology
Connecting 34 European Countries and 30 National R&E Networks
European connectivity to over 3000 R&E institutions
Advanced Services:IPv6Premium IPMulticast v4 + v6Best EffortLess Than Best EffortLayer 2 VPN
GEANT2 / Dante
5Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Global Connectivity
6Copyright © 2006 Juniper Networks, Inc. www.juniper.net
IP QoS Services on GÉANT
Premium IPupper-bounded one-way delayupper-bounded IPDVnegligible packet lossguaranteed capacity
Less than Best Effortclass of traffic using the un-utilised Best Effort and higher classes of service bandwidth
http://www.geant.net/server/show/nav.00700a009
7Copyright © 2006 Juniper Networks, Inc. www.juniper.net
AGENDA
GÉANT Network and ServicesPremium IPLess than Best EffortQueuing on GÉANT and statusRouter ConfigurationPremium IP Management
8Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Premium IP Model
End-to-end service across multiple management domains
using diffserv, ATM CBR or over-provisioning(!)packet tagged DSCP 46 (EF - 101110) destination aware servicepacket with other DSCP are left untouched (packets from other service)Premium IP bandwidth limited to 10% of the link capacity can cope with 20% in case of circuit failure
9Copyright © 2006 Juniper Networks, Inc. www.juniper.net
GEANT and IP Premium Service
Source: http://www.dante.net/sequin
10Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Premium IP on GEANT
Protection of authorised Premium IP trafficunder normal circumstances, the Premium IP traffic of a circuit is limited to 10% of the circuit capacity
20% in case of another circuit failurebullet-proof all the GÉANT accesses against unauthorisedPremium IP traffic (tagged DSCP 46) on all the ingress interfaces
if DSCP 46 packet arrives on GÉANT and part of an unauthorised flow: classify the packet into the Best Effort queue and remark it as Best Effort (DSCP 0)if DSCP 46 packet arrives on GÉANT and is part of an authorised flow: check against policer according capacity requested in the SLA (in-profile accepted, out-of-profile dropped)
11Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Premium IP on GEANTProtection of authorised Premium IP traffic [cont]
per next AS rate-limitation (implemented by Juniper for GÉANT)can also do source-destination IP addresses when NREN don’t do it. (NREN = National Research & Education Network, a Dante customer network directly connected to GEANT)
Trust the Premium IP traffic received from a GÉANT backbone interface.
Traffic checked at the GÉANT ingresses.
Configure queuing mechanism on the backbone and access interfaces.
strict-high priority is configured to the Premium IP queue.Don’t forget that the amount of Premium traffic expected in the Premium IP queue is 10% of the link capacity (service over-provisioned by a factor 9); this is assured by ingress policing.
90% for the BE and 5% for the network control (and 5% for LBE)
12Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Jitter distribution in VBR traffic - BE & Premium IP
0%
10%
20%
30%
40%
50%
60%
70%
0.00
0.19
0.38
0.56
0.75
0.94
1.13
1.31
1.50
1.69
1.88
2.07
2.25
2.44
2.63
2.82
3.01
3.19
3.38
3.57
3.76
Premium IP BE jitter[ms]
perc
enta
ge o
f pac
kets
Avg. jitter vs. packet size - BE & Premium IP
02468
101214
78 171
202
262
443
520
747
853
985
1234
1330
1355
1382
1384
1401
1426
1445
1450
1480
1490
Premium IP BE
avg.
jitt
er [m
s]
packet size [bytes]
Test result end-to-end IP Premium
13Copyright © 2006 Juniper Networks, Inc. www.juniper.net
AGENDA
GÉANT Network and ServicesPremium IPLess than Best EffortQueuing on GÉANT and statusRouter ConfigurationPremium IP Management
14Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Less than Best Effort
Class of traffic using the un-utilised Best effort and higher classes of service bandwidth
in case of competition for resources, the LBE traffic will de discarded before any Best-Effort or higher classes of traffic.use the DSCP 8 (001000) - same as Internet2 scavenger service.
Congestion on an interface due to LBEshould be transparent to the BE or higher classes of servicesno BE or higher classes of services packet loss
15Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Less than Best Effort
No end-to-end guaranteesno metric needed to quantitatively describe the service
Can be supported on one interfaceanywhere else, the LBE tagging should be passed transparently.
Application scenariosmirroring, test traffic, some GRID data transfers, network backups, protection of research traffic from student dormitory one.
16Copyright © 2006 Juniper Networks, Inc. www.juniper.net
LBE Queuing Technique
For algorithm with bandwidth shared assignment, as Weighted Wound Robin and Weighted Fair Queuing, a very small bandwidth share is allocated to the LBE queue.
Typically between 0% and 5%
17Copyright © 2006 Juniper Networks, Inc. www.juniper.net
LBE: Measurement with congestion
One-way delay Increase of LBE maximum one-way delay of 1.5msIncrease of BE maximum one-way delay of 400µs
18Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Normal Traffic
Normal Traffic +
Less Than Best Effort2.0 Gbit/s
Normal Traffic +
Radio Astronomy Data500 Mbit/s
Normal Traffic +
Radio Astronomy Data +
Less Than Best Effort2.0 Gbit/s
LBE live test:ER2002 Demo - VLBI - dataGRID
19Copyright © 2006 Juniper Networks, Inc. www.juniper.net
AGENDA
GÉANT Network and ServicesPremium IPLess than Best EffortQueuing on GÉANT and statusRouter ConfigurationPremium IP Management
20Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Queuing Technique
WRR - Juniper M-series
Weight Assure the queue to be given a minimum amount of bandwidth proportional to the weight.
Priority queue with high priority are served before the low priorityallow the BE (and other high priority queues) to be served firstuntil empty before serving the LBE one.
WREDis used to limit the queuing delay in case of congestionuse to protect one class of traffic over the other within a queue.
21Copyright © 2006 Juniper Networks, Inc. www.juniper.net
DSCP/ToS Values used by GEANT
Service DSCP value ToS value Juniper alias ToS (hex) DSCP-ToSbinary
Premium IP 46 184 ef B8 101110 -101110xx
LBE 8 32 cs1 20 001000 -001000xx
DWS 32 128 cs4 80 100000 -100000xx
Network control 1
48 192 cs6 C0 110000 -110000xx
Network control 2
56 224 cs7 E0 111000 -111000xx
The DSCP/ToS values used in GÉANT to classify the traffic of the different QoS classes are shown in the table below. In addition to the three service classes offered to transiting traffic there is a DWS (IP commodity service) and a Network Control class, which are traffic classes used internally to the GÉANT network.
22Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Juniper Networks and CoS ServicesGEANT with IP Premium + LBE Service
Junos CoS features include policing, (strict) priority queuing, weighted round robin (WRR), precedence/DSCP field rewrite, and random early drop RED. On a Juniper M-series Router each port has 4 Queues Weighted Round Robin Percentages can be set for each QueueNew generation Q-PICs offers multiples queues per logical interfaces (Ethernet VLAN, ATM PVC, etc.)
WRRWRR
Source: http://www.dante.net/nep/geantqos/ and http://www.dante.net/tf-ngn/activities.html
Queue FC LP Service DSCP Weight Priority Buffer low Best Effort / 0 BE high DWS 32
90% Low 50%
low Premium IP 46 1 EF high / /
N/A Strict-high
15%
low Less than BE 8 2 LBE high Retag to BE 0
5% low 30%
low Network control 48 3 NC high Network control 56
5% high 5%
23Copyright © 2006 Juniper Networks, Inc. www.juniper.net
QoS Configuration on GEANTThe configuration has completed on most of the GEANT routers allowing Premium IP, BE and LBE to co-exist.
The routers where the three services have been enable are represented as green on the following map.
The routers coloured yellow are Juniper routers where “old” FPCs have been re-used from TEN-155 (1999) These old FPC’s that do not allow for the full functionality of QoS.
As such BE is not ideally protected by LBE and the bandwidth is effectively shared. Premium IP only is supported.
24Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Current QoS Configuration on GEANT
25Copyright © 2006 Juniper Networks, Inc. www.juniper.net
AGENDA
GÉANT Network and ServicesPremium IPLess than Best EffortQueuing on GÉANT and statusRouter ConfigurationPremium IP Management
26Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Router Configuration
Each router in the GÉANT network contains certain QoSbuilding blocks in order to configure Per Hop Behaviors (PHB). The configuration shown here is taken from a Juniper M160 router with JUNOS 5.7 and with E-FPC (enhanced FPCs) and SDH interfaces.
Classifiers, schedulers and rewrite rules can be associated to each interface. In GÉANT two types of interface configurations are used for QoS
a backbone interfacean access interface (i.e. the interface where the traffic from an NREN is entering GÉANT)
27Copyright © 2006 Juniper Networks, Inc. www.juniper.net
DSCP and ToS Values
Type of Service field illustration
Illustration of DSCP Type of Service field configuration
Service DSCP value ToS value Juniper alias ToS (hex) DSCP-ToS binary
Premium IP 46 184 ef B8 101110 - 101110xx
LBE 8 32 cs1 20 001000 - 001000xx
DWS 32 128 cs4 80 100000 - 100000xx
Network control 1 48 192 cs6 C0 110000 - 110000xx
Network control 2 56 224 cs7 E0 111000 - 111000xx
28Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Router Interfaces
Backbone interface
so-7/0/0 {scheduler-map MAP-BASIC;unit 0 {
classifiers {dscp backbone-classifier;
}rewrite-rules {
dscp basic-rewrite-rules;}
}}
Access interface
so-0/2/3 {scheduler-map MAP-BASIC;unit 0 {
classifiers {dscp access-classifier;
}rewrite-rules {
dscp basic-rewrite-rules;}
}}
In addition, the access interface may contain filters in order to classify and police Premium IP traffic.The following configurations apply to all (access and backbone) interfaces.
29Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Drop ProfilesDrop profile define the parameters used by the Random Early Detection (RED) mechanism that MAY be used in a queue
dws-drop-profile {fill-level 35 drop-probability 10;fill-level 40 drop-probability 100;
}be-drop-profile {
fill-level 15 drop-probability 30;fill-level 19 drop-probability 50;fill-level 24 drop-probability 70;fill-level 30 drop-probability 100;
}less-than-be-drop-profile {
fill-level 25 drop-probability 30;fill-level 30 drop-probability 50;fill-level 40 drop-probability 70;fill-level 50 drop-probability 100;
}
30Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Queues and Schedulers1. Associate a name with each queue
Note: a queue is sometimes also called a forwarding class
forwarding-classes {queue 0 best-effort;queue 1 expedited-forwarding;queue 2 less-than-best-effort;queue 3 network-control;
}
The Premium IP traffic is classified into the expedited-forwarding queue. The naming of the queues is performed once and applies to all interfaces of the router.
31Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Queues and Schedulers2. Define scheduler configurations
scheduler weight, queue size and priority as set at the GÉANT router for each queue:sch-best-effort {
transmit-rate percent 90;buffer-size percent 50;priority low;
}sch-expedited-forwarding {
buffer-size percent 15;priority strict-high;
}sch-less-than-best-effort {
transmit-rate percent 5;buffer-size percent 30;priority low;drop-profile-map loss-priority low protocol any drop-profile less-than-be-drop-profile;drop-profile-map loss-priority high protocol any drop-profile be-drop-profile;
}sch-network-ctrl {
transmit-rate percent 5;buffer-size percent 5;priority high;
}
32Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Queues and Schedulers3. Associate a scheduler with a queue (FC)
The main advantage of the scheduler-map is that it can be applied to more than one interface.
MAP-BASIC {forwarding-class best-effort scheduler sch-best-effort;forwarding-class expedited-forwarding scheduler sch-expedited-forwarding;forwarding-class less-than-best-effort scheduler sch-less-than-best-effort;forwarding-class network-control scheduler sch-network-ctrl;
}
33Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Classification
The classifier is a functional block located at the input interface that sets three internal bits for each IP packet
Two bits that select the output-queue. There are four output queues, also called forwarding-classes, per port.One bit to indicate the loss-priority, the packets classified in an output-queue can have two different values of loss-priority (low or high).
Best Effort (BE) and Less than Best Effort (LBE) traffic is classified by the classifier rules as shown below. Note that it is also possible to classify packets by means of an input firewall filter
This is used for classifying Premium IP traffic according to the source/destination address and optionally the DSCP value of the packet. Excess Premium IP traffic is discarded (policer)
Backbone Classifierdscp backbone-classifier {
import default;forwarding-class best-effort {
loss-priority low code-points [ af11 af12 af13 ];loss-priority high code-points cs4;
}forwarding-class less-than-best-effort {
loss-priority low code-points cs1;}
}
Access Classifierdscp access-classifier {
import default;forwarding-class best-effort {
loss-priority low code-points [ af11 af12 af13 ];}forwarding-class less-than-best-effort {
loss-priority low code-points cs1;loss-priority high code-points [ ef cs4 ];
}
34Copyright © 2006 Juniper Networks, Inc. www.juniper.net
MarkingThe marking of packets with a DSCP value is the last QoSaction performed before the transmission of the packet (hence after firewall filter evaluation)
dscp basic-rewrite-rules {forwarding-class best-effort {
loss-priority high code-point cs4;}forwarding-class expedited-forwarding {
loss-priority low code-point ef;}forwarding-class network-control {
loss-priority low code-point nc1;loss-priority high code-point nc2;
}forwarding-class less-than-best-effort {
loss-priority high code-point be;loss-priority low code-point cs1;
}}
Service Incoming DSCP value
New DSCP value
Authorised Premium IP 46 46/drop
Un-authorised Premium IP 46 0/5
DWS 32 0
LBE 8 8
Network Control 48/56 48
Best Effort other values Unchanged
35Copyright © 2006 Juniper Networks, Inc. www.juniper.net
AGENDA
GÉANT Network and ServicesPremium IPLess than Best EffortQueuing on GÉANT and statusRouter ConfigurationPremium IP Management
36Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Useful ToolFeature of the NANOG traceroute to discover the DSCP changes along the path (Simon Leinen from Switch:-)
[root]# ./traceroute -t 184 193.171.2.1 traceroute to 193.171.2.1 (193.171.2.1), 30 hops max, 40 byte packets 1 css7-ATM4-0-0-101-dmsk.man.poznan.pl (150.254.160.62) 1 ms 1 ms 1 ms 2 150.254.163.118 (150.254.163.118) 2 ms 2 ms 2 ms 3 z-pozmanu-oc3.poznan-gw.pol34.pl (212.191.127.49) 2 ms 2 ms 2 ms 4 pol-34.pl1.pl.geant.net (62.40.103.109) 2 ms 2 ms 2 ms 5 pl.cz1.cz.geant.net (62.40.96.45) 22 ms (TOS=0!) 22 ms 22 ms 6 cz.de1.de.geant.net (62.40.96.38) 30 ms 30 ms 30 ms 7 de1-1.de2.de.geant.net (62.40.96.130) 30 ms 30 ms 31 ms 8 de.at1.at.geant.net (62.40.96.5) 43 ms 43 ms 43 ms 9 aconet-gw.at1.at.geant.net (62.40.103.2) 43 ms 43 ms 43 ms 10 193.171.2.1 (193.171.2.1) 45 ms * 45 ms
37Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Need for Automation
Service management was done manuallyService set up, maintenance and termination was done by phone calls and emailsConsiderable manual effort requiredComplexity in keeping track of:
Path informationCurrent and future reservationsPremium IP utilisation levelsChanges in network topology
Multi-party communication
38Copyright © 2006 Juniper Networks, Inc. www.juniper.net
System Architecture
Java web-based architectureUsing Apache 1.3 web server, Tomcat servlet container & MySQL Database
39Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Premium IP Reservation Tool Features (1)
Authentication & AuthorisationPath Finder
Find shortest path between two end pointsDynamic based upon configured IS-IS cost
Utilisation MonitoringCheck Premium IP reservation levels on each intermediate link along the pathTake into account all active reservation during the given time periodDisplay the available Premium IP capacity
40Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Premium IP Reservation Tool Features (2)
Reservation ManagementView, Request, Modify, CancelBased upon available Premium IP capacity
Contact managementIP Address management
Automated email notificationto User, DANTE - Premium IP team, NOCupon Reservation Request, Modification, Cancellation
Router Configuration update (script)Others: IS-IS cost, Reports, Archival, System Administration...
41Copyright © 2006 Juniper Networks, Inc. www.juniper.net
42Copyright © 2006 Juniper Networks, Inc. www.juniper.net
Thank you
Jean-Marc UzéLiaison Research & Education, EMEA
Mobile: +3361543251231 Place Ronde, 92986 Paris-La-Defense, France