International Telecommunication Union
Workshop on End-to-End Quality of Service.What is it? How do we get it?Geneva, 1-3 October 2003
IP Performance IP Performance Metrics:Metrics:
Definitions and Implementation ExamplesDefinitions and Implementation Examples Al MortonAT&T Labs
21-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Outline
o Performance Management Framework• Relationship to the E2E QoS goal
o IP Parameters/Metrics Summaryo In-progress Metric Developmento Implementations
1. Service Providers2. Customers3. 3rd Parties
1. Performance for MPLS-enabled IP Nets
31-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Network Performance Management Framework
o Fault Monitoring -- failure detectiono Passive Info Collection (single point)
• Read MIB counters or control data• Sample Traffic
o Active Measurements• Synthetic Traffic Dedicated to meas.
o Customer Measurements• Live or Synthetic traffic
41-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Relationship to E2E QoS: Provide answers to ...
o Network Provider• Is the design meeting requirements for
various traffic classes or applications?• How can I demonstrate the superior
performance of my service offering?o Customer
• Is Network Performance Agreement?o 3rd Parties
• What does “the net” look like? Hot spots?
• What Network Provider is “best”?
51-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Packet
Perf
. P
ara
mete
rsIPRE1
t Tmax
t Tmax
t > Tmax
ingress MP egress MP
SuccessfulIP packet outcome
ErroredIP packet outcome
SpuriousIP packet outcome
LostIP packet outcome
LostIP packet outcome
IPRE2Corrupted header or
errored payload
IPRE2(Note)
IPRE1
NOTE – Outcome occurs independent of IP packet contents
Never delivered ordelivered to an unpermitted
egress MP
IPRE1
IPRE1
IPRE2Valid header anderror-free payload
IPRE2(Note)
61-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Metric/Parameter Definition Summary
IETF IPPM RFCs ITU-T Recs.Framework 2330 Y.1540 cl 1 thru 5Sampling 2330 Poisson
3432 Periodic(future work inSG4 ?)
Loss 2680 Y.1540 cl 5.5.6Delay 2679 (1-way)
2681 (Round Trip)Y.1540 cl 6.2
Delay Variation 3393 Y.1540 cl 6.2.2
Availability 2678 Y.1540 cl 7Bulk Transfer Cap 3148Loss Patterns 3357 Possibly in G.IPP
71-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Comparison of IETF and ITU-T Delay Variation Metrics
Tx Rcv Playout
Time spent in: Transit Rcv Buffer
1
2
3
4
t
1
2
3
4
Inter packetarrival time,longer thansend interval
IETF IPDV is a measure of transfer delay variation w.r.t. previous packet.For Packet n,IPDV(n) = Delay(n) - Delay(n-1) or = R(n) - R(n-1) - T(n) - T(n-1)If the nominal transfer time is =10msec, and packet 2 is delayed in transit for an additional 5 msec, then two IPDV values will be affected.IPDV(2) = 15 - 10 = 5 msecIPDV(3) = 10 - 15 = -5 msecIPDV(4) = 10 - 10 = 0 msec
ITU-T SG 13 PDV is delay w.r.t. a reference, usually minimum delay.PDV(n) = Delay(n) - Min[Delay(*)]PDV(1,3,4)=0 PDV(2)=5
81-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Transient Delay Variation caused by burst traffic
-20
-10
0
10
20
30
40
50
60
70
80
90
100
120 220 320 420 520 620 720 820 920 1020 1120
Time in ms
Del
ay o
r Ji
tter
(m
s)
delta ms IPDV ms RT-delay msITU PDV IETF
91-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Packet Metrics for VoIP and other voiceband applications
o *new* metrics in G.IPP• Consecutive Packet Loss• Degraded Seconds• Short-term Delay Variation• Overall VoIP Parameters
o Alan Clark’s Presentation
101-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
What is Packet Reordering?
Packets arrive at Dst, but not in send order.
1, 2, 3, 7, 8, 9, 10, 11,... Loss,no reordering
1, 2, 3, 7, 8, 9, 4, 5, 6, 10, 11,...reordering
In the “world of order” all these packets are of interest.
1, 2, 3, 7, 8, 9, 4, 5, 6, 10, 11,... | Early | Late |
No reordering until Late Packets Arrive# of Early Packets => Reordering Extent
111-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Affect of Reordered Packets on most applications
o Receivers must perform work to restore order
1, 2, 3, 7, 8, 9,10, 4, 5, 6, 11, 12,...
| Buffered ||Reordered|
1 2 3 7 8 9 10 4 5 6 11
4 5 6, 11 (& 7 to 10)
1 2 3
Dst Time axis
Higherlayers
121-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Definition of Reordered Packeto Packet n is designated reordered
when its sequence number is less than the Next Expected threshold (set by the arrival of a previous packet).
1
2
3
7
8
9
10
4
5
6
11
0
2
4
6
8
10
12
1 2 3 4 5 6 7 8 9 10 11
Arrival Order (i)
Se
nd
Ord
er
(s)
Next Expected
131-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Failure Recovery Time
o When recovery was a simple outage, characterization was simple, too.
o IETF Benchmarking Methodology WG has identified 5 possible recovery scenarios:Lost packets Induced delay
Out-of-order packetsDuplicate packets123 45 67123456 4
1267 1234
Errored packets
1267
141-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Implementations: Customers, Service Providers & 3rd Parties
o Select Ping Target - make Round-trip connectivity and RTT measurement
o Accuracy Issues include path through router, path through net (asymmetries), response time at target, sampling rates
o Compare to current perf. to “normal”
$ ping R2 (or R3)
R1 R2
R3
R4or
or
151-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Beyond ping: ICMP Timestamp or Timestamp Reply Message
o Time spent processing packet at target can be removed, for more accurate RTT.
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Code | Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identifier | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Originate Timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Receive Timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Transmit Timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
R1 R2
R3
R4
OriginateCode=13
Receive TransmitCode=14
161-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Implementations:AT&T Global IP Measurements
BR(City 1)
MEASUREMENTCOLLECTION
SERVER (MCS)
BR(City 2)
BR(City n)
Measurementprobe
AT&T GLOBAL IP BACKBONE INFRASTRUCTURE
WEBCLIENTS(for reportviewing)
MEASUREMENTAGGREGATION& REPORTINGSERVER (MRS)
AGGREGATEDMEASUREMENT
DATA
Measurement Probes
http://www.att.com/ipnetwork
171-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
AT&T’s IP Measurement Design
o Poisson Sequence (RFC2330)• 15 minute duration = 0.3 pkts/sec• Type UDP, IPv4• 278 bytes total• ~300 packets sent• unbiased sample
o Periodic Sequence (RFC3432)• 1 minute duration • Random Start Time• 20 ms packet spacing• Type UDP, IPv4 • 60 bytes total• ~3000 packets sent
24 hours. . .
15 minutes
181-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Technical Collaborators at AT&T
o Len Ciavattoneo George Holubeco Madhukar Kshirsagaro Ron Kulpero Arvind Ramarajano Gomathi Ramachandran
191-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
New Measurement Challenges for MPLS-enabled IP Networks
o Most (all?) IP/Packet Network challenges
o Two main categories of MPLS Domains:• LDP-based, connection-less• Traffic Engineering, connection oriented
o Label Switched Paths are Unidirectionalo point to point and multi-point to pointo Many options for Failure Recoveryo LSP identity optionally removed (PHP)o Work in progress in SG 13 =
Y.MPLSperf
201-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
New Measurement Challenges for MPLS-enabled IP Networks
Networksection
Networksection
Networksection
MPLS Edge Node, or MPLS Ingress Node, or
LSR if both IP and MPLS are enabled
MPLS Node
Exchangelink
Exchangelink
Exchangelink
Exchangelink
Exchangelink
Network Section Ensemble (NSE)
Label Switched Paths
Scope of OA&M Measurements: single Network Section or MPLS Domain
MPLS Domain MPLS Domain
MPLS Network
211-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
New Measurement Challenges for MPLS-enabled IP Networks
o New Protocols = New Opportunities to Blackhole Traffic
o Detect this new class of failures with• Y.1711 MPLS OA&M Connectivity Verific.
• First version approved, adding fast failure detection
• LSP-Ping, Like ICMP Echo Request, plus• One-way Delay measurement possible• LSP Traceroute possible
221-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
New Measurement Challenges for MPLS-enabled IP Networks
o New Availability Definition? Crossroad:• Connection-Oriented Transport has
used a 10 second sliding window• Connection-Less Packet Transport has
used a 5 minute fixed windowo MPLS Networks => both transport
types• When Connection-oriented Services use
a Connection-less transport, which precedent should the Availability Definition follow?
231-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Summary
o Performance Management Framework• Measurement Systems are a key step
toward the goal of E2E QoS
o Summary of existing Parameters/Metrics
o In-progress Metric Developmento Active Measurement Implementations
• Ping for connectivity and ...• Dedicated Measurement Systems
o Parameter Framework for MPLS has new challenges
241-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Resources and References
o L. Ciavattone, A. Morton and G. Ramachandran, "Standardized Active Measurements on a Tier 1 IP Backbone," IEEE Communications Magazine, June 2003.
o Geoff Huston, “Measuring IP Network Performance,” The Internet Protocol Journal, vol 6, no.1, March 2003 http://www.cisco.com/ipj
o X.Xiao, et al., “A Practical Approach for Providing QoS in the Internet Backbone,” IEEE Communications Magazine, December 2002.
o D. Meyer, et al., “Trends in Measurement and Monitoring of Internet Backbones,” Panel at NANOG 26, slides etc. at http://www.nanog.org/mtg-0210/measurement.html
o ITU-T Rec. Y.1540, “Internet Protocol Data Communication Service – IP Packet Transfer and Availability Performance Parameters,” 2003.
o IETF IP Performance Metrics Working Group, links to RFC 2330, other IPPM RFCs and Internet Draft on Reordering: http://www.ietf.cnri.reston.va.us/html.charters/ippm-charter.html
o Draft New Recommendation Y.MPLSperf, “Performance and Availability Parameters for MPLS Networks”
o Draft New Recommendation G.IPP, “Performance Parameter Definitions for Quality of Speech and other Voiceband Applications Utilising IP Networks”
o RFC 792, “Internet Control Message Protocol,” J. Postel, September 1981.