48
INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s): Q13, Q14 and Q17 STUDY GROUP 12 Source: Chairman TIA TR30.3 Subcommittee Title: Updated Liaison presentation from TIA TR30.3 concerning proposed revision of G.1050 ABSTRACT As indicated in liaison presentations to SG12 over the last 2 years, TIA TR30.3 has been working on a proposed new revision to TIA-921 (Project: PN-3-0062-RV2) and the ITU-T version of the document G.1050, Network Model for Evaluating Multimedia Transmission Performance Over the Internet Protocol. TIA TR30.3 has now completed this work and has prepared a draft of TIA-921-B for TIA ballot. TR30.3 has also prepared a harmonized draft of ITU-T G.1050 Revised. TR30.3 and submits this draft document as a liaison to the upcoming Q13/12 meetings in Geneva. We request comments and feedback during the session of the attached draft of G.1050 Revised with a goal of achieving possible consent during January session. As Chairman of TR30.3, I am submitting the proposed revision to G.1050 and electronic attachments for review and consideration by SG12 Question 13 as an update to Recommendation G.1050 at the January2011 meetings in Geneva. TIA TR-30.3 has approved submitting this document and associated materials to ITU-T SG12. This presentation provides an overview of the proposed revision of G.1050 and electronic attachments. TD 435rev1 (GEN/12) Contacts Jack Douglass, Chairman TIA TR30.3 PacketStorm Communications Tel: +1 949-436-0663 C Email: [email protected] Chip Webb Anue System, USA Tel: +1 512-600-5403 Email: [email protected] Les Wu Alcatel-Lucent, USA Tel: +1 (908) 582-4964 Email: [email protected] Attention: This is not a publication made available to the public, but an internal ITU-T Document intended only for use by the Member States of ITU, by ITU-T Sector Members and Associates, and their respective staff and collaborators in their ITU related work. It shall not be made available to, and used by, any other persons or entities without the prior written consent of ITU-T.

INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Embed Size (px)

Citation preview

Page 1: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

INTERNATIONAL TELECOMMUNICATION

TELECOMMUNICATIONSTANDARDIZATION SECTOR

STUDY PERIOD 2009-2012

January 18-27, 20110English only

Original: EnglishQuestion(s): Q13, Q14 and Q17

STUDY GROUP 12

Source: Chairman TIA TR30.3 Subcommittee

Title: Updated Liaison presentation from TIA TR30.3 concerning proposed revision of G.1050

ABSTRACTAs indicated in liaison presentations to SG12 over the last 2 years, TIA TR30.3 has been working on a proposed new revision to TIA-921 (Project: PN-3-0062-RV2) and the ITU-T version of the document G.1050, Network Model for Evaluating Multimedia Transmission Performance Over the Internet Protocol.  TIA TR30.3 has now completed this work and has prepared a draft of TIA-921-B for TIA ballot. TR30.3 has also prepared a harmonized draft of ITU-T G.1050 Revised. TR30.3 and submits this draft document as a liaison to the upcoming Q13/12 meetings in Geneva. We request comments and feedback during the session of the attached draft of G.1050 Revised with a goal of achieving possible consent during January session. As Chairman of TR30.3, I am submitting the proposed revision to G.1050 and electronic attachments for review and consideration by SG12 Question 13 as an update to Recommendation G.1050 at the January2011 meetings in Geneva. TIA TR-30.3 has approved submitting this document and associated materials to ITU-T SG12.

This presentation provides an overview of the proposed revision of G.1050 and electronic attachments.

TD 435rev1 (GEN/12)

Contacts Jack Douglass, Chairman TIA TR30.3PacketStorm Communications

Tel: +1 949-436-0663 CEmail: [email protected]

Chip WebbAnue System, USA

Tel: +1 512-600-5403Email: [email protected]

Les WuAlcatel-Lucent, USA

Tel: +1 (908) 582-4964Email: [email protected]

Attention: This is not a publication made available to the public, but an internal ITU-T Document intended only for use by the Member States of ITU, by ITU-T Sector Members and Associates, and their respective staff and collaborators in their ITU related work. It shall not be made available to, and used by, any other persons or entities without the prior written consent of ITU-T.

Page 2: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Proposed Revision to G1050Network Model for Evaluating Multimedia

Transmission Performance over the Internet Protocol

Liaison to ITU-T SG12 – Q13, Q14, Q17

TIA TR30.3 Committee

Jack Douglass, Chairman, TIA TR30.3

PacketStorm Communications

January 18 to 27, 2011

Page 3: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Agendao G.1050-2007 / TIA-921Ao Summary of Changes from G.1050 (2007)o Proposed G.1050 Revisiono IP Network Simulation Source Codeo IP Network Simulationo Network Models: Core, Core to LAN, LAN to LANo Impairment Combination Tableso Inputs to IP Network Simulationo IP Network Simulation Outputso Electronic Attachmento Possible Uses for IP Network Impairment Simulationo Future TIA-921/G.1050 Worko Requests for SG12

Page 4: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Summary of Changes from G.1050 (2007)

o Proposed revision to G.1050 replaces G.1050 (2007)o G.1050 (2007)

Gilbert-Elliott mathematical model that fit certain observed network behavior but was not easily extended to other scenarios

o Revised G.1050 – Provide more realism than the earlier version Discrete Event Simulation Models the mechanisms using that contribute to packet delay, jitter, and loss:

interfering streams, queue delays in network elements, and the characteristics of specific access technologies.

Greatly reduce the number of standard test cases (43 test cases) True bidirectional model Removed LOO “likelihood of occurrence” concept Impairment levels are updated to keep current with evolving IP networks Users can customize test cases to fit their specific needs

Page 5: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

ITU-T G.1050 and TIA-921Test Profiles based on ITU-T Y1541

Different test profiles for different Service Level Agreements (SLAs)

Impairment Type Units Profile AWell‑ManagedRange (min to max)

Profile BPartially-ManagedRange (min to max)

Profile CUnmanagedRange (min to max)

One Way Latency ms 20 to 100 (regional)90 to 300 (intercontinental)

20 to 100 (regional)90 to 400 (intercontinental)

20 to 500

Jitter (peak to peak) ms 0 to 50 0 to 150 0 to 500

Sequential Packet Loss ms Random loss only (except when link failure occurs)

40 to 200 40 to 10,000

Rate of Sequential Loss sec-1 Random loss only (except when link failure occurs)

< 10-3 < 10-1

Random Packet Loss % 0 to 0.05 0 to 2 0 to 20

Reordered Packets % 0 to 0.001 0 to 0.01 0 to 0.1

Page 6: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

ITU-T G.1050-2007 and TIA-921A Network Topologies

LAN ACore IPNetwork

DUTA Route flapping

Link Failures

One-way delay

Jitter

Packet loss

Packet Reorder

LocalAccess

A

DUTB

LAN BLocalAccess

B

Access Rates

----------------

Occupancy level

Qos Edge Router

Access Rates

----------------

Occupancy level

Qos Edge Router

LAN Rates

-----------------

Occupancy level

Packet loss

Core IPNetwork

Route flapping

Link Failures

One-way delay

Jitter

Packet loss

Packet Reorder

DUTB

LAN BLocalAccess

B

DUTA

Access Rates

----------------

Occupancy level

Qos Edge Router

LAN Rates

-----------------

Occupancy level

Packet loss

LAN-to-LAN IP Network Model

Core-to-LAN

LAN Rates

-----------------

Occupancy level

Packet loss

Page 7: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

ITU-T G.1050-2007 and TIA-921A

Impairment Severity Table LAN / Access Rate Tables

IP Impairment Combinations (Delay and Loss)

1344 General Model120 Core to LAN ≤ 3 Mbit/s

48 Core to LAN > 3 Mbit/s (IPTV)

Page 8: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Proposed G.1050 Revisiono G.1050 proposed update to Recommendation

Proposed ITU-T G.1050 Revised Recommendation harmonized with draft of TIA-921 Ballot

Request comments and feedback during the session Achieve possible consent during January session

o Electronic Attachments C++ Source Code and Related Files Simulator Inputs, Outputs and Results

Page 9: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

IP Network Model Simulation Source Code

o C++ Windows XP / 7: Cygwin 1.7 Linux: OpenSUSE 11.3 (i386), Fedora 9

o Other Linux and Windows Tools TCL Versions 8.4 and 8.5 Perl 5.10 GNUPlot 4.4

o Simulation environment running on PC Sample data, voice, video or audio PCAP files can be run through the

model and captured the output as PCAP files for analysis and comparison

o The Software Copyright Holder (Anue Systems) will grant a free license based on Option 1.3 of ITU Software Copyright Statement and Licensing Declaration – Form has been filed

Page 10: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

IP Network Model Simulation

o General purpose discrete Event Simulatoro Event Models simulate Network Elements

Switches Routers Access Technologies Firewalls Buffers Connections via wires

o Types of Events for Network Elements Packet Arrival Packet Departure Other types of events are also possible

Page 11: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Model Basic IP Network Element

Third Interferer

Second Interferer

+Test Stream Input to This Stage

Link Latency

Test Stream Output from This Stage

Disturbance Packets

Packet Priority Queues with Loss

Store and

Forward

Interfering Stream Generator

Page 12: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

CORE Simulation Model

Core Network

Interfering Streams

(pcap files)

Interfering Streams

(pcap files)

Test Stream

Test Stream

1 Gbit/s 1 Gbit/s

# SwitchesQueueDelay

Reorder

Page 13: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

CORE-to-LAN Simulation Model

Core Network

Interfering Streams

(pcap files)

Interfering Streams

(pcap files)

Test Stream

Test Stream

Access Network1 Gbit/s

Page 14: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

LAN-to-LAN Simulation Model

Interfering Streams

(pcap files)

Test Stream

Access Network

1 Gbit/s

Access Network

1 Gbit/s Core Network

Core Network

1 Gbit/s

Interfering Streams

(pcap files)

Interfering Streams

(pcap files)

Interfering Streams

(pcap files)

Test Stream

Page 15: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Well Managed Network

Impairment / InterfererPCAP Avg BW

(Mb/s) w1 w2 w3 w4 w5 w6 w7 w8Core

Number of Switches 3 4 5 6 7 8 9 10Total Core Link Delay (ms) 10 25 50 75 85 100 125 150

Access (pick one technology)GPON Access Rate Down (Mbit/s) 50 50 35 35 25 25 25 15

Access Rate Up (Mbit/s) 25 25 25 25 25 25 25 5Residual BER 1.E-12 1.E-12 1.E-11 1.E-10 1.E-10 1.E-09 1.E-09 1.E-09

DSL Access Rate Down (Mbit/s) 33 33 33 22 16 16 16 10Access Rate Up (Mbit/s) 3 3 3 2 1 1 1 1Residual BER 1.E-08 1.E-08 1.E-08 1.E-08 1.E-07 1.E-07 1.E-07 1.E-07

Managed Bandwidth(QoS Voice=1

Video=2)IPTV HD Stream 1 - CBR (qty) 8 1 1 1 1 1 1 1 1IPTV HD Stream 2 - VBR (qty) 8 1IPTV SD Stream 1 - CBR (qty) 2 1 1 1 1 1IPTV SD Stream 2 - VBR (qty) 2 1 1 1VoIP/Fax (qty) 0.064/0.064 1 1 1 1 1 1 1 1

Residual Bandwidth QoS = 7Peer-to-peer Rate Down 1.89Peer-to-peer Rate Up 0.285HTTP Rate Down 1.58HTTP Rate Up 0.66OTT1 Rate Down 6.678OTT1 Rate Up 0.099OTT2 Rate Down 2.66OTT2 Rate Up 0.051VoIP/Fax Rate Down 0.064 100% 100% 100% 100% 100% 100% 100% 100%VoIP/Fax Rate Down 0.064 100% 100% 100% 100% 100% 100% 100% 100%

Well ManagedNetwork Element

Page 16: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Partially Managed Network

Impairment / InterfererPCAP Avg BW

(Mb/s) p1 p2 p3 p4 p5 p6Core

Number of Switches 3 4 5 8 10 12Total Core Link Delay (ms) 10 25 50 75 100 200

Access (pick one technology)GPON Access Rate Down (Mbit/s) 35 35 25 15 5 15

Access Rate Up (Mbit/s) 35 15 25 5 2 5Residual BER 1.E-12 1.E-12 1.E-11 1.E-09 1.E-09 1.E-09

DSL Access Rate Down (Mbit/s) 24 18 12 6 3 6Access Rate Up (Mbit/s) 3 1.5 1.5 1 1 1Residual BER 1.E-08 1.E-07 1.E-07 1.E-06 1.E-06 1.E-06

Managed Bandwidth(QoS Voice=1

Video=2)IPTV HD Stream 1 - CBR (qty) 8IPTV HD Stream 2 - VBR (qty) 8IPTV SD Stream 1 - CBR (qty) 2 1 1 1 1 0 1IPTV SD Stream 2 - VBR (qty) 2 1 1 1 1 1 1VoIP/Fax (qty) 0.064/0.064 1 1 1 1 1 1

Residual Bandwidth QoS = 7Peer-to-peer Rate Down 1.89 10% 50% 25% 20% 25%Peer-to-peer Rate Up 0.285 135% 135% 135% 135% 135%HTTP Rate Down 1.58 10% 100% 40% 20% 40%HTTP Rate Up 0.66 10% 50% 50% 60% 50%OTT1 Rate Down 6.678 100% 100%OTT1 Rate Up 0.099 100% 100%OTT2 Rate Down 2.66 100% 100% 100%OTT2 Rate Up 0.051 100% 100% 100%VoIP/Fax Rate Down 0.064 100% 100% 100% 100% 100% 100%VoIP/Fax Rate Down 0.064 100% 100% 100% 100% 100% 100%

Network Element

Partially Managed

Page 17: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Unmanaged & Core Network Core

Impairment / InterfererPCAP Avg BW

(Mb/s) u1 u2 u3 u4 u5 u6 u7 c1Core

Number of Switches 3 4 5 8 10 12 15 3Total Core Link Delay (ms) 10 25 50 75 100 200 250 150

Access (pick one technology)GPON Access Rate Down (Mbit/s) 35 35 25 15 5 15 5 1000

Access Rate Up (Mbit/s) 35 15 25 5 2 5 2 1000Residual BER 1.E-12 1.E-12 1.E-11 1.E-09 1.E-09 1.E-09 1.E-09 1.E-12

DSL Access Rate Down (Mbit/s) 24 18 12 6 3 6 3 1000Access Rate Up (Mbit/s) 3 1.5 1.5 1 1 1 1 1000Residual BER 1.E-08 1.E-07 1.E-07 1.E-06 1.E-06 1.E-06 1.E-06 1.E-12

Managed Bandwidth(QoS Voice=1

Video=2)IPTV HD Stream 1 - CBR (qty) 8 10IPTV HD Stream 2 - VBR (qty) 8 10IPTV SD Stream 1 - CBR (qty) 2 10IPTV SD Stream 2 - VBR (qty) 2 10VoIP/Fax (qty) 0.064/0.064 10

Residual Bandwidth QoS = 7Peer-to-peer Rate Down 1.89 10% 50% 25% 20% 25% 20% 1000%Peer-to-peer Rate Up 0.285 135% 135% 135% 135% 135% 135% 2000%HTTP Rate Down 1.58 10% 100% 40% 20% 40% 20% 1000%HTTP Rate Up 0.66 10% 50% 50% 60% 50% 60% 1000%OTT1 Rate Down 6.678 100% 100% 500%OTT1 Rate Up 0.099 100% 100% 500%OTT2 Rate Down 2.66 200% 200% 200% 100% 50% 100% 50% 500%OTT2 Rate Up 0.051 200% 200% 200% 100% 50% 100% 50% 500%VoIP/Fax Rate Down 0.064 100% 100% 100% 100% 100% 100% 100% 1000%VoIP/Fax Rate Down 0.064 100% 100% 100% 100% 100% 100% 100% 1000%

UnmanagedNetwork Element

Page 18: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Inputs to Network SimulationNetwork / Interferer PCAPs

o Network / Interferer Traffic IPTV Video, VoIP, Over-the-Top Video, Peer-to-Peer, HTTP, FoIP Standard PCAPs or user provided PCAPs

o PCAP – Packet Events drive Discrete Events Simulator Packet size Packet Timestamps

o PCAP Payload Payloads if present in the PCAP are carried through the simulator.

Page 19: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Input PCAP File Characteristics

QoS PCAP File Data Size Packets Duration Date TimeAvg Pkt

SizePacket Rate

Avg Bitrate 5s peak 1s peak Description

2 HD_cbr 275756340 201282 142 05/03/10 23:50:12 1370 1419 15555964 15558816 15574160 High Definition IPTV Flow, Constant Bit Rate2 HD_vbr 196918938 143737 142 05/03/10 23:34:54 1370 1014 11112049 13112544 18555280 High Definition IPTV Flow, Variable Bit Rate2 SD_cbr 274844350 200617 285 05/04/10 1:03:49 1370 703 7702130 7704880 7715840 Standard Definition IPTV Flow, Constant Bit Rate2 SD_vbr 197616080 144246 285 05/04/10 1:14:50 1370 507 5553410 6608880 8789920 Standard Definition IPTV Flow, Variable Bit Rate7 OTT1_Down 119929381 79619 144 12/07/09 18:47:41 1506 554 6678690 7160352 73277607 OTT1_Up 1777873 32832 144 12/07/09 18:47:41 54 229 99008 115257 1218247 OTT2_Down 110344529 73522 331 09/25/10 11:32:07 1501 222 2662988 12364163 139382727 OTT2_Up 2100467 36356 332 09/25/10 11:32:07 58 110 50663 229643 2726087 P2P_Down 34546941 30000 147 09/09/09 21:32:08 1152 205 1885957 3261356 39036087 P2P_Up 5412102 25500 152 09/09/09 21:32:07 212 168 285154 530008 7034167 HTTP_Down 27377604 20000 138 12/31/69 19:00:24 1369 145 1582495 2167475 46465127 HTTP_Up 1451588 15000 176 12/31/69 19:00:26 97 85 65997 89577 1936247 ipfax_v34_t38_fwd 53619 278 49 10/12/09 11:17:46 193 6 8809 50820 880007 ipfax_v34_t38_rev 31376 156 46 10/12/09 11:17:49 201 3 5468 45320 856001 voip_g729_fwd 603120 7180 215 01/22/10 14:38:24 84 33 22406 22444 228481 voip_g729_rev 602784 7176 215 01/22/10 14:38:24 84 33 22403 22444 22848

Fax-over-IP

G.729 Voice-over-IP

Progressive Downloaded Over-The-Top High Definition VideoAdaptively Streamed Over-the-Top Standard Definition Video

Peer-to-Peer File Download and Sharing

Web Browser Transactions

Page 20: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Input PCAP FilesManaged Bandwidth Characteristics

IPTV HD IPTV SD

VoIP

Page 21: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Input PCAP FilesResidual Bandwidth Characteristics

Adaptive Streaming (OTT2)

Progressive Download (OTT1)

FoIPHTTP

Peer-to-Peer

Page 22: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Aggregate Bandwidth Characteristics

0

2000000

4000000

6000000

8000000

10000000

12000000

14000000

16000000

18000000

0 60 120 180 240 300 360 420 480 540 600

Agg

rega

te B

W (b

its/s

)

sec

Dw Series Tests

Dw1 Dw2 Dw3 Dw4 Dw5 Dw6 Dw7 Dw8

0

2000000

4000000

6000000

8000000

10000000

12000000

14000000

16000000

18000000

20000000

0 60 120 180 240 300 360 420 480 540 600

Agg

rega

te B

W (b

its/s

)

sec

Dp Series Tests

Dp1 Dp2 Dp3 Dp4 Dp5 Dp6

0

5000000

10000000

15000000

20000000

25000000

0 60 120 180 240 300 360 420 480 540 600

Agg

rega

te B

W (b

its/s

)

sec

Du Series Tests

Du1 Du2 Du3 Du4 Du5 Du6 Du7

0

100,000,000

200,000,000

300,000,000

400,000,000

500,000,000

600,000,000

0 15 30 45

bit

s/s

sec

Aggregate Core Bandwidth

Well Managed Partially Managed

Unmanaged Core

Page 23: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Simulation Outputs

o Output and Result files .out (CSV) Time Series of Packet Loss and Delay Packet Delay Variation (PDV) Histogram Cumulative Distributive Functions (CDF) Summary of results

o Output PCAPs Analysis of impaired packets can be performed if

payload is included.

Page 24: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.out File (CSV)Example: HDTV1 / DSL / Well Managed Network

Description: Set Top Box #1 IPTV DownstreamContent-Encoding : ASCIIDelay Unit: ms

Time Delay Drop5 27.84455091 0

5.028805447 27.50758121 05.030184825 27.50758121 05.031562257 27.50758121 05.032941634 27.50758121 05.034321012 27.50758121 05.035698444 27.50758121 05.037077821 27.50758121 05.038455253 27.50758121 1

5.03983463 27.50758121 15.041214008 27.50758121 0

5.04259144 27.50758121 05.043970817 27.50758121 05.045350195 27.50758121 05.046727626 27.50758121 05.048107004 27.50758121 05.049486381 27.50758121 05.050863813 27.50758121 05.052243191 27.50758121 15.053620623 27.50758121 0

Page 25: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.out File Time Series of Packet Loss and DelayExample: HDTV1 / DSL / Well Managed Network

Dw1 Dw2 Dw3

Dw4 Dw5 Dw6

Dw7 Dw8

Page 26: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.out File PDV HistogramsExample: HDTV1 / DSL / Well Managed Network

Dw1 Dw2 Dw3

Dw4 Dw5 Dw6

Dw7 Dw8

Page 27: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.out File PDV Histograms and CDFsExample: HDTV1 / DSL / Well Managed Network

Page 28: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Simulation Results SummaryExample: HDTV1/ DSL / Well Managed Network

Test Case File Name

Number of

PacketsMin

DelayMax

DelayDrop

(Loss)SeqLoss

drop %(Loss)

seq %(Loss)

max-min(Jitter)

Dw1 HDTV1_Down 434003 12.50 13.97 47 0 0.011% 0.000% 1.47

Dw2 HDTV1_Down 433993 27.51 29.31 47 0 0.011% 0.000% 1.81

Dw3 HDTV1_Down 433970 52.52 67.52 52 0 0.012% 0.000% 15.01

Dw4 HDTV1_Down 433944 77.70 106.95 59 0 0.014% 0.000% 29.25

Dw5 HDTV1_Down 433522 87.89 94.71 474 2 0.109% 0.000% 6.82

Dw6 HDTV1_Down 433541 102.90 153.04 444 8 0.102% 0.002% 50.14

Dw7 HDTV1_Down 433486 127.92 178.05 481 8 0.111% 0.002% 50.14

Dw8 HDTV1_Down 433491 153.34 164.97 456 2 0.105% 0.000% 11.63

Page 29: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.pcap fileExample: HDTV1 (Payloads Truncated)

Page 30: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.out File (CSV)Example: SDTV2 / DSL / Well Managed Network

Description: Set Top Box #3 IPTV DownstreamContent-Encoding : ASCIIDelay Unit: ms

Time Delay Drop5 53.27604133 0

5.084155125 53.15206458 05.213908587 53.09975333 05.300238227 53.09975333 05.429842105 53.97893607 15.516254848 53.09975333 0

5.64598892 53.09975333 05.732518006 53.98514513 05.818867036 53.09975333 05.948515235 53.09975333 06.034911357 53.09975333 0

6.12133241 54.01707403 06.251094183 53.62069871 06.380631579 53.09975333 16.467027701 53.09975333 06.596878116 53.09975333 06.683282548 53.09975333 06.769584488 53.14577862 16.899157895 53.2229226 06.985570637 53.09975333 0

Page 31: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.out File Time Series of Packet Loss and DelayExample: SDTV2 / DSL / Partially Managed Network

Dp1 Dp2 Dp3

Dp4 Dp5 Dp6

Page 32: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.out File PDVExample: SDTV2 / DSL / Partially Managed Network

Dp1 Dp2 Dp3

Dp4 Dp5 Dp6

Page 33: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.out File PDV Histograms and CDFsExample: SDTV2 / DSL / Partially Managed Network

Page 34: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Simulation Results SummaryExample: SDTV2/ DSL / Partially Managed Network

Test Case File Name

Number of

PacketsMin

DelayMax

DelayDrop

(Loss)SeqLoss

drop %(Loss)

seq %(Loss)

max-min(Jitter)

Dp1 SDTV2_Down 110327 12.62 15.45 12 0 0.011% 0.000% 2.83

Dp2 SDTV2_Down 110207 27.78 45.76 129 0 0.117% 0.000% 17.98

Dp3 SDTV2_Down 110196 53.10 119.95 134 0 0.121% 0.000% 66.85

Dp4 SDTV2_Down 109015 79.05 213.13 1309 69 1.186% 0.063% 134.09

Dp5 SDTV2_Down 108813 105.89 375.24 1505 168 1.364% 0.152% 269.35

Dp6 SDTV2_Down 109009 204.09 338.18 1288 49 1.168% 0.044% 134.09

Page 35: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.pcap fileExample: SDTV1 (With Payload)

Page 36: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.out File Time Series of Packet Loss and DelayExamples: Managed Bandwidth

HDTV1

SDTV2

HDTV2

SDTV1

VoIP

Page 37: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.out File PDV HistogramsExamples: Managed Bandwidth

HDTV1

SDTV2

HDTV2

SDTV1

VoIP

Page 38: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.out File Time Series of Packet Loss and DelayExamples: Residual Bandwidth

OTT1

Peer-to-Peer

OTT2 HTTP

FoIP

Page 39: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

.out File PDV HistogramsExamples: Residual Bandwidth

OTT1

Peer-to-Peer

OTT2 HTTP

FoIP

Page 40: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Electronic Attachment

ISO 9660 format file that is downloadable from the ITU web site Part of the Recommendation Simulation Code – C++ Code G.1050 Document (PDF) Input PCAP files (without payloads) TCL files for 43 Test Case scenarios .out files Output PCAPs PNGs Results Summary (.csv) Impairment Combination Table

Page 41: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Possible Uses for IP Network Impairment Simulation

o Simulate Real-World IP network impairments PDV Histogram and Packet Loss characteristics

o Test any type of IP stream(s) (PCAP) under simulated network conditions Standard Test Cases or user define simulated network

conditions Test with or without payload Capture output PCAP for analysis and comparison

o Test any type of IP stream using Hardware Emulation of simulated network model Standard Test Cases or user define simulated network

conditions

Page 42: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Example CORE-to-LAN Simulation Test Stream(s)

Interfering Stream(s)

HD VBR IPTV

2

QoS

InterferersBandwidth

Interferers PDV & Loss

MOS Calculated from captured PCAP

OTT1

OTT2

SD VBR IPTV

HTTP

HD CBR IPTV

QoS

1

2

7

2

7

7

VoIP Down

Page 43: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Example MOS Calculation Derived from Simulation Output PCAP

Simulation Output PCAP

Video Stream Analysis:=============================================Stream source: 192.168.69.110:49713Stream destination: 234.5.6.101:2345Stream duration: 00:09:59.936Stream CODEC: H.264Stream transport: RTP (SSRC=0x11223344)Stream transport: MP2TS (PID=481, Ctrl=0x0000)Packet counts: received 2469009

lost 649 (0.00026)discarded 819 (0.00032)out-of-seq 6 (0.00000)duplicate 0 (0.00000)corrected 0 (0.00000)

MOS-V (min/avg/max/inst): 2.74/3.93/4.56/3.19MOS-AV (min/avg/max/inst): 2.89/3.82/4.32/3.29

Captured output PCAPs can be use to analyze and compare competing algorithms

Simulation .out Packet Delay and Loss Characteristic can be used to cause

impairments on test streams

Description: Dw4-HDTV1 Content-Encoding : ASCII Delay Unit: ms Time Delay Drop

21.33439 77.69556 0 21.33571 77.69556 0 21.33703 78.00174 0 21.33835 77.69556 0 21.33967 77.69556 0 21.34099 77.69556 0 21.34363 77.69556 1 21.34363 77.69556 0 21.34495 77.69556 0

Page 44: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Example MOS V Plots Derived from Simulation Output PCAP

MOS V (1s Intervals) for HDTV (CBR) Well Managed DLS 1 to 8

MOS V (1s Intervals) for SDTV (VBR) Partially Managed DLS 1 to 6

Page 45: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Hardware Emulation from IP Network Simulation

Description: Dw4-HDTV1 Content-Encoding : ASCII Delay Unit: ms Time Delay Drop

21.33439 77.69556 0 21.33571 77.69556 0 21.33703 78.00174 0 21.33835 77.69556 0 21.33967 77.69556 0 21.34099 77.69556 0 21.34363 77.69556 1 21.34363 77.69556 0 21.34495 77.69556 0

Device Under Test A (Video Server)

Impairment Emulator (.out files)

Device Under Test B (Video Client STB)

Video Stream Video Stream

• Select a standard test case that includes a technology similar to the test stream

• Configure the emulator with the standard .out file for the selected test case

• Run the test stream through the emulator, and evaluate the effect of the network impairments on the stream

Page 46: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Advanced Use of the Network ModelUser-Defined Test Cases

o Replace any of the standard interferer pcap files with user interfere pcap files

o Change the traffic mixo Change network conditions

number of switches bit error ratio access technology parameters, etc.

Page 47: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Future TIA-921/G.1050 Work

o Active Network Emulation using Interferer PCAP files

o Add Cable Access Characteristicso Add Wireless Access Characteristicso Update as IP Network evolves

Page 48: INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13,

Requests to SG12

o Provide comments and feedback on G.1050 Revised during session Document Simulation and Results Electronic Attachments

o Consider possible Consent of G.1050 Revised during the January session