Telecommunications Industry AssociationTR-30.3/10-04-011 By Teleconference April 26, 2010

Telecommunications Industry Association TR41.N.n-YY-MM-XXX

Document Cover Sheet

Project Number PN-3-0062-RV2

Document Title 31004011 VidTran10 TIA-921 - G.1050 Presentation

Source PacketStorm

Contact Name: Jack Douglass Complete Address:

Phone: Cell: 949-436-0663 Home/Office: 949-480-1220 Fax: 949-480-1220 Email: [email protected]

Distribution TR-30.3

Intended Purpose of Document (Select one)

For Incorporation Into TIA Publication For Information

X Other (describe) - Liaison

The document to which this cover statement is attached is submitted to a Formulating Group or sub-element thereof of the Telecommunications Industry Association (TIA) in accordance with the provisions of Sections 6.4.1–6.4.6 inclusive of the TIA Engineering Manual dated October 2009, all of which provisions are hereby incorporated by reference.

Abstract

Video Services Forum (VSF) VidTrans10 presentation.

Telecommunications Industry Association TR-30.3/10-04-011By Teleconference April 26, 2010

Jack Douglass

VP of Marketing and Business Development

Chairman of TIA TR30.3 Committee

Standards Based Network Model for Evaluating Multimedia Transmission Performance Over IP

Agenda

• Standards Based IP Network Model– ITU-T G.1050 / ANSI TIA-921

• G.1050-2007 / TIA-921A– Top Down Approach

• G.1050–201X / TIA-921B (proposed revision)– Bottom-Up Approach

• Example Results• Applications for G.1050/TIA-921 IP Network Model

Standards Based IP Network ModelITU-T G.1050 / ANSI TIA-921

• Committees– TIA TR30.3 Committee– ITU-T SG12 Question 14

• Range of test scenarios – Video, VoIP and Data – “Real-World” Network Characteristics

• Service Level Agreements (SLAs) based impairments (ITU-T Y.1541)– Well-Managed– Partially-Managed – Unmanaged conditions

Top Down ApproachG.1050-2007 / TIA-921A (Current Model)

• Surveyed many networks to evaluate the jitter and loss characteristics (Bursty, Not Random)

• Level of the impairment characteristics were adjusted to match the service levels in Y.1541

• Created impairment combinations based on Impairment Severity Levels and LAN/Access Rates

• Multiple Two State Time Series Modified Gilbert-Elliott / Markov Models were used to emulate loss and delay characteristics– Emulated bursty packet loss and jitter that are similar to the surveyed

network characteristics

– Time based model with pre-calculated delay and loss

Network Impairments

• Network Architecture • Types of Access Links• QoS controlled Edge

Routing• MTU Size• Packet Loss (Frame

Loss)• Out of order packets• One Way Delay

(Latency)

• Variable Delays (Jitter) • Occupancy (Background Traffic,

Congestion, Network Load)• Route Flapping• Network faults• Link Failures

Network TopologiesITU-T G.1050-2007 / TIA-921A

LAN ACore IPNetwork

DUTA Route flapping

Link Failures

One-way delay

Jitter

Packet loss

Packet Reorder

LocalAccess

A

DUTB

LAN BLocalAccess

B

Access Rates

----------------

Occupancy level

Qos Edge Router

Access Rates

----------------

Occupancy level

Qos Edge Router

LAN Rates

-----------------

Occupancy level

Packet loss

Core IPNetwork

Route flapping

Link Failures

One-way delay

Jitter

Packet loss

Packet Reorder

DUTB

LAN BLocalAccess

B

DUTA

Access Rates

----------------

Occupancy level

Qos Edge Router

LAN Rates

-----------------

Occupancy level

Packet loss

LAN-to-LAN IP Network Model

Core-to-LAN

LAN Rates

-----------------

Occupancy level

Packet loss

Test Profiles based on ITU-T Y1541ITU-T G.1050-2007 / TIA-921A

Different test profiles for different Service Level Agreements (SLAs)

Impairment Type Units Profile AWell‑ManagedRange (min to max)

Profile BPartially-ManagedRange (min to max)

Profile CUnmanagedRange (min to max)

One Way Latency ms 20 to 100 (regional)90 to 300 (intercontinental)

20 to 100 (regional)90 to 400 (intercontinental)

20 to 500

Jitter (peak to peak) ms 0 to 50 0 to 150 0 to 500

Sequential Packet Loss ms Random loss only (except when link failure occurs)

40 to 200 40 to 10,000

Rate of Sequential Loss sec-1 Random loss only (except when link failure occurs)

< 10-3 < 10-1

Random Packet Loss % 0 to 0.05 0 to 2 0 to 20

Reordered Packets % 0 to 0.001 0 to 0.01 0 to 0.1

8 Impairment Severity CombinationsITU-T G.1050-2007 and TIA-921A

Impairment Severity Combinations

Severity=> A B C D E F G H*

Impairment Units

Profie A LOO % 50 30 15 5 0 0 0 0

Profile B LOO % 5 25 30 25 10 5 0 0

Profile C LOO % 5 5 10 15 20 25 15 5

Source Location (A) Parameters

LAN A Occupancy % 1 2 3 5 8 12 16 20

Access A Occupancy % 0 1 2 4 8 15 30 50

MTU A bytes 512 512 1508 1508 1508 1508 1508 1508

Core Network Impairments

Route flap interval seconds 0 3600 1800 900 480 240 120 60

Route flap delay ms 0 2 4 8 16 32 64 128

Delay ms 4 8 16 32 64 128 256 512

Jitter (peak to peak) ms 5 10 24 40 70 100 150 500

Link fail interval seconds 0 3600 1800 900 480 240 120 60

Link fail duration ms 0 64 128 256 400 800 1600 3000

Packet loss % 0 0.01 0.02 0.04 0.1 0.2 0.5 1

Reordered packets % 0 0.00025 0.0005 0.001 0.005 0.01 0.05 0.1

Destination Location (B) Parameters

Access B Occupancy % 0 1 2 4 8 15 30 50

MTU B bytes 512 512 1508 1508 1508 1508 1508 1508

LAN B Occupancy % 1 2 3 5 8 12 16 20

LAN-Access Rate CombinationsITU-T G.1050-2007 and TIA-921A

6 IPTV Core-to-LAN (> 3 Mbit/s) Rate Combinations 48 Impairment Combinations (6 rates * 8 Severities)

168 LAN-to-LAN Scenario Rate Combinations1344 Impairment Combinations (168 rates * 8 Severities)

15 Core-to-LAN (Excluding IPTV≤ 3 Mbit/s) Rate Combinations120 Impairment Combinations (15 rates * 8 Severities)

Testing with Fixed Levels of IP Impairments

• Real IP Network Impairments Vary Over Time significantly over time and are bursty in nature

• IP Impairment network emulators exist today that provide fixed or semi-fixed levels of IP impairments.

Fixed Jitter & Packet Loss

Fixed Packet Jitter

Real World Impairment Characteristics

Good

Bad

Severe

1344 General Model120 Core to LAN ≤ 3 Mbit/s48 Core to LAN > 3 Mbit/s

Link FailureRoute Flap

IP Impairments IPTV Testing G.1050-2007/TIA-921A

1XA

1XB

1XC

1XD

1XE

1XF

1XG

1XH

48 Core to LAN > 3 Mbit/s Impairment Combinations

Reasons Revising G.1050-2007 / TIA-921A

• Increased Realism • Keep in step with evolving IP network• Reduced number of test cases

– Small set of Standard Long Duration Test Cases

– Focused Test Cases for transient impairments such as link failures and route flaps

– Short burst tests (15 seconds) for voice/video subjective testing and fax

• True bi-directional model• Ability to test with mixed traffic• Statistically repeatable tests• Ability for users to customize test cases and use their own .pcap

files to model interferers.• Methods for verifying that test equipment implementations match

the model

Bottom-Up ApproachG.1050-201X / TIA-921B (Revised Model)

• Model interfering traffic – Capture different types of real network traffic

– Create Packet Delay Variation (PDV) Histograms from captured traffic

– Make interferers scalable

• Model the Mechanisms that cause Jitter, Delay and Loss– Packet loss and delay are caused by interferers and self-interferers

• Model of each Network Element (Core, Access, LAN)– Capture characteristics of each Network Elements

– Capture characteristics of Access and LAN technologies • DSL, Cable, Fiber Optic, WiMAX, LTE; LAN – Wi-Fi, Ethernet, HPNA

– Bandwidth Throttling, Smoothing and Shaping Effects of network elements

– Bi-directional characteristics

• Model Quality of Service (QoS)

• Adjust and scale the amounts and types of interfering traffic– Match Surveyed Network Characteristics

– Match Service Levels in Y.1541

• A similar model is being considered in ITU-T SG15 Question 13 – G.8261

Test Profiles based on ITU-T Y1541 ITU-T G.1050-201X / TIA-921B (Revised Model)

Different test profiles for different Service Level Agreements (SLAs)

Impairment Type Units Profile AWell‑ManagedRange (min to max)

Profile BPartially-ManagedRange (min to max)

Profile CUnmanagedRange (min to max)

One Way Latency ms 20 to 100 (regional)90 to 300 (intercontinental)

20 to 100 (regional)90 to 400 (intercontinental)

20 to 500

Jitter (peak to peak) ms 0 to 50 0 to 150 0 to 500

Sequential Packet Loss ms Random loss only (except when link failure occurs)

40 to 200 40 to 10,000

Rate of Sequential Loss sec-1 Random loss only (except when link failure occurs)

< 10-3 < 10-1

Random Packet Loss % 0 to 0.05 0 to 2 0 to 20

Reordered Packets % 0 to 0.001 0 to 0.01 0 to 0.1

Impairment Combination TableResidential Impairment Combinations in the direction from the core

Network Element Well Managed Partially Managed Unmanaged

Impairment Sev1 Sev2 Sev3 Sev4 Sev5 Sev6 Sev7 Sev8 Sev9QoS

IPTV 2 2 2 2 2 2 3 3 3 VoIP (primary) 1 1 1 1 2 3 3 3 3 Data 3 3 3 3 3 3 3 3 3

Test stream Y Y Y ? ? N N N N LAN - wired Effective Rate (Mbit/s) 100 100 100 100 100 100 100 100 100 Peer-to-peer 0 0 1 POP3 0 0 1 HTTP 0 1 1 Web video 0 0 0 VoIP (secondary) 0 1 0 FoIP (secondary) 0 0 1 Gaming 0 0 0 SAP 0 0 0 LAN - wireless Effective Rate (Mbit/s) 20 20 20 4 Peer-to-peer 0 0 0 POP3 1 1 1 HTTP 1 1 2 Web video 0 0 1 VoIP (secondary) 0 0 1 FoIP (secondary) 0 0 0 Gaming 0 1 1 SAP 0 0 1 Firewall Buffer size 64 KB 64 KB 64 KB Rate (Mbit/s) 100 10 Proc delay (µs) 100 100 100 "Modem"

Buffer size 64 KB 64 KB 64 KB Proc delay 0 0 0

VoIP (primary) FoIP (primary) IPTV service Access (pick one technology)

DSL Access Rate (Mbit/s) 33 22 14 7 Cable Access Rate (Mbit/s) 100 100 50 ONT Access Rate (Mbit/s) 100 100 100 100 100 100

Network TopologiesITU-T G.1050-201X / TIA-921B (Revised Model)

Core-to-LAN

LAN-to-LAN IP Network Model

LAN and Access Block Diagram ITU-T G.1050-201X / TIA-921B (Revised Model)

Test Streams• IPTV• Web Video• VoIP / FoIP• DATA

Network Interferers• TCP• Peer-to-Peer• POP3 email protocol• HTTP web browsing• Web Video (You Tube, Hulu)• VoIP• FoIP• Gaming

Network Interferers• TCP• Peer-to-Peer• POP3 email protocol• HTTP web browsing• Web Video (You Tube, Hulu)• VoIP• FoIP• Gaming

LANDSLAM / CMTS /

OLT

DSL Modem / Cable Modem /

ONTFirewallEdge Router Wired

Wireless

Residential Gateway

Capturing “Real-World” Characteristics

• Capture Packet Delay Variation (PDV) characteristics– Access and LAN Technologies– Different types of Interferers (network traffic ) using .pcap (from Wireshark)

• Result is composite Packet Delay Variation (PDV) for combined Access/LAN and Interferers

Disturbanceload

generatorDisturbance

loadgenerator

Basic Model Element ITU-T G.1050-201X / TIA-921B

21

Disturbanceload

generator

+Input

packetsOutput packets

LinkLatency

Simulated Packet Queue with Loss

Store/Fwd

Network Model Structure (Simplified)

HDTV

InterferersFrom PCAP

IPTV

HTTP

HTTP

POP3

Stream Under Test

EdgeRouter

DSLAM/CMTSOLT

DSL Modem/Cable Modem/

ONT

Firewall/Router

LAN100M

20Mb/sor

30Mb/s 100M1G

1G

64kB buffer

64kB buffer

64kB buffer

64kB buffer

1ms100ns

0ns

Stream Under Test

Composite Interference

STB

HTTP Interference Characteristics

Delay (milliseconds)

Packet Delay Variation (PDV) Histogram Delay vs. Time

IPTV Interference (VBR) Characteristics



POP3 Interference Characteristics



Example #1

Test StreamsSTB#1 IPTV – SD 2Mb/s – QoS 1

Access DSL

20Mb/sNetwork InterferersPC#1, HTTP web browsing – QoS 2

PC#2, POP3 email protocol – QoS 2

STB#2, IPTV – SD (2 Mb/s) – QoS 1

STB#3 , IPTV – SD (2 Mb/s) – QoS 1

Network InterferersPC#1, HTTP web browsing – QoS 2




LANDSLAM / CMTS /

OLT



Wireless

Residential Gateway

Example #1

Network Interferers PC#1, HTTP web browsing – QoS 2




Test StreamsSTB#1, IPTV – SD 2Mb/s – QoS 1

Access DSL

20Mb/s

Example #1 Test Stream


Test StreamsSTB#1, IPTV – SD 2Mb/s – QoS 1

Access DSL

20Mb/s

Example #2

Test StreamsSTB#1, IPTV – HD 8Mb/s – QoS 1

Access DSL

20Mb/s

LANDSLAM / CMTS /

OLT



Wireless

Residential Gateway









Example #2






Access DSL

20Mb/s

Example #2 Test Stream



Access DSL

20Mb/s

Example #3


Access DSL

30Mb/sNetwork InterferersPC#1, HTTP web browsing – QoS 2








LANDSLAM / CMTS /

OLT



Wireless

Residential Gateway

Example #3






Access DSL

30Mb/s

Example #3Test Stream



Access DSL

30Mb/s

ApplicationsG.1050/TIA-921– Compare the Encoders and STBs

Video Encoders

A

B

C

A

B

C

STBs

G.1050-2007 / TIA-921A Impairmentso Compatibility of Encoder and Decodero Effectiveness Decoders to conceal network errorso Stability of STB under network conditionso Regression Testing o Standard, Definitive, Repeatable set of tests representing Real-World IP Network Conditions

Imp Gen

ApplicationsG.1050/TIA-921– Emulate Field Problems, Error Correcting, Server Loading

AServer

G.1050-2007 / TIA-921A Impairments

DServer

RUDP

o Emulate field problemso Emulate effectiveness of error correcting schemeo Test Loading of A&D Server in presence of Network Errorso Test ability of system to meet required specifications

• Standard, Definitive, Repeatable set of tests representing Service Levels

Imp Gen

ApplicationsG.1050/TIA-921 – Effectiveness of FEC

VideoServer

G.1050-2007 / TIA-921A Impairments

o Emulate effectiveness of Forward Error Correcting (FEC) Techniques and Equipmento Test ability of FEC to meet required specifications

• Standard, Definitive, Repeatable set of tests representing Real-World IP Network Conditions

FEC Equipment

FEC Equipment

Imp Gen

ApplicationsG.1050/TIA-921 – General Network Emulation

DUTA

DUTBImp Gen

Test Any Type of Network Equipment under Real-World Network Characteristics

DUTB

TR30.3 and ITU-T SG12 Welcome Your Input to Make

G.1050 More Accurate

Documents

Telecommunications Industry AssociationTR-30.3/10-04-011 By Teleconference April 26, 2010