1.2 QoS Measurements Methods and Tools

QoS Measurements Methods and Tools

Contact:Jarmo prokkola

[email protected]: +358 20 722 2346

VTT Technical Reseach Centre of Finland

Easy Wireless Workshop,IST Summit, Budapest, 05.07.2007

mailto:[email protected]

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

2Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola

Network performance vs. QoS

•Network traffic measurements are employed to study the pure networkperformance

•Active measurements are performed by injecting traffic with knownproperties into the network

•Passive measurements consists of monitoring the existing trafficflow(s) at one or more points

•QoS measurements lie logically above network traffic measurements,and relate to the performance of networking applications.

•Objective QoS relates to something concrete and quantitative•Packet loss, Delay, Jitter, Connection break length…

•Subjective QoS corresponds to the service quality from the userperspective.

•Mean Opinion Score (MOS) tests are often used•Subjective QoS can be estimated within certain limits from the

basis of objective QoS (e.g., PESQ algorithm)



The purpose of QoS measurements & monitoring

•Answers to the question: How do the applications see the network?

•Realtime QoS monitoring benefits•Network administrators•Network equipment manufacturers (testing)•Networking application developers•Operators•End users•Networking performance

•QoS aware applications (e.g., adapting video traffic flow)•Handover triggering•Congestion control management

•Accurate QoS measurements can be used by•Network equipment manufacturers (testing)•Networking application developers•Researchers



Common QoS Measurement Architecture

•QoS measurement architecture includes:•(a) Measurement points

•Located at network nodes(e.g., hosts, routers, firewalls)

•(b) Traffic measurement tool•Captures packets and collects

information of the desired traffic flow•(c) QoS analysis tool

•Analyzes the collected data andcalculates the actual QoS statistics

•(d) QoS monitor or database•Analysis can be done in realtime,

and/or after the traffic traces have beencollected.

•Analysis can be centralized to aspecific server, or distributed to thenetwork.

Trafficmeasurement tool

Measurement point(network node)

Measurement point(network node)

Trafficmeasurement tool

QoS analysistool

…

QoS(realtime monitor

or database)

a

b

c

d

Figure 1. Functional decomposition of QoSmeasurement architecture.



Singlepoint measurements

•Measurement tool is operating in asingle network node

•Round trip (RT) performance•RT delay, RT jitter, RT packet loss•Service response time

•E.g., HTTP, or commonly TCPperformance

•Singlepoint traffic statistics•Offered traffic load, throughput,

number of packets, packet sizes…

•Oneway performance can not bemeasured

Round trip performance

Figure 2. A singlepoint measurementarchitecture.



Twopoint measurements

•If the measurement tool is connected toboth communicating nodes, this is calledendtoend measurement.

•Oneway performance can be measuredseparately for UL and DL

•Delay, jitter, packet loss, …•Directional traffic flow performance,

e.g., realtime traffic flows, UDP•Traffic statistics from both points

•Realtime monitoring requires controltraffic

•Enables also twoway monitoringpossibility in a single point

•Requires flow identification and timingsynchronization (for delay) betweenmeasurement points

endtoend performance

Uplink (UL) performance

Downlink (DL) performance

Figure 3. An endtoend measurementarchitecture.



Multipoint Measurements

•Endtoend performance can be split to network segment performances.•E.g., what is the access network performance vs. core network performance?

•Enables bottleneck identification (e.g., how do individual network devices work?)

endtoend performance

Network segment 1 performance Network segment 2 performance

•Measurement tool unitscould be passivecomponents, and e.g.,network administratorcould activate monitoringwhen needed

•Analysis and resultcollection is challenging

•Measurement units coulde.g., communicate with acentralized analysis tool

Figure 4. An example of a 3point measurement architecture.



QoS measurement tools used in EW: MOSET

• MOSET (VTT’s freeware) is an activemobile service testing tool

•A testing application for mobilephone

•Execution of measurements /tests

•Instant results•A testing server

•Collect & manage results•Distribute test applications and

tests• Service response time measurements

•Connection establishment•Retrieval of service content from

network•Measures HTTP/TCP performance

from the user point or viewFigure 5. MOSET architecture.



QoS measurement tools used in EW: QoSMeT

•QoSMeT (VTT) is a passive tool for measuringoneway endtoend network QoS from theapplication’s point of view

•Realtime measurements & accurate perpacket statistics

•Is able to measure QoS of realtimeapplications (e.g., VoIP, video conferencing),but other networking applications can bemeasured as well.

•The tool works practically over any kind ofnetwork as long as IP is supported

•QoSMeT can be run in the same device withthe measured application or within the networkpath at desired points

•Measures e.g.: Delay, Jitter, Packet loss,Connection break duration (e.g., during ahandover), Throughput and Offered load, andthe volume of data sent/received

•GPS is used for clock synchronizationFigure 6. QoSMeT architecture.



QoS measurement tools used in EW: M5 MultiAnalyzer

•M5 (NetHawk) enables protocol monitoring, call and session tracing, keyperformance indicator (KPI) analysis, QoS measurements, and radiooptimization measurements

•Monitors and analyzes UTRAN, GERAN, UMAN, IMS, WiMAX and corenetwork interfaces at the same time.

•Supports all common transport techniques –STM1/OC3, STM4/OC12, E1/J1/T1 in different modes and Ethernet

•Product components: Standard PC, NetHawk Adapters & cables andNetHawk M5 analysis SW

•Enables multipoint flow measurements with QoSMeT•QoSMeT handles the endpoints, while M5 takes traces within the

network path and decodes QoSMeT’s control packets



QoS measurement tools used in EW: Agent based solution

• The EW agent (Moviquity) is a software componentattached to the terminal and server software:

•Measurement of main E2E QoS parameters (BW,delay, jitter, packet loss)

•Action control (switch to preferred network, codecchange)

• Three kinds of agents are required:•Server Agent (EWSA) including Central Agent•Network/Domain Agent (EWDA)•User/Mobile Agent (EWMA)

• Via continuous feedback, it is possible to monitoractively the communications environment.

• Complete knowledge of the network and service statushelps the server:

•To share and administrate resources with betterefficiency

•Provide the maximum possible quality to the usertaking into account the network the applicationservice and the user profileFigure 7. Agents taking delay measurements: It is

seen that during video reception there is clearlymore delay in the network.



EW Measurement Examples:VHO Performance

•A simple handover scenario (VoIP application using Mobile IP):LAN WLAN 3G WLAN LAN

•Great differences in delay and jitter between network technologies(e.g., 3G onewaydelay is only barely acceptable for VoIP)

•Connection break length is on the order of seconds during handovers

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0.0 50.0 100.0 150.0 200.0 250.0Time [s]

Del

ay [s

]

0.01

0.1

1

10

0.0 50.0 100.0 150.0 200.0 250.0

Time [s]

Con

nect

ion

loss

leng

th [s

]

Delay Connection break length



0.0

200.0

400.0

600.0

800.0

1000.0

1200.0

10 100 1000 10000 100000 1000000 10000000UDP packet size [B], TCP downloaded file size [B]

Goo

dput

[kbi

t/s] UDP HSDPA

UDP 3G

TCP HSDPA

TCP 3G

UDP HSDPA Throughput

EW Measurement Examples:HSDPA vs. WCDMA performance in a live network

•A great improvement withHSDPA is got as compared tothe basic 3G/WCDMA

•With UDP, goodput near to theoperator’s limit (1 Mbit/s) isachieved with reasonably smallpacket sizes.

•However, good TCPperformance still requires quitelarge packet size

• RT delay is high also inHSDPA, thus limiting theperformance of TCP

=> A typical WWW user doesnot see the real potential ofHSDPA!

Goodput

Downlink goodput/throughput performance as a function of data packet size



EW Measurement Examples:Multipoint Measurements in VTT’s CNL 3G Network

•Uplink delay behavior indifferent sections of thenetwork

•Lightly loaded network,to minimize the effectsof queuing delay

•Most of the total delay iscaused by the radioaccess part

•The effect of laboratoryLAN is practicallymeaningless

0.000

0.010

0.020

0.030

0.040

0.050

0.060

0.070

0.080

0.090

0.100

0.0 5.0 10.0 15.0 20.0Measurement time [s]

Del

ay [s

]

EndtoendRadio accessRNC simulatorEthernet (after RNC)



Conclusions

•The complexity of QoS measurements grow when increasing thenumber of measurement points

•While single point measurements are simple, the informationprovided is very limited

•Analysis of even a large monitoring network can be simple ifonly traffic statistics are needed. However, traffic flow behavioranalysis is complicated and needs also clock synchronization ifdelay measurements are desired.

•QoS measurements play important role in EW, several tools areused, and new tools have been developed

•The tools are used in revealing the performance of the existingnetworks, giving feedback to applications, and in measuring theperformance of the developed QoS methods.

•Interesting performance behaviors have been found in numerousmeasurements in EW, and several publications have been done.

Documents

1.2 QoS Measurements Methods and Tools