IxChariot Training

IxChariot

Course # 985-0300

Copyright 2009

Introduction to IxChariot

©2007 Ixia. All rights reserved.

Console

Endpoint 1

Endpoint 2

What is IxChariot?

IxChariot is a software-based network performance testing and

measurement tool

Tests data, video, voice, broadband and wireless transport networks

Emulates application traffic and measures end-to-end performance and

quality

Used to…

Emulate applications to ensure the integrity, performance and reliability of a

network before application deployment

Benchmark network performance and troubleshoot connectivity issues

Test functionality of network equipment under varying traffic profiles

©2007 Ixia. All rights reserved.

Application Testers (e.g. AppTest)

Application-level Network Testers (IxChariot)

Packet Generators (e.g., IxExplorer)

Application

Presentation

Session

Transport

Network

Data Link

Physical

What Layers are Tested by IxChariot?

{ {

©2007 Ixia. All rights reserved.

Qualifying hardware and software

Profiling a new application to predict its impact

Benchmarking to establish Service-Level

Agreements

Verifying the network after changes are made

Voice over IP assessment testing

Multicast interoperability testing

Triple-Play testing with VoIP, Video, and Data

traffic

Wireless network testing

How is IxChariot used?

©2007 Ixia. All rights reserved.

How is IxChariot Used in Wireless?

Tests wireless roaming performance

IxChariot is part of the official test plans for:

– Wi-Fi 802.11a/WPA™ Test Plan

– Wi-Fi 802.11b/WPA Test Plan

– Wi-Fi 802.11g/WPA Test Plan

– Wi-Fi PDA Test Plan

– Wi-Fi WMM™ Test Plan

– Wi-Fi WPA2 Test Plan

Using IxChariot

©2007 Ixia. All rights reserved.

Test Instructions

Application Scripts

TCP, UDP, RTP

IPX, SPX

IPv4, IPv6

VoIP, Multicast

TCP, UDP, RTP

IPX, SPX

IPv4, IPv6

VoIP, Multicast

Test Results

IxChariot’s Basic Design

Results Reporting

• .html

• .csv

• .txt

©2007 Ixia. All rights reserved.

Performance Endpoints

Installed as a service or daemon

Execute scripts

Measure performance

Return observed data

30+ Operating systems supported

Including Windows (including Win CE), Solaris, Linux, AIX, MVS,

OS/2, NetWare, Tru64 UNIX, FreeBSD, SCO UnixWare, IRIX

©2007 Ixia. All rights reserved.

Lab 1 Application Rollout

10

Lab Objectives

Students who complete this lab will be able to:

1. Create a Basic IxChariot test.

2. Understand what StackManger is and what it is used for.

3. Replicate a pair and run a multi-pair IxChariot test.

4. Edit an IxChariot Script.

5. Begin to Interpret IxChariot test results.

©2007 Ixia. All rights reserved.

From Address

To Address

Protocol

Script

Modify Script

Service Quality

Defining the Endpoint Pair

©2007 Ixia. All rights reserved.

Selecting Script for Endpoint Pair

©2007 Ixia. All rights reserved.

Types of Application Scripts

IPX, RTP, or UDP

protocols only

No acknowledgments or

separate connections

made

Bader Benchmark scripts

Internet scripts

Business scripts

©2007 Ixia. All rights reserved.

Application Script

Run on Endpoint Pairs to emulate application traffic

©2007 Ixia. All rights reserved.

Choosing Run Options

Ending a Test Run

Run Until. . .

Reporting Timings

Real-Time or Batch Reporting

Polling Endpoints

Whether and how often

©2007 Ixia. All rights reserved.

Real-Time versus Batch Reporting

Real-Time Reporting

Lets you see results as they happen but

Test data flows may skew the test results

Batch Reporting

Minimized Console traffic doesn’t skew the results but

You must poll to monitor a currently-running test

©2007 Ixia. All rights reserved.

Lab 2 Voice over IP (VoIP)

17

©2007 Ixia. All rights reserved.

Lab 2 Voice over IP (VoIP)

Lab Objectives

Students who complete this lab will be able to:

1. Create a basic IxChariot VoIP test

2. Learn what MOS means and how IxChariot calculates MOS

3. Generate a full duplex VoIP call

4. Analyze call quality using IxChariot VoIP metrics

VoIP Testing with IxChariot

©2007 Ixia. All rights reserved.

How does IxChariot test VoIP?

IxChariot tests the network’s ability to handle VoIP traffic

Real Time Protocol

Voice Quality

Measure E-model and R value

VoIP Standards

©2007 Ixia. All rights reserved.

VoIP Overview

Real Time Protocol

Voice Quality

E-model and R value

VoIP Standards

©2007 Ixia. All rights reserved.

Voice Quality

Voice Quality is a complex issue: It depends upon the Codec used

It depends upon any QoS settings

It depends upon consistency of reception

It depends upon lost data

It depends upon throughput

It has several possible scoring methods

Scoring methods: MOS (ITU P.800)

Mean Opinion Score

PSQM (ITU P.861) /PSQM+

Perceptual Speech Quality Measure

MNB (ITU P.861)

Measuring Normalized Blocks

PESQ (ITU P.862)

Perceptual Evaluation of Speech Quality

PAMS (British Telecom)

Perceptual Analysis Measurement System

The E-model (ITU G.107)

©2007 Ixia. All rights reserved.

Why is Quality so Important?

Voice traffic is small compared to data traffic.

A voice conversation is a two-way exchange, but

voice traffic is two one-way unicasts.

The traffic is time sensitive – delays can make the

conversation walkie-talkie like

Lost data does not get retransmitted because it is

streaming media

Consecutive lost data can create large gaps in the

conversation

©2007 Ixia. All rights reserved.

Voice Quality Measurements

PSQM (Perceptual Speech Quality Measurement)

PSQM+

PESQ (Perceptual Evaluation of Speech Quality)

MNB (Measuring Normalized Blocks)

PAMS (Perceptual Analysis Measurement System)

British Telecom variation, requires licensing fee

MOS (Mean Opinion Score)

Subjective – based upon representative samples

©2007 Ixia. All rights reserved.

E-model and R value

E model – ITU G.107

R value maps to a MOS score

©2007 Ixia. All rights reserved.

IxChariot’s Role in VoIP Testing

©2007 Ixia. All rights reserved.

Using the VoIP Features of IxChariot

©2007 Ixia. All rights reserved.

Special VoIP Pair Settings

Special Features for VoIP

Five different codec types, emulating Different compression algorithms

Data rates

Datagram sizes

Flexibility in changing datagram sizes

Silence suppression & voice activity

rate

Configurable jitter buffer

Quality of service

Performance and quality metrics

Scoring methods: Jitter

lost data

consecutive lost datagrams

one-way delay

Mean Opinion Score

©2007 Ixia. All rights reserved.

Lab 3 Video and Multicast

29

©2007 Ixia. All rights reserved.

Lab 3 Video and Multicast

Lab Objectives

Students who complete this lab will be able to:

1. Use IxChariot to emulate real video traffic

2. Understand IxChariot’s video metrics

3. Analyze and trouble-shoot video quality issues

Video and Multicast testing with IxChariot

©2007 Ixia. All rights reserved.

IP Video testing with IxChariot

Tests the network’s ability to deliver IP Video Quality of

Experience (QOE)

IP Video Metric Ideal

Established Video Metrics

Media Delivery Index (Delay Factor / Media Loss Rate)

©2007 Ixia. All rights reserved.

Evaluate Network Performance with Datagrams

Datagram behavior can indicate the following

disorders:

Retransmissions (Duplicates Sent)

Network Loops (Duplicate packets received, packets arriving out of

order)

Lost packets vs. packets received late in either direction

Multiple paths (packets arriving out of order)

Frame size sensitivity

©2007 Ixia. All rights reserved.

RTP: Real Time Transport Protocol

Used by many leading voice and video applications

Provides end-to-end transport functions suitable for

applications transmitting real-time data

Independent of the transport and network layers

Documented in RFC 1889

©2007 Ixia. All rights reserved.

B

A

C

B C

D1 D2=D1 D3¹D2

E1 sends

E2 receives

Network

E1 E2

A

Jitter

©2007 Ixia. All rights reserved.

Jitter

©2007 Ixia. All rights reserved.

Test multicast interoperability among different devices

Datagrams

Sending/Receiving IP Multicast Traffic

Setting Up Multicast Groups

Evaluating Network Performance with Datagrams

Tuning Datagram Performance

Multicast testing with IxChariot

©2007 Ixia. All rights reserved.

Datagrams

Sending/Receiving IP Multicast Traffic

Setting Up Multicast Groups

Evaluating Network Performance with Datagrams

Tuning Datagram Performance

Multicast and Datagram Statistics

©2007 Ixia. All rights reserved.

10.20.134.7

10.20.134.3

10.20.134.4

10.20.134.5

10.20.134.6

10.20.134.8

10.20.134.9

10.20.134.10

10.20.134.11

10.20.134.12

10.1.14.254

Traditional Traffic Flows

©2007 Ixia. All rights reserved.

10.1.14.254 225.1.1.1

Applications that Use IP Multicast

©2007 Ixia. All rights reserved.

Multicast Routing

Protocol

Eth

ern

et

Eth

ern

et

Eth

ern

et

Router

Router

Router

Router

Router

Router

Tower

box

Eth

ern

et

Receiving IP Multicast Traffic

©2007 Ixia. All rights reserved.

Multicast

Sender

Multicast

Receivers

X Single

Data Send

data propagated

to receivers

Multicast

Routers

Eth

ern

et

Eth

ern

et

Eth

ern

et

Eth

ern

et

Router

Router

Router

Router

Router

Router

Tower

box

replicated

data send

data pruned where no receivers

Sending IP Multicast Traffic

©2007 Ixia. All rights reserved.

Performance Considerations

Is my network ready? Raw capacity

Upgrades

Configuration

How much IP Multicast traffic? Receiver’s quality

• Send data rate

• Buffering

• Lost data rate

• Other network traffic

How many simultaneous senders?

What happens during congestion? TCP applications vs. UDP applications

Does QoS or bandwidth management help?

©2007 Ixia. All rights reserved.

Lab 4 Triple Play

44

©2007 Ixia. All rights reserved.

Lab Objectives

Students who complete this lab will be able to:

1. Comprehensively assess the performance of the Triple Play

network

2. Interpret Voice, Video, and Date metrics

3. Generate HTML reports

Lab 4 Triple Play

©2007 Ixia. All rights reserved.

Designing a Good Test

The Ideal Test:

Runs long enough for small relative precision

Avoids timing records shorter than 10ms.

Doesn’t generate too many timing records

But, generates enough records to show fluctuations

©2007 Ixia. All rights reserved.

Console

E1 E2 Firewall

Admissible port =

explicit port number in script

Testing through a Firewall

©2007 Ixia. All rights reserved.

Console

E1 E2

Firewall

• For setup: 10115 – default but now

configurable

• For routing timing records --

User configurable

(Change User Settings notebook)

Test Data through a Firewall

©2007 Ixia. All rights reserved.

Test Results

Interpreting Test Results

Timing Records

Measured Time

Confidence intervals

Relative precision

Three primary statistics

Throughput (really “Goodput” – RFC 3511)

Average throughput from Layer 7

Transaction rate

Number of script transactions executed each second

Response time

Time (in seconds) required for one transaction

©2007 Ixia. All rights reserved.

IxChariot Main Results Window

All results from tests are viewed in both graphical and

tabular format

©2007 Ixia. All rights reserved.

Granular data on each timing record available

Timing Records

©2007 Ixia. All rights reserved.

Feeling Good about Test Results

95% Confidence Interval

Well-known statistical measurement for the

reliability of the calculated average

95% likelihood that the actual average lies

between the lower and upper bound indicated

by the 95% Confidence Interval

Relative Precision

"Good" Relative Precision value is 10.00 or less

Can compare relative precision values regardless of

type of script run

©2007 Ixia. All rights reserved.

Compare Tests

©2007 Ixia. All rights reserved.

Working With Graphs

Creating Bar Graphs

Creating Pie Charts

Creating Histograms

Recognizing a Problem

©2007 Ixia. All rights reserved.

Graph Types

©2007 Ixia. All rights reserved.

Modifying the Axis

The graph axis scale is created automatically,

sometimes overemphasizing measurements

Adjusting the scale can clarify measurements

©2007 Ixia. All rights reserved.

Using Pie Charts

Pie Charts highlight Throughput and Transaction rate

differences