Descriptive Statistical Transport Stream Analyzer

Alexe Radu Andrei

Telecommunications Department

Politehnica University of Bucharest

Bucharest, Romania

Abstract—This paper presents the design and implementation of

a new type of transport stream analyzer based on metrics derived

from the field of descriptive statistics. The method used ensures a

detailed analysis of the input stream and presents the result of

the analysis in a clear and clutter free manner. In addition it can

be modified and tailored to different situation, by defining and

using new custom metrics, yielding thus a flexibility not given by

any other analyzing system.

Keywords-Transport stream, analyzer, descriptive statistics

I. INTRODUCTION

Many of today’s real-time MPEG2-TS stream analyzers [2] [3] are geared towards extracting the PSI/SI tables and presenting a map of programs contained by the transport stream (TS). This information is located in the payload of the consecutive transport stream packets that make up the stream. The data stored in the header of the transport stream packets make up the transport layer and provide the means by which the payload data is transmitted over a channel. Therefore at the receiving end, this data is used by the analyzers but rarely presented to the user who is usually interested in the content. This situation, although very common, is not always true: e.g. in academic or research institutes were an in-depth analysis is needed of all the layers of the target transport stream [1]. At the same time we must draw attention to the trade-off between the depth of the analysis, the large quantity of data presented at the input of the system and the limited computing power available. All those limitations, and many other factors, prevent us from using a straightforward solution, where the full content of the stream is presented to the viewer (like in the case of a file analyzer). Instead we took a different approach by defining descriptive statistics metrics that quantitatively describe the main features of the stream without getting into unneeded details. Such a solution provides a good balance between the depth of the analysis and the quantity of the computational resources needed.

II. SYSTEM DESCRIPTION

A. Hardware setup

We have designed The Descriptive Statistics Transport Stream Analyzer as a software application that can run in any environment (any operating system on any hardware platform).

Input Adapter

PCI / USB MPEG2-TS Stream

Analyzer

Figure 1. Hardware setup

This goal has been achieved by using a development framework which is independent platform-wise (Qt/C++). The access to the stream is possible via an input hardware adapter which is connected to the computer through the USB or PCI interface. The API supplied by the adapter’s manufacturer provides the means to access the device’s capabilities. Although the API may contain from several to tens of functions, the analyzing application, at its core, needs only the stream data, the timestamp associated with the received data and the input stream bit rate, which are all accessible through basic read/get functions.

B. Software architecture

In Fig. 2 the analyzing system architecture is presented. It contains a series of functional interconnected blocks that offers an insight about the main stages of the stream analyzing process and the in-between data transfer. The first block of the architecture provides the analyzing system with a simple and clean interface to the underlying physical layer by encapsulating the native adapter API and offering the stream data as a succession of 188 byte long buffers representing the transport stream packets. Additionally a timestamp, representing the instant the first byte of packet was received, and an estimate of the input bit rate are provided for every transport stream packet.

buffer

Interface MPEG2-TS

TS analyzer buffer[188]

PID-stream statistics

timestamp TSPkt

inputBitrate PSI/SI section

analysis

PAT analyzer PMT analyzer

buffer

Display & Command Control

Figure 2. Software architecture

978-1-4673-1176-2/12/$31.00 ©2012 IEEE

PidTable

PidTableLine PID:0

StreamType Bitrate Other statistics

PidTableLine PID:1

StreamType Bitrate Other statistics

PidTableLine PID:M

StreamType Bitrate Other statistics

Column 1 Column 2 Column N

Figure 3. A representation of the PID table

The 188 byte packet buffer is received by the TS analyzer which parses the data into a structure (named/denoted/further referred to as TSPkt) that contains the same information as the transport stream buffer but in a more accessible way in which every field is easily addressable. The transport stream packet, associated timestamp and bit rate represent the inputs of the descriptive statistical analyzing block, called packed identifier (PID) stream statistics, which represents the core of the application. Every transport stream packet contains a 13-bit long field called PID which is used to identify a PID stream. Because most, if not all, the transport stream packet fields have a meaning only within a PID stream the result of the analysis is a dynamic resizable table which may contain up to a maximum of 2

13 = 8192 lines, every line containing a series of statistical

metrics which describe the evolution of the values of the transport stream packet fields (see Fig.3). In addition, every line contains a buffer to store the payloads from consecutive received packets. Once an access unit (PES packet or PSI/SI section) is fully received it is flushed out and, in the case of a PSI/SI section, sent for further analysis to the PSI/SI section analysis block. Here the header of the section is analyzed and its payload is sent to the appropriate table (PAT or PMT) analyzing block according to the value of the table id field found in the current section. The Display and Command Control is a generic block which offers an interface for the user. It has connections with all the major blocks of the analyzer enabling analysis results to be displayed and configuration parameters to be set.

III. DESCRIPTIVE STATISTICAL METRICS

In order to provide a detailed enough analysis of a transport stream and yet to present the result of the analysis clear and clutter free we have opted for defining an accumulation interval (or timeslot) and a series of descriptive statistical metrics that capture the main aspects of the variation of the values taken by the fields of the transport stream packets received during the timeslot. The metrics we have defined in this paper are grouped around either every field of the transport stream packet (which is usually the case) or around a group of semantically related fields. In addition two more separate metrics have been provided, namely the stream type and the bit rate.

A. Stream Type

This propriety is divided into three main metrics:

• Main Type – provides a course division of the stream into a few basic categories

• Subtype – provides additional division, on a finer level, of several of the main categories

• Description – string-like propriety that offers a more detailed description of the stream

Table 1 presents the main stream types/subtypes defined.

The placement of the stream in one category or another is

done in three main ways:

• statically – some streams have a statically assigned PID by the standard

• implicitly – all the streams which do not have a PID statically assigned are initially declared as of type Unknown

• dynamically – as a result of analyzing the PAT or PMT tables

B. Bit rate statistics

This statistic defines two metrics:

• Slot packet counter – the number of TS packets received during a timeslot

• Bit Rate – the PID stream bit rate computed as bit rate = (slot packet counter x packet length) / timeslot length

C. Transport Error Indicator Statistics

This group of statistics takes into consideration the transport error indicator field of the TS packet and defines the following metrics:

• Slot Error Counter / Percent – the number / percent (out of slot packets) of TS packets received during the current timeslot with the transport error indicator field set

• Total Error Counter – the number of TS packets received so far with the transport error indicator field set

TABLE I. MAIN TYPES / SUBTYPES

Main Type Sub-

type Assigned Details

Unknown Implicit Assigned initially

Reserved Implicit Assigned by the standard

PAT Statically PID 0x00

Table PMT Dynamic

As a result of analyzing the

PAT table

Video Dynamic As a result of analyzing the

PMT table

Audio Dynamic As a result of analyzing the

PMT table

PCR Dynamic As a result of analyzing the

PMT table

Other Dynamic As a result of analyzing the

PMT table

D. Payload Unit Start Indicator Statistics

This group of statistics refers to the payload unit start indicator field of the TS packet and defines the following metric:

• Payload Unit Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the payload unit start indicator field set

E. Transport Priority Statistics

This group of statistics refers to the transport priority field of the TS packet and defines the following metrics:

• High Priority Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the transport priority field set

• Low Priority Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the transport priority field reset

F. Transport Scrambling Statistics

This group of statistics refers to the transport scrambling control field of the TS packet and defines the following metrics:

• No Scrambling Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the transport scrambling control field set to ‘00’

• Type 1 Scrambling Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the transport scrambling control field set to ‘01’

• Type 2 Scrambling Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the transport scrambling control field set to ‘10’

• Type 3 Scrambling Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the transport scrambling control field set to ‘11’

G. Continuity/Discontinuity Statistics

This group of statistics refers to the continuity counter (CC) and discontinuity indicator fields of the TS packet and defines the following metrics:

• CC Range – it is represented as [First CC Value – Last CC Value] i.e. the value of the first and last continuity counter field received in the current timeslot

• CC Wrap Around Counter – the number of times the continuity counter wraps around its maximum value during the current timeslot

• CC Error Counter – the number of erroneous uses of the continuity counter: when a discontinuity of the continuity counter happens without being signaled by the discontinuity indicator and when more than two consecutive identical packets (with the same continuity counter) are received for the same PID

• Duplicate Packet Counter / Percent – the number of duplicate packets received during the current timeslot

• Discontinuity Counter – the number of packets received during the current timeslot with the discontinuity indicator set

H. Adaptation Field Statistics

This group of statistics refers to the adaptation field and adaptation field length fields of the TS packet and defines the following metrics:

• Reserved Packet Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the adaptation field control field set to ‘00’

• Adaptation Field Only Packet Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the adaptation field control field set to ‘01’

• Payload Only Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the adaptation field control field set to ‘10’

• Adaptation Field and Payload Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the adaptation field control field set to ‘11’

• Payload Bit Rate – the number of payload data bytes received during the current timeslot averaged over the length of the timeout interval

I. Random Access Statistics

This group of statistics refers to the random access indicator and payload unit start indicator fields of the TS packet and defines the following metrics:

• Random Access Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the random access indicator field set

• Random Access Error Counter – the number of TS packets received so far which have the random access indicator set but the payload unit start indicator reset

J. Elementary Stream Priority Statistics

This group of statistics refers to the elementary stream priority indicator field of the TS packet and defines the following metrics:

• Elementary Stream High Priority Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the elementary stream priority indicator field set

• Elementary Stream Low Priority Counter / Percent – the number / percent (out of the slot packets) of TS packets received during the current timeslot which have the elementary stream priority indicator field reset

K. Program Clock Reference Statistics

This group of statistics refers to the program clock reference flag and program clock reference (PCR) field of the TS packet and defines the following metric:

• PCR Range – it is represented as [First PCR Value – Last PCR Value] i.e. the value of the first and last program clock reference received in the current timeslot

L. Original Program Clock Reference Statistics

This group of statistics refers to the original program clock reference flag and original program clock reference (OPCR) field of the TS packet and defines the following metric:

• OPCR Range – it is represented as [First OPCR Value – Last OPCR Value] i.e. the value of the first and last original program clock reference received in the current timeslot

M. Transport Private Data Statistics

This group of statistics refers to the transport private data flag, transport private data length and transport private data fields of the TS packet and defines the following metrics:

• Private Data Packet Counter / Percent – the number / percent (out of slot packets) of TS packets received during the current timeslot which contain private data (have the private data flag set)

• Private Data Bit Rate – the number of transport private data bytes received during the current timeslot averaged over the timeout interval

N. Splice Statistics

This group of statistics refers to the splicing point flag and splices countdown fields of the TS packet and defines the following metric:

• Splice Range – it is represented as [First Splice Value – Last Splice Value] i.e. the value of the first and last splice countdown received in the current timeslot

IV. TESTS AND RESULTS

To test the capabilities of our transport stream analyzer against another commercially available real-time stream analyzer [2] we have provided a test stream which contains only the PAT table with no additional elementary streams or tables.

Figure 4. Comparison Test

This stream has a bit rate of approximately 15 kbps and is formed by looping over a twenty packet transport stream file. Two of these twenty packets have the transport error indicator set and the stream contains two duplicate packets. The timeslot interval has been set for both analyzers to be 2 seconds long. Fig. 4 shows the result of the analysis for both analyzers. The superiority of our solution is evident because:

• it provides a complete analysis – our analyzer provides metrics that describe all the transport stream packet fields;

• it offers detailed analysis for the defined metrics – even for the common defined metrics (e.g. the continuity counter) the level of details is superior for our solution.

In addition the information have been presented in a layered manner offering the possibility to vary the degree of details viewed for the result of the analysis.

V. CONCLUSION

The descriptive statistical transport stream analyzer described in this paper provides a detailed customizable analysis of the transport stream. The architecture and solutions used for developing this analyzing system offer a great flexibility in portability on any existing platform with great performance over a wide range of general purpose devices. In addition, because the system was designed to work with a transport stream, it can be easily ported so that it may serve as an analyzer for a stream that complies with any standard derived from MPEG2-TS (e.g. DVB, ATSC).

REFERENCES

[1] ISO/IEC International Standard 13818-1, Generic coding of moving picture and associate audio information, Part 1: Systems, 2000

[2] DEKTEC Corporation, DTC-320 StreamXpert, http://www.dektec.com/Products/Apps/DTC-320

[3] GkWare Corporation, StreamGuru MPEG Stream Analyzer, http://www.streamguru.de/mpeg-analyzer/