13

PIcTzlRE QUALITY ISSUES IN DIGITAL VIDEO COMPRESSION.

INTRODUCTION:

The picture quality aspects of digital television are of growing importance especially because of the increasing use of compression throughout the broadcast chain and other uses of compressed video on other media.

. .

pi- .......................... I Q4 .. . . . . . . . . . . . . . . . . .................... I I I Figure1 TbeBdcastChain.

In the broadcast case, illustrated by Figure 1, there are several stages between the studio and the home; for analogue technology the most variation, and, hence, greatest part of the impairment budget, is allocated to the latter stages of the process. This model does not necessarily describe digital compression systems because the type and allocation of impairments are not the same.

Broadcasting processes, each with its own quality needs, which already employ compression are:

Non-Linear Editing, Compressed signals for storage in hard disc servers, Production VTRs - Digital Betacam, Ampex DCT etc., Distribution to Satellite, Cable and Terrestrial transmitters, Direct transmission to the home - DTHIDBS.

The consumer electronics and particularly the computer industries, have been driven by their own, often proprietary, standards of quality. The broadcast community has always been concerned with picture quality, its assessment and quantification and has worked towards concensus on quality issues. This paper explores aspects of this assessment problem particularly as they arise in transmission systems.

QUALITY ASSESSMENT METHODS:

When early analogue TV signals began to be distributed widely, the specifications of the systems which conveyed them were developed using extensive suhjective testing. These studies identified the key parameters, placed limits on their variation by understanding the way in which they accumulate in a network, distrihuted hudgeted limits throughout a system and, finally, defined test signals for use in objective tests. The proven correlations between the observed suhjective picture quality and ohjective measurement of analogue impairments then led. over many years, to CCIR Recommendations 500-4 [methodologies] and 567-3 [test signalsl. The success of these methods has led to the routine use ot Insertion Test Signals [ITS] in the Vertical Blanking Interval [VBI] of an analogue video waveform which allows in-service quality monitoring, provided the ITSIVBI undergoes the same experience as the picture.

This has only been possihle for analogue signals because both studio and transmission impairments are of the same kind eg the addition of noise or linear distortions. Digital Compression and Transmission defects are distinct in their effect on picture quality; so much so that they cannot be added on any practical basis. Transmission and Recording systems are designed to be substantially error free hut their failure is very sudden [see below] and the de-compression process is then severely affected and so snme error resilience in the compression algorithm is clearly valuable. However, this value is limited by the speed of failure and careful management of channel operating characteristics is vital.

Parameters applying to analogue signal interfaces, the latter three added for the composite case, are:

Signal Amplitudes and Timing, K Factors, Luminance non-linearity, Luminance/ChrominanceGain/DelayInequality , Noise, Differential Gain and Phase (DGDP), Sub-carrier Features, Chrominance Phase Noise.

These objective parameters have stood the test of time

lnternatlonal Broadcasting Convention. 14-18 September 1995 Conference Publcation No. 413, &? IEE 1995

14

but the advent of digital methods of coding seriously challenge them and, particularly with coming of compression systems, it has become clear that these parameters have serious, sometimes profound, limitations in properly assessing the performance of digital television systems. For example, none of them reflect accurately the effect of digit errors on the picture, the highly picture dependent motion processing of a high ratio compression system or standards conversion whose artifacts remain among the worst picture defects today. These have been tolerated [a] because the problem is difficult to solve, [b] only a small proportion of viewing is affected by the defects and [c] performance is still limited by economic factors. With the prospect of a variety of compression systems permeating every stage in the broadcast chain for all programming, one crucial question is: Can this situation be left to a similar fate where picture quality becomes uncontrollable?

Figure 2 Generic Analogue TV TrnnWisSian and Testing.

Figure 2 illustrates the measurement of analogue television systems; Figure 3 is a more complete scheme where A/D and DIA and other processes are involved, one example of which could be compression. Equipment suitable for this latter case could do some kind of signature analysis but how this might relate directly to picture quality i s not clear. Such measurements have value simply because they give some confidence that the equipment is working within its design limits. Current analogue methods also do this but the design limits relate directly to picture quality. The 'Garbage In, Garbage Out' [GIGO] phenomenon applies well to compression, illustrating the limits of engineering input to the broader quality debate.

The subject cannot be treated exhaustively here; there is a considerable amount of work needed to understand and resolve the many issues involved. Subjective testing remains the only reliable means of picture quality assessment in the digital era and will remain so until proven correlations between test parameters and picture quality can be found. Encouraging signs suggest that the matter is beginning to receive attention; one is the

European RACE project MOSAIC, established to study methodologies whereby digital impairments can be assessed and subsequently measured. Another is that the early adopters of the technology, both manufacturers and users, are beginning to understand the processes and a dialogue has been established. Various useful

to focus the public debate. published articles have appeared '. '. 3. 4* '. 6.

ASSESSING VIDEO COMPRESSION SCHEMES:

Pmcessing:

Compression comprises the following processing stages, as illustrated by FiEure 4, all of which are needed to achieve the highest compression ratios:

(IIxwMm'ALcUnur

Figure 4

* Input processing, including synchronisation, any composite decoding and blanking suppression, * AID conversion of input analogue signals, * Temporal and/or Spatial redundancy reduction; the latter alone for mild degrees of compression eg DVTR or Non-Linear editors, * Network adaptation.

Practical equipment needs resilience to abnormal conditions by providing for:

Block Diagram of Compression C"

* Hum Tolerance * Overload margin * Coding delay [much larger for Temporal processes] * Relative Audio/Video delay * Composite sub-camer stability * Response to short interruptions in transmission path.

The measured performance of the same compression process will differ depending upon the actual interfacing format used for the presentation of picture material and the corresponding test signals. The question of composite signal decoding and re-coding during compression [and other] component based processes must be a serious element in the quality assessment, a point illustrated well in results reported in ' where the ultimate perfonnance is limited by defects in composite signals.

Because noise is unstructured, compression cannot achieve good results with noisy signals. Some input noise reduction may therefore be useful to optimise the performance of the subsequent compression process provided that its artifacts are more benign than those they attempt to avoid.

A/D comprises Sampling and Quantising which can cause impairment due to inadequate sampling, breaking Nyquist's rule, or inadequate quantising, too few amplitude levels. The bit rate, the product of the sampling rate and the number of bits per sample, is minimised by minimising each component. Sampling synchronisation to the PAL or NTSC colour sub-carrier produces benefits in signal quality. A three times rate is adequate but a rate of four times has greater advantages for processing and is common in studio equipment. Coding of the component signals is defined in CCIR Recommendations 601 and 656.

Quantising occurs when a continuous amplitude range is approximated by a sufficient number of permitted levels which allow the eye to perceive a continuous amplitude range. Video signals which have been gamma corrected at source require 7 or 8 bits for one coding stage but, for more demanding processing in studios often 9 or 10 bits are used. Quantising can result in a noise-like impairment. Provided the television signal has random statistical properties, the quantising error is also noise- like and is measurable reliably in a similar way. If the signal is not random, this impairment is best described as distortion and is not as readily quantifiahle objectively.

If the blanking periods are not transmitted [they can be reinserted] the actual source bit rates are reduced by

about 20%. For transmission to the viewer or for recording the removal of the blanking is appropriate but for inter-studio transmission VBI [and ITS] retention is often preferred.

compressioa Algorithms:

i t 1 1 0 Y im la m

Figure 5 Bit Rntes for various Applications.

Figure 5 illustrates the way in which channel capacity, expressed as the bit rate, varies with the quality required, expressed in the form of the application. Although this is not an accurate graph, the greater rate of change of quality with hit rate at the lower bit rates, ie less than IO MBitis, is a good reflection of practical experience.

All the algorithms currently defined by CCIR and MPEG for transmission rates between 2 and 140 MBitls, as well as many widely available proprietary algorithms, use techniques which exploit strong spatial and temporal correlations in images. That is, each frame of a typical picture is very similar to its previous and subsequent neighbours [Temporal Redundancy] and that particular areas of a given frame can be very similar [Spatial Redundancy]. These similarities allow the complete image to he described by fewer bits per picture element since differences between elements, rather than absolute values, can be coded by virtue of these similarities. Whilst, on average, compression processing produces savings in bit rate, the actual savings vary instantaneously from frame to frame and within each frame depending on the current degree of correlation. The demand for channel capacity therefore varies instantaneously.

The amount of reduction also varies depending upon application; in cases where editing is to be done on compressed data little o r no Temporal redundancy, the most profitable, can be used since the frame sequences and boundaries have to he preserved and so only a small reduction ratio is possible if quality i s to be maintained. To achieve the lowest bit rates consistent with broadcast quality, temporal procasing IS essential and additional data reduction methods are also invoked such as Entropy

16

coding, examples of which are Huffman and Morse Codes, also generally known as Variable Length Coding or VLC.

Fixed and Variable Bit rates:

Where the reduced bit rate signal is to be transmitted by means of a channel employing fixed bit rates there is a need to regulate the variable demand of the compression process by means of a buffer store. The size of the buffer must be consistent with the fixed bit rate available in the channel and the variability of the demand; the rate control of the buffer, such that it neither fills or empties, is one of the major aspects of encoder design. As the buffer fills, feedback control is used to reduce the flow of bits from the compression process by quantising the coded data more coarsely thus reducing the demand on the buffer capacity [see w e 6J but also causing the picture coding accuracy to be degraded, visibly at times, perhaps requiring more bit rate to cure [see Figure 71. The constant bit rate constraint can thus lead to variable quality. If enough [say 8-10 or more] different coded pictures are multiplexed the process can effectively trade-off the short term demands of the several sources so that a degree of statistical sharing can improve quality for all sources.

I I 1 I

I I Figure 6 Block DingrPm of COmpressioo coder.

If variable bit rate is available, such as that possible by telecommunications networks using Asynchronous Transfer Mode [ATM], the peak and mean rates acceptable by the network may allow the compression process to use the network for short term excess capacity demand as might happen when more difficult pictures require coding. Whilst this might seem better than fixed bit rate cases, where buffer management may seem more difficult, there are ATM network control aspects which are important to optimise for the maintenance of broadcast picture quality. ATM networks have not yet been fully defined or implemented widely and there is still much work remaining to obtain the appropriate features and

safeguards for broadcasters' needs. Variable bit rates may also be possible on some recording media such as magnetic or optical discs. Variable bit rates can help support constant picture quality.

-Q- I

I I F i p 7

Degree of Compression:

Qual~ty Variatinns in C o t q " Coding.

The process of compression is potentially 'lossy', meaning that the video or audio signal being compressed may be reconstructed with some distortion, probably of a non-linear nature. At mild compression ratios, say less than lO: l , compression can be a substantially lossless process even with only spatial redundancy reduction. At greater ratios some loss must occur and the design of any system is directed towards minimising the visibility or audibility of the loss by, for example, exploiting the failings of the human visual or auditory systems respectively. Thus, where a compressed video signal is destined for viewing only and the features of the eye can be exploited, relatively more compression can be applied than if the signal is destined for further digital processing. Numerical fidelity of the signal in this latter case is very important and so milder compression is advised.

For analogue systems additive impairments such as noise and distortion are always present more or less to the same degree all the time and all over the picture area so that the eye may get used to them to some extent. Random noise, especially if shaped to shift its energy into the high frequencies, is particularly acceptable to the eye. Shaping digital impairments to be similar to noise seems a useful ohjective. In contrast to this, the performance of compression systems is picture dependent and digital impairments vary with time [ie picture coding difficulty1 and within the picture area. The viewing experience with compression is therefore quite different from analogue and the limits of acceptability are largely defined by those rare, critical picture sequences which are mostly unpredictable, certainly for live programmes. This probabilistic element in coding design can be avoided by maintaining the highest practicable bit rates. In the case of

17

compressing existing recorded material there can be an element of ’adjustment’ during multiple passes, augmented perhaps by human or computer intervention, during the transfer process. Indeed, this is the method used to prepare films and other material for transfer to optical or magnetic disc stores and for CDI and CDV.

Given the above, the use of full-field test signals, ie still pictures, means that any compression system is only ever evaluated objectively in its still picture mode; such a test is also done out of service. This may he adequate for compression systems exploiting spatial redundancy only. Any other modes, for example used for moving pictures with rapid motion [cuts etc.], are left unused during such tests. Appropriate new forms of moving test signals need devising which are subtle but simple such that they can be applied and used effectively.

If several compression systems, either similar or not, are cascaded in a broadcast chain, the impairment which accumulates is not fully understood as yet and the rules by which predictions of performance can be made with some confidence are not clear.

One way to preserve numerical fidelity may be to avoid the analogue domain; the use of composite interfaces between codecs introduces the opportunity for further impairments. Other contributory causes include cascaded anti-alias and reconstruction filters in the AID and DIA processes in the coders and decoders respectively; frequency response errors are not generally the result of impairments arising in compression processes. However, a degree of ‘dither’, as would be caused by low level analogue noise, could he beneficial in smearing the numerical processes by introducing an heuristic element into the subsequent compressors thereby avoiding repeated reinforcement of particular undesirable picture artifacts.

There is reason to suppose that, once the first compression process in a chain of similar systems has processed a given video signal, further degradation is minor if the signal interface between the compression systems is digital. With digital interfaces, all codecs after the first will nominally have the same numerical data to process and so the build up of any compression related impairment should be slow. There is some evidence to support this assertion for some compression schemes but, depending on the algorithm and bit rate, however, it is true to different degrees; it is unrealistic to expect high performance after several tandem 2 MBit/s codecs designed for non-contribution applications.

Applications such as Contribution, where further signal processing may take place at a destination studio, require a higher degree of numerical fidelity from a compression system than do Distribution applications. The EBU and CCIR have established quality criteria for contribution based on 2 codecs in tandem for 34 MBitls [complex hybrid in Figure 61 and 3 in tandem for 140 MBit/s [intra-frame DPCM]. The interface connections between codecs at the intermediate video level were expected to conform to CCIR Recommendation 656. More than this number of codecs in tandem was never envisaged at the time and few currently available codecs of any kind were designed specifically for many tandem connections. Simple algorithms and high bit rate give headroom which can absorb the effects of many tandem compressors; the excessive removal of redundancy makes the resultant hit stream more vulnerable and thus needs to be done with great care.

In studio systems the temporal compression element is avoided because of its effect on editing and other frame conscious processing. The use of repeated & processing eg DCT [JPEG] or Wavelet [proprietary] in non-linear editors or DVTRs, will therefore be of greatest interest. The result of DCT processing defects is ‘blockiness’; for a complex scene, it is possible for adjacent reconstructed pcture blocks to differ visibly at the block edges. The design of coefficient management and mild compression ratios is the key to successful processing but making i t capable of higher compression requires more complication. Where successive DCT processing uses the same block size, and is synchronised such that the blocks are exactly aligned. the same defect may be generated at each stage with potential for a growing error. One way to avoid this is to change the block sizes between processes; this could smooth over the error structure and render i t more noise-like. A

~ small amount of noise in the original video could also be helpful. This is somewhat impractical to arrange by design but may happen in the case of systems from different suppliers where different parameter choices conspire to achieve incidental smoothing. A second way would be to ensure that video synchronisation between successive processes is offset so that it causes the block structures to overlap. This would also cause a degree of randomisation in successive stages which should lead to a smoothing.

Wavelet processing is a form of Sub-Band coding and is an alternative to DCT with which it has some similarities. Here, however, the 2-D picture Tiles can be of variable shapes depending upon the frequency content of the image. This gives a better match to the changing image characteristics and avoids the visible block boundaries of DCT. The processing is marginally more complex in practice and results are encouraging; some claim better subjective results than DCT. The use of diverse systems connected by digital interfaces may

18

be a workable solution; there are several proprietary systems already in use and so this may be the de-facto solution anyway. By the GIGO principle, it is essential that quality is maintained at the early stages because, once lost, it is irretrievable in later stages. Furthermore, as with all things digital, the failure, when it comes, is abrupt and not gradual.

CHANNEL CODING PERFOMANCE:

Given the vagaries of radio waves or recording media, one role for technology in broadcasting is to mitigate their worst effects of on picture quality. For guided transmission media this factor is not so important. Satellite broadcasting, although a spectrum user, has distinctly better propagation stability and so does not suffer the same difficulties as terrestrial broadcasting. Firmre 8 illustrates the failure mode of digital channels where the abruptness needs careful channel coding design including appropriate error correcting codes. Such codes improve the channel characteristics at the expense of the abruptness of failure.

1 ... , .................... : ........ ....................... ....... ...... ...... .......,.... " n D M I y :,,. ." A m -

The quest for a means to manage the sudden failure characteristic of digital channels leads to the idea of graceful failure. One approach is where services can he maintained at more than one level. For example, a high resolution service available on receivers with appropriate processing can be supported from the same transmission as a reduced resolution service on a simpler and cheaper receiver. This is known as hierarchical coding.

The MPEG system also includes the concept of Scalability which allows for hierarchical transmissions. Because of the propagation vagaries described above, broadcasting media, above all others, need scalability in order to he competitive and to overcome inherent technical disadvantages. Scalability is also the means whereby, in the future, High Definition services could be introduced in a compatible way.

CONCLUSIONS:

The paper has reviewed a numher of quality issues and has emphasised the need to devise objective measures of performance which apply to compression processes, since those adapted from analogue are often inappropriate. With DTH broadcast systems, means exist to avoid the major pitfalls of quality variations; several channels of programming in multiplex gives broadcasters flexibility to trade Choice for Quality.

If picture redundancy is over-exploited, there will he an erosion of quality during programme production. It is important for the future that precipitate steps are not taken until the accumulation processes are better understood; high hit rate and modest levels of compression should he employed in studios. It might also he wise to use a variety of systems to assist in smoothing defects. I t is fortunate that temporal processing is inappropriate in studios because the use of high ratio compression needs to he exploited to the full in the final stage to the viewer.

REFERENCES:

Lodge, N K, and Wood, D, 1994, IEE Conf 397 [IBC94], 313-339.

Histed, C, TV Buyer. U, 14-18.

Mitchell, J and Feme. A. 1994, Int B'ing, U, 26-30.

Wilkinson J H . and Stone, J J , March 1994, IEE Coll. Rep 1994. BKSTS J, 128-134.

Wood, D, 1993. EBU Rev, 256, 9 - 15.

ZOU, W Y, 1994, SMPTE J , 103.12, 795-800

Zou, W Y, et al. 1994, SMPTE J, 103.12, 789-794.

CCIR Study Period 1990 - 1994, CMTT Document 36/E , Information on results of 34 MBit/s tests.