An Improved Scalar Quantization-based Digital VideoWatermarking Scheme for 1.264/AVG

Adarsh Golikeri, Panos Nasiopoulos and Z. Jane WangDepartment of Electrical and Computer Engineering

University of British ColumbiaVancouver, BC, Canada V6T 1Z4

Email: {adarshg, panosn, zjanew}gece.ubc.ca

Abstract- Digital video watermarking has attracted a great levels of spatial and temporal distortion, while a built-in bit-deal of research interest in the past few years in applications rate control mechanism is used to ensure optimumsuch as digital fingerprinting and owner identification. watermark bit allocation. We designed our method to workH.264/AVC is the latest and most advanced video coding within the H.264/AVC video standard, the latest and moststandard, but to this date, there are very few watermarking advanced video coding standard [4]. This paper is organizedschemes designed for it. This is mainly due to its complexity as follows: Section II offers a brief overview of SCS. Sectionand compression efficiency which presents a major challenge III describes our watermarking scheme. Section IV showsfor any video watermarking approach. We developed a new, how our method is adapted to work with H.264/AVC.quantization-based video watermarking scheme, which is Experimental results are presented in Section V anddesigned to work with H.264/AVC. Our scheme offers constant conclusions in Section VI.robustness at all compression rates without affecting theoverall bite rate and quality of the video stream. Experimentalresults show that, compared to existing methods, our scheme II. TUE SCALARCOSTA SCHEME (SCS)significantly outperforms existing methods under compression, The watermarking process can be considered as atranscoding, filtering, scaling, rotation and collusion attacks. communications system with side-information at the encoder

side (see Fig. 1) [2],[3]. Henceforth, bold text (x) denotes aI. INTRODUCTION vector, while normal text (x) and italics (x) denote a scalar.

In the past few years, the need for watermarking has Using a secure key K, the watermark message m is

gained significant attention due to the spread of illegal embedded into the host signal x of variance ox2. Theredistribution and unauthorized use of digital multimedia [1]. watermark is defined as w = s-x and has a variance 7w2. TheA great variety of watermarking schemes has been proposed watermarked signal s is then transmitted over a channel,in the literature. The Scalar Costa Scheme (SCS) is a reliable which introduces an additive white Gaussian noise (AWGN)information embedding technique, which is based on Costa's v of variance 0v2, resulting in an attacked work r. Theoriginal, theoretical scheme [2],[3]. SCS outperforms the Watermark-to-Noise Ratio (WNR) is defined asrelated Dither Modulation (DM) techniques for low l0*logl0(ow2/ov2). The decoder receives signal r and, usingWatermark-to-Noise Ratios (WNR) and performs the same key K which was used during embedding, extractssignificantly better than the state-of-the-art blind Spread the watermark message estimate m' (Fig. 1). The ScalarSpectrum (SS) watermarking [3]. SCS, however, has certain Costa Scheme (SCS) uses a structured codebook, constructedinherent limitations when used for video watermarking, such by a concatenation of scalar uniform quantizers [3]. For aas fixed watermarking embedding strength and no Rate- Costa-type embedding of a watermark message m, SCSDistortion optimization. Moreover, SCS does not provide determines an intermediate sequence q which is nearlyany way of controlling the spatial and temporal distortions orthogonal to the cover work x. This message m is encodedcaused by the watermark insertion. Finally, the distribution into watermark letters d that belong to a D-ary alphabet (e.g.,of the watermark bits is not dependent on the regional visual D=2 represents a binary alphabet). For embedding aimportance ofthe input signal. watermark, the following sample-wise operation is

In this paper, we propose a new watermarking method performed:which is specifically designed for video. Our method qn = QA{xn - A(dn/D + kn)} - (xn - A(dn/D + kn)), (1)borrows ideas from SCS and is extremely robust tocompression, transcoding, filtering, scaling, rotation and whereqspetxl,Qdand k are elements of the vectors q, x, d andcollusion. This is achieved by offering a locally adaptive k respectively, QA{.} denotes scalar uniform quantizationwatermark embedding strength and a scheme for optimum with step size A, and knE [0,1) are the elements of a secureRate-Distortion. A unique perceptual mask controls the pseudo-random sequence k, derived

0-7803-9390-2/06/$20.00 ©)2006 IEEE 1434 ISCAS 2006

X v consists of a frequency sensitivity function, luminance andcontrast masking components [1]. The frequency sensitivityis a table defined by the model, with each table entry

M s r m:l

encoder decoder

W' d(u) 11Figure 1. Typical blind watermarking scenario

from the watermark key K. The embedded watermarksequence is given by

w = uxq, (2) Figure 2. Spread Transform (ST) watermarking

where cx (Oc<c1) is the watermark scale factor. SCS doesnot provide an analytical formula for determining the representienthetmagni e of hei cesponding.optimum value of a. Instead, an optimum a (in the capacity DC coefficient in a block that can be perceived by the eye.

sese is deie by the folwn numeric........ We use the standard frequency sensitivity table used in [1].sneidiexpression: Luminance masking accounts for the effect of the DC-oc = I ((w2 / ((0w + 2.71ov )). (3) component (i.e. the average brightness of the block) on the

frequency sensitivity table. Contrast masking takes intoThe final watermarked data is represented by: account the effect of visibility of a change in one frequency

s = x + w = x + cxq. (4) due to the energy present in that particular frequency. Afteraccounting for these effects, the thresholds or slacks for the

For watermark decoding, the received data r is quantized individual DCT coefficients are obtained. These slacksto the nearest codebook entry. The sample-wise extraction represent the amounts by which the individual coefficientsrule is: maybe changes, before resulting in a perceptible change in

the block. These slacks represent the perceptual mask,Yn-QA{rn - Ak11 - (r11 - Akn) (5) denoted by p. For Intra macroblocks the perceptual mask

where yn, rn1 and kn are elements of y, r and k, respectively. sequence t is given by:For binary SCS, Jy,.J should be close to zero if d1,= 0 and close t = p / IPI (6)to A/2 for d1 = 1. This is known as minimum distancedecoding. In case of Inter macroblocks, the effect of both spatial

and temporal masking must be considered. Previous researchIII. PROPOSED WATERMARKING SCHEME has shown that watermark artifacts, such as "mosquito"

effects and flicker, are visible in the fast moving regions of aLately, Spread Transform coding [5] has been used in frame [6]. These artifacts correspond to regions with a large

combination with traditional SCS (known as ST-SCS) to motion vector values. For this reason, the strength of theimprove the bit-error rate of watermarking (Fig. 2). We have watermark should be reduced in such regions. This isdeveloped a new watermarking method which borrows ideas achieved by weighting the perceptual mask by the inverse offrom Spread Transform coding and SCS and is specifically the motion vector magnitude. Thus, for Inter macroblocks,designed for video. In traditional Spread Transform Scalar the perceptual mask is given by:Costa Scheme (ST-SCS) watermarking, the host signal x isprojected onto a pseudo-random vector. The disadvantage of t p / my, (7)this approach is that it does not account for perceptual where p is the Watson's perceptual mask and lmvl is themasking effects of the Human Visual System (HVS). In our absolute magnitude of the motion vector. The elements of tscheme, we derive a unique perceptual mask sequence t in (7) are then normalized.(derived from the host signal x itself) in order to achieveimperceptibility. Once the perceptual mask t is generated, the projection of

x onto t is found. This operation yields a scalar quantity:The generation of t differs depending on which type of T

macroblock is used. In case of Intra macroblocks, spatial x t. (8)masking effects are considered. First, a Gaussian low-pass In our scheme, x represents the transform domainfilter is applied to the macroblock in order to mitigate the coefficients of Intra and Inter coded macroblocks. Theeffect of noise. Then, for each given macroblock, a watermark keyKis used to generate therandom scalar valueperceptual mask is computed using the Watson's model. k E [0,1). For binary ST-SCS, the equation for embedding aWatson's model estimates the perceptibility of changes in the '0' bit is obtained by putting d-0 and D=2 in equation (1):coefficients of the block-based DCT of an image. This model

1435

s'= QA{X' - Ak} - (x'- Ak). (9) perceptual mask t, which results in the scalar e'. The

Similarly, embedding a '1' bit is possible by setting d=1 q e dand D=2 in (1): Dd= QA{e'-Ak}-(e'-Ak). (13)

s'= QAX' - A(0.5+k)} - (x'- A(0.5+k)). (10) For each macroblock, we compute the value of x whichminimizes the Lagrangian cost function (11). This value ofEquto (9) and (10) represent subtractive dithered uisteRe-soronpimmwemakcleftr

quantization. The components of x that are orthogonal to t wicis the Rate-Distortion optimum watermark scale factorare equal to x-x't. These components are not altered during which we use m our methodthe embedding process and are for this reason they are added Watermarked video may require many more bits thanback to the watermark data. Therefore, the final watermarked unwatermarked video, especially at low video bit-rates.data s is obtained by combining (4) with the orthogonal Therefore, it is desirable to have a bit-rate control scheme incomponents: order to find the optimum trade-off between the fidelity of

s = (x'+ocs')t+ (x-x't). (11) the watermark and that of the host signal. This schemeshould determine the best allocation of available watermarkTraditional ST-SCS uses a fixed cx that is computed from bits between different watermarked macroblocks . In video

global statistics (see equation (3)). In contrast, our method coding, the most important factor for controlling the bit-rateuses a locally adaptive value for cx which is computed in is the residual signal coding fidelity, which is controlled byreal-time from a combination of local and global statistics. choosing a suitable quantization step-size for the transformAs a result, we obtain stronger control over the watermark coefficients. Our scheme is designed in such a way that itscale factor, which makes our watermark to adapt better to achieves watermark bit-rate control simply by changing thethe host signal characteristics. As a result, our method is quantization step A which is used to embed the watermark inmuch more robust than traditional ST-SCS. (9), (10) and (13). Therefore, our scheme has a built-in

mechanism for watermark bit-rate control, through theDuring video encoding, several coding parameters such parameters A and 4,. This is an important advantage over

as macroblock prediction modes, motion vectors and existing schemes, such as [6], which require an explicit bit-transform coefficient quantization levels have to be rate controller. In Section 4, we explain how bit-rate controldetermined. Since natural video has widely varying spatial is achieved when we implement our scheme on H.264/AVC.and temporal (motion) content, the selection of differentcoding options for different parts of the image becomes Another important advantage of our method is that thenecessary. Therefore, the task of the video coder is to find a watermark can be decoded from the partially decompressedset of coding parameters so that a trade-off between the video bit stream, since the watermark is embedded in thevideo bit-rate and distortion (R-D) is achieved. This means transform coefficients. Decoding of the watermark requiresthat for a given video bit-rate, the encoder has to find the knowledge of the secure key K, which is needed to generatecombination of coding options that minimizes the distortion. the pseudorandom scalar k. The perceptual mask t isLagrangian bit-allocation techniques for R-D coding have computed for this macroblock as explained at the beginningbeen widely accepted in recent video codec development, of this section. The transform coefficients are projected ontodue to their effectiveness and simplicity. Adding a t to obtain the scalar projection y'. This projection is thenwatermark to a video stream may also affect the bit rate and quantized using (13) and simple hard decision decoding isquality of the image. It is, therefore, highly desirable that the used to extract the message m'.watermark embedding procedure incorporates R-Doptimized coding in order to compute the optimum IV. WATERMARKING OF H.264/AVC VIDEOwatermark for different regions of a video frame. One of the challenges for designing watermarking

To this end, we use the Lagrangian multiplier technique schemes for H.264 is that even the Intra-frames consistto compute the locally optimum value of cx at the macroblock mainly of residual data. This means that adding a watermarklevel. The simplified Lagrangian cost function for a without affecting the picture quality or the bit rate isparticular value of cx is: extremely difficult. H.264 achieves bit rate reduction for the

Intra-frames by using spatial prediction for IntraJa= Du+ kwEun (12) macroblocks, a major departure from the previous coding

where Da is the distortion (sum of squared differences or standards. Intra macroblocks can be predicted usingSSD) between the host signal x and the watermarked work s Intra 16x16 (entire macroblock predicted from top and left

4 is the Lagrangian parameter and is dependent on the neighbors) or Intra 4x4 (each 4x4 luma block separatelychoice of the video standard used for encoding [7], and E, is predicted from its neighbors) modes. Inter macroblocks use

Dd''A'A ---II- De W define Dd s A variable block-size motion compensation. The different sizesthe decoding error = ld Dl.W eieD sthe decoded include 16X16, 16X8, 8X16 and 8X8. The 8X8 partition can bedistance and De as the expected distance. De iS equal to 0 if fute diie 'no84 x r4x lcs hsamtothe embedded message bitd0, anditis equal to±A/2 ifd=1 vector is transmitted for each partition, adding moreTo obtain Dd, the watermarked data s is projected onto the complexity to the coding scheme.

1436

Another challenge that H.264 poses to watermarking is shown in [7] that there is a strong relationship between thethat it uses an integer transform. This is a major problem for Lagrangian parameter X used for R-D coding and QPH.264:traditional Spread Spectrum watermarking schemes whichembed watermarks drawn from a Gaussian distribution. = 0.85 * pow(2, (QPH.264 -12)/3) (19)

Our algorithm addresses all the above issues and is Therefore, bit-rate control in H.264 is conducted bydesigned to work efficiently within H.264. First, in order to controlling QPH.264 and accordingly adjusting the value of k.take into account the variety of motion vectors while Similarly, in our method, the watermark bit allocation isderiving the perceptual mask sequence t for a subdivided controlled by choosing the step size ofthe quantizer QPw (ormacroblock in (7), we divide the Watson's perceptual mask equivalently, A) and adjusting the value of 2w used in (11).p by the motion vectors of the corresponding regions. The Lagrangian parameter 2w is given by:

We consider watermarking the luma components of both kw = 0.85 * pow(2 (QPw -12)/3). (20)the Intra and Inter macroblocks, operating on the integer 0transform coefficients of the macroblock residual data. This We then use this 2w in (12) to determine the locallyis possible since we designed our method without making optimum watermark scale factor oc. By selecting this valueany assumptions about the nature of the host signal x. Formacroblocks predicted using the Intra_16x16 mode, the oicm we ensure that our watermark will not affect the R-DHadamard coefficients are watermarked, because they optimized coding decisions of the H.264 encoder. When thecontain most of the energy of the macroblock. For those Hv264 encoder varies QPH.264 to achieve the desired overallpredicted using the Intra_4x4 mode and all the Inter videobit-rate, QPw changesproportionallysinceitisrelatedmacroblocks, we watermark the integer transform to QPH.264 through (15) and (16). Thus, the watermark bitscoefficients. are allocated in proportion to the H.264 encoder's bit-rate

control algorithm. This ensures that the overall video bit-Let QPH.264 denote the Quantization Parameter (QP) va.lue rtisnot adversel fetd

used in H.264, Qstep denote the H.264 quantization step size, rate 1 ely affected.QPw denote our watermark quantization parameter and V. EXPERIMENTAL RESULTSA denote our watermark quantization step-size. The QPH.264 The performance of our scheme was tested on 10 videoand Qstp are related as follows:andQ,,pare related as follows:

sequences (Carphone, Coastguard, Football, Foreman,Qstep =0.6282 * exp(QPH.264*0.1155), 0 < QPH.264 < 51. (14) Flower Garden, Mother Daughter, News, Paris, Tempete and

TbsQPH264 in Tennis) that represent all aspects of content. These are:The behavior of (14) iS such that for values of QH24m Under the same picture quality (PSNR and bit rate), wethe range 0-30, the corresponding Qstep changes by very compared the robustness of our method against thesmall amounts. However, for QPH.264between 31-51 the Qstep traditional ST-SCS scheme for several different attacks.increases very rapidly. Our main objective is to control the We used the H.264/AVC reference software version JM\c9.3distribution of the watermark bits in such a way that for our implementation [8].minimum Bit-Error Rate (BER) is achieved, independent ofthe video bit-rate. We achieve this by establishing a Fig. 3 shows the Bit Error Rates (BER) caused by H.264relationship between our watermark step size QPwand compression at various bit-rates, for 2 representative streams.QPH.264 Evaluations over a large set of video sequences We observe that our method significantly outperforms ST-indicated that the minimum BER is obtained when: SCS. Moreover, the robustness of our method is constant at

all the bit-rates, and results in dramatic improvements overQP = 48, 0 < QPH.264 < 30, and (15) ST-SCS. This indicates that the watermark bit-rate control in

QP = 1.329* QPH264+ 6.768, 31 < QPH.264 < 51. (16) (15) and (16) and the Rate-Distortion optimization in (20)are performing very well by adapting the watermark step size

The relationship between QPw and A is exactly the same A and the watermark scale factor oc to the compression rate.as that between QPH.264 and Qstep. Therefore, equivalently, On the other hand, the performance of traditional ST-SCS

A = 60~ <_PH.64 <30 ,and 17)suffers in spite of using the same watermark power w

A 1l 60, 0 < QPH.264 <30 , and (1l7) during embedding.A= 0.6882 * exp(QPH.264*0. 1686), 31 < QPH.264 < 51. (18) Fig. 4 shows the Watermark-to-Noise Ratios (WNRs)Thus, there is a close relationship between (14) and (18). against BERs (at 512kb/s) for two extreme cases. On

Both equations represent exponential curves, with an initial average, our scheme requires about 2 decibels less WNR toslow ascent between 0-30 and then a rapid increase in the achieve minimum BER. We also observe that, for the samerange 31-51. This guarantees that our watermark robustness WNR, the BER achieved by our scheme is about 2 orders ofis constant at all different compression rates. magnitude lower than ST-SCS.

Our watermark is embedded on the transform coefficients For the same picture quality (PSNR), Table I shows thebefore the Quantization process. In H.264, bit-rate control is results for transcoding (sequences recompressed at the sameachieved through proper selection ofthe QPH.264. It has been bit rate but with a different GOP structure), Gaussian

1437

filtering (3x3, variance 0.5), 7500 scaling, 5° rotation with [7] T. Wiegand, H. Schwarz, A. Joch, F. Kossentini, and G. J. Sullivan,bilinear sampling, and collusion attacks (5 different "Rate-Constrained Coder Control and Comparison of Video Coding

watermarked copies averaged at 512 kb,s). We observe Standards," IEEE Transactions on Circuits and Systems, vol. 13, no.watermarked copies were averaged at 512 kb/s). We observe 7, July 2003.that, on average, our scheme is 3 times better than ST-SCS [8] Karsten Suihring, H.264/AVC Software Coordination,when transcoding is performed. Under Gaussian low-pass http://bs.hhde/-.suehring/tml/.filtering, downscaling and rotation attacks, our scheme BER v OPyields BERs less than half those of ST-SCS. Finally, the 0045average BER obtained by our scheme after collusion attacks 0.04 E/ -e-- Proposed scheme. Tenniis0.04 C~~~~~~~-0-Existing scheme, Tennisis less than 1/4th that of ST-SCS. All in all, our scheme -A--- Proposed scheme, Footballsignificantly outperforms ST-SCS in all the above attack I- Exstingscheme,Footballcategories. w 0.03

VI. CONCLUSION , 0.025coiW 0.02We have presented a new video watermarking scheme, _

which is designed to work for H.264/AVC. Our scheme 0.01Econsists of a locally adaptive, R-D optimized watermark . awhich is inserted in the transform coefficients ofmacroblock 0 Xresiduals. A unique perceptual mask limits distortion, while a 0 o0o xbuilt-in bit-rate control mechanism ensures optimum E010 20 owatermark bit allocation. In the category of compression- ° Qua20ztio 40 P0 G0decompression, our scheme yields bit-error rateimprovements of more than two orders of magnitude Figure 3. Bit error rates after H.264 compression attack at different bit-compared to traditional ST-SCS. Our scheme achieves the ratessame BER as ST-SCS, using 2 decibels less Watermark-to- BER vs Watermark-to-Noise RatioNoise (WNR). In addition, our scheme significantlyoutperforms ST-SCS after geometric and collusion attacks.

10ae

REFERENCES

[1] I.J. Cox, M.L. Miller and J.A. Bloom, Digital Watermarking, co10lAcademic Press, 2002. A

[2] M. H. M. Costa, "Writing on dirty paper," IEEE Trans. Info. Thy, vol.29, no. 3, pp. 439--441, May 1983. ,104

[3] J. J. Eggers, R. Buml, R. Tzschoppe, and B. Girod, "Scalar costa ischeme for information embedding," IEEE Trans. Signal Process., 0 - o evol. 15, pp. 1003--1019, Apr. 2003. fA Proposed scheme. News

- L- Existing scheme, News[4] Draft ITU-T Recommendation and Final Draft International Standard -E-- Proposed scheme. Paris

of Joint VideoSpecification(ITU-T Rec. H.264 ISO/IEC 14496-10 - 0- Existing scheme, Paris l,IAVC), Joint Video Team (JVT), Mar. 2003, Doc. JVT-GO50. 10-10 -9.5 -9 -8.5 -6 -7.5 -7 -6.5 -6

[5] B. Chen, Design and analysis of digital watermarking, information Watermark-to-Noise Ratio (dB) ->embedding and data hiding systems, PhD thesis, MIT, Cambridge, Figure 4. Bit error rates at different Watermark-to-Noise-RatiosMA, June 2000.

[6] A. M. Alattar, E. T. Lin, and M. U. Celik, "Digital watermarking oflow bit-rate advanced simple profile MPEG-4 compressed video,"IEEE Trans. Circuits Syst. Video Technol., vol. 13, pp. 787--800,Aug. 2003.

TABLE I. BIT ERROR RATES AFTER VARIOUS WATERMARK ATTACKS

Transcoding Gaussian filtering 75% downscaling 50 rotation CollusionBER x 10 BER x 10 BER x 10 BER x 10 BER x 10

Proposed Existing Proposed Existing Proposed Existing Proposed Existing Proposed ExistingFootball 6.5724 48.5791 4.1212 40.6060 2.0471 38.8821 4.4714 40.2290 5.1313 67.475

News 15.6296 36.8014 26.6126 46.3030 16.7137 37.9125 27.4411 47.5220 31.010 47.131

Paris 19999 39.5348 24437 48.1076 177573 40.8130 267581 48.8612 83967 69.554

LTennis 1984 52.7621 3744 62.5000 3724 62.9026 1765 49.3765 2129 8.5

1438