Embed Size (px)
Citation preview
Image Processing Theory, Tools and Applications
Enhancing Perceptual Quality of Watermarked High
Definition Video through Composite Mask
Tae-Woo Oh\ Kyung-Su Kim\ Hae-Yeon Lee2, Ji-Won Lee' and Heung-Kyu Lee'
1 Department of EECS, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
e-mail: [email protected]@[email protected]@mmc.kaist.ac.kr
2 School of Computer and Software Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea
e-mail: [email protected]
Abstract-This paper proposes a composite mask based watermarking method, which improves perceptual quality of videos watermarked by traditional video watermarking scheme. The composite mask including noise visibility function (NVF) mask, adaptive dithering mask, and contour mask considers human visual system (HVS). The adaptive dithering mask solves the block artifact problem caused by expanding the watermark basic pattern for the robustness against various downscaling attacks. The contour mask makes up for the disadvantage of NVF mask, which cannot handle separately high textured regions and contour or edge regions. Extensive experiments prove that the watermarking method based on the proposed composite mask satisfies imperceptibility, robustness against downscaling as well as common video processing, and real-time performance.
Keywords-Blocking artifact, Composite mask, Dithering, Human visual system, Image contour, Video watermarking.
1. INTRODUCTION
In accordance with the growth of infrastructures and industries for
digital video contents, illegal copies and distributions of the video contents increase because they can be easily processed, delivered and stored. Since the illegal acts lead to great financial harm to contents providers and the related market, video watermarking as digital right management system becomes important. The principle of invisible video watermarking method is to imperceptibly embed a signal into an original video content. The inserted signal stands for copyright information and makes illegally reproduced copies to be traced back to the receiver from which they originated.
Recently, the spread of high resolution video contents ask the video watermarking system for several necessary conditions:
(a)
invisibility, robustness, real-time processing [1]. First, the watermark embedded into the video contents should be imperceptible to human observer. The higher the quality of the video contents is, the more important the invisibility is. Second, the high resolution video contents generally undergo several manipulations to be adapted for
various display devices such as LCD TV, portable multimedia player,
and high performance cell phone. In practical situation, the manipulations mostly include downscaling, trans-coding to various formats, and frame rate conversion. The embedded watermark should be stably detected in spite of the main manipulations. Finally, video watermarking system should be processed at low computational cost. Especially, real-time video watermarking systems are required for the protection of high quality video on demand (VOD) services.
In this paper, we propose a new composite mask for practical video watermarking scheme. The watermarking scheme based on the
mask is robust to scaling attack as well as general video processing such as trans-coding and frame rate conversion. At the same time, the scheme satisfies the invisibility by exploiting the composite mask based on HVS. The composite mask for controlling the watermark embedding strength consists of NVF mask, adaptive dithering mask,
and contour mask. Since the algorithm of the proposed method is simple, real-time processing in high-definition (HD) video contents is possible.
The paper is organized as follows. In Sec. II, we state the problem of traditional watermarking scheme and basic ideas as its solution. In
Sec. III, we explain about watermark embedding and detecting procedure based on the proposed composite mask. Experimental
results are presented in Sec. IV and Sec. V concludes.
(b) (c)
Fig. 1. The comparison of the invisibility according to the extension of watermark pattern - (a) original image / (b) image that non-extended
pattern is embedded / (c) image that 8 times extended pattern is embedded.
978-1-4244-7249-9/101$26.00 ©2010 IEEE
II. PROBLEM STATEMENT AND BASIC IDEA
A. Block artifacts removing
The traditional watermarking methods [2][3] embed 2-D watermark pattern on the basis of spread spectrum method [4][5] and redundant embedding method [6]. In these methods, each element of watermark pattern to be embedded should be repetitively expanded to be stably detected in the watermarked video scaled to small size.
However, the invisibility of the embedded watermark pattern is worsened because of the block artifacts caused by reinforcing the low frequency of the extended pattern. Figure I(b) and (c) show that the image of Fig. I(a) is watermarked by non-extended and 8 times extended pattern, respectively. The block artifacts are clearly shown in the case of 8 times extension. Although the peak signal-to-noise ratio (PSNR) of Fig. I(b) and Fig. I(c) is equal as 38 dB, it is obvious that
the watermark invisibility of (c) is lower.
This paper proposes adaptive dithering mask to solve the block artifact problem. The dithering mask makes the imperceptibility of the extended pattern better by increasing the internal frequency of the
extended watermark pattern. However, if all watermark patterns of the embedded regions are dithered by equal masking degree without considering the property of the cover work, there is the limit to controlling the watermark embedding strength and consequently the performance for the imperceptibility decreases. Therefore, the proposed dithering mask is adaptively built based on the analysis of the cover work. Since the extended watermark pattern is strongly dithered in flat regions and weakly in high activity regions by the proposed dithering mask, the results adaptive to the cover work are obtained. The detail of the mask is described in Sec. IILA.I.
B. Contour detecting
There are several HVS models built for watermarking system [7][8][9]. The HVS is able to be utilized to improve the robustness while still maintaining the imperceptibility. A large number of traditional watermarking methods exploit the noise visibility function (NVF) masking [7] based on HVS, too. Through the NVF, the watermark can be embedded more strongly in contour or texture regions than in flat regions. However, contour or edge regions are more sensitive to noise addition than texture regions but less than flat region according to the HVS represented in [10]. Thus, the traditional masks such as the NVF mask have the problem because they do not distinguish between the contour region and high texture region. If the traditional mask is used directly, the contour region can be easily corrupted and it results in severe distortions to the video. Therefore, when the watermark is embedded into a video, the watermark in contour regions should be more weakly embedded than in high texture regions, but more strongly than in flat region.
In order to make up for the drawback of the NVF system, a new contour masking scheme can be exploited in the traditional watermarking method. The proposed contour masking is processed at YUV domain, which is directly obtained from the output of general video decoders. In general, the transition of pixel value in luminance channel Y is faster than it in color channel, U or V [II]. Thus, the
(a) Y channel (b) U chaJUlel (c) V channel
Fig. 2. Each channel of the YUV domain.
high activity region (i. e. shawl area of lena image) at U or V channel is relatively dimmer and flatter than that at Y channel as shown in Fig. 2. By using this tendency at YUV domain, the proposed contour mask makes it possible to distinguish between contour regions and high texture regions. The specific process of the mask is explained in Sec. IILA.2.
III. WATERMARKING PROCESS
A. Embedding procedure
The embedding method through the proposed mask adds 2-D watermark pattern W to the luminance channel Y of each frame by spread spectrum scheme. Before directly adding the watermark into spatial domain of each frame, a local scaling factor is considered by using masks based on HVS in order to improve invisibility and
robustness. The embedding process per each frame is represented by
Y' (i, j) = Y(i, j) + A(i, j). Wei, j) = Y(i, j) + a(i, j). P(i, j). y(i, j). Wei, j) (I)
where Y' is the inserted luminance channel of a frame and (i, j) is the
spatial coordinate within the watermarking ROL A is the composite
mask including NVF mask a , adaptive dithering mask P , and
contour mask y. This process is repeated during predefined time per
a watermark pattern. The specific processes about the watermark generation and each masking are as follows.
First, a basic watermark pattern W of the size MjxM2 is generated
with a secret key. This pattern follows standard normal distribution, N(O, I). Each element of the basic pattern w is repetitively extended by k times to vertical and horizontal direction in order to improve the robustness against scaling attack as well as general video processing attacks. As a consequence, the size of the watermark pattern W to be
embedded is Mj·k x M2·k.
The NVF mask a is calculated by
aU,}) = ( (1 - n fi(i,}»).So + n fi(i,})· Sl (2)
where So is the upper bound in edged and texture regions and S j is the lower bound of visibility in flat and smooth regions. The nvf is described by
I nvf ( i, j) = --,------:----
1+ ( D / (J"�" )'(J"'(i,j) (3)
where D is a scaling constant and (J"�., is the maximum of the local
variance.
The adaptive dithering mask P and contour mask y are
specifically explained in following subsections, respectively.
1) Adaptive dithering mask: The block artifacts caused by the
expansion of the watermark pattern are diminished by the adaptive dithering mask. Unlike general dithering masks, the proposed dithering mask is adaptive to the cover contents. Since the block
artifacts in high activity regions are less perceptible to human vision than in flat regions [12], the dithering is more weakly performed in
high activity regions than in flat regions by using the dithering mask based on NVF. The size of the adaptive dithering mask is equal to the
1 ,8(0,1) 1 ,8(0,3 1
,8(1,0 1 ,8(1,2 1 ,8(1,4)
1 P(2,1) 1 ,8(2,3 1
,8(3,0 1 ,8(3,2 1 P(3,4)
1 ,8(4,1) 1 ,8( 4,3 1 .
.
.
.
.
Fig. 3. The component of the adaptive dithering mask.
watermark pattern size M/·k x M2·k. The components of the mask are
like Fig. 3. The mask is given by
l-a.:....(i,:..:. j.:.... )-_ S...2.. , .(B -B.)+B., ifi+j=odd ,8U, j) = So -S,
mu mm mm
I , if i + j = even
(4)
where Bmax and Bmil1 are respectively the max and min boundary and
the range is 0 � Bm
i" < B� � 1. The variable value f3(i, j) is close to
Bmil1 in flat area and Bmax in edge or high texture area. Figure 4 is the result that 8 times extended watermark is embedded in the still image of Fig. I(a) after adaptive dithering masking. The figure shows the block artifacts decrease compared to Fig. l(c).
2) Contour mask: The contour mask separates the contour or edge
regions from the results of the NVF mask that doesn't distinguish between contour and high texture regions. The mask is obtained by a composite operation at Y, U, and V channels. In the case of general video decoders, the size of U or V channel should be scaled up to the size of Y channel because the size of Y channel is bigger than it of U and V channels. After the scaling, in the case of U or V channel,
Fig. 4. The result image that 8 times expanded watermark pattern is
embedded in the still image of Fig. I(a) after adaptive dithering.
Fig. 5. Masking results - left column (only NVF mask) / right
column (NVF mask and contour mask).
following process is performed. First, noisy and high activity region is suppressed by mean filter. Second, the binarization result composed of o and 255 is obtained by Otsu thresholding method [13]. Third, erosion and dilation operation as morphological algorithms are processed to remove partially remained high activity area. Fourth, the
gradient of the morphological result is calculated by using the Sobel edge detector [14] and then the magnitude of the gradients is binarized to 0 or I. The results of each U and V channel are denoted by U E and V E, respectively. Finally, the combined result, C E, is obtained by inclusive OR operation between U E and V E. The C E includes mostly edge and contour region. In order to clarifY the CE, dilation operation is applied and the result is denoted by CED. Next, in case of Y channel,
the gradient of Y channel is obtained by Sobel edge detector and then normalized to [0, 255] range. The normalized result is binarized to 0 or 1 by Otsu thresholding method. The binarization result is denoted by Y E and represents edge and high textured regions. Finally, the contour or edge region result FE is approximated by AND logical operation between Y E and C ED. And the contour mask is given by
y(i,j) = 1- FE(i,j)x m
(5)
where m is weight factor and empirically set to 0.5. Since FE is
composed of 0 and I, 0 < y(i, j) � 1 according to m. In other words,
y(i, j) is 1 in flat or high activity region and the value between 0 and
I in contour or edge region.
Figure 5 shows the performance of the proposed contour mask. The figures are the scaled images after each masking for viewing purposes. The results that the proposed contour mask and NVF mask are applied together become dim in contour or edge region compared to only NVF masking results. Therefore, the strength of the watermark embedding at contour or edge region can be controlled by the proposed contour mask.
B. Detection procedure
The input of the watermark extraction process is the watermarked ROI of each frame luminance channel. Since the watermarking system
.'
., 1
ot>' ./
(a) Documentary (b) Sho\\ (c) Action Mo,·ie
Fig. 6. Test videos.
uses a blind watermark detector, the embedded watermark should be estimated from the given watermarked video. We estimate the watermark by exploiting the adaptive Wiener filter [15] as a de-nosing filter. The watermark estimated from each frame is accumulated during predefined time to increase the accuracy of the estimated watermark. After the accumulation, the normalized correlation is
computed between the accumulated estimation result and the original
watermark generated by the secret key. If the correlation value exceeds a preset threshold, the hidden messages are correctly extracted. We determine the preset threshold depending on approximate Gaussian method [16].
IV. EXPERIMENTAL RESULTS
To measure the fidelity, robustness, and simplicity of
watermarking scheme based on our masking method, several experiments were performed in three MPEG-2 Full-HD resolution
(I 920x 1080) videos, which are 40 seconds in length and 30 fps in
frame rate, from various genres as shown in Fig. 6. The accumulative time for the correlation was 2 second and the watermark pattern, which was embedded into the center of each video frame, was
288x288 size by repetitively expanding a 48x48 basic watermark
pattern by 6 times to vertical and horizontal direction. The parameters Din Eq. (3) and So, Sj, Bm;II> Bmax in Eq. (4) were set to 150, 4, I, 0.15, and 0.85, respectively.
A. Fidelity test
The average PSNR and the average wPSNR are used as objective
measures. The wPSNR is closer to perception than the PSNR. The wPSNR which reflects the HVS of [10] can be calculated by
following equations.
S - aU, j) . rU, j) n( i, j) = -,,-0 __ '----_-.C.-
So (6)
( 255' )
wPSNR = IOloglo --wMSE (8)
We compared the fidelity of our scheme with the traditional masking method [7] using only NVF mask. For the fairness of the test, we evenly coordinated the watermark embedding strength of the both cases. The strength of the watermark is its standard deviation [7]. In addition, we SUbjectively evaluated the quality of the watermarked videos using the ITU-R Rec. 500-11 quality rating scale [1][18]. The test involved ten participants who are familiar with the details of the watermark algorithm and are able to detect visual artifacts. The test videos were displayed on SAMSUNG PAVV 650 LCD TV in the
room of 10 Ix brightness. Also, they viewed the watermarked videos at the five times distance of the LCD screen height according to the preferred viewing distance rule of the ITU-R Rec. 500-11 [18]. Table
I shows the test results. The PSNR and wPSNR are the results in ROI region. Although PSNR results between both methods are similar, wPSNR and subjective test results prove that the proposed masking
scheme produces high fidelity watermarked videos and has no obvious processing artifacts compared to the traditional masking method.
Table I. Results of fidelity test.
�e Docum-Show
Action Method entary Movie
PSNR Proposed mask 47.8 47.1 48.3 (dB) Traditional mask 47.7 46.8 47.9
wPSNR Proposed mask 50.7 50.2 50.9 (dB) Traditional mask 48.2 47.8 48.4
Subjective Proposed mask 4.5 4.7 4.6 score Traditional mask 3.9 4.3 4.0
B. Robustness test
To evaluate the robustness of the watermarking method using the proposed composite mask, experiments were performed on the videos, which are watermarked by embedding strength used in Sec. IV.A. We focused the robustness of our watermarking system against combined attacks commonly happened to high quality videos under practical situations. The watermarked Full-HD MPEG-2 videos, which are between at 25.8 Mbps and at 42.9 Mbps, were processed by various manipulations including arbitrary-ratio down scaling, trans-coding to MPEG-4, and frame rate conversion from 30 fps to 24 fps. Table 2 shows the results as normalized correlation. The multiple attacks mean the frame rate conversion and the trans-coding. Threshold is 0.09 that satisfies false positive error probability of 10.8. All correlation values sufficiently exceed the threshold. Although the results of the traditional case using only NVF mask are not described in this table, the results are almost similar with those of the proposed method. The experimental result implies that watermarking scheme through the presented masking is robust against various downscaling attacks as well as common signal processing manipulations.
Table 2. Correlation results of watermark detection.
�e Docum-Show
Action Attack entary movie
1280x720 + Multiple attacks 0.41 0.47 0.39 640x480 + Multiple attacks 0.32 0.36 0.33 480x360 + Multiple attacks 0.30 0.31 0.28 320x180 + Multiple attacks 0.21 0.22 0.25
C. Real-time performance test
We implemented the proposed scheme by using the Intel integrated performance primitives library and tested on Intel(R) Core2 Duo CPU 2.0 GHz, 2GB RAM. In general, three sub-functions for decoding video streams, embedding or detecting watermarks, and displaying marked video should be processed in 0.03 sec/frame for real-time watermarking system [17]. The process time of the watermarking algorithm is important because the decoding and displaying process time are constant. Table 3 shows the processing time results of Full-HD videos. The time unit is second. These results prove that the proposed masking method has very simple and consequently the watermarking system has real-time performance.
Table 3. Results of computational complexity test.
Decoding Watermarking Display Total Time Process Time Time Time
Embeddin� 0.0112 0.0095 0.0038 0.0245 Detecting 0.0012 0.0162
v. CONCLUSION
In this paper, we propose a new composite mask for a video watermarking scheme that keeps the invisibility in spite of the extension of watermark pattern for the robustness against scaling attacks. This composite mask is built by multiple operation of adaptive dithering mask, contour mask, and NVF mask, which consider HVS. Experimental results show that the proposed method is good at invisibility of the embedded watermark while it still makes the video watermarking system robust to various scaling attacks as well as common video processing manipulations. Also, the real-time performance of the method makes practical utilization possible.
ACKNOWLEDGMENT
This research is supported by Ministry of Culture, Sports and
Tourism(MCST) and Korea Culture Content Agency(KOCCA) in the Culture Technology(CT) Research & Developement Program 2009, and by NRL(Nationai Research Lab)program through the National Research Foundation of Korea funded by the Ministry of Education, Science and Technology (No. ROA-2007-000-20023-0).
REFERENCES
[1] I. 1. Cox, M. L. Miller, J. A. Bloom, 1. Fridrich, and T. Kalker. Digital Watermarking and Steganography. Morgan Kaufinann Publisher, USA, 2008.
[2] C. H. Lee, H. K. Lee, and Y. H. Suh. Image watermarking resistant to geometric attack combined with removal attack. Int. J of Image and Graphics, vol. 5, pp. 37-65, Feb. 2005.
[3] K. S. Kim, M. 1. Lee, and H. K. Lee. A practical real-time video watermarking scheme robust against scaling attack. in Proc. Int. Workshop on Digital Watermarking, Guangzhou, China, 2006.
[4] I. 1. Cox, S. Member, 1. Kilian, F. T. Leighton, and T. Shamoon. Secure spread spectrum watermarking for multimedia. IEEE Trans. on Image Processing, vol. 6, pp. 1673-1687, Dec. 1997.
[5] F. Hartung, 1. K. Su, and B. Girod. Spread spectrum watermarking: malicious attacks and counterattacks. in Proc. SPIE, San Jose, CA, 1999.
[6] T. Y. S. T. I. Echizen, Y. Fujii and H. Yoshiura. Use of statistically adaptive accumulation to improve video watermark detection. Trans. Of Information Processing Society of Japan, vol. 2, pp. 2440-2453, 2006.
[7] S. Voloshynovskiy, A. Herrigel, N. Baumgaertner, and T. Pun. A stochastic approach to content adaptive digital image watermarking. in Proc. Information Hiding, London, UK, 1999.
[8] M. Kutter and S. Winkler. A vision-based masking model for spread spectrum image watermarking. IEEE Trans. on Image Processing, vol. II, pp. 16--25, Jan. 2002.
[9] A. Koz and A. A. Alatan. Oblivious spatio-temporal watermarking of digital video by exploiting the human visual system. IEEE Trans. on Circuits and Systems for Video Technology, vol. 18, pp. 326--337, Mar. 2008.
[10] M. Barni and F. Bartolini. Watermarking System Engineering Enabling Digital Assets Security and Other Applications. CRC Press, Boca Raton, 2004.
[11] A. Anwander, B. Neyran, and A. Baskurt. Multiscale colour gradient for image segmentation. in Proc. Knowledge-based Intelligent Engineering System and Allied Technologies, pp. 369-372, Brighton, UK, 2000.
[12] S. Singh, V. Kumar, and H. K. Verma. Reduction of blocking artifacts in JPEG compressed images. Digital Signal Processing, vol. 17, pp. 225-243, Jan. 2007.
[13] N. Otsu. A threshold selection method from gray level histograms. IEEE Trans. on System, Man and Cybernetics, vol. 9, pp. 62-66, Jan. 1979.
[14] M. Nixon, A. S. Aguado. Feature extraction & image processing. Academic Press, 2007.
[15] I. G. Karybali and K. Berberidis. Efficient spatial image watermarking via new perceptual masking and blind detection
schemes. IEEE Trans. on Information Forensics and Security, vol. I, pp. 256--274, Jun. 2006.
[16] M. L. Miller and 1. A. Bloom. Computing the probability of false watermark detection. in Proc. Information Hiding, London, UK, 1999.
[17] I. K. Kang, D. H. 1m, Y. H. Suh, and H. K. Lee. Real-time watermark embedding for high resolution video watermarking. in Proc. Int. Workshop on Securuty, Kyoto, Japan, 2006.
[18] lTD. Methodology for the subjective assessment of the quality of television pictures. Recommendation ITU-R BT.500-11, Jun. 2002.
