Perceptual Watermarks for Perceptual Watermarks for Digital Image and VideoDigital Image and Video

ECE 738 paper ECE 738 paper

presentationpresentation

Pei QiPei QiECE at UW-MadisonECE at UW-Madison

pqi@cae.wisc.edupqi@cae.wisc.edu

What is ‘perceptual’ What is ‘perceptual’ watermarkwatermark

Prior knowledgePrior knowledge

Perceptual watermarkPerceptual watermark

Prior knowledgePrior knowledge

Additive watermarkAdditive watermark

Ideal watermarkIdeal watermark Three principlesThree principles

- Transparency or imperceptibility- Transparency or imperceptibility

- Robustness- Robustness

- Capacity- Capacity

Challenging problemChallenging problem

- Conflicts- Conflicts

- Tradeoff between transparency and - Tradeoff between transparency and

robustnessrobustness

Prior knowledgePrior knowledge

Human visual systemHuman visual system Three propertiesThree properties of the human visual system of the human visual system

1.1. Frequency sensitivityFrequency sensitivity

What’s freq. sensitivityWhat’s freq. sensitivity

Freq. sensitivity describes the human eye’s sensitivity to sine wave Freq. sensitivity describes the human eye’s sensitivity to sine wave gratings at various freq. Given that the minimum viewing distance is fixed, it’s gratings at various freq. Given that the minimum viewing distance is fixed, it’s possible to determine a static just noticeable difference threshold for each freq. possible to determine a static just noticeable difference threshold for each freq. band.band.

JND thresholdJND threshold

The JND threshold is such that changes in the frequency content in the The JND threshold is such that changes in the frequency content in the image in the particular frequency band below the threshold are not noticeableimage in the particular frequency band below the threshold are not noticeable

Prior knowledgePrior knowledge

Human visual systemHuman visual system Three propertiesThree properties of the human visual system of the human visual system

2.2. Luminance sensitivityLuminance sensitivity

What’s luminance sensitivityWhat’s luminance sensitivity

Luminance sensitivity measures the effects of the detectability threshold Luminance sensitivity measures the effects of the detectability threshold of noise on a constant background, which is a nonlinear function and depends of noise on a constant background, which is a nonlinear function and depends on local image characteristics.on local image characteristics.

3. Contrast masking3. Contrast masking

Contrast masking allows more dynamic control of the JND threshold Contrast masking allows more dynamic control of the JND threshold levels. Contrast masking refers to the detectability of one signal in the levels. Contrast masking refers to the detectability of one signal in the presence of another signal.presence of another signal.

Prior knowledgePrior knowledge

SummarySummary What is our goal to introduce human visual system in watermarking application?What is our goal to introduce human visual system in watermarking application?

1. 1. Determine if a watermark inserted into a image is invisible or not Determine if a watermark inserted into a image is invisible or not

2. We are always trying to insert the maximum strength and maximum length watermarks into an 2. We are always trying to insert the maximum strength and maximum length watermarks into an image, SINCE more watermarks are insertedimage, SINCE more watermarks are inserted

- more robust to attacks- more robust to attacks

- more likely to be detected- more likely to be detected

Make use of properties of human visual system to adjust the watermark so that it’s perfect for both Make use of properties of human visual system to adjust the watermark so that it’s perfect for both robustness and transparencyrobustness and transparency

JNDsJNDs

JNDs generated from different properties provide the quantized thresholds for embedding JNDs generated from different properties provide the quantized thresholds for embedding watermarks. watermarks.

- upper bounds on watermark strength levels- upper bounds on watermark strength levels

- upper bounds on watermark length (capacity)- upper bounds on watermark length (capacity)

Note: JND thresholds are NOT a fixed value, which depend on different images and approachesNote: JND thresholds are NOT a fixed value, which depend on different images and approaches

Perceptual watermark Perceptual watermark techniquestechniques

Image-Independent watermarkImage-Independent watermark

Image-dependent or Image-adaptive Image-dependent or Image-adaptive

watermarkwatermark

Image-Independent Image-Independent watermarkwatermark

A typical method (Cox approach)A typical method (Cox approach) Key pointsKey points

• Place watermark in perceptually significant components (low Place watermark in perceptually significant components (low frequency) (for robustness)frequency) (for robustness)

– Modify by a small amount below Just-noticeable-difference (JND)Modify by a small amount below Just-noticeable-difference (JND)

• Use long random vector as watermark to avoid artifactsUse long random vector as watermark to avoid artifacts

• Any difference if using other watermark instead (w-b images, logo)Any difference if using other watermark instead (w-b images, logo)(for imperceptibility & robustness)(for imperceptibility & robustness)

Embedding Embedding v’v’i i = v= vii + + v vi i wwii = v = vii (1+ (1+ w wii) )

• Perform DCT on entire image and embed watermark in DCT Perform DCT on entire image and embed watermark in DCT coefficientscoefficients

• Choose N=1000 largest AC coeff. and scale {vi} by a random factorChoose N=1000 largest AC coeff. and scale {vi} by a random factor DetectionDetection

Block diagram of Cox’s Block diagram of Cox’s schemescheme

2D DCT

sort v’=v (1+ w) IDCT & normalize

Original image

N largest coeff.

other coeff.

marked image

random vector generator

wmk

seed

ImplementationImplementation

Avoiding to change the corresponding location Avoiding to change the corresponding location of each coefficient in the image, when you sort the of each coefficient in the image, when you sort the vector projected from matrix of DCT coefficientsvector projected from matrix of DCT coefficients

Challenging problemChallenging problem

How to improve Cox approachHow to improve Cox approach• Global scaling factor is not suitable for all coefficientsGlobal scaling factor is not suitable for all coefficients

- Maybe beyond the threshold in some areas of image, especially - Maybe beyond the threshold in some areas of image, especially obvious in the smooth background areaobvious in the smooth background area

• More explicitly compute Just-noticeable-difference More explicitly compute Just-noticeable-difference

– JND ~ max amount each frequency coefficient can be JND ~ max amount each frequency coefficient can be modified imperceptiblymodified imperceptibly

– Use Use ii for each coefficients for each coefficients finely tune watermark strength finely tune watermark strength

• OverheadOverhead

- Cost of computation of thresholds for each coefficient- Cost of computation of thresholds for each coefficient

Image-dependent orImage-dependent orImage-adaptive Image-adaptive watermarkwatermark

Block-based DCT approachBlock-based DCT approach

Wavelet DWT approachWavelet DWT approach

Image-Adaptive Image-Adaptive watermarkwatermark

General Image-Adaptive watermark General Image-Adaptive watermark

schemescheme X*u,v : The watermarked image

Xu,v : The original image

Wu,v : The sequence of watermark values

Ju,v : The computed JND for each coefficient

Question:Question: Why Xu,v > Ju,v (from local image, considering properties of HVS)

otherwiseX

JifXwJXX

vu

vuvuvuvuvuvu

,

,,,,,*,

,

Block-based DCT approachBlock-based DCT approach

Nonoverlapping 8x8 blocksNonoverlapping 8x8 blocks DCT applied to each block independentlyDCT applied to each block independently XXu,v,bu,v,b: The DCT coefficients: The DCT coefficients

XX**u,v,bu,v,b: The watermarked DCT coefficients: The watermarked DCT coefficients

WWu,v,bu,v,b: The sequence of watermark values: The sequence of watermark values

ttCCu,v,bu,v,b: The computed JND calculated from the visual model: The computed JND calculated from the visual model

Key pointsKey points Block-by-block DCTBlock-by-block DCT

How to derive tHow to derive tCCu,v,bu,v,b

bvu

Cbvubvubvu

Cbvubvu

bvu X

tifXwtXX

,,

,,,,,,,,,,*,,

,

Block-based DCT approachBlock-based DCT approach

ttFFu,v u,v : a frequency threshold value, which is : a frequency threshold value, which is

an 8x8 matrix values for each DCT basis an 8x8 matrix values for each DCT basis functionfunction

ttLLu,v,bu,v,b : Luminance sensitivity estimated by : Luminance sensitivity estimated by

the formula.the formula.

XX0,0,b0,0,b: DC coeff. for block b: DC coeff. for block b

XX0,00,0: DC coeff. Corresponding to the : DC coeff. Corresponding to the

mean luminance of the displaymean luminance of the display

a: parameter controlling the degree of a: parameter controlling the degree of luminance sensitivity (empirical luminance sensitivity (empirical value=0.649)value=0.649)

ttCCu,v,b u,v,b :Contrast masking threshold, where w :Contrast masking threshold, where w

between 0 and 1, a empirical value for w between 0 and 1, a empirical value for w is 0.7is 0.7

abFvu

Lbvu

X

Xtt )(

0,0

,0,0,,,

])(,[ ,, 1,,,,,,,,

vuvu wLbvu

w

bvuL

bvuC

bvu tXtMaxt

Block diagram of IA-DCT Block diagram of IA-DCT approachapproach

CalculateJNDs

CalculateJNDs

WatermarkInsertion

WatermarkInsertionDCTDCT

Original image X(i,j) X(u,v)

Watermark sequence W(u,v)

Watermarked image X*(u,v)

J(u,v)

Wavelet DWT approachWavelet DWT approach

Key pointKey point Hierarchy DecompositionHierarchy Decomposition

The upper left corner: Lowest The upper left corner: Lowest

frequency band.frequency band.

l: resolution level 1, 2, 3, 4l: resolution level 1, 2, 3, 4

F: frequency orientation 1, 2, 3F: frequency orientation 1, 2, 3

Much simpler than DCT app.Much simpler than DCT app.

- Cost of computing JNDs- Cost of computing JNDs

Wavelet DWT approachWavelet DWT approach

XXu,v,l,fu,v,l,f : wavelet coefficient at position(u,v) in resolution level l and frequency : wavelet coefficient at position(u,v) in resolution level l and frequency

orientation forientation f

XX**u,v,l,fu,v,l,f : watermarked wavelet coefficient : watermarked wavelet coefficient

WWu,v,l,fu,v,l,f : watermark sequence : watermark sequence

ttFFl,fl,f : computed frequency weight at level l and frequency orientation f, which : computed frequency weight at level l and frequency orientation f, which

could be further refined by adding image-dependent components like DCT could be further refined by adding image-dependent components like DCT

approachapproach

flvu

Fflflvuflvu

Fflflvu

flvu X

tifXwtXX

,,,

,,,,,,,,,,,*,,,

,

DetectionDetection

Detection scheme for Block-based DCTDetection scheme for Block-based DCT 1. Based on classical detection theory as SS detection (Cox)1. Based on classical detection theory as SS detection (Cox)

- Original image is subtract from watermarked image and correlation between - Original image is subtract from watermarked image and correlation between

the signal difference and the watermark sequence is determinedthe signal difference and the watermark sequence is determined

- The correlation value is compared to a threshold to determine whether the - The correlation value is compared to a threshold to determine whether the

received image contains the watermark. received image contains the watermark.

Testing IA-DCT without original Testing IA-DCT without original imageimage

Key pointsKey points1.1. Assume original image has been Assume original image has been

JPEG compressedJPEG compressed

2.2. Feature vector {XFeature vector {Xff}, X}, XDD is greater is greater

than ½ of its corresponding than ½ of its corresponding

quantization table value Qquantization table value Q

3.3. W is only inserted in {XW is only inserted in {Xff}}

4.4. A correlation measure c is found A correlation measure c is found

between {Zbetween {Zff} and W} and W

5.5. A threshold test is performed on c to A threshold test is performed on c to

determine if the W under test is determine if the W under test is

present in Zpresent in Z

DetectionDetection

Detection for WaveletDetection for Wavelet

1. First, the correlation is performed 1. First, the correlation is performed

separately at each level separately at each level

2. Second, We calculate the average 2. Second, We calculate the average

for each resolution level l and freq. for each resolution level l and freq.

orientation forientation f

3. At last, we choose the maximum 3. At last, we choose the maximum

correlation value over all the possible correlation value over all the possible

levels as well as freq. locationslevels as well as freq. locations

ComparisonComparison

Image-Independent vs Image-AdaptiveImage-Independent vs Image-Adaptive

Image QualityImage Quality

All acceptable, but SS watermark is most visible in the smooth All acceptable, but SS watermark is most visible in the smooth

background area.background area.

Robustness to Compression and CroppingRobustness to Compression and Cropping

Winner: IA-WWinner: IA-W

Robustness to ScalingRobustness to Scaling

Winner: IA-W againWinner: IA-W again

Robustness to shiftRobustness to shift

Only IA-W survivesOnly IA-W survives

Video watermarksVideo watermarks

Unique requirements for watermarksUnique requirements for watermarks

Extension of the IA-DCT Technique to Extension of the IA-DCT Technique to

VideoVideo

Watermarking of MPEG-2Watermarking of MPEG-2

Scene-Adaptive Video WatermarkingScene-Adaptive Video Watermarking

Watermarking StandardsWatermarking Standards

Key points in paperKey points in paper

What’s the perceptual watermarkWhat’s the perceptual watermark

How does HVS work for watermark How does HVS work for watermark

applicationsapplications

Three typical watermarking techniquesThree typical watermarking techniques

PapersPapers

Perceptual Watermarks for Digital Image and Perceptual Watermarks for Digital Image and

videovideo RAYMOND B. WOLFGANGRAYMOND B. WOLFGANG

CHRISTINE I. PODILCHUK AND EDWARD J. DELPCHRISTINE I. PODILCHUK AND EDWARD J. DELP

Image-Adaptive Watermarking Using Visual Image-Adaptive Watermarking Using Visual

ModelsModels CHRISTINE I. PODILCHUK AND WENJUN ZENGCHRISTINE I. PODILCHUK AND WENJUN ZENG

Thank you for your Thank you for your attentionattention

PPT file and papers can be downloaded from PPT file and papers can be downloaded from

websitewebsite

http://www.cae.wisc.edu/~pqi/ece738/presentation/http://www.cae.wisc.edu/~pqi/ece738/presentation/

Contact info:Contact info:

Name: Qi, Pei Email: pqi@cae.wisc.eduName: Qi, Pei Email: pqi@cae.wisc.edu