1 Error Concealment for Shape in MPEG-4 Object-Based Coding Student: Jieh-Yuean Hsu Advisor: Prof....

1

Error Concealment for Shape in MPEG-4 Object-Based

Coding

Student: Jieh-Yuean Hsu

Advisor: Prof. David W. Lin

2

Reference

[1]Meng-Yuan Liu, “Real-Time Implementation of MPEG-4 Video Encoder Using SIMD-Enhanced Intel Process” , M.S. thesis, Department of Electrical Engineering, Nation Chiao Tung University, Hsinchu, Taiwan, July 2004

[2]Cheng Huang and Paul Salama,” Error Concealment for Shape in MPEG-4 Object-Based Video Coding”, IEEE Transactions on Image Processing, Vol. 14, No. 4, April 2005.

3

Reference

[3]Guido M. Schuster, Aggelos K. Katsaggelos,” Motion Compensated Shape Error Concealment”, IEEE Transactions on Image Processing, Vol. 15, NO. 2, February 2006

4

Outline Introduction of MPEG-4 Object-

Based Coding Error Concealment

Boundary Extraction Boundary patching Boundary Filling

Experimental Result Conclusion

5

Outline Introduction of MPEG-4 Object-

Based Coding Error Concealment

Boundary Extraction Boundary patching Boundary Filling

Experimental Result Conclusion

6

VO and VOP

VO: Video Objects VOP: Video Object Planes A sequence of VOPs is referred to as

a VO I-VOP: Intra-coded P-VOP: Predictive-coded B-VOP: Bidirectionally predictive-coded

7

VO and VOP

8

VOP information

Spatial (Intra) Shape (Object-Based Only) Texture

Temporal (Inter) Motion

9

Structure of VO Encoder

10

Shape Information

Alpha plane Binary alpha plane Gray scale alpha plane

11

Binary Mask

16

16BAB

Transparent

Opaque

Boundary

12

Outline Introduction of MPEG-4 Object-

Based Coding Error Concealment

Boundary Extraction Boundary patching Boundary Filling

Experimental Result Conclusion

13

Basic Assumption The shape of the objects does not

drastically change between the current VOP and the reference VOP

It is assumed that the reference VOP is undamaged

14

Boundary Extraction

Use the 8-neighborhood If any pixel in the 8-neighborhood of a

current pixel doesn’t belong the object boundary

The boundary will be 4-connected

15

Boundary Extraction

16

4-connected Boundary

17

Boundary Patching Find the number of endpoints

A boundary pixel having only one 4-connected neighbor

The number of endpoints must be even

Partition the endpoints in the current VOP

18

Boundary Patching

Endpoints

Aa

c

CB

b

19

Boundary Patching Patching by Global Motion

Compensation Some parameter:

Scale:

Centroid:

,

,

2( 2)

( 1) 1

i ji j

i ji j

barea VOP

scalearea VOP b

( , )x y

,( )

( )

i ji jb j

xarea VOP

,( )

( )

i ji jb i

yarea VOP

20

Boundary Patching orientation angle:

Rotation angle:

2, ,( )i j i ji j

a x x b , , ,( )( )i j i j i ji j

b x x y y b 2

, ,( )i j i ji jc y y b

1tan ( )( )ba c

cur ref

21

Boundary Patching

Current VOP Reference VOP

[( ) cos( ) ( ) sin( )]cur cur cur curref ref

x x y yx x

scale

[( ) cos( ) ( ) sin( )]cur cur cur curref ref

y y x xy y

scale

22

Boundary Patching

Reference VOP Current VOP

[( ) cos( ) ( ) sin( )]cur ref ref ref ref curx x x y y scale x

[( ) cos( ) ( ) sin( )]cur ref ref ref ref cury y y x x scale y

23

Boundary Patching

Mapping

Aa

c

CB

b

A

a

c

CB

b

Current VOP Reference VOP

24

Boundary Patching Traverse the boundary of the ref.

VOP, and record the order of endpoints

ex:

Map each pixel in curve whose endpoints belong to different pair

ex , ,

A a c C B b

a c C B b A

25

Boundary Patching

Reference VOP Current VOP

a

A

c

C

B

b

Recovered pixel

26

Boundary Filling Filling in the closed boundary of the

VOP with opaque pixels After filling, we can get the recovered

VOP

Recovered VOP

27

Outline Introduction of MPEG-4 Object-

Based Coding Error Concealment

Boundary Extraction Boundary patching Boundary Filling

Experimental Result Conclusion

28

Experimental Result brea_qcif, 30 fps

29

Experimental Result

30

Experimental Result Akiyo_cif, 10fps

31

Experimental Result

32

Experimental Result brea_qcif, 10 fps

33

Outline Introduction of MPEG-4 Object-

Based Coding Error Concealment

Boundary Extraction Boundary patching Boundary Filling

Experimental Result Conclusion

34

Conclusion The method for shape concealment based on

global motion compensation It works well for video sequence with slow motion

and high frame rate It can achieve good result, even the shape is

severely damaged Extra information added resulted in 5% increase

for stream size, and it is acceptable