Method and Architecture Design for Motion Compensated Frame Interpolation in High-Definition Video Processing

Yen-Lin Lee and Truong NguyenECE Dept., UCSD, La Jolla, CA 92093-0407Method and Architecture Design for Motion Compensated Frame Interpolation in High-Definition Video ProcessingMotion compensated frame interpolation (MCFI)Enhance the quality of reconstructed video

Work on how to accurately estimate true motionsConsiders spatial and temporal correlation, searches with block matching algorithm (BMA)proposed method and architecture adopt one-pass and low complexity design

The proposed method(1/2)Adopts a motion-compensated approach to insert one or several interpolated frames between any two contiguous original frames

The proposed method(2/2)Three major parts: True motion engineMotion estimation, true motion decision, output true MV to interpolator Block-based interpolatorobtains motion information performs a motion-compensated procedure with weighted valuesDeblocking filterSmooth the blocking artifact where a neighboring interpolated block with larger sum of absolute difference (SAD) shows up

Bilateral SearchAdopt motion re-estimation with bilateral search with original uncompressed frames from a decoder

directly seeks two similar blocks from a zero distance of the interpolated block

1.Naturally, a method without motion re-estimation should have lower complexity, but it is arduous to look for true motions when motion vector field is inaccurate and irregular.2. mv f and mv b represent two different directional motion vectors, a forward motion and a backward motion.

5Multi-directional Enlarged Matching AlgorithmLarger matching block size could improve the accuracy of searching true motion ( N*N M*M )larger part of a particular object is considered.Nine types of enlarged matching directions are definedkeep the completeness of the front edge for the moving object.

the results suggest that using smaller matching size is worse and easily obtain wrong motion with better pixel difference instead of a true motion, especially for blocks with less texture.6One-pass Process with Multi-grids ClassificationSimplify the process and reduce the storage size for motion vector fieldDivides a searching window into multiple areas Each area has one motion candidate.

Temporal Object InformationOne-pass processing would not provide complete info.acquire some temporal informationFor objects with constant speed, especially in panning scenes.When the camera pans slowly across a scene, most interpolated would move consistently except real moving objects.Remove those inaccurate motion vectorsInterpolated blocks with a larger pixel variance are considered, which means that there are more textures.

THE PROPOSED ARCHITECTUREwhite color : functional elementsgray color : storage elements

True Motion Engine(1/3)it sets nine-grids classification, searching window from -7 to +7 for a down-sampled frame, 88 block size, 1616 enlarged block size, and four candidates for each MSEA sub-block.

True Motion Engine(2/3)Multi-level successive eliminate algorithm (MSEA)A fast full-search block matching algorithm (FFSBMA)Following a successive elimination algorithm (SEA)

Avoid local trap problem

True Motion Engine(3/3)The fourth step : Probable sum of absolute difference (PSAD)MSEA engine decides four highly possible candidates for each directional areaPSAD engine only needs to check SAD values for these four vectors instead of a full search with SAD.The fourth step : true motion selection from these nine motion candidates by pre-defines conditions, and neighboring and temporal information.The fifth step : The proposed architecture would perform a refining procedure by probable refining engine (PREF)Implementationdouble the frame rate for :HDTV1080p (19201080) 30fps video at 180MHz HDTV 720p (1280720) 30fps video at 83MHzthe architecture is designed in VHDL and implemented with TSMC 90-nm technology

PERFORMANCE AND EXPERIMENTAL RESULT(1/5)

PERFORMANCE AND EXPERIMENTAL RESULT(2/5)

PERFORMANCE AND EXPERIMENTAL RESULT(3/5)

PERFORMANCE AND EXPERIMENTAL RESULT(4/5)

PERFORMANCE AND EXPERIMENTAL RESULT(5/5)More result:http://videoprocessing.ucsd.edu/~yenlinlee/mcfi/

CONCLUSIONThe method employs a unique true motion engine with an adaptive overlapped block matching algorithmmulti-directional enlarged matching algorithmone-pass process.The proposed architecture employs a modified multi-level successive eliminate algorithm has capabilities to reduce the heavy computation.Experimental results show that the proposed algorithm provides better video quality than conventional methods