StochasticSIFT: hi · StochasticSIFT：確率的枠組によるSIFTの般化の一般化目的ノイズや急激な明度変化に対する頑健性を向上させる特長特徴点抽出

h iIS1-80

StochasticSIFT:Interest point detection based on Interest point detection based on stochastically-derived stability

Ukrit Watchareeruetai(1,2), 木村昭悟(1),Cheng Bao(1,3), 川西隆仁(1), 柏野邦夫(1)g , ,

(1) 日本電信電話（株）コミュニケーション科学基礎研究所(2) I t ti l C ll(2) International College,

King Mongkut’s Institute of Technology Ladkrabang(3) Dept. Eng. Physics, University of British Columbia

Abstract 映像を対象とした新しい特徴点抽出法の提案

確率的枠組によるSIFTの般化 StochasticSIFT：確率的枠組によるSIFTの一般化

目的イズや急激な明度変化に対する頑健性を向上させるノイズや急激な明度変化に対する頑健性を向上させる

特長特徴点抽出ためタ出力を確率変数と捉る

（DoGフィルタ）（Gaussian）特徴点抽出のためのフィルタ出力を確率変数と捉える

特徴点の「安定性」を考慮、信頼性の低い候補を排除

安定性不安定安定顕著性非顕著顕著

StochasticSIFT: Interest point detector based on stochastically-derived stability 2

Interest point detectors Recently, local interest features have been

successfully used in many computer vision successfully used in many computer vision systems Image indexingg g Stereo matching Object recognition

Advantages Robust against partial

l iocclusion Do not require segmentation

processprocess [Schmid 2000]


SIFT [Lowe 2004]

One of the most appeal interest point (IP) detector and is widely used.

High distinctive and efficient to compute Robust against scale and rotation changes Robust against scale and rotation changes Partially tolerant to illumination and affine

geometry changesgeometry changes Might not be robust against

sudden illumination changes or noisessudden illumination changes or noises


Scope of work

Goal: object recognition in video sequence

Idea: Idea: stochastic model, temporal smoothness

and other information (e.g. movement) and other information (e.g. movement) may be used to improve SIFT detector

Scope: Scope: focus on IP localization


Overview

Interest point

),( yxDoG Di i i ti

Interest point mapU(t-1) U(t) U(t+1)

),( yxDoG Discrimination map (optional)F(t-1) F(t) F(t+1) ),,( tS x ),,( tS x

( DoG value )

Stochastic feature map (SFM)S(t-1) S(t) S(t+1)

Deterministic feature map(DoG)

(t-1) (t)(t+1)

S SSEstimate DoG value based on

Kalman filtering

I(t-1)Input video

I(t) I(t+1)


Stochastic feature map (SFM)

Kalman filtering State noise State propagation step co-variance

Ob ti

State estimation stepSFM

Observation(DoG)

VARt

ttststtt s

s

2212

1))|,,(ˆ),,(()1,,()1(

),,(

xxx

x

Observe noiseco-variance (Adaptive computation of variances)


Considering optical flowOptical flow is needed because objectsusually are not at the same locationusually are not at the same location Modified state propagation step

Modified state estimation step (Motion-considered position) Modified state estimation step (Motion considered position)

(Gaussian pyramids are used for multi-scale calculation)


Kalman filter reset

For some areas,optical flow is

Kalman filter estimate SFM based on data from different areaoptical flow is

not reliable (e.g., very large flow)

from different area

Reset Kalman filter Reset criterion 1. or

Reset Kalman filter

2. is at the outside of image

(e.g., 30 for 320 x 240)


IP selection strategy Based on the IP stability, defined as

Sum of Fisher discriminantsagainst neighboring pixels and scalesagainst neighboring pixels and scales

Intuitive description of the IP stability

[Horizontal] DoG values[Vertical] Prob. taking the value


Experimental set-up Matching test image with video frame

Datasets

Test image

Video sequence Datasets

5 video sequences (101 frames, 320×240) Rotation, affine, scale, illumination changes, , , g Create ground truths (homography) from markers

d Y i B d P f bC l d

StochasticSIFT: Interest point detector based on stochastically-derived stability 11NTT card Yurica Bus card Perfume boxCalendar

Evaluation measures

Repeatability ratePosition matching (regardless of descriptor)

Matching rate [Schmid 2000]

M t hi b th iti d d i tMatching both position and descriptor


Evaluation: Repeatability rate• Our method provides better scores for all the datasets than SIFT.• More noises added, worse scores, but larger differences of scores

Blue: SIFTPi k P d

YurikaPink: Proposed

NTT cardNTT card

Bus card CalendarPerfume box


Matching rate• Our method provides better scores than SIFT except “Bus card”

YurikaBlue: SIFTPink: Proposed

NTT cardca d

Bus card Calendar Perfume box


Error case Why only the “Bus card” dataset provides

worse results? Due to a plastic case?worse results? Due to a plastic case?


Conclusion Proposed a novel framework applying

to h ti model fo IP dete tiona stochastic model for IP detection StochasticSIFT:

a generalized variant of SIFT IP detectora generalized variant of SIFT IP detector Consider IP stability derived from a

stochastic framework to remove IP stochastic framework to remove IP candidates polluted by noises/distortions.

Results suggest that Results suggest that The proposed method has advantages

for noise robustness.for noise robustness.


Thank you for your coming!!

Corresponding author

Akisato Kimura, Ph.D @ NTT CS Labs.E-mail: [email protected] Twitter ID: @_akisato


Documents

StochasticSIFT: hi · StochasticSIFT：確率的枠組によるSIFTの般化の一 般化 目的 ノイズや急激な明度変化に対する頑健性を向上させる 特長 特徴点抽出

StochasticSIFT: hi · StochasticSIFT：確率的枠組によるSIFTの般化の一般化目的ノイズや急激な明度変化に対する頑健性を向上させる特長特徴点抽出