Progress ResportBy shooting

2009/10/1

outlineimTop overviewimTop detection

FingerMobile

Finger detection evaluationMobile detection improvement

imTop Architecture

SceneTouch/MobileDetection

View

Application (oForm)

Node

OtherSensors

Projector

Projector

PTU

Speaker

Socket

Window Message

COM Message Parser OpenGL c#

DeviceControl

Virtual Director (oDirector)

Event queue

LED Light

CamNode

Background Subtraction

OpeningDown-scale

Integral Image

ConnectedComponent

Palm Tracking

ConnectedComponent

PCA Analysis

Palm-fingerAssociation

Finger Tracking

ConnectedComponent

PCA Analysis

PalmAssociation

FingerFusion

-

--

finger palm

Camera Synchronize manager

Capture camera(Source filter)

Capture camera(Source filter)

imTopFilter(Transform

Filter)

Merge finger results

Match current result to previous

Process un-matched finger points

Smooth finger results

imTopFilter(Transform

Filter)

Show detection results(Render filter)Send current results

Peripheral Projector

Fovea Projector

IR Camera

IR Illuminator

Mirror

Pan-tilt Unit

56”

96 ppi24 ppi

Undistortion

PreprocessBuild integral image

Smooth image

Background subtraciton

Finger detection

Finger result

Detect lighter spot

Connected component

Finger analyze

Kernel scan

Sufficient energy

Appropriate center

imTopFilter(Transform

Filter)

Flow chartLoad Image

Undistotion

Pre-process

Finger detection

Show resultSend Result to imTop

Calculate Background image by

10 initial frames(initial 10 frames)

Background subtraction

(exceed 10 frames)

Finger detection

Detect lighter spot

Connected component

finger analyze

Set finger result

Image preprocessing

Mobile detection

Detect contour

mobile analyze

Set finger result

Image preprocessing

Result management

Match current result to the previous

Process for un-matched finger points/mobile contours

Smooth the detection results

Send results

Copy current results from each camera

outlineimTop overviewimTop detection

FingerMobile

Finger detection evaluationMobile detection improvement

Image preprocessingSmooth current image

Smooth Image by Gaussian filter (kernel 3x3)Build integral image

Calculate the M+2 by N+2 integral image of current M by N background subtracted image

Detect lighter spotSearching for region where the average

intensity of inner part is higher enough then that of outer part

Corners & Finger-size regions will be detected

Connected componentUsing previous finger candidates Using queueCalculate the mean position of each

connected candidates Mark each connected component with

groupID which is the position of the initial candidate

Finger analyzeCalculate region center and energy of each

connected component Accept as a Finger input if

Sufficient EnergyThe Center is within its region

outlineimTop overviewimTop detection

FingerMobile

Finger detection evaluationMobile detection improvement

Mobile detectionImage preprocessing

Smooth current image Smooth Image by Gaussian filter (kernel 3x3)

Detect contourMobile Analyze

Filter contour Not rectangle Inappropriate area

Detection result

outlineimTop overviewimTop detection

FingerMobile

Finger detection evaluationMobile detection improvement

Single-tapping testFixed Target

Appear sequentially Calculate accuracy

Variation between Target center & finger detection point

+

++

+

Finger-moving test Demo a video of moving a targetTell user try to move this target as demo

videoRecord finger detection point Calculate variation between target center &

finger detection point

+

outlineimTop overviewimTop detection

FingerMobile

Finger detection evaluationMobile detection improvement

Microsoft’s tag

Thanks for listening