SM1205 Interactivity Topic 09: Motion Tracking Part I Spring 2010SCM-CityU1

SM1205 Interactivity

Topic 09: Motion Tracking Part I

Spring 2010 SCM-CityU 1

Demos First

• Snow sweeper by PlaydoCAM (link) – Make sure webcam is ready


Demos First

• Eyekanoid by PlaydoCAM (link)


Demos First

• PlaydoJam by PlaydoCAM (link)


How Come?

• Reason: you are watched by computers via webcams

• Computers know your movement

– Relying on technology, called motion tracking


Motion Tracking (wiki)

• To locate a moving object or several ones – Output: the location of moving objects

• How can a program know if there are moving objects and where they are?

• Idea: use special image processing algorithms to get regions of moving objects– Many algorithms exist for this purpose


Solution I

• Find the difference between current frame and a given background – Disadvantage: background must be static & fixed • Disallow camera movement • Disallow changes of lighting condition

Current frame Static background

Solution II

• Find difference between consecutive frames– E.g., difference between frames i and i+1– A more flexible solution!


Current frame Previous frame

Note: white colors correspond to boundary of moving objects

Main References

• Motion tracking in ActionScript – Ostrich Flash (link) – Motion tracker from soulwire (link) – Play with demos in W drive


Motion Tracking Library

• In cityu.scm package (developed by Hongbo Fu)

– Webcam class • An easy-to-use wrapper for webcam access

– MotionTracker class• Main class for motion tracking

– MotionTrackerSettingUI class• User interface for parameter settings

– MotionTrackerButton class • Use motion to trigger button clicking


Note: classes are just special data types, which usually have their own properties & methods/functions

Note: classes are just special data types, which usually have their own properties & methods/functions


Webcam Class


Webcam Class • Allow you to access streaming video from webcam• Original webcam source has size 320 x 240• But you can scale up to any size, e.g., 640 x 480


Webcam Class • To use Webcam class, you first need to tell Flash where

the corresponding source code (Webcam.as) is– Under cityu/scm/ folder

• This is done by import statement

• Make sure your Flash application is under the same folder as cityu folder


Note: those folders contain all necessary source code for motion tracking

Webcam Class

• Initializing webcam takes time

• Webcam class provides a special event type, called Webcam.READY – To make sure you’re always using an initialized

webcam

• Only within event listener for this event, you can guarantee webcam is ready to use


Webcam Class

• Basic usage of Webcam class


import cityu.scm.Webcam; // import Webcam class

// create a webcam with size 400 x 300

var webcam:Webcam = new Webcam(400, 300);

// webcam will dispatch READY event when the webcam is ready to use

webcam.addEventListener(Webcam.READY, onCameraReady);

function onCameraReady(e:Event): void {addChild(webcam);

}

import cityu.scm.Webcam; // import Webcam class

// create a webcam with size 400 x 300

var webcam:Webcam = new Webcam(400, 300);

// webcam will dispatch READY event when the webcam is ready to use

webcam.addEventListener(Webcam.READY, onCameraReady);

function onCameraReady(e:Event): void {addChild(webcam);

}

Default Values for Function Parameters

• You can define functions with default values for function parameters – Advantage: you don’t need to pass all values every time

when using functions • E.g., function definition of Webcam

usage of Webcam


function Webcam(w:Number=640, h:Number=480) {...

}

function Webcam(w:Number=640, h:Number=480) {...

}

// create a webcam with size 640 x 480 (default size)var webcam:Webcam = new Webcam(); // equivalent: Webcam(640, 480)// create a webcam with size 640 x 480 (default size)var webcam:Webcam = new Webcam(); // equivalent: Webcam(640, 480)

Webcam Class

• Like other symbol instances, Webcam instances provide a lot of display properties – E.g.,


webcam.alpha = 0.4;webcam.x = 200;webcam.y = 30;

Webcam Class

• Use Webcam’s sendToBack method to avoid occlusion of stage objects by webcam – webcam.sendToBack();



MotionTracker Class


MotionTracker Class• Default visualization for tracked regions • Able to track multiple objects • Each tracked object can have its own clipping region – Motion happened outside clipping region will be ignored


Clipping regions

Simple Usage of MotionTracker

• Step 1: import MotionTracker class– I.e., import cityu.scm.MotionTracker;

• Step 2: initialize a MotionTracker object – E.g.,


var tracker:MotionTracker=new MotionTracker(webcam);var tracker:MotionTracker=new MotionTracker(webcam);


Center of Tracked Region• Use ColorTracker’s properties – trackX: Number; trackY: Number• Center of tracked region • With respect to top-left corner of webcam (instead of stage)

– E.g., tracker.trackX

Spring 2010

SCM-CityU 23

(0, 0)

(trackX, trackY)


trackX = 10 trackY = 20

Class Exercise

• Create a symbol instance on stage • Make this shape follow center of moving region– Use trackX and

trackY


Coordinate System Transformation

• Use DisplayObject’s methods – localToGlobal, globalToLocal

• Stage to Webcam – var ptWebcam:Point =

webcam.globalToLocal(ptStage);

• Webcam to Stage – var ptWebcam:Point =

webcam.localToGlobal(ptStage);


Hide Default Visualization • Use hideTrackedArea and hideTrackedCenter methods– E.g.,


tracker.hideTrackedArea(); tracker.hideTrackedCenter();tracker.hideTrackedArea(); tracker.hideTrackedCenter();

Collision with Tracked Regions

• Use MotionTracker’s methods– hitTestObject• E.g. tracker.hitTestObject(cursor);

– hitTestPoint • E.g., tracker.hitTestPoint(cursor.x, cursor.y, true);

• You can also use isTracked method to perform collision detection only when object is tracked– isTracked return true if object is tracked; false,

otherwise.Spring 2010 SCM-CityU 28


• Usage of MotionTracker’s hitTestObject is very similar to that of DisplayObject’s

• Only difference: order matters – Wrong: cursor.hitTestObject(tracker);– Correct: tracker.hitTestObject(cursor);



• Usage of MotionTracker’s hitTestPoint is exactly the same as that of DisplayObject’s– hitTestPoint checks if a given point is within regions

of pixels with white color


Example: Shape Sweeper


Class Exercise

• Create a large number of circles on the stage and store them into an array for future use– Random size– Random position– Random opacity

• Tips:– addChild– push


Webcam Interaction • For each shape which falls into tracked region, we

remove it from stage – Use removeChild method – Need for loop

• How about corresponding elements in the array?– How to remove them from the array?


Webcam Interaction • Removing array elements one by one in a for loop is

tricky – Consider the following example

– Can you expect the result?• Array length and possibly element indices change over

time/iterations


var a:Array = [2, 3, 5, 7, 9];for (var i:uint = 0; i < a.length; i++) {

a.pop(); // pop the last element of the current array }trace(a);

var a:Array = [2, 3, 5, 7, 9];for (var i:uint = 0; i < a.length; i++) {

a.pop(); // pop the last element of the current array }trace(a);

Webcam Interaction

• Instead of removing array elements one by one, let’s reset the array to empty when all shapes have been removed from the stage– shapes = [];

• To achieve this, – Use numOfShapesOnStage variable to maintain the number

of shapes left on the stage – Use contains method to check if array element shapes[i] is

still on the stage• if (contains(shapes[i])) {…}


Documents

SM1205 Interactivity Topic 09: Motion Tracking Part I Spring 2010SCM-CityU1