SM1205 Interactivity Topic 10: Motion Tracking Part II Spring 2010SCM-CityU1

SM1205 Interactivity

Topic 10: Motion Tracking Part II

Spring 2010 SCM-CityU 1

Motion Tracking Library

• In cityu.scm package (developed by Hongbo Fu)

– Webcam class • An easy-to-use wrapper for webcam access


import cityu.scm.Webcam; // import Webcam class

// create a webcam with size 400 x 300var webcam:Webcam = new Webcam(400, 300);

// webcam will dispatch READY event when the webcam is ready to usewebcam.addEventListener(Webcam.READY, onCameraReady);

function onCameraReady(e:Event): void {addChild(webcam);

}

import cityu.scm.Webcam; // import Webcam class

// create a webcam with size 400 x 300var webcam:Webcam = new Webcam(400, 300);

// webcam will dispatch READY event when the webcam is ready to usewebcam.addEventListener(Webcam.READY, onCameraReady);

function onCameraReady(e:Event): void {addChild(webcam);

}

Motion Tracking Library

• In cityu.scm package (developed by Hongbo Fu)

– MotionTracker class


var tracker:MotionTracker=new MotionTracker(webcam);var tracker:MotionTracker=new MotionTracker(webcam);

MotionTracker Class• Properties– trackX: Number; trackY: Number• Center of tracked region • With respect to top-left corner of webcam (instead of stage)

• Methods– hitTestObject

– hitTestPoint – hideTrackedArea– hideTrackedCenter


Spring 2010

4

(0, 0)

(trackX, trackY)

Demos


Clipping Regions• By default, motion tracking is done over the entire

video domain – Check MotionTracker-Skeleton.fla


Clipping Regions • What if motion only within specific regions (called

clipping regions) are needed?– Motion outside those clipping regions will be ignored

• Demo


clipping regions

Clipping Regions • Specify a clipping region when creating a motion tracker

– cl, ct, cr, cb are with respect to top-left cornerof webcam

– trackX, trackY are also withrespect to top-left cornerof webcam

– You can have only ONE clipping region for one tracker


tracker = new MotionTracker(webcam,

cl, ct, cr, cb);tracker = new MotionTracker(webcam,

cl, ct, cr, cb);

cl

cr

ct cb

Clipping Region

Multiple Clipping Regions • To have multiple clipping regions, you can create

multiple trackers – All trackers share the same Webcam instance – Each tracker has its own associated clipping region


// initialize 1st motion tracker tracker = new MotionTracker(webcam, 0, 0, 200, 200);

// initialize 2nd motion tracker tracker2 = new MotionTracker(webcam, 300, 300, 500, 500);

// initialize 1st motion tracker tracker = new MotionTracker(webcam, 0, 0, 200, 200);

// initialize 2nd motion tracker tracker2 = new MotionTracker(webcam, 300, 300, 500, 500);

Class Exercise • Extend the previous example to have two clipping regions

as follows– Hint: create two trackers with their own clipping regions– How trackers work at the overlapped areas of clipping regions?


Example: Ping-Pong Game


box

barL

ball

barR

box2

Example: Ping-Pong Game• Use tracker and tracker2 to control barL and barR,

respectively


Class Exercise

• Make ball bounce against stage borders – Change ball’s x and y properties continuously in onTimer

event listener – Check if ball hits

stage borders


Hit Test between Ball and Bars

• Let’s just use hitTestObject method


if (ball.hitTestObject(barR)) { trace("barR got hit");

}

if (ball.hitTestObject(barL)) { trace("barL got hit");

}

if (ball.hitTestObject(barR)) { trace("barR got hit");

}

if (ball.hitTestObject(barL)) { trace("barL got hit");

}

Hit Test between Ball and Bars

• Collision response


Case 1: Ball is on left sideof barR

Case 2: Ball is on right sideof barR

Hit Test between Ball and Bars• But how to determine which case to handle?• Solution: use signs of ball’s speedX– speedX > 0 Case 1

• Response: put ball exactly at the left sideof barR

– speedX < 0 Case 2• Response: put ball

exactly at the right sideof barR


Class Exercise

• Add sound effects when ball hits stage borders/bars – You can use Flash built-in sounds

• Hints– Export sounds for

ActionScript


var ws:WoodSound = new WoodSound(); ws.play();var ws:WoodSound = new WoodSound(); ws.play();

Moving Speed • Use MotionTracker’s speedX and speedY to get the

moving speed of tracked object

• speedX (in pixels)– Positive value if object moves along positive x-axis – Negative value if objects moves along negative x-axis

• speedY (in pixels)– Positive value if object moves along positive y-axis – Negative value if objects moves along negative y-axis


trace(tracker.speedX, tracker.speedY);trace(tracker.speedX, tracker.speedY);

Example • Transfer moving speed of tracked object to ball– E.g., hand

• Demo


Example • Let’s modify the previous ping-pong example– Keep one tracker only, without any clipping region– No vertical bars– Add friction


Example • Change ball’s speeds only if there is some intersection

between ball and moving object


if (tracker.isTracked() && tracker.hitTestPoint(ball.x, ball.y, true)) {

speedX += tracker.speedX; speedY += tracker.speedY;

}

if (tracker.isTracked() && tracker.hitTestPoint(ball.x, ball.y, true)) {

speedX += tracker.speedX; speedY += tracker.speedY;

}

Example

• Interact with more balls (demo)


Assignment 3

• Topic: Webcam Interaction with Motion

• Requirements– Use motion tracker – Interact with motion – Have music/sound elements – Meaningful system – Documentation (in Word format)• Title, motivation, how-to-use, screenshots, references


Assignment 3

• Submission – Via ACS • Max total file size: 50M• .fla, .swf, .doc (documentation) files and other files

– Deadline: 23:59:59, 12 May (Wed), 2010• No late submission is allowed

• In-class presentation, 12 May, 2010– Each student has roughly 5 minutes for presentation


Assignment 3

• Assessment – Based on • Implementation difficulty + UI design + creativity (80%)• Presentation performance (10%)• Documentation (10%)

– Note: if you use any technique which has not been taught in this course, you must state its source or reference in the documentation clearly • Otherwise, some marks will be deducted


Summary of MotionTracker

• Make sure your Flash application is under the same folder as the tracking library



• Without clipping region

• With clipping region

– cl, ct, cr, cb are with respect to top-left cornerof webcam


tracker = new MotionTracker(webcam);tracker = new MotionTracker(webcam);

tracker = new MotionTracker(webcam,

cl, ct, cr, cb);tracker = new MotionTracker(webcam,

cl, ct, cr, cb);


• Properties – trackX, trackY: center of tracked region • With respect to top-left corner of webcam

– speedX, speedY• Moving speed of

tracked object

Spring 2010 SCM-CityU

2828

Spring 2010

28

(0, 0)

(trackX, trackY)


• Methods– hitTestObject• To test if bounding boxes of display object and tracked

region intersect with each other

– hitTestPoint• To test if a given point is within the tracked region

– isTracked• To check if any moving object is tracked

– hideTrackedArea– hideTrackedCenter


Documents

SM1205 Interactivity Topic 10: Motion Tracking Part II Spring 2010SCM-CityU1