Upload
ann-jenkins
View
214
Download
0
Embed Size (px)
Citation preview
Clipping Regions• By default, motion tracking is done over the entire
video domain – Check out MotionTracker-Skeleton.fla
Spring 2012 SCM-CityU 2
Clipping Regions • What if motion only within specific regions (called
clipping regions) are needed?– Motion outside those clipping regions will be ignored
• Demo
Spring 2012 SCM-CityU 4
clipping regions
Clipping Regions • Specify a clipping region when creating a motion tracker
– cl, ct, cr, cb are with respect to top-left cornerof webcam
– trackX, trackY are also withrespect to top-left cornerof webcam
– You can have only ONE clipping region for one tracker
Spring 2012 SCM-CityU 5
tracker = new MotionTracker(webcam,
cl, ct, cr, cb);tracker = new MotionTracker(webcam,
cl, ct, cr, cb);
cl
cr
ct cb
Clipping Region
Multiple Clipping Regions • To have multiple clipping regions, you can create
multiple trackers – All trackers share the same Webcam instance – Each tracker has its own associated clipping region
Spring 2012 SCM-CityU 6
// initialize 1st motion tracker tracker = new MotionTracker(webcam, 0, 0, 200, 200);
// initialize 2nd motion tracker tracker2 = new MotionTracker(webcam, 300, 300, 500, 500);
// initialize 1st motion tracker tracker = new MotionTracker(webcam, 0, 0, 200, 200);
// initialize 2nd motion tracker tracker2 = new MotionTracker(webcam, 300, 300, 500, 500);
Class Exercise • Extend the previous example to have two clipping regions
as follows– Hint: create two trackers with their own clipping regions– How trackers work at the overlapped areas of clipping regions?
Spring 2012 SCM-CityU 7
Example: Ping-Pong Game• MotionTracker-PingPong-Skeleton.fla in W drive
Spring 2012 SCM-CityU 8
box
barL
ball
barR
box2
Class Exercise • Use tracker and tracker2 to control barL and barR,
respectively
Spring 2012 SCM-CityU 9
Class Exercise
• Make ball bounce against stage borders – Please refer to Ping-Pong example in Topic 05
Spring 2012 SCM-CityU 10
Hit Test between Ball and Bars
• Let’s just use hitTestObject method
Spring 2012 SCM-CityU 11
if (ball.hitTestObject(barR)) { trace("barR got hit");
}
if (ball.hitTestObject(barL)) { trace("barL got hit");
}
if (ball.hitTestObject(barR)) { trace("barR got hit");
}
if (ball.hitTestObject(barL)) { trace("barL got hit");
}
Hit Test between Ball and Bars
• Collision response
Spring 2012 SCM-CityU 12
Case 1: Ball is on left sideof barR
Case 2: Ball is on right sideof barR
Hit Test between Ball and Bars• But how to determine which case to handle?• Solution: use signs of ball’s speedX– speedX > 0 Case 1
• Response: put ball exactly at the left sideof barR
– speedX < 0 Case 2• Response: put ball
exactly at the right sideof barR
Spring 2012 SCM-CityU 13
ball.x = barR.x - barR.width * 0.5 - rball.x = barR.x - barR.width * 0.5 - r
Class Exercise
• Add sound effects when ball hits stage borders/bars – You can use Flash built-in sounds
• Hints– Export sounds for
ActionScript
Spring 2012 SCM-CityU 14
var ws:WoodSound = new WoodSound(); ws.play();var ws:WoodSound = new WoodSound(); ws.play();
Moving Speed • Use MotionTracker’s speedX and speedY to get the
moving speed of tracked object
• speedX (in pixels)– Positive value if object moves along positive x-axis – Negative value if objects moves along negative x-axis
• speedY (in pixels)– Positive value if object moves along positive y-axis – Negative value if objects moves along negative y-axis
Spring 2012 SCM-CityU 15
trace(tracker.speedX, tracker.speedY);trace(tracker.speedX, tracker.speedY);
Example • Transfer moving speed of tracked object to ball– E.g., hand
• Demo
Spring 2012 SCM-CityU 16
Example • Let’s modify the previous ping-pong example– Keep one tracker only, without any clipping region– No vertical bars– Add friction
Spring 2012 SCM-CityU 17
Example • Change ball’s speeds only if there is some intersection
between ball and moving object
Spring 2012 SCM-CityU 18
if (tracker.isTracked() && tracker.hitTestPoint(ball.x, ball.y, true)) {
ball_speed_x += tracker.speedX;ball_speed_y += tracker.speedY;
}
if (tracker.isTracked() && tracker.hitTestPoint(ball.x, ball.y, true)) {
ball_speed_x += tracker.speedX;ball_speed_y += tracker.speedY;
}
Summary of MotionTracker
• Without clipping region
• With clipping region
– cl, ct, cr, cb are with respect to top-left cornerof webcam
Spring 2012 SCM-CityU 20
tracker = new MotionTracker(webcam);tracker = new MotionTracker(webcam);
tracker = new MotionTracker(webcam,
cl, ct, cr, cb);tracker = new MotionTracker(webcam,
cl, ct, cr, cb);
Summary of MotionTracker
• Properties – trackX, trackY: center of tracked region • With respect to top-left corner of webcam
– speedX, speedY• Moving speed of
tracked object
Spring 2012 SCM-CityU
2121
Spring 2010
21
(0, 0)
(trackX, trackY)
Summary of MotionTracker
• Methods– hitTestObject• To test if bounding boxes of display object and tracked
region intersect with each other
– hitTestPoint• To test if a given point is within the tracked region
– isTracked• To check if any moving object is tracked
– hideTrackedArea– hideTrackedCenter
Spring 2012 SCM-CityU 22
Button Click • Can we click digital buttons by our fingers directly?
• Answer: Yes!– Through motion
– Of course, you canalso rely on touchsensors• Need extra hardware
Spring 2012 SCM-CityU 25
Motion Tracking Library
• In cityu.scm package (developed by Hongbo Fu)
– Webcam class • An easy-to-use wrapper for webcam access
– MotionTracker class• Main class for motion tracking
–MotionTrackerButton class • Use motion to trigger button clicking
Spring 2012 SCM-CityU 27
Key Idea• Analyze motion over a button on the stage– If motion is detected over it, dispatch OVER event
• Cf. MouseEvent.MOUSE_OVER
– If motion continuously exists for a while, dispatch CLICKED event• Cf. MouseEvent.MOUSE_DOWN• To avoid undesired clicks after dispatching CLICKED event, clicking is
disabled for a short period (say 2 sec)
– If clicking is re-enabled, dispatch CLICKABLE event• Cf. MouseEvent.MOUSE_UP
Spring 2012 SCM-CityU 28
• To create a motion-driven button, you need – Step 1: create some display object (e.g., btn) on stage
• By either design tool or ActionScript programming
– Step 2: create a MotionTrackerButton object and associate it with btn
– Step 3: add event listeners for motionBtn • MotionTrackerButton.OVER MotionTrackerButton.CLICKED MotionTrackerButton.CLICKABLE
MotionTrackerButton Class
Spring 2012 SCM-CityU 29
var motionBtn:MotionTrackerButton = new MotionTrackerButton(webcam,
btn);
var motionBtn:MotionTrackerButton = new MotionTrackerButton(webcam,
btn);
Step 1: Create Display Object• MotionTracker-Skeleton.fla in W drive – Draw some shape on stage– Convert it to symbol – Assign an instance name (e.g., btn)
Spring 2012 SCM-CityU 30
Step 2: Create MotionTrackerButton Object
• Before adding a MotionTrackerButton object, we need to initialize a Webcam object– We don’t need to explicitly create a MotionTracker object,
since it is automatically generated for the associated button
Spring 2012 SCM-CityU 31
import cityu.scm.Webcam; // import Webcam class
var webcam:Webcam = new Webcam(640, 480); webcam.addEventListener(Webcam.READY, onCameraReady);
function onCameraReady(e:Event): void { addChild(webcam); // to avoid occlusion of stage objects by webcam webcam.sendToBack(); webcam.alpha = 0.4;
}
import cityu.scm.Webcam; // import Webcam class
var webcam:Webcam = new Webcam(640, 480); webcam.addEventListener(Webcam.READY, onCameraReady);
function onCameraReady(e:Event): void { addChild(webcam); // to avoid occlusion of stage objects by webcam webcam.sendToBack(); webcam.alpha = 0.4;
}
Step 2: Create MotionTrackerButton Object
• First import MotionTrackerButton class
• Then create a button with motion control
– 3rd parameter: amount of motion needed to activate clicking• How easy a button can be clicked
– 4th parameter: time (in milliseconds) needed to re-enable clicing after dispatching CLICKED event • How often a button can be clicked
Spring 2012 SCM-CityU 32
import cityu.scm.MotionTrackerButton; import cityu.scm.MotionTrackerButton;
var motionBtn:MotionTrackerButton =
new MotionTrackerButton(webcam, btn, 10, 500); var motionBtn:MotionTrackerButton =
new MotionTrackerButton(webcam, btn, 10, 500);
Step 2: Create MotionTrackerButton Object
• Motion detection is clipped to the button area only – To see tracking visualization, make the button a bit
transparent
Spring 2012 SCM-CityU 33
Step 3: Add Event Listeners• Event types – MotionTrackerButton.OVER– MotionTrackerButton.CLICKED– MotionTrackerButton.CLICKABLE
Spring 2012 SCM-CityU 34
motionBtn.addEventListener(MotionTrackerButton.OVER, onButtonOver);
motionBtn.addEventListener(MotionTrackerButton.CLICKED, onButtonClick);
motionBtn.addEventListener(MotionTrackerButton.CLICKABLE, onButtonClickableAgain);
motionBtn.addEventListener(MotionTrackerButton.OVER, onButtonOver);
motionBtn.addEventListener(MotionTrackerButton.CLICKED, onButtonClick);
motionBtn.addEventListener(MotionTrackerButton.CLICKABLE, onButtonClickableAgain);
function onButtonClick(e:Event): void {
trace("button clicked"); }
function onButtonClick(e:Event): void {
trace("button clicked"); }
Class Exercise• Task 1: when button is clicked by motion – If btn’s current frame label is “up”, go to frame “down”– If btn’s current frame label is “down”, go to frame “up”– Tips
• Add code in onButtonClick • Use currentLabel, gotoAndStop
Spring 2012 SCM-CityU 35
Class Exercise• Task 2: play with different values for 3rd and 4th
parameters of MotionTrackerButton. – 3rd parameter: how easy a button can be clicked– 4th parameter: how often a button can be clicked
• Compare – MotionTrackerButton(webcam, btn, 1, 500);– MotionTrackerButton(webcam, btn, 20, 500);
• Compare – MotionTrackerButton(webcam, btn, 5, 50);– MotionTrackerButton(webcam, btn, 5, 2000);
Spring 2012 SCM-CityU 36
Pausing, Resuming and Removing of MotionTrackerButtons
• Like MotionTracker’s pause, resume and dispose methods, MotionTrackerButton class has same set of methods for similar functionalities
Spring 2012 SCM-CityU 37
Pausing, Resuming and Removing of MotionTrackerButtons
Spring 2012 SCM-CityU 38
btnResume.addEventListener(MouseEvent.CLICK, buttonClick); btnPause.addEventListener(MouseEvent.CLICK, buttonClick); btnNextFrame.addEventListener(MouseEvent.CLICK, buttonClick);
function buttonClick(e:MouseEvent): void { if (e.target == btnResume) {
motionBtn.resume(); } else if (e.target == btnPause) {
motionBtn.pause(); } else if (e.target == btnNextFrame) {
motionBtn.dispose();
removeChild(webcam); webcam.dispose();
gotoAndStop(2); }
}
btnResume.addEventListener(MouseEvent.CLICK, buttonClick); btnPause.addEventListener(MouseEvent.CLICK, buttonClick); btnNextFrame.addEventListener(MouseEvent.CLICK, buttonClick);
function buttonClick(e:MouseEvent): void { if (e.target == btnResume) {
motionBtn.resume(); } else if (e.target == btnPause) {
motionBtn.pause(); } else if (e.target == btnNextFrame) {
motionBtn.dispose();
removeChild(webcam); webcam.dispose();
gotoAndStop(2); }
}
Example: Piano with Motion Control
• How to interact multiple MotionTrackerButton objects?
Spring 2012 SCM-CityU 39
Music Notes• Like previous digital piano example, we use an array to
store music notes
Spring 2012 SCM-CityU 40
var notes:Array = [new S1(), new S2(), new S3(), new S4(),
new S5(), new S6(), new S7()];
var notes:Array = [new S1(), new S2(), new S3(), new S4(),
new S5(), new S6(), new S7()];
Create Motion Buttons • We use arrays to store both piano keys and their
corresponding motion buttons – Corresponding key and motion button have same index
Spring 2012 SCM-CityU 41
// a list of motion buttons. e.g., keysMB[0] is for keys[0] var keysMB:Array = new Array(); var keys:Array = [key1, key2, key3, key4, key5, key6, key7];
// a list of motion buttons. e.g., keysMB[0] is for keys[0] var keysMB:Array = new Array(); var keys:Array = [key1, key2, key3, key4, key5, key6, key7];
for (var i:uint = 0; i < keys.length; i++) { keysMB.push(new MotionTrackerButton(webcam, keys[i], 5, 500)); }
for (var i:uint = 0; i < keys.length; i++) { keysMB.push(new MotionTrackerButton(webcam, keys[i], 5, 500)); }
Event Listeners
Spring 2012 SCM-CityU 42
for (var i:uint = 0; i < keys.length; i++) { // motion-driven
keysMB[i].addEventListener(MotionTrackerButton.CLICKED, onKeyPressed);
keysMB[i].addEventListener(MotionTrackerButton.CLICKABLE, onKeyReleased);
}
for (var i:uint = 0; i < keys.length; i++) { // motion-driven
keysMB[i].addEventListener(MotionTrackerButton.CLICKED, onKeyPressed);
keysMB[i].addEventListener(MotionTrackerButton.CLICKABLE, onKeyReleased);
}
function onKeyPressed(e:Event): void { for (var i:uint = 0; i < keys.length; i++) {
if (keysMB[i] == e.target) { trace("index of pressed key", i);
} }
}
function onKeyPressed(e:Event): void { for (var i:uint = 0; i < keys.length; i++) {
if (keysMB[i] == e.target) { trace("index of pressed key", i);
} }
}