SM1205 Interactivity Topic 10: Motion Tracking Part II Spring 2010SCM-CityU1

Motion Tracking Library In cityu.scm package (developed by Hongbo Fu) – Webcam class An easy-to-use wrapper for webcam access Spring 2010SCM-CityU2 import cityu.scm.Webcam; // import Webcam class // create a webcam with size 400 x 300 var webcam: Webcam = new Webcam (400, 300); // webcam will dispatch READY event when the webcam is ready to use webcam.addEventListener( Webcam.READY, onCameraReady); function onCameraReady(e:Event): void { addChild(webcam); } import cityu.scm.Webcam; // import Webcam class // create a webcam with size 400 x 300 var webcam: Webcam = new Webcam (400, 300); // webcam will dispatch READY event when the webcam is ready to use webcam.addEventListener( Webcam.READY, onCameraReady); function onCameraReady(e:Event): void { addChild(webcam); }

Motion Tracking Library In cityu.scm package (developed by Hongbo Fu) – MotionTracker class Spring 2010SCM-CityU3 var tracker:MotionTracker=new MotionTracker(webcam);

MotionTracker Class Properties – trackX: Number; trackY: Number Center of tracked region With respect to top-left corner of webcam (instead of stage) Methods – hitTestObject – hitTestPoint – hideTrackedArea – hideTrackedCenter Spring 2010SCM-CityU4 Spring (0, 0) (trackX, trackY)

Demos Spring 2010SCM-CityU5

Clipping Regions By default, motion tracking is done over the entire video domain – Check MotionTracker-Skeleton.fla Spring 2010SCM-CityU6

Clipping Regions What if motion only within specific regions (called clipping regions) are needed? – Motion outside those clipping regions will be ignored Demo Spring 2010SCM-CityU7 clipping regions

Clipping Regions Specify a clipping region when creating a motion tracker – cl, ct, cr, cb are with respect to top-left corner of webcam – trackX, trackY are also with respect to top-left corner of webcam – You can have only ONE clipping region for one tracker Spring 2010SCM-CityU8 tracker = new MotionTracker(webcam, cl, ct, cr, cb ); tracker = new MotionTracker(webcam, cl, ct, cr, cb ); cl cr ct cb Clipping Region

Multiple Clipping Regions To have multiple clipping regions, you can create multiple trackers – All trackers share the same Webcam instance – Each tracker has its own associated clipping region Spring 2010SCM-CityU9 // initialize 1st motion tracker tracker = new MotionTracker(webcam, 0, 0, 200, 200); // initialize 2nd motion tracker tracker2 = new MotionTracker(webcam, 300, 300, 500, 500); // initialize 1st motion tracker tracker = new MotionTracker(webcam, 0, 0, 200, 200); // initialize 2nd motion tracker tracker2 = new MotionTracker(webcam, 300, 300, 500, 500);

Class Exercise Extend the previous example to have two clipping regions as follows – Hint: create two trackers with their own clipping regions – How trackers work at the overlapped areas of clipping regions? Spring 2010SCM-CityU10

Example: Ping-Pong Game Spring 2010SCM-CityU11 box barL ball barR box2

Example: Ping-Pong Game Use tracker and tracker2 to control barL and barR, respectively Spring 2010SCM-CityU12

Class Exercise Make ball bounce against stage borders – Change ball’s x and y properties continuously in onTimer event listener – Check if ball hits stage borders Spring 2010SCM-CityU13

Hit Test between Ball and Bars Let’s just use hitTestObject method Spring 2010SCM-CityU14 if (ball.hitTestObject(barR)) { trace("barR got hit"); } if (ball.hitTestObject(barL)) { trace("barL got hit"); } if (ball.hitTestObject(barR)) { trace("barR got hit"); } if (ball.hitTestObject(barL)) { trace("barL got hit"); }

Hit Test between Ball and Bars Collision response Spring 2010SCM-CityU15 Case 1: Ball is on left side of barR Case 2: Ball is on right side of barR

Hit Test between Ball and Bars But how to determine which case to handle? Solution: use signs of ball’s speedX – speedX > 0  Case 1 Response: put ball exactly at the left side of barR – speedX < 0  Case 2 Response: put ball exactly at the right side of barR Spring 2010SCM-CityU16

Class Exercise Add sound effects when ball hits stage borders/bars – You can use Flash built-in sounds Hints – Export sounds for ActionScript Spring 2010SCM-CityU17 var ws:WoodSound = new WoodSound(); ws.play();

Moving Speed Use MotionTracker’s speedX and speedY to get the moving speed of tracked object speedX (in pixels) – Positive value if object moves along positive x-axis – Negative value if objects moves along negative x-axis speedY (in pixels) – Positive value if object moves along positive y-axis – Negative value if objects moves along negative y-axis Spring 2010SCM-CityU18 trace(tracker.speedX, tracker.speedY);

Example Transfer moving speed of tracked object to ball – E.g., hand Demo Spring 2010SCM-CityU19

Example Let’s modify the previous ping-pong example – Keep one tracker only, without any clipping region – No vertical bars – Add friction Spring 2010SCM-CityU20

Example Change ball’s speeds only if there is some intersection between ball and moving object Spring 2010SCM-CityU21 if (tracker.isTracked() && tracker.hitTestPoint(ball.x, ball.y, true)) { speedX += tracker.speedX; speedY += tracker.speedY; } if (tracker.isTracked() && tracker.hitTestPoint(ball.x, ball.y, true)) { speedX += tracker.speedX; speedY += tracker.speedY; }

Example Interact with more balls (demo) Spring 2010SCM-CityU22

Assignment 3 Topic: Webcam Interaction with Motion Requirements – Use motion tracker – Interact with motion – Have music/sound elements – Meaningful system – Documentation (in Word format) Title, motivation, how-to-use, screenshots, references Spring 2010SCM-CityU23

Assignment 3 Submission – Via ACS Max total file size: 50M.fla,.swf,.doc (documentation) files and other files – Deadline: 23:59:59, 12 May (Wed), 2010 No late submission is allowed In-class presentation, 12 May, 2010 – Each student has roughly 5 minutes for presentation Spring 2010SCM-CityU24

Assignment 3 Assessment – Based on Implementation difficulty + UI design + creativity (80%) Presentation performance (10%) Documentation (10%) – Note: if you use any technique which has not been taught in this course, you must state its source or reference in the documentation clearly Otherwise, some marks will be deducted Spring 2010SCM-CityU25

Summary of MotionTracker Make sure your Flash application is under the same folder as the tracking library Spring 2010SCM-CityU26

Summary of MotionTracker Without clipping region With clipping region – cl, ct, cr, cb are with respect to top-left corner of webcam Spring 2010SCM-CityU27 tracker = new MotionTracker(webcam); tracker = new MotionTracker(webcam, cl, ct, cr, cb ); tracker = new MotionTracker(webcam, cl, ct, cr, cb );

Summary of MotionTracker Properties – trackX, trackY: center of tracked region With respect to top-left corner of webcam – speedX, speedY Moving speed of tracked object Spring 2010SCM-CityU 28 Spring (0, 0) (trackX, trackY)

Summary of MotionTracker Methods – hitTestObject To test if bounding boxes of display object and tracked region intersect with each other – hitTestPoint To test if a given point is within the tracked region – isTracked To check if any moving object is tracked – hideTrackedArea – hideTrackedCenter Spring 2010SCM-CityU29