SM1205 Interactivity Topic 09: Motion Tracking Part I Spring 2010SCM-CityU1

Demos First Snow sweeper by PlaydoCAM (link)link – Make sure webcam is ready Spring 2010SCM-CityU2

Demos First Eyekanoid by PlaydoCAM (link)link Spring 2010SCM-CityU3

Demos First PlaydoJam by PlaydoCAM (link)link Spring 2010SCM-CityU4

How Come? Reason: you are watched by computers via webcams Computers know your movement – Relying on technology, called motion tracking Spring 2010SCM-CityU5

Motion Tracking (wiki)wiki To locate a moving object or several ones – Output: the location of moving objects How can a program know if there are moving objects and where they are? Idea: use special image processing algorithms to get regions of moving objects – Many algorithms exist for this purpose Spring 2010SCM-CityU6

Solution I Find the difference between current frame and a given background – Disadvantage: background must be static & fixed Disallow camera movement Disallow changes of lighting condition Current frame Static background

Solution II Find difference between consecutive frames – E.g., difference between frames i and i+1 – A more flexible solution! Spring 2010SCM-CityU8 Current frame Previous frame Note: white colors correspond to boundary of moving objects

Main References Motion tracking in ActionScript – Ostrich Flash (link)link – Motion tracker from soulwire (link)link – Play with demos in W drive Spring 2010SCM-CityU9

Motion Tracking Library In cityu.scm package (developed by Hongbo Fu) – Webcam class An easy-to-use wrapper for webcam access – MotionTracker class Main class for motion tracking – MotionTrackerSettingUI class User interface for parameter settings – MotionTrackerButton class Use motion to trigger button clicking Spring 2010SCM-CityU10 Note : classes are just special data types, which usually have their own properties & methods/functions

SM1205 Interactivity Webcam Class Spring 2010SCM-CityU11

Webcam Class Allow you to access streaming video from webcam Original webcam source has size 320 x 240 But you can scale up to any size, e.g., 640 x 480 Spring 2010SCM-CityU12

Webcam Class To use Webcam class, you first need to tell Flash where the corresponding source code (Webcam.as) is – Under cityu/scm/ folder This is done by import statement Make sure your Flash application is under the same folder as cityu folder Spring 2010SCM-CityU13 import cityu.scm.Webcam; // import Webcam class Note: those folders contain all necessary source code for motion tracking

Webcam Class Initializing webcam takes time Webcam class provides a special event type, called Webcam.READY – To make sure you’re always using an initialized webcam Only within event listener for this event, you can guarantee webcam is ready to use Spring 2010SCM-CityU14

Webcam Class Basic usage of Webcam class Spring 2010SCM-CityU15 import cityu.scm.Webcam; // import Webcam class // create a webcam with size 400 x 300 var webcam: Webcam = new Webcam (400, 300); // webcam will dispatch READY event when the webcam is ready to use webcam.addEventListener( Webcam.READY, onCameraReady); function onCameraReady(e:Event): void { addChild(webcam); } import cityu.scm.Webcam; // import Webcam class // create a webcam with size 400 x 300 var webcam: Webcam = new Webcam (400, 300); // webcam will dispatch READY event when the webcam is ready to use webcam.addEventListener( Webcam.READY, onCameraReady); function onCameraReady(e:Event): void { addChild(webcam); }

Default Values for Function Parameters You can define functions with default values for function parameters – Advantage: you don’t need to pass all values every time when using functions E.g., function definition of Webcam usage of Webcam Spring 2010SCM-CityU16 function Webcam(w:Number=640, h:Number=480) {... } function Webcam(w:Number=640, h:Number=480) {... } // create a webcam with size 640 x 480 (default size) var webcam:Webcam = new Webcam() ; // equivalent: Webcam(640, 480) // create a webcam with size 640 x 480 (default size) var webcam:Webcam = new Webcam() ; // equivalent: Webcam(640, 480)

Webcam Class Like other symbol instances, Webcam instances provide a lot of display properties – E.g., Spring 2010SCM-CityU17 webcam.alpha = 0.4; webcam.x = 200; webcam.y = 30;

Webcam Class Use Webcam’s sendToBack method to avoid occlusion of stage objects by webcam – webcam.sendToBack(); Spring 2010SCM-CityU18

SM1205 Interactivity MotionTracker Class Spring 2010SCM-CityU19

MotionTracker Class Default visualization for tracked regions Able to track multiple objects Each tracked object can have its own clipping region – Motion happened outside clipping region will be ignored Spring 2010SCM-CityU20 Clipping regions

Simple Usage of MotionTracker Step 1: import MotionTracker class – I.e., import cityu.scm.MotionTracker; Step 2: initialize a MotionTracker object – E.g., Spring 2010SCM-CityU21 var tracker:MotionTracker=new MotionTracker(webcam);

Spring 2010SCM-CityU22

Center of Tracked Region Use ColorTracker’s properties – trackX: Number; trackY: Number Center of tracked region With respect to top-left corner of webcam (instead of stage) – E.g., tracker.trackX Spring 2010 SCM-CityU23 (0, 0) (trackX, trackY)

Spring 2010SCM-CityU24 trackX = 10 trackY = 20

Class Exercise Create a symbol instance on stage Make this shape follow center of moving region – Use trackX and trackY Spring 2010SCM-CityU25

Coordinate System Transformation Use DisplayObject’s methods – localToGlobal, globalToLocal Stage to Webcam – var ptWebcam:Point = webcam.globalToLocal(ptStage); Webcam to Stage – var ptWebcam:Point = webcam.localToGlobal(ptStage); Spring 2010SCM-CityU26

Hide Default Visualization Use hideTrackedArea and hideTrackedCenter methods – E.g., Spring 2010SCM-CityU27 tracker.hideTrackedArea(); tracker.hideTrackedCenter();

Collision with Tracked Regions Use MotionTracker’s methods – hitTestObject E.g. tracker.hitTestObject(cursor); – hitTestPoint E.g., tracker.hitTestPoint(cursor.x, cursor.y, true); You can also use isTracked method to perform collision detection only when object is tracked – isTracked return true if object is tracked; false, otherwise. Spring 2010SCM-CityU28

Collision with Tracked Regions Usage of MotionTracker’s hitTestObject is very similar to that of DisplayObject’s Only difference: order matters – Wrong: cursor.hitTestObject(tracker); – Correct: tracker.hitTestObject(cursor); Spring 2010SCM-CityU29

Collision with Tracked Regions Usage of MotionTracker’s hitTestPoint is exactly the same as that of DisplayObject’s – hitTestPoint checks if a given point is within regions of pixels with white color Spring 2010SCM-CityU30

Example: Shape Sweeper Spring 2010SCM-CityU31

Class Exercise Create a large number of circles on the stage and store them into an array for future use – Random size – Random position – Random opacity Tips: – addChild – push Spring 2010SCM-CityU32

Webcam Interaction For each shape which falls into tracked region, we remove it from stage – Use removeChild method – Need for loop How about corresponding elements in the array? – How to remove them from the array? Spring 2010SCM-CityU33

Webcam Interaction Removing array elements one by one in a for loop is tricky – Consider the following example – Can you expect the result? Array length and possibly element indices change over time/iterations Spring 2010SCM-CityU34 var a:Array = [2, 3, 5, 7, 9]; for (var i:uint = 0; i < a.length; i++) { a.pop(); // pop the last element of the current array } trace(a); var a:Array = [2, 3, 5, 7, 9]; for (var i:uint = 0; i < a.length; i++) { a.pop(); // pop the last element of the current array } trace(a);

Webcam Interaction Instead of removing array elements one by one, let’s reset the array to empty when all shapes have been removed from the stage – shapes = []; To achieve this, – Use numOfShapesOnStage variable to maintain the number of shapes left on the stage – Use contains method to check if array element shapes[i] is still on the stage if (contains(shapes[i])) {…} Spring 2010SCM-CityU35