1 Mobile Computing Advanced Touching Copyright 2014 by Janson Industries Assg Part1Assg Part1 AssgPart2AssgPart2

Copyright 2014 by Janson Industries 2 Objectives ▀ Explain u Motion events u Multi-touch u Gestures u Creating gestures

Copyright 2014 by Janson Industries 3 Hardware ▀ The touch screen is constructed to pick up pressure on the screen and convert that to data u Screen coordinates u Pressure u Size of touch u Time ▀ All info stored in a MotionEvent object

Copyright 2014 by Janson Industries 4 Motion Sequence Actions ▀ The action property holds value that identifies action type u action = 0 (ACTION_DOWN) is the initial touch u action = 2 (ACTION_MOVE) any up, down, or sideways movement on the screen u action = 1 (ACTION_UP) is when finger is lifted off the screen

Copyright 2014 by Janson Industries 5 n onTouch or onTouchEvent method called when motion action occurs u A MotionEvent object passed to these methods n For classes that implement onTouchListener, onTouch called n For subclasses of View, onTouchEvent called Motion Sequence

Copyright 2014 by Janson Industries 6 Example n Will create a view subclass called TouchTest n Its onTouchEvent will get info from the MotionEvent object and display in LogCat n Will also use the switch structure

Copyright 2014 by Janson Industries 7 Nested if/else if (month == 1) { monthTV.setText(“Jan”);} else {if (month == 2) { monthTV.setText(“Feb”);} else { if (month == 3) { monthTV.setText(“Mar”);} else { if (month == 4) {……… Nested if ▮ Placing an if statement as one of the statements to be executed in an if/else clause

Copyright 2014 by Janson Industries 8 Switch switch (Month) { case 1: monthTV.setText(“Jan”); break; case 2: monthTV.setText(“Feb”); break; case 3: monthTV.setText(“Mar”); break; : : : : : default: System.out.println(“Not a valid month!”); } ▮ Instead of complicated nested ifs, can use a switch

Copyright 2014 by Janson Industries 9 Switch ▮ Need the break statements because once the condition is true, all subsequent statements are executed ▮ In the example, this means ▮ The label is set to Dec ▮ And the “Not a valid month” message displayed

Copyright 2014 by Janson Industries 10 TouchTest n TouchTest (a subclass of View) will: n Check for touches n When a touch occurs, display a variety of info about the touch

Copyright 2014 by Janson Industries 11 TouchTest package my.touch.com; import android.content.Context; import android.util.AttributeSet; import android.view.MotionEvent; import android.view.View; public class TouchTest extends View { public TouchTest(Context context, AttributeSet attrs) { super(context); } n Created new TouchProj project, my.touch.com package and Main activity, then create new class TouchTest

Copyright 2014 by Janson Industries 12 TouchTest public boolean onTouchEvent(MotionEvent event) { int action = event.getAction(); switch (action) { case 0: System.out.println("Action: " + event.getAction()); System.out.println("Location: " + event.getX() + " x " + event.getY() + " y"); System.out.println("Pressure: " + event.getPressure()); System.out.println("Size: " + event.getSize()); System.out.println("Down time: " + event.getDownTime() + " ms"); System.out.println("Event time: " + event.getEventTime() + " ms"); System.out.println("Elapsed: " + (event.getEventTime() - event.getDownTime()) + " ms"); break;

Copyright 2014 by Janson Industries 13 TouchTest case 1: System.out.println("Action: " + event.getAction()); System.out.println("Location: " + event.getX() + " x " + event.getY() + " y"); System.out.println("Pressure: " + event.getPressure()); System.out.println("Size: " + event.getSize()); System.out.println("Down time: " + event.getDownTime() + " ms"); System.out.println("Event time: " + event.getEventTime() + " ms"); System.out.println("Elapsed: " + (event.getEventTime() - event.getDownTime()) + " ms"); break; } return (true); }

Copyright 2014 by Janson Industries 14 main.xml <LinearLayout xmlns:android=" android:orientation="vertical" android:layout_width="fill_parent" android:layout_height="fill_parent"> <my.touch.com.TouchTest android:layout_width="wrap_content" android:layout_height="wrap_content" /> n Change activity_main.xml to add TouchTest to main screen

Copyright 2014 by Janson Industries 15 TouchTest n Run n Click & hold n Release

Copyright 2014 by Janson Industries 16 Touch Test :42:54.086: I/System.out(18244): Action: :42:54.086: I/System.out(18244): Location: x y :42:54.097: I/System.out(18244): Pressure: :42:54.097: I/System.out(18244): Size: :42:54.127: I/System.out(18244): Down time: ms :42:54.127: I/System.out(18244): Event time: ms :42:54.167: I/System.out(18244): Elapsed: 0 ms :42:56.106: I/System.out(18244): Action: :42:56.106: I/System.out(18244): Location: x y :42:56.117: I/System.out(18244): Pressure: :42:56.131: I/System.out(18244): Size: :42:56.131: I/System.out(18244): Down time: ms :42:56.131: I/System.out(18244): Event time: ms :42:56.131: I/System.out(18244): Elapsed: 2020 ms

Copyright 2014 by Janson Industries 17 Touch Test n In emulator, pressure and size will always be 1 and 0 n If you click and drag, a series of move actions (2) will be generated n We’ll add a check for an action 2 in the switch

Copyright 2014 by Janson Industries 18 TouchTest n Took out the pressure and size displays n Click and drag quickly to right case 2: System.out.println("Action: " + event.getAction()); System.out.println("Location: " + event.getX() + " x " + event.getY() + " y"); System.out.println("Down time: " + event.getDownTime() + " ms"); System.out.println("Event time: " + event.getEventTime() + " ms"); System.out.println("Elapsed: " + (event.getEventTime() - event.getDownTime()) + " ms");

Copyright 2014 by Janson Industries 19 Touch Test :48:09.017: I/System.out(18304): Action: :48:09.017: I/System.out(18304): Location: x y :48:09.037: I/System.out(18304): Down time: ms :48:09.037: I/System.out(18304): Event time: ms :48:09.037: I/System.out(18304): Elapsed: 1638 ms :48:09.129: I/System.out(18304): Action: :48:09.129: I/System.out(18304): Location: x y :48:09.137: I/System.out(18304): Down time: ms :48:09.137: I/System.out(18304): Event time: ms :48:09.137: I/System.out(18304): Elapsed: 1751 ms :48:09.150: I/System.out(18304): Action: :48:09.157: I/System.out(18304): Location: x y :48:09.157: I/System.out(18304): Down time: ms :48:09.157: I/System.out(18304): Event time: ms :48:09.169: I/System.out(18304): Elapsed: 1761 ms

Copyright 2014 by Janson Industries 20 Implementing Touch n Create a View subclass called Square n When screen touched a square will appear around the touch n If movement, square will follow the movement n When touch lifted square disappears

Copyright 2014 by Janson Industries 21 Square package my.touch.com; import android.content.Context; import android.graphics.Canvas; import android.graphics.Color; import android.graphics.Paint; import android.util.AttributeSet; import android.view.MotionEvent; import android.view.View; public class Square extends View { private float x = -30; // Set initial location off screen private float y = -30; private Paint myPaint; public Square(Context context, AttributeSet attrs) { super(context, attrs); myPaint = new Paint(); myPaint.setColor(Color.BLUE); }

Copyright 2014 by Janson Industries 22 Square public boolean onTouchEvent(MotionEvent event) { int action = event.getAction(); switch (action) { case MotionEvent.ACTION_DOWN: // In both cases these case MotionEvent.ACTION_MOVE: // statements are run x = event.getX(); y = event.getY(); break; case MotionEvent.ACTION_UP: x = -30; // set the location outside the screen area y = -30; break; } return (true); } public void draw(Canvas canvas) { canvas.drawRect(x-15, y-15, x+15, y+15, myPaint); invalidate(); }

Copyright 2014 by Janson Industries 23 activity_main.xml <LinearLayout xmlns:android=" android:orientation="vertical" android:layout_width="fill_parent" android:layout_height="fill_parent"> <my.touch.com.Square android:layout_width="wrap_content" android:layout_height="wrap_content" />

Copyright 2014 by Janson Industries 24 Click Drag Release

Copyright 2014 by Janson Industries 25 Assignment - Part 1 n Create Square such that the square is dragged around (as shown earlier) n But when released, square returns to the original clicked location

Copyright 2014 by Janson Industries 26 Velocity n Can measure using a VelocityTracker n VelocityTracker initially obtained then u Specify pixels per time period in milliseconds F Per second means 1000 u Feed the events u Retrieve the x and y speeds

Copyright 2014 by Janson Industries 27 VelocityEx package my.touch.com; import android.content.Context; import android.util.AttributeSet; import android.view.MotionEvent; import android.view.VelocityTracker; import android.view.View; public class VelocityEx extends View { private VelocityTracker vTracker = null; public VelocityEx(Context context, AttributeSet attrs) { super(context, attrs); }

Copyright 2014 by Janson Industries 28 VelocityEx public boolean onTouchEvent(MotionEvent event) { int action = event.getAction(); switch (action) { case 0: if (vTracker == null) { //Get the VelocityTracker object or clear out old data vTracker = VelocityTracker.obtain(); } else { vTracker.clear(); } //Feed initial click down event to set starting point vTracker.addMovement(event); break;

Copyright 2014 by Janson Industries 29 VelocityEx case 2: vTracker.addMovement(event); // Sets speed per second vTracker.computeCurrentVelocity(1000); // Displays the x and y velocity System.out.println("X velocity is " + vTracker.getXVelocity() + " pixels per second"); System.out.println("Y velocity is " + vTracker.getYVelocity() + " pixels per second"); break; } return (true); }}

Copyright 2014 by Janson Industries 30 VelocityEx <LinearLayout xmlns:android=" android:orientation="vertical" android:layout_width="fill_parent" android:layout_height="fill_parent"> <my.touch.com.VelocityEx android:layout_width="wrap_content" android:layout_height="wrap_content" /> n Change activity_main.xml to display VelocityEx

Copyright 2014 by Janson Industries 31 When run

Copyright 2014 by Janson Industries 32 Touch n Can put listener on any view subclass u onTouch method will be invoked n Add a TextView to the layout n Main activity must u Implement onTouchListener u Get the text view and add the listener to it

Copyright 2014 by Janson Industries 33 Touch <LinearLayout xmlns:android=" android:orientation="vertical" android:layout_width="fill_parent" android:layout_height="fill_parent"> <TextView android:layout_height="wrap_content" android:text="Example text in text view" android:layout_width="wrap_content" android:textSize="35dp"> <my.touch.com.VelocityEx android:layout_width="wrap_content" android:layout_height="wrap_content" /> n New TextView

Copyright 2014 by Janson Industries 34 Touch ::: import android.view.MotionEvent; import android.view.View; import android.view.View.OnTouchListener; import android.widget.TextView; public class Main extends ActionBarActivity implements OnTouchListener { TextView tv; public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); //TextView retrieved and listener added tv = (TextView)this.findViewById(R.id.tV); tv.setOnTouchListener(this); } public boolean onTouch(View v, MotionEvent event) { System.out.println("**** Main's onTouch invoked"); return false; } :::

Copyright 2014 by Janson Industries 35 onTouch & onTouchEvent ▀ Must return a Boolean value ▀ If it returns true, it means: u The action has been consumed u No other view needs to be notified ▀ If it returns false, it means: u The action has not been consumed u The method is not interested in any other future events for this action u Other views should be notified

Copyright 2014 by Janson Industries 36 onTouch returns false so if we click, drag, and then release on the TextView, only one message printed If clicked again, one message will be displayed because it’s a new action

Copyright 2014 by Janson Industries 37 onTouch & onTouchEvent ▀ But specifying true also has implications ▀ We will put a checkbox on the main screen ▀ onTouch returns false and the checkbox works like normal <CheckBox android:text="example" android:layout_width="wrap_content" android:layout_height="wrap_content" android:textSize="25dp"/>

Copyright 2014 by Janson Industries 38 Clicked and released and two messages are displayed

Copyright 2014 by Janson Industries 39 CheckBox is checked

Copyright 2014 by Janson Industries 40 Change it to true, click and release, and two messages are displayed

Copyright 2014 by Janson Industries 41 But the CheckBox is not checked. True says don’t notify other View methods, in this case, the CheckBox’s onTouchEvent method which displays the green check mark and changes the state

Copyright 2014 by Janson Industries public boolean onTouch(View v, MotionEvent event) { tv.setText("Pressure: " + String.valueOf(event.getPressure()) + " Size: " + String.valueOf(event.getSize())); System.out.println("**** Main's onTouch invoked"); return true; } 42 Can sense pressure and size of touch but must run on a device (not emulator)

Copyright 2014 by Janson Industries 43

Copyright 2014 by Janson Industries 44 Assignment – Part 2 ▀ In Square, make the square size relative to the size and pressure of initial touch ▀ Make sure square is easy to see regardless of size and pressure ► Please, no microscopic squares

Copyright 2014 by Janson Industries 45 MultiTouch ▀ Complicated and doesn’t always work the same way on different devices and Android implementations ▀ For example, getPointerCount returns the number of fingers touching the screen u On some devices it only reports some of the fingers!

Copyright 2014 by Janson Industries 46 MultiTouch ▀ Assuming getPointerCount returned 3, the three touches are referenced by a pointer index ▀ In this case, the pointer index has pointer ids of 0, 1, and 2 located at index locations 0, 1, and 2 u Calls to get info about a touch must include a pointer index location ► getX(pointerIndex);

Copyright 2014 by Janson Industries 47 MultiTouch ▀ In addition when the second finger touch occurs the action code is 261 u A third finger 517 ▀ Android returns the pointer index (1) and action code (5) for an additional touch u A hex 0105 equals 261 ► Third finger is 2 and 5, 0205 = 517.

Copyright 2014 by Janson Industries 48 MultiTouch ▀ To prove must create new project with latest build level u 1.6 doesn’t support getPointerCount ▀ Will print out the action code and the count of number of touches u Will touch the screen with 3 fingers one after another

Copyright 2014 by Janson Industries 49 Touch package com.example.multitouchproj; import android.content.Context; import android.util.AttributeSet; import android.view.MotionEvent; import android.view.View; public class MultiTouchTest extends View { int action, count; public MultiTouchTest(Context context, AttributeSet attrs) { super(context); } public boolean onTouchEvent(MotionEvent event) { action = event.getAction(); count = event.getPointerCount(); System.out.println("Action: " + action); System.out.println("Count: " + count); return true; }

Copyright 2014 by Janson Industries 50 Initial touch and count 2nd touch and count 3rd touch and count

Copyright 2014 by Janson Industries 51 MultiTouch ▀ If the first finger gets removed the pointer ids are shifted left in the index u Index 0 holds pointer id 1 u Index 1 holds pointer id 2 ▀ If the first finger touches again it is assigned pointer id 0 in index 0

Copyright 2014 by Janson Industries 52 Touch ::: public class MultiTouchTest extends View { int action, count; float xLoc, yLoc; ::: public boolean onTouchEvent(MotionEvent event) { action = event.getAction(); count = event.getPointerCount(); System.out.println("Action: " + action); System.out.println("Count: " + count); for (int counter = 0; counter < count; counter++){ xLoc = event.getX(counter); yLoc = event.getY(counter); System.out.println("Location of " + counter + " touch x:" + xLoc + " y:" + yLoc); } return true; }

Copyright 2014 by Janson Industries 53 MultiTouch ▀ Will touch once then twice and then remove first touch ▀ Notice what index is displayed and what the location is

Copyright 2014 by Janson Industries 54 1st 2nd Removed first finger

Copyright 2014 by Janson Industries 55 MultiTouch ▀ Keeping track complicated! u Thank the Lord for gestures ▀ Gestures are recognizable motions u Been around a long time

Copyright 2014 by Janson Industries 56 Gestures ▀ Pinch not supported until 2.2 ▀ Create a new project (GestureProj), package (my.gestures.c om), and set build target to 2.2

Copyright 2014 by Janson Industries 57 Gestures ▀ Specify u Activity name u Layout name

Copyright 2014 by Janson Industries 58 Gestures ▀ Recognizable recorded motions ▀ Can be single or multi-stroke ▀ When defining, the order of the strokes make a difference ▀ Creating a T gesture like this Different than this

Copyright 2014 by Janson Industries 59 Gestures ▀ Can create new ones with Gestures Builder ▀ Start the emulator and display all applications (center icon) ▀ Double click ▀ A blank screen with two buttons on the bottom will be displayed

Copyright 2014 by Janson Industries 60

Copyright 2014 by Janson Industries 61 Gestures Builder ▀ If it’s not on your device, download it from ▀ On phone need to change download settings to allow unknown source u Settings, Security, tap “Unknown sources ” name=com.android.gesture.builder.apk&can=2&q

Copyright 2014 by Janson Industries 62

Copyright 2014 by Janson Industries 63 Gestures ▀ Click the Add gesture button and move the cursor in the shape of the gesture you want to create ▀ In the data entry field called name, type in a name for the gesture and click Done u Location where it's stored will be displayed ► mnt/ sdcard/gestures

Copyright 2014 by Janson Industries 64

Copyright 2014 by Janson Industries 65

Copyright 2014 by Janson Industries 66 Gestures ▀ If you don’t like the gesture, click the discard button and try again ▀ You can record many gestures with the same name u That’s how you capture all the different ways of drawing a gesture

Copyright 2014 by Janson Industries 67 Once saved the gestures will be listed Click/tap and hold a gesture to see a menu of functions that can be performed against one (delete, rename)

Copyright 2014 by Janson Industries 68 Deciphering Gestures ▀ Need a GestureOverlayView on the screen ▀ Class must u Implement OnGesturePerformedListener u Add listener to the GestureOverlayView u Have an onGesturePerformed method ► Will be invoked when a gesture is performed

Copyright 2014 by Janson Industries 69 Deciphering Gestures ▀ onGesturePerformed will be passed a Gesture object ▀ Class needs a GestureLibrary u Accepts a gesture object u Compares it to all the recorded gestures u Returns an ArrayList of predictions ► Prediction consists of the gesture name and a score

Copyright 2014 by Janson Industries 70 Deciphering Gestures ▀ The array list has the predictions in order from most to least likely ▀ So the gesture in position 0 of the ArrayList is what the system thinks is the correct gesture

Copyright 2014 by Janson Industries 71 Gestures <LinearLayout xmlns:android=" android:orientation="vertical" android:layout_width="fill_parent" android:layout_height="fill_parent"> <TextView android:layout_width="fill_parent" android:layout_height="wrap_content" android:text="Draw gestures and I'll guess what they are" /> <android.gesture.GestureOverlayView android:layout_width="fill_parent" android:layout_height="fill_parent" android:gestureStrokeType="multiple" android:fadeOffset="1000" /> If multi-stroke possible must specify

Copyright 2014 by Janson Industries 72 Gestures package my.gestures.com; import java.util.ArrayList; import android.app.Activity; import android.gesture.Gesture; import android.gesture.GestureLibraries; import android.gesture.GestureLibrary; import android.gesture.GestureOverlayView; import android.gesture.Prediction; import android.gesture.GestureOverlayView.OnGesturePerformedListener; import android.os.Bundle; import android.util.Log; import android.widget.Toast; public class GestureProjActivity extends Activity implements OnGesturePerformedListener {

Copyright 2014 by Janson Industries 73 Gestures GestureLibrary gestureLib = null; Prediction prediction; public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.main_gesture); gestureLib = GestureLibraries.fromFile("/sdcard/gestures"); if (!gestureLib.load()) { Toast.makeText(this, "Could not load /sdcard/gestures", Toast.LENGTH_SHORT).show(); finish(); } GestureOverlayView gestureView = (GestureOverlayView) findViewById(R.id.gestureOverlay); gestureView.addOnGesturePerformedListener(this); }

Copyright 2014 by Janson Industries 74 Deciphering Gestures ▀ If you want to package the gestures with your app u Create the raw folder in res u Copy the gestures file into raw u Create the GestureLibrary as follows ► Instead of creating it from sdcard gestureLib = GestureLibraries.fromRawResource(this, R.raw.gestures);

Copyright 2014 by Janson Industries 75

Copyright 2014 by Janson Industries 76

Copyright 2014 by Janson Industries 77 Gestures public void onGesturePerformed(GestureOverlayView view, Gesture gesture) { ArrayList predictions = gestureLib.recognize(gesture); if (predictions.size() > 0) { prediction = predictions.get(0); if (prediction.score > 1.0) { Toast.makeText(this, prediction.name, Toast.LENGTH_SHORT).show(); } Toast will display the gesture name

Copyright 2014 by Janson Industries 78 Initial screen

Copyright 2014 by Janson Industries 79

Copyright 2014 by Janson Industries 80

Copyright 2014 by Janson Industries 81 Gestures for (int i = 0; i < predictions.size(); i++) { System.out.println("prediction " + predictions.get(i).name + "'s score = " + predictions.get(i).score); } n Too see all the predictions scores, add this after the statement that displays the toast