1. Mobile Device Development Camera and Sensors Dr.YingLiang Ma

2. UFCFX5-15-3Mobile Device Development Agenda Android Camera Overview Simple android app using camera Using Camera API Using

3. UFCFX5-15-3Mobile Device Development Camera in android mobile

4. UFCFX5-15-3Mobile Device Development Camera Specifications 1.Camera resolutions (image sensors) 2.Optical Zoom or Digital Zoom 3.Xenon flash or LED flash 4.Video capture resolutions (4K??) 5.Face recognition 6. Low-light performance 7.Anti-shake performance

5. UFCFX5-15-3Mobile Device Development Camera Quality variation 1.Camera resolutions 2.With or without flash 3.No face-forward camera 4.No advance features

6. UFCFX5-15-3Mobile Device Development Simple Camera App Using existing android camera application in our application Intent intent = new Intent(android.provider.MediaStore.ACTION_IMAGE_CAPTURE); 1ACTION_IMAGE_CAPTURE_SECURE It returns the image captured from the camera, when the device is secured 2ACTION_VIDEO_CAPTURE It calls the existing video application in android to capture video 3EXTRA_SCREEN_ORIENTATION It is used to set the orientation of the screen to vertical or landscape 4EXTRA_FULL_SCREEN It is used to control the user interface of the ViewImage

7. UFCFX5-15-3Mobile Device Development UI Design <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" tools:context=".MainActivity"> <ImageView android:id="@+id/imageView1" android:layout_width="match_parent" android:layout_height="match_parent" android:contentDescription="@string/hello_world" android:src="@drawable/ic_launcher" /> <TextView android:id="@+id/textView1" android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_alignParentTop="true" android:layout_alignRight="@+id/imageView1" android:text="@string/tap" android:textAppearance="?android:attr/textAppearanceLarge" />

8. UFCFX5-15-3Mobile Device Development String definition Camera Settings Hello world! Tap the image to open the camera!!

9. UFCFX5-15-3Mobile Device Development Main Activity public class MainActivity extends Activity { ImageView imgFavorite; @Override protected void onCreate(Bundle savedInstanceState) { imgFavorite.setOnClickListener(new OnClickListener() { @Override public void onClick(View v) {open(); } }); } public void open(){ Intent intent = new Intent(android.provider.MediaStore.ACTION_IMAGE_CAPTURE); startActivityForResult(intent, 0); } @Override protected void onActivityResult(int requestCode, int resultCode, Intent data) { // TODO Auto-generated method stub super.onActivityResult(requestCode, resultCode, data); Bitmap bp = (Bitmap) data.getExtras().get("data"); imgFavorite.setImageBitmap(bp); }

10. UFCFX5-15-3Mobile Device Development Result

11. UFCFX5-15-3Mobile Device Development Camera API app We will be using the camera API to integrate the camera in our application Class Description Camera It is used to control the camera and take images or capture video from the camera SurfaceView This class is used to present a live camera preview to the user.

12. UFCFX5-15-3Mobile Device Development UI Design <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="fill_parent" android:layout_height="fill_parent" android:orientation="horizontal" > <LinearLayout android:layout_width="wrap_content" android:layout_height="match_parent" android:layout_weight="0.30" android:orientation="vertical" > <FrameLayout android:id="@+id/camera_preview" android:layout_width="fill_parent" android:layout_height="400dp" /> <Button android:id="@+id/button_capture" android:layout_width="match_parent" android:layout_height="wrap_content" android:onClick="snapIt" android:text="@string/Capture" /> <ImageView android:id="@+id/imageView1" android:layout_width="fill_parent" android:layout_height="fill_parent" android:scaleType="fitXY" android:src="@drawable/ic_launcher" />

13. UFCFX5-15-3Mobile Device Development UI

14. UFCFX5-15-3Mobile Device Development Android Manifest file <manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.example.camera1" android:versionCode="1" android:versionName="1.0" > <uses-sdk android:minSdkVersion="8" android:targetSdkVersion="17" /> <application android:icon="@drawable/ic_launcher" android:label="@string/app_name" > <activity android:name="com.example.camera1.MainActivity" android:label="@string/app_name" android:configChanges="orientation" android:screenOrientation="portrait">

15. UFCFX5-15-3Mobile Device Development Structure MainActivity ShowCamera Preview windows Main UI FrameLayout AddView PictureCallback Take photo

16. UFCFX5-15-3Mobile Device Development Show Camera Activity public class ShowCamera extends SurfaceView implements SurfaceHolder.Callback { private SurfaceHolder holdMe; private Camera theCamera; public ShowCamera(Context context,Camera camera) { super(context); theCamera = camera; holdMe = getHolder(); holdMe.addCallback(this); } @Override public void surfaceCreated(SurfaceHolder holder) { try { theCamera.setPreviewDisplay(holder); theCamera.startPreview(); } catch (IOException e) { } @Override public void surfaceDestroyed(SurfaceHolder arg0) { }

17. UFCFX5-15-3Mobile Device Development Main Activity public class MainActivity extends Activity { private Camera cameraObject; private ShowCamera showCamera; private ImageView pic; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); pic = (ImageView)findViewById(R.id.imageView1); cameraObject = isCameraAvailiable(); showCamera = new ShowCamera(this, cameraObject); FrameLayout preview = (FrameLayout) findViewById(R.id.camera_preview); preview.addView(showCamera); }

18. UFCFX5-15-3Mobile Device Development Camera thread private PictureCallback capturedIt = new PictureCallback() { @Override public void onPictureTaken(byte[] data, Camera camera) { Bitmap bitmap = BitmapFactory.decodeByteArray(data, 0, data.length); if(bitmap==null){ Toast.makeText(getApplicationContext(), "not taken", Toast.LENGTH_SHORT).show(); } else { Toast.makeText(getApplicationContext(), "taken", Toast.LENGTH_SHORT).show(); pic.setImageBitmap(bitmap); } cameraObject.startPreview(); } };

19. UFCFX5-15-3Mobile Device Development Save the picture Put SD card write permission in android Manifest file In PictureCallback function Toast.makeText(getApplicationContext(), "taken", Toast.LENGTH_SHORT).show(); pic.setImageBitmap(bitmap); String path = Environment.getExternalStorageDirectory().toString(); String filename = path + "/SpyApp"+String.format("%d.jpg", System.currentTimeMillis()); FileOutputStream outStream = null; try { outStream = new FileOutputStream(filename); outStream.write(data); outStream.close(); } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } finally { }

20. UFCFX5-15-3Mobile Device Development See the picture files Download File Manage from Google Play Store

21. UFCFX5-15-3Mobile Device Development See the picture files

22. UFCFX5-15-3Mobile Device Development See the picture files

23. UFCFX5-15-3Mobile Device Development Record audio If you want to have more control over audio recording you could use the MediaRecorder.

24. UFCFX5-15-3Mobile Device Development Example codes Set audio permission in android Manifest file MediaRecorder recorder = new MediaRecorder(); recorder.setAudioSource(MediaRecorder.AudioSource.MIC); recorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP); recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB); recorder.setOutputFile(PATH_NAME); recorder.prepare(); recorder.start(); // Recording is now started... recorder.stop(); recorder.reset(); // You can reuse the object by going back to setAudioSource() step recorder.release(); // Now the object cannot be reused

25. UFCFX5-15-3Mobile Device Development Add them to MainActivity Set audio permission in android Manifest file protected void onCreate(Bundle savedInstanceState) { ….. ///record audio String path = Environment.getExternalStorageDirectory().toString(); String filename = path + "/SpyAudio"+String.format("%d.3gp", System.currentTimeMillis()); recorder = new MediaRecorder(); recorder.setAudioSource(MediaRecorder.AudioSource.MIC); recorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP); recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB); recorder.setOutputFile(filename); try { recorder.prepare(); } catch (IllegalStateException e) {e.printStackTrace();} recorder.start(); // Recording is now started

26. UFCFX5-15-3Mobile Device Development Stop recording @Override public boolean onKeyDown(int keyCode, KeyEvent event) { //Handle the back button if(keyCode == KeyEvent.KEYCODE_BACK) { recorder.stop(); recorder.reset(); // You can reuse the object by going back to setAudioSource() step recorder.release(); // Now the object cannot be reused return true; } else return super.onKeyDown(keyCode, event); }

27. UFCFX5-15-3Mobile Device Development Use Sensor in Android App Motion sensors These sensors measure acceleration forces and rotational forces along three axes. This category includes accelerometers, gravity sensors, gyroscopes, and rotational vector sensors. Environmental sensors These sensors measure various environmental parameters, such as ambient air temperature and pressure, illumination, and humidity. This category includes barometers, photometers, and thermometers. Position sensors These sensors measure the physical position of a device. This category includes orientation sensors and magnetometers.

28. UFCFX5-15-3Mobile Device Development Use Sensor in Android App You can access sensors available on the device and acquire raw sensor data by using the Android sensor framework. Determine which sensors are available on a device. Determine an individual sensor's capabilities, such as its maximum range, manufacturer, power requirements, and resolution. Acquire raw sensor data and define the minimum rate at which you acquire sensor data. Register and unregister sensor event listeners that monitor sensor changes.

29. UFCFX5-15-3Mobile Device Development Sensor availablity Few Android-powered devices have every type of sensor. For example, most handset devices and tablets have an accelerometer and a magnetometer, but fewer devices have barometers or thermometers. Also, a device can have more than one sensor of a given type. For example, a device can have two gravity sensors, each one having a different range. Identifying Sensors and Sensor Capabilities private SensorManager mSensorManager; mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE); List deviceSensors = mSensorManager.getSensorList(Sensor.TYPE_ALL);

30. UFCFX5-15-3Mobile Device Development Monitoring Sensor Events public class SensorActivity extends Activity implements SensorEventListener { private SensorManager mSensorManager; private Sensor mLight; @Override public final void onSensorChanged(SensorEvent event) { // The light sensor returns a single value. // Many sensors return 3 values, one for each axis. float lux = event.values[0]; // Do something with this sensor value. }

31. UFCFX5-15-3Mobile Device Development Handling Different Sensor Configurations private SensorManager mSensorManager;... mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE); if (mSensorManager.getDefaultSensor(Sensor.TYPE_PRESSURE) != null){ // Success! There's a pressure sensor. } else { // Failure! No pressure sensor. } Using Google Play filters to target specific sensor configurations <uses-feature android:name="android.hardware.sensor.accelerometer" android:required="true" />

32. UFCFX5-15-3Mobile Device Development Best Practices for Accessing and Using Sensors Unregister sensor listeners Don't test your code on the emulator Don't block the onSensorChanged() method (do as little as possible within the onSensorChanged(SensorEvent) method as sensor data can change at a high rate) Avoid using deprecated methods or sensor types Verify sensors before you use them Choose sensor delays carefully

33. UFCFX5-15-3Mobile Device Development Using the Proximity Sensor private SensorManager mSensorManager; private Sensor mSensor;... mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE); mSensor = mSensorManager.getDefaultSensor(Sensor.TYPE_PROXIMITY); @Override public final void onSensorChanged(SensorEvent event) { float distance = event.values[0]; // Do something with this sensor data. }

34. UFCFX5-15-3Mobile Device Development Using the Orientation Sensor The orientation sensor lets you monitor the position of a device relative to the earth's frame of reference (specifically, magnetic north). private SensorManager mSensorManager; private Sensor mSensor;... mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE); mSensor = mSensorManager.getDefaultSensor(Sensor.TYPE_ORIENTATION); @Override public void onSensorChanged(SensorEvent event) { float azimuth_angle = event.values[0]; float pitch_angle = event.values[1]; float roll_angle = event.values[2]; // Do something with these orientation angles. }

35. UFCFX5-15-3Mobile Device Development Using the Step Counter Sensor The step counter sensor provides the number of steps taken by the user since the last reboot while the sensor was activated. The step counter has more latency (up to 10 seconds) but more accuracy than the step detector sensor. private SensorManager mSensorManager; private Sensor mSensor;... mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE); mSensor = mSensorManager.getDefaultSensor(Sensor.TYPE_STEP_COUNTER);

36. UFCFX5-15-3Mobile Device Development Environment Sensors Unlike most motion sensors and position sensors, which return a multi-dimensional array of sensor values for each SensorEvent, environment sensors return a single sensor value for each data event. TYPE_AMBIENT_TEMPERATURE event.values[0]°C Ambient air temperature. TYPE_LIGHT event.values[0] lx Illuminance. TYPE_PRESSURE event.values[0 hPa or mbar Ambient air pressure. TYPE_RELATIVE_HUMIDITYevent.values[0]% Ambient relative humidity. TYPE_TEMPERATUREevent.values[0]°C Device temperature.