1. OpenCV for Android
Ke Wang

2. Agenda Computer vision in 5 minutes (instead of 5 years of PhD)
OpenCV in 5 minutes
OpenCV4Android

3. Agenda
Computer vision in 5 minutes
OpenCV in 5 minutes
OpenCV4Android

4. What is Computer Vision?

5. What a person sees

6. What a computer sees

7. Goal of Computer Vision
The goal of computer vision is to give computers visual perception
camera
computer
scene

8. Vision and Graphics? Vision Graphics Simulation Estimation projection
3D geometry
physics
Simulation
projection
3D geometry
physics
Estimation

9. Vision is Hard

10. Illusory motion
Copyright A.Kitaoka 2003

11. Shading

12. Cast shadows

13. CV with Other Fields Robotics Artificial Intelligence Image Processing
Compute Vision
Computational Photography
Computer Graphics
Human Perception
Neuroscience

14. Object Recognition Toshiba Tech IS-910T DataLogic LaneHawk LH4000 2013
2012

15. Automated visual inspection

16. Face detection
Age recognition
Sony Cyber-shot
Smile recognition

17. First-down line

18. Image stitching

19. Robotics

20. Autonomous vehicles

21. Image search

22. Augmented/Virtual reality
Occulus Rift: 2 billion
MagicLeap: .5 billion second stage funding (record)
Sports analytics
Nice explanation of first-down line on

23. Optical character recognition
Digit recognition, AT&T labs
License plate readers

24. Agenda
Computer vision in 5 minutes
OpenCV in 5 minutes
OpenCV4Android

25. What is OpenCV? Open Source Computer Vision Library
C++, C, Python and Java interfaces
Supports Windows, Linux, Mac OS, iOS and Android
Supports GPU with CUDA/OpenCL
Designed for computational efficiency and with a strong focus on real- time applications

26. OpenCV Modules

27. Basic Operations on Images
Image ROI

28. Basic Operations on Images
Padding

29. Arithmetic Operations on Images
Blending

30. Arithmetic Operations on Images
Bitwise operations: AND, OR, NOT, XOR
I want to put OpenCV logo above an image. If I add two images, it will change color. If I blend it, I get an transparent effect. But I want it to be opaque. If it was a rectangular region, I could use ROI as we did in last chapter. But OpenCV logo is a not a rectangular shape. So you can do it with bitwise operations as below:

31. Image Processing: Change Colorspace
RGB to HSV

32. Geometric Transformation of Images
Affine transformation: all parallel lines in the original image will still be parallel in the output image
Need a 2x3 matrix

33. Geometric Transformation of Images
Perspective transformation: straight lines will remain straight even after the transformation
Need a 3x3 matrix

34. Image Thresholding

35. Adaptive Thresholding

36. Image Blurring
Averaging

37. Image Blurring
Gaussian Blur

38. Image Blurring
Median filtering
Can be used for denoising

39. Edge Detection

40. Feature Detection and Matching

41. Pose Estimation

42. Face Detection

43. Agenda
Computer vision in 5 minutes
OpenCV in 5 minutes
OpenCV4Android

44. Two Paradigms of Using OpenCV on Android
Java binding
C++ binding with JNI

45. Step 1: Get OpenCV4Android SDK
Option 1: download pre-built SDK
Option 2: build from source

46. Step 2: Create Android Studio Project

47. Step 2: Create Android Studio Project

48. Step 2: Create Android Studio Project

49. Step 2: Create Android Studio Project

50. Step 3: Import OpenCV4Android SDK
File -> New -> Import Module

51. Step 3: Import OpenCV4Android SDK
Source directory: <OpenCV-android-sdk>\sdk\java

52. Step 3: Import OpenCV4Android SDK

53. Step 3: Import OpenCV4Android SDK
Missing platforms and sync projects?
Modify the OpenCV320Library320 build.gradle
Change the SDK versions to be the same as the app build.gradle
Sync gradle

54. Step 4: Add OpenCV as Dependency
File -> Project Structures -> Modules -> app -> Dependencies

55. Step 4: Add OpenCV as Dependency
Module dependency

56. Step 4: Add OpenCV as Dependency
Add openCV320Library

57. Step 5: Install OpenCV Manager App
OpenCV Manager is a service managing OpenCV libraries
Less memory usage: shared by all apps using OpenCV
Hardware specific optimization
Trusted OpenCV library source
Option 1: install pre-built apk using adb (recommended)
adb install <opencv-sdk>/apk/OpenCV_3.2.0_Manager_3.20_<arch>.apk
Option 2: install from Google Play

58. Usage Model for OpenCV Manager

59. API Workflow

60. Java OpenCV Loader
static boolean initAsync(String Version, Context AppContext, LoaderCallbackInterface Callback)
Loads and initializes OpenCV library using OpenCV Manager
Parameters:
Version – OpenCV Library version, OPENCV_VERSION_3_2_0.
AppContext – application context for connecting to the service.
Callback – object, that implements LoaderCallbackInterface for handling connection status.
Return type: boolean;
Returns:
returns true if initialization of OpenCV starts successfully.

61. Tutorial: Edge Detection on Camera Stream
Setup UI
Initialize OpenCV
Read camera frames
Detect edge on frames

62. Step 0: Request Camera Permission
Declare camera permission in AndroidManifest.XML
<uses-permission android:name="android.permission.CAMERA"/>

63. Step 1: Setup UI Add view in the layout XML file
<org.opencv.android.JavaCameraView
android:layout_width="fill_parent"
android:layout_height="fill_parent"
app:camera_id="any"
app:show_fps="true" />

64. Step 1: Setup UI
Associate camera view just like any other Android UI widget
private CameraBridgeViewBase mCameraView;
mCameraView = (CameraBridgeViewBase)
findViewById(R.id.opencv_java_camera_view);

65. Step 2: Initialize OpenCV
private BaseLoaderCallback mOpenCVLoaderCallback = new BaseLoaderCallback(this) {
@Override
public void onManagerConnected(int status) {
switch (status) {
case LoaderCallbackInterface.SUCCESS:
mCameraView.enableView();
mCameraView.enableFpsMeter();
break;
default:
super.onManagerConnected(status); } };

66. Step 2: Initialize OpenCV
@Override
protected void onResume() {
super.onResume();
Log.i(TAG, "Loading OpenV");
if (!OpenCVLoader.initDebug()) {
Log.i(TAG, "Using OpenCV Manager for initialization.");
if (!OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_2_0,
this, mOpenCVLoaderCallback)) {
Log.e(TAG, "Cannot load OpenCV library."); }
} else {
Log.i(TAG, "Using internal OpenCV libraries.");
mOpenCVLoaderCallback.onManagerConnected(
LoaderCallbackInterface.SUCCESS);

67. Step 2: Initialize OpenCV
@Override
protected void onDestroy() {
super.onDestroy();
if (mCameraView != null) {
mCameraView.disableView();
mCameraView.disableFpsMeter(); }
protected void onPause() {
super.onPause();
if(mCameraView != null) {

68. Step 3: Read Frames from Camera
Implement interface CameraBridgeViewBase.CvCameraViewListener2
@Override public void onCameraViewStarted(int width, int height) { }
@Override public void onCameraViewStopped() { }
@Override public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
return inputFrame.rgba(); }

69. Step 3: Read Frames from Camera
Set the listener for camera view in OnCreate()
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
mCameraView = (CameraBridgeViewBase) findViewById(R.id.opencv_java_camera_view);
mCameraView.setCvCameraViewListener(this); }

70. Canny Edge Detection Step 1: Gaussian kernel to smooth image.
Step 2: Compute gradient on images.
Step 3: Non-maximum suppression
Step 4: Hysteresis:
If pixel gradient higher than high threshold, accept as edge
If pixel gradient lower than low threshold, reject

71. Step 4: Edge Detection private Mat mEdge; @Override
public void onCameraViewStarted(int width, int height) {
mEdge = new Mat(width, height, CvType.CV_8UC1); }
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
Imgproc.Canny(inputFrame.gray(), mEdge, 80, 100);
return mEdge;

72. Resources OpenCV OpenCV4Android SDK Computer vision http://opencv.org/
ge/O4A_SDK.html ml
Computer vision
Computer Vision: Algorithms and Applications by Richard Szeliski

73. Tips
Quiz covered in the slides