CIS 470 Mobile App Development Lecture 20 Wenbing Zhao Department of Electrical Engineering and Computer Science Cleveland State University wenbing@ieee.org 11/8/2018 CIS 470: Mobile App Development

Google Mobile Vision API Barcode Reader 11/8/2018 CIS 470: Mobile App Development

CIS 470: Mobile App Development Barcode Reader <?xml version="1.0" encoding="utf-8"?> <manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.wenbing.barcodereader"> <uses-feature android:name="android.hardware.camera" /> <uses-permission android:name="android.permission.CAMERA" /> <application android:allowBackup="true" android:icon="@mipmap/ic_launcher" android:label="@string/app_name" android:supportsRtl="true" android:theme="@style/AppTheme"> <meta-data android:name="com.google.android.gms.version" android:value="@integer/google_play_services_version" /> <meta-data android:name="com.google.android.gms.vision.DEPENDENCIES" android:value="barcode" /> <activity android:name=".MainActivity" android:label="@string/title_activity_main" > <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LAUNCHER" /> </intent-filter> </activity> <activity android:name=".BarcodeCaptureActivity" android:label="Read Barcode"/> </application> </manifest> Create a new app and name it BarcodeReader Modify manifest 11/8/2018 CIS 470: Mobile App Development

CIS 470: Mobile App Development Barcode Reader apply plugin: 'com.android.application' android { compileSdkVersion 24 buildToolsVersion "25.0.0" defaultConfig { applicationId "com.wenbing.barcodereader" minSdkVersion 9 targetSdkVersion 24 versionCode 1 versionName "1.0" testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner" } buildTypes { release { minifyEnabled false proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro' } } } dependencies { compile fileTree(dir: 'libs', include: ['*.jar']) compile 'com.android.support.constraint:constraint-layout:1.0.2' compile 'com.google.android.gms:play-services-vision:9.4.0+' compile 'com.android.support:design:24.2.0' compile 'com.android.support:support-v4:24.2.0' testCompile 'junit:junit:4.12' androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', { exclude group: 'com.android.support', module: 'support-annotations' }) } Modify build.gradle (Module: app) 11/8/2018 CIS 470: Mobile App Development

CIS 470: Mobile App Development Barcode Reader <?xml version="1.0" encoding="utf-8"?> <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" android:paddingLeft="@dimen/activity_horizontal_margin" android:paddingRight="@dimen/activity_horizontal_margin" android:paddingTop="@dimen/activity_vertical_margin" android:paddingBottom="@dimen/activity_vertical_margin" tools:context="com.wenbing.barcodereader.MainActivity"> <TextView android:layout_width="match_parent" android:layout_height="wrap_content" android:textAppearance="?android:attr/textAppearanceLarge" android:text="@string/barcode_header" android:id="@+id/status_message" android:layout_alignParentRight="true" android:layout_alignParentEnd="true" android:layout_centerHorizontal="true" /> <TextView android:layout_width="wrap_content" android:layout_height="wrap_content" android:textAppearance="?android:attr/textAppearanceLarge" android:id="@+id/barcode_value" android:layout_below="@+id/status_message" android:layout_alignParentLeft="true" android:layout_alignParentStart="true" android:layout_marginTop="110dp" android:layout_alignRight="@+id/status_message" android:layout_alignEnd="@+id/status_message" /> Add 7 Java classes: CameraSource, CameraSourcePreview, GraphicOverlay, BarcodeCaptureActivity, BarcodeGraphic, BarcodeGraphicTracker, BarcodeTrackerFactory. Populate all classes with the java files posted on the web Modify activity_main.xml layout: 11/8/2018 CIS 470: Mobile App Development

CIS 470: Mobile App Development Barcode Reader <Button android:layout_width="wrap_content" android:layout_height="wrap_content" android:text="@string/read_barcode" android:id="@+id/read_barcode" android:layout_alignParentBottom="true" android:layout_centerHorizontal="true" /> <CheckBox android:layout_width="wrap_content" android:layout_height="wrap_content" android:text="@string/auto_focus" android:id="@+id/auto_focus" android:layout_below="@+id/barcode_value" android:layout_alignParentLeft="true" android:layout_alignParentStart="true" android:layout_marginTop="66dp" android:checked="false" /> <CheckBox android:layout_width="wrap_content" android:layout_height="wrap_content" android:text="@string/use_flash" android:id="@+id/use_flash" android:layout_alignTop="@+id/auto_focus" android:layout_alignParentRight="true" android:layout_alignParentEnd="true" android:checked="false" /> </RelativeLayout> Modify activity_main.xml layout: 11/8/2018 CIS 470: Mobile App Development

CIS 470: Mobile App Development Barcode Reader <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:id="@+id/topLayout" android:orientation="vertical" android:layout_width="match_parent" android:layout_height="match_parent" android:keepScreenOn="true"> <com.wenbing.barcodereader.CameraSourcePreview android:id="@+id/preview" android:layout_width="match_parent" android:layout_height="match_parent"> <com.wenbing.barcodereader.GraphicOverlay android:id="@+id/graphicOverlay" android:layout_width="match_parent" android:layout_height="match_parent" /> </com.wenbing.barcodereader.CameraSourcePreview> </LinearLayout> Add another layout resource named barcode_capture.xml 11/8/2018 CIS 470: Mobile App Development

CIS 470: Mobile App Development Barcode Reader <resources> <!-- Default screen margins, per the Android Design guidelines. --> <dimen name="activity_horizontal_margin">16dp</dimen> <dimen name="activity_vertical_margin">16dp</dimen> </resources> Add dimens.xml in values resource Modify strings.xml <resources> <string name="app_name">BarcodeReader</string> <string name="ok">OK</string> <string name="permission_camera_rationale">Access to the camera is needed for detection</string> <string name="no_camera_permission">This application cannot run because it does not have the camera permission. The application will now exit.</string> <string name="low_storage_error">Face detector dependencies cannot be downloaded due to low device storage</string> <string name="title_activity_main">Barcode Reader Sample</string> <string name="barcode_header">Click "Read Barcode" to read a barcode</string> <string name="read_barcode">Read Barcode</string> <string name="auto_focus">Auto Focus</string> <string name="use_flash">Use Flash</string> <string name="barcode_success">Barcode read successfully</string> <string name="barcode_failure">No barcode captured</string> <string name="barcode_error">"Error reading barcode: %1$s"</string> </resources> 11/8/2018 CIS 470: Mobile App Development

Google Mobile Vision Barcode Detection https://codelabs.developers.google.com/codelabs/bar-codes/#0 Google Play services 7.8 added Mobile Vision barcode detection APIs Classes for detecting and parsing bar codes are available in the com.google.android.gms.vision.barcode namespace The BarcodeDetector class is the main workhorse -- processing Frame objects to return a SparseArray<Barcode> types In the case of 1D barcode such as UPC codes, this will simply be the number that is encoded in the bar code. This is available in the rawValue property, with the detected encoding type set in the format field Frame frame = new Frame.Builder().setBitmap(myBitmap).build(); SparseArray<Barcode> barcodes = detector.detect(frame); Barcode thisCode = barcodes.valueAt(0); TextView txtView = (TextView) findViewById(R.id.txtContent); txtView.setText(thisCode.rawValue); 11/8/2018 CIS 470: Mobile App Development

Google Mobile Vision Barcode Detection For 2D bar codes that contain structured data, such as QR codes -- the valueFormat field is set to the detected value type, and the corresponding data field is set For example, if the URL type is detected, the constant URL will be loaded into the valueFormat, and the Barcode.UrlBookmark will contain the URL value Beyond URLs, there are lots of different data types that the QR code can support Barcode detection works in all orientations, code parsing is done locally 11/8/2018 CIS 470: Mobile App Development

Google Mobile Vision Barcode Detection https://developers.google.com/vision/android/multi-tracker-tutorial The barcode detector detects barcodes and creates a collection of barcode instances A multi-processor instance keeps track of each barcode that is currently active. It uses a factory to create a new graphic tracker instance per barcode As barcodes are tracked across video frames, the multi-processor sends updates to the corresponding barcode tracker instances // A barcode detector is created to track barcodes. An associated multi-processor instance // is set to receive the barcode detection results, track the barcodes, and maintain // graphics for each barcode on screen. The factory is used by the multi-processor to // create a separate tracker instance for each barcode. BarcodeDetector barcodeDetector = new BarcodeDetector.Builder(context).build(); BarcodeTrackerFactory barcodeFactory = new BarcodeTrackerFactory(mGraphicOverlay, this); barcodeDetector.setProcessor(new MultiProcessor.Builder<>(barcodeFactory).build()); 11/8/2018 CIS 470: Mobile App Development

Google Mobile Vision Barcode Detection class BarcodeTrackerFactory implements MultiProcessor.Factory<Barcode> { private GraphicOverlay<BarcodeGraphic> mGraphicOverlay; private Context mContext; public BarcodeTrackerFactory(GraphicOverlay<BarcodeGraphic> mGraphicOverlay, Context mContext) { this.mGraphicOverlay = mGraphicOverlay; this.mContext = mContext; } @Override public Tracker<Barcode> create(Barcode barcode) { BarcodeGraphic graphic = new BarcodeGraphic(mGraphicOverlay); return new BarcodeGraphicTracker(mGraphicOverlay, graphic, mContext); } } 11/8/2018 CIS 470: Mobile App Development

CIS 470: Mobile App Development Exercise Assume that the barcode you scanned is a 2d barcode that contains a URL Add another activity or fragment to display the URL content (i.e., the webpage) Add a button to launch the activity or fragment 11/8/2018 CIS 470: Mobile App Development