Image Processing Platform Team 04: Meiyi Yang, Hao Wu, Junran Liu, Xiangchen Zhao, Xinhui Liu, Yifan Liu, Vinny DeGenova

Project Description Client: Ripple Goyal & Eder Figueroa Our project: Machine learning algorithms. Fetching images from API like Instagram. User friendly interface. What they already have is a platform. By implementing Natural Language Processing, this platform could fetch data from social media, like Twitter, in real-time, and analyze the data for public safety. For instance, users like DPS officer could be alerted if there is a post containing sensitive topics. A pipeline that could recognize and detect certain topics by using machine learning algorithms. A pipeline that could fetch images from social media like Instagram in real-time. A pipeline that is easy to operate. Users can upload images and choose a certain topic to detect on UI.

Core Capabilities Image recognition Uploading images to be detected Fetching images from API Output images with selected tag Core capabilities: 1. By implementing machine learning algorithms, it builds detector to recognize certain topic on images. 2. Users are allowed to upload images for detection with a certain topic. 3. Users are allowed to choose images fetched from social media like Instagram through API. 4. The system could output images with label attached.

Risks Algorithms Instagram API integration Why? - Determine feasibility of the algorithms - Not sure how many images are needed to train a feasible model - Not sure how does Instagram API work Since using machine learning algorithms to train image recognition model is the most important function of our project, so how to implement such algorithms and how to train detector is the top risk of our project.

Algorithm Risk Personnel Shortfalls Dataset is small For the algorithm part, not all team members have had experiences in working with machine learning, so it is not realistic for us to implement the algorithms from scratch。 Besides, it would be hard for us to use incomplete algorithms. If we do that, it will take much longer time for us to learn and adjust the algorithm. After a several times of discussion, we thought that using a well-built machine learning algorithm should be a feasible choice.

Our Algorithm Part Algorithm: Convolutional Neural Network (deep learning method) Framework: TensorFlow Language: Python

Algorithm: Why Deep Learning? Much better performance in image recognition On CIFAR-10 dataset, CNN : 96.5 %; traditional method: lower than 75%[1] Make features automatically as part of the training Don’t need SIFT features any more. GPU can speed computations many time over On CIFAR-10 dataset, CNN : 96.5 %; traditional method: lower than 75%[1] Don’t need SIFT features any more. c

Why TensorFlow? open sourced under the Apache 2.0 open source license. So we can use it in a commercial product easy-to-use and easy-to-learn It takes only a few lines of command to train a new model true portability Allow us to use our own laptop without need of any special hardware supported by Google, has an excellent prospect This source is under the Apache 2.0 open source license, so we can use it in a commercial product. It takes only a few lines of command to train a new model. Allow us to use our own laptop without need of any special hardware. Don’t need to change any code if we want to train that model faster on GPU. It is supported b Google.

Algorithm Inception (Tensorflow) Pre-trained model have 1000 classes Figure 1: visualization of the model architecture Inception (Tensorflow) Pre-trained model have 1000 classes 5.1% top-5 error rate Figure 2: results from AlexNet

Algorithm Step 1 : Try the pre-trained model Figure 3: Usage with Python APIc

Algorithm Step 2: Retrain the model with our own dataset Prepare our image[2] # bazel build tensorflow/examples/image_retraining:retrain # bazel-bin/tensorflow/examples/image_retraining/retrain --image_dir ~/{own_dataset} Train our dataset Modify the retrain.py Classify the image Modify the classify_image.py

Reduce Algorithm Risk Do distortion (increase the number of image) Use the function: add_input_distortions(flip_left_right, random_crop, random_scale, random_brightness). Modify the parameter in the function.

API Integration Requirement: Risk: Why: “The pipeline shall get the images from the Facebook/Twitter via Facebook/Twitter's API” Risk: Fetch images from API in real-time Why: External API Not sure how to process images in real-time 1. By using API connector provided by the client, we can fetch images in real-time. 2. Problem: How many images? How long does it take? 3. If we get a large amount of images at a time, or the frequency of getting images is too high - run out of memory - unable to process concurrently.

API Integration In order to reduce this risk, we build this user interface and work flow that users can decide when to connect to the API to catch images, And when to stop catching. By clicking the button, users can choose to get images from Facebook or Instagram.

API Integration After fetching a set of images, users can decide when to stop. Then, the images will be uploaded to our system for detection.

API Integration Output will be images with selected tag.

Motivation & User scenario Persona Name: Leonie Hanne Customer Segment: LAX Airport security Age: 28 years old Gender: Male Location: LAX Airport Who are they? LAX Airport security Has been working at LAX airport for 3 years Goals & Pain points: Using metal detector and X-ray devices Inspecting passengers. Detect dangerous and sensitive good Motivation & User scenario Keep real-time situational awareness of events as they occur, factoring in policing goals and priorities.

Motivation & User scenario Persona Name: Philip Exampleton Customer Segment: DPS Staff Age: 35 years old Gender: Male Location: USC campus Motivation & User scenario protect against opportunistic and intelligent criminals. Our technology anticipates the reactions of criminals—and generates optimized real-time security strategies. Who are they? USC’s campus police Has been working at DPS in USC for 6 years Goals & Pain points: force tackles a wide range of duties over the course of a day – and a night. Preventing accident

Thank you End

Reference [1]http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html [2]https://www.tensorflow.org/versions/r0.9/how_tos/image_retraining/index.html#training-on-flowers Figure1:https://github.com/tensorflow/models/tree/master/inception Figure2:https://www.tensorflow.org/versions/r0.10/tutorials/image_recognition/index.html Figure3:https://www.tensorflow.org/versions/r0.10/tutorials/image_recognition/index.html