1. THE ASSISTIVE SYSTEM SHIFALI KUMAR BISHWO GURUNG JAMES CHOU
SENIOR DESIGN II
FINAL PRESENTATION
THE ASSISTIVE SYSTEM
SHIFALI KUMAR
BISHWO GURUNG
JAMES CHOU

2. Project Goal
To develop an object recognition system which utilizes audio input and output to assist visually impaired individuals.

3. Methods Tested Algorithms: SIFT, SURF, HOG Models: Bag of Features
Classifiers: SVM, Random Forest

4. Methods Abandoned SIFT
Needs to be built from scratch. No native function.
SURF with Bag of Features
Very accurate but slow. (~ 2 minutes for 3 categories)
Randomforest
Easy to tune, requires several trees for more accuracy. 50+ trees for good accuracy but still slow. (~ 2 minutes for 6 categories)

5. HOG with SVM(~15 classes)
Generally,
Works in 30 seconds.
Fast and fairly accurate.
Feature extraction in 10 seconds.
Classifier trained in 20 seconds.
SVM doesn’t require a large dataset.
Doesn’t require too much tuning.

6. User supervised learning

7. Text to Speech in Windows

8. Speech Recognition - Made progress but still requires development.
- Researched several sources and methods to find a way incorporate speech to text.
- Attained the first few steps which will help us later such as playing an audio file and recording voice input.

9. Audio Progress 1

10. Audio Progress 2

11. Accomplishments Local and global feature extraction of the images.
Classifiers built and trained to predict the correct label of the image.
Text-to-speech feature added in both Mac OS X and Windows system.
Machine learning supervised by the user for better predictions.

12. Future Plans
- Finishing speech-to-text feature and having it work reliably.
- Working with classes that result in better time efficiency.
- Getting more accurate predictions from images captured by the camera.
- Adding more query options along with the corresponding results
- Implementing neural network models.

13. Conclusion
We’ve learned a lot about the field of computer vision where none of us have had prior experience in. We observed the performances of various algorithms and classifiers.
We learned a lot about supervised machine learning.
Speech recognition as a means of communication for the user to interact with the system was a more difficult task than anticipated.

14. Questions & Concerns are welcome.
Thank You!
Questions & Concerns are welcome.