Assistive Technology for Patients with Medical Face Blindness Project 10 Timothy Chia - tchia2 Jeffrey Luan - jrluan2

Introduction Prosopagnosia - A neurological condition which affects the ability to recognize familiar faces As a result, social interactions are adversely affected Workplace Family and Friends Safety

Objective Facilitate social interaction for individuals suffering from prosopagnosia Utilize existing smartphone technology as effectively as possible, while remaining unobtrusive

High Level Modules Camera Processing Wristband Capture and transmit images to the processing module Processing Identify faces in photos User interface Communication hub for other modules Wristband Display names and vibrate in response to information from processing module Button for user to request a recognition cycle

Design

Software Cycle

Project Build

Additional Design Considerations Low cost Familiar user interface Portability Battery Life Development Time Iterations

Modularity Hardware Software Breadboard Arduino Computer instead of Android phone Battery Circuit Independence Software Testing/Training of API on computer Android app functional independence from hardware modules

Stage Targets Recognition and database capabilities on computer Hardware component functionality achieved on breadboard Android app functional without hardware Android app communicating with hardware Android app fully functional with hardware Android app fully functional with PCB hardware

Recognition Design Kairos API Inputs, Outputs, Interface, Capabilities Threshold control Training Calibration: Priority on minimization of false positives

Facial Recognition Requirements False Positive < 5% Training Set: 7 individuals, 15 photos Testing Set: 13 individuals, 1 photo Constraints: Distance = 1.25m Angle = Direct Lighting Conditions = Indoor, all lights on.

Receiver Operating Characteristic Curve

Challenges

Bluetooth Receiver Issue Android Buffer Issue For unknown reasons, bluetooth inputstream consistently crashes after receiving about 15,000 bytes, refusing to accept more bytes from the HC - 05. Diagnosis Attempt transmission of various size arrays from HC-05, also varying the android side handling. Attempted several varieties of continuous reading, buffered reading, blocking reading, parallel thread reading etc. Significance 320 * 240 = 76,800 > 10,000 Conclusion Must settle for 10 kB chunks

Bluetooth Transmitter Issue HC - 05 Speed Issue Serial input below datasheet spec. Maximum of 115,200 baud instead of 1,000,000 baud. Diagnosis AT command rejected. Significance Images received by computer from camera become garbage at this speed. Interrupt Handler Interference Images became garbage after implementing asynchronous command reception from phone module. Can selectively disable interrupts during pixel capture.

Bluetooth Solution Buffer Issue: No memory in PCB layout. Downscaling by 1/16 is 4800 bytes per frame. 4800 < 10,000, but only have ~1000 bytes of memory in microcontroller. 80x60 below API recommended resolution. Speed Issue: Cannot transmit 4800 bytes synchronously either. No time for hardware solution. SPI SRAM ordered, circuit revision drafted in case, code drafted for TIMER1 and SPI pin shuffling.

Pixel Timing Original driver transmitted each byte synchronously. (blocking on UCSR). Low speed means transmission desyncs from PCLK.

Row Timing Use PCLK to capture ¼ pixels in this row. Resynchronize. For our speed, need 2 rising edges. Asynchronously transmit during next 3 rows.

Example Camera Issues: Pixel synch lost due to low speed serial transmission Correct # of HREF blocking waits Row synchronization incorrect: (synch, asynch without HREF, asynch with wrong # HrEF

Hardware Challenges PCB Voltages below specification Diagnosis/Potential Causes Multimeter on regulator circuits without load, and stable 5V input. Voltage regulator damage (possibly due to static discharge or other human error) Conclusion No time to replace due to small form factor. Bypass our power circuitry entirely. Camera circuit not returning data Potential Causes Mismatch between PCB design and breadboard prototype Initially, used breadboard to reorder pins and add pullup resistors Camera eventually went completely dead, even when returned to original circuit; unknown cause, but most likely damage inflicted onto camera during testing

Hardware Challenges (continued) Oversights in PCB design Diagnosis After receiving PCBs, the following errors were noticed Certain traces to microcontroller were wrongly placed USB port placed out of place Conclusion While not an fatal error, took time to troubleshoot Would have corrected provided more time/board submissions

Conclusions Communication between all modules Image capture and recognition Phone App for database management State machine and command transmission for modules 3000 lines of code ready for next iteration of hardware modules

Future Work Size and Enclosure Image Quality Battery Circuit Delays encountered in PCB development Would have selected smaller components and optimized our PCB design to be smaller (for realistic portable usage) Image Quality Because data errors occurred at 320x240, we had to downscale images. Severe loss of accuracy. Battery Circuit Voltage regulator issues prevented battery use Troubleshooting required Replacing regulator(s) Redesigning circuit