1. Pedro Rivera Torres Ammar Almani Srrah Algheithy Thad Guriel Andy Duann Hao Chen

2.  PDR Revision  Functional decomposition (hardware)  Hardware subcomponents/implementation  Functional decomposition (software)  Software subcomponents/implementation  Current project status  Goals (milestone 1, 2, expo)  Technicalities (schedule, budget) Thad Guriel

3.  Sam Siewart  Google IO interface  Jim Pilkington  Assistive Technology Specialist  Colorado Department of Human Services  Division of Vocational Rehabilitation  Catherine Kilcommons  Assistive technology lab coordinator  Disability services - University of Colorado Boulder Thad Guriel

4. Android Keypad CPU RF Receiver IR Receiver RF Comm IR Navigation RF Comm IR Navigation User Info Maps Info Audio Output User Thad Guriel

7. HardwareDeviceNavigationIR ReceiverCommunicationTransmitterReceiverMicroprocessorAudioTTS 256Phone analogDoorNavigationIR TransmitterCommunication Transmitter Receiver Thad Guriel

8. Is there a signal? Query signal sent from device Xbee Xbee checks for signal Xbee establishes communication with device Xbee sends interrupt signal to processor No Yes Andy Duann

9. Xbees establish communication Processor wakes up Processor kicks out sign data Processor turns on IR Xbee cyclic sleep Send data Talk to device XBee Turn off IR Start clock Processor goes to sleep Andy Duann

11. Cyclic sleep Cycle wake up Xbee establishes communication Interrupt from processor Send signal to wake door XBee Transmit data Was there a signal? YesNo Andy Duann

13. No audio signal uP outputs logic low to switches Turns on Android switch Turns off synthesizer switch Andy Duann

15. Phone audio Comparator outputs logic high uP turns off synthesizer interrupts Outputs logic low to switches Comparator goes low when phone audio gone Andy Duann

17. uP sends interrupt enable to synthesizer uP sets logic high to switches Turns android off Turns synthesizer on Synthesizer audio Synthesizer sends interrupt request when data present uP checks android interrupt Is it low? NoYes Andy Duann

22.  Locate building entrance after hand-off from GPS  Read signs from a distance and determine its relative direction Hao Chen

23. GPS tells user they are in the vicinity of Engineering entrance, and “Engineering Building; to your northeast”, for example Device tells user “Engineering lobby; to your north” and “elevators; to your West”. Hao Chen

24.  3 infrared LED’s are mounted on the entrance, to send directional information at a wide angle  The information of the building is transmitted with XBee Hao Chen

25.  5 IR receivers are mounted on the device  IR receivers are sensitive to 950 nm wavelengths and 38 kHz carrier only  The direction of the entrance can be determined based on which IR receiver(s) received the signal Hao Chen

26.  In the above picture, receiver 4 picks up the signal while the rest don’t, therefore the microprocessor determines the sign is at northeast direction Hao Chen Reader 1 2 3 5 4

27.  Motivation: battery runs out quickly if IR LED’s are on all the time  Solution: only turn transmitter on when user close  Device uses XBee to trigger transmitter  The transmitter turns off after set amount of time Hao Chen

28. User approaches entrance, transmitter turns on XBee transmits data to device Device determines direction of entrance Device outputs information and direction through speaker Sign automatically turns off after set time Hao Chen

29. Srrah Algheithy

30. Level 3 Level 2 Level 1 Level 0 SoftwareAndroidAndroid app GPS/location information Directional information Android ADKPIC24FGB110Keypad USB initialization Xbee information Android to ADK USB interface Srrah Algheithy

31. Android app to access GPS and Compass Android phone 2.3.4 gingerbread USB interface Android ADK

32. Continuously given information about surroundings Building names Street names Current location continuously updated Navigate to points of interest Bus stops Srrah Algheithy

33. GPS App Android Location Manager Location Listener Proximity Alert Geocoding

34.  requestLocationUpdates()  Location provider  Time between updates  Distance between updates  LocationListener()  ItemizedOverlay  OverlayItem() ▪ Specify and name points of interest  Use built in compass for direction Srrah Algheithy

35.  Communicate effectively with the keypad system.  Coordinate Keypad output with Android apps.  Control internal functionalities of the Android phone  Manage operation and information coming form Xbee  Coordinate Xbee info to turn on IR sensors.  Manage the audio system. Ammar Almani

36.  Using PIC24F j256 GB110  Features:  16-bit: similar to DSP  Still low power (3.3V)  Programming Memory: 256 KB  Full Speed USB operation in Device  USB on-The-Go  3 ADCs: 500K conversion / second. Ammar Almani

37. Main()AudioAndroid/GPSSensorsXbee infoKeypad Voice Recognition (Android) Bluetooth Connection Ammar Almani

38.  Coordination Module  Using the keypad encoder to generate interrupts when a button is pressed. ModuleKeypad_IR() Inputs-An interrupt signal from chip - 4 binary digits Outputs- The appropriate command to both Android phone and Xbee FunctionalityTo coordinate between user and peripherals. Test ProcedureTest to control android app through PIC24F android chip Ammar Almani

39.  Input/output module Ammar Almani Modulegps_Info() Inputs- Destination specified by user through keypad Outputs- Control audio system to issue guidance commands FunctionalityTakes destination as input uses GPS in Android to output guidance voice commands Test ProcedureMore than one

40.  Composite module Ammar Almani Modulehandle_sensors() Inputs-Receive signal from GPS to turn on sensors -If analog needed use ADCs Outputs- Output a specific direction for audio system FunctionalityTakes IR information from five sensors and use transmitted data to perceive the sign’s location. Test Procedure- Once HW ready, PIC24 kit is used to test.

41.  Input Module Pedro Rivera Modulereceive_xbee() Inputs-Wake up xbee signal from GPS Outputs-information in buffer to the audio function FunctionalityTakes data from xbee, store it in a buffer, make sure all data is transmitted; and then send it to the audio function. Test Procedure- Use UART utility to test function in 2 levels

42.  A system that outputs audio signals from both Android device and text-to-speech chip Pedro Rivera Moduleoutput_audio() Inputs-Synthesizer interrupt request -Android Interrupt Outputs-Logical output to analog audio switches FunctionalityTakes both data from xbee and main and an audio interrupts form android; then decides the priority and behave accordingly. Test Procedure- More than one test

43. Kyepad_I R gps_infoAudioBluetoothVoice_recognitionXbee_receiveSensorsExtra sensors Pedro Rivera

44.  Requirements for Open Accessory Hardware  Making PIC the host ▪ Being host entails: -Initiating communication -Knowing what all devices are doing Pedro Rivera

45.  Open accessory Handshake (When accessory is connected, the accessory will perform these steps):  Send “Get Protocol” command  Send device info to identify the accessory to the Android Device  Send “Start” Command  The Android device should re-enumerate in accessory mode and launch an app Pedro Rivera

46.  USB Accessory Handshake, part 2 Accessory mode if: Vendor ID is 0x18D1 (Google) Product ID is 0x2D00 or 0x2D01 Read configuration descriptors Look for first bulk IN and first bulk OUT endpoints Set configuration to 1 Endpoints are now ready for communication Pedro Rivera

48. Thad Guriel HardwareSoftware Audio circuit built & testing in progress Android app for location written (testing in progress) Keypad circuit built & testedADC driver working for up to 80KHz IR built & testing in progressInterrupt is pending XBee testing in progressUART driver pending

49. Thad Guriel HardwareSoftware Successfully implements the “wake up” of signs when user is near App provides current location, destination, arrival, direction Successfully generates 38 kHz modulated infrared signal and receives it at a 5 meter distance USB driver working Successfully limits the range of XBEE to 10 meters and able to transmit information from one XBEE to the other. Keypad, press a button on keypad and it controls something on the android PCB is designed and orderedXbee receives signal, gets information and saves it (doesn’t have to be correct) UART driver working

50. Thad Guriel HardwareSoftware 3 IR LEDs are used to complete the signsInterfacing between keypad and GPS All 5 IR receivers are mounted on the device & successfully determines the direction of the sign App gives directions to destination Received XBEE sign information is outputted through speaker Xbee functioning PCB revision builtAudio under testing

51.  All chips are mounted on PCB, device is fully functioning.  The project is demoed through video recording. Thad Guriel

52. TaskThadHaoSrrahAndyPedroAmmar GPS AppPrimarySecondary USB interfaceSecondaryPrimarySecondary MicroprocessorSecondary Primary CommunicationPrimarySecondary Primary IRSecondaryPrimarySecondaryPrimary PowerSecondary Primary KeypadSecondary PrimarySecondaryPrimary AudioPrimarySecondary Primary PCBPrimary SecondaryPrimarySecondary Hardware ----- Software -----

53. Thad Guriel PartCost PIC test board$60 XBee$50 Speakjet$24.95 TTS 256$21.95 Audio Amp$3 Keypad encoder$10 Voltage regulator$2 Flip flop$2 IR$10 Arduino$30 Headphone jack$5 Total-$218.90 UROP funding$1000 Remaining budget$781.1

54.  Wall e (slide 5) - http://www.robotbooks.com/robot_toys_II.htm  User (slide ) - http://platinumjellybean.tumblr.com/post/1005345702/this-is-a-stick-man  Device for visually impaired (slide 6) - http://www.humanware.ca/web/en/newsletter/240.htm  Microchip (slide 36) - http://www.edn.com/article/520757- Microchip_targets_active_current_reduction_in_PIC24F_expansion.php  Android logo (slide 33 & 34) – http://blogs.vertigo.com/personal/ericl/Blog/Lists/Photos/Forms/DispForm.aspx?ID=12  Android logo (slide 32) – http://www.digitaltrends.com/mobile/do-developers-prefer-apples-or-androids/  Gingerbread Android logo (slide 31) – http://www.vpsems.com/android-gingerbread.html  Visually impaired navigation (slide 22) - http://www.teu.ac.jp/hashino/research/index.html  Clip art (slide 27, 26, 24,23 & 56) – Microsoft PowerPoint clipart  Android/USB – http://www.everythingusb.com/android-open-accessory-kit-21147.html Thad Guriel

55. Andy Duan

56. Thad Guriel