1. Towards a Practical Pedestrian Distraction Detection Framework using Wearables
Nisha Vinayaga-Sureshkanth† Anindya Maiti†‡ Murtuza Jadliwala†
Kirsten Crager‡ Jibo He‡ Heena Rathore*
† University of Texas at San Antonio, USA
‡ Wichita State University, USA
* Hiller Measurements, USA
Monday, March 19, 2018

2. Pedestrian Safety
Monday, March 19, 2018

3. One Main Cause - Distraction
Monday, March 19, 2018

4. Pedestrian Safety Tools
Existing tools detect hazardous contexts using customized or inbuilt sensors.
Users’ smartphone camera:
Resource-intensive.
Unreliable if the camera is obstructed.
Users’ smartphone microphone and/or GPS:
Only useful for detecting outdoor traffic-related hazards.
Specialized sensors [A][B][C][D]:
Additional expense.
Non-ubiquitous.
Hincapié-Ramos, Juan David, and Pourang Irani. "CrashAlert: enhancing peripheral alertness for eyes-busy mobile interaction while walking." Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2013.
Ahn, Euijai, and Gerard J. Kim. "Casual video watching during sensor guided navigation." Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry. ACM, 2013.
Wen, Jiaqi, Jiannong Cao, and Xuefeng Liu. "We help you watch your steps: Unobtrusive alertness system for pedestrian mobile phone users." Pervasive Computing and Communications (PerCom), 2015 IEEE International Conference on. IEEE, 2015.
Jain, Shubham, et al. "Lookup: Enabling pedestrian safety services via shoe sensing." Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 2015.
Monday, March 19, 2018

5. Our Approach – Detect Pedestrian Distraction
Detect and prevent inattentiveness among pedestrians.
After all, if pedestrians are not distracted they will be able to easily navigate away from obstacles and other hazards.
The problem of detecting distracted pedestrians can be generalized as a concurrent activity recognition (CAR) problem!
How?
Detect concurrent pedestrian activities of being mobile (e.g., walking, running or climbing/descending stairs) and being distracted (e.g., texting, eating or reading).
Use ubiquitous mobile and wearable devices’ motion sensor data for CAR.
Monday, March 19, 2018

6. Generic Pedestrian Safety System for Mobile Devices
Research Challenges:
Well-known CAR models are resource-intensive, thus unsuitable for usage on mobile and wearable devices.
CAR models have not been previously tested for effectiveness in detecting distracted pedestrian activities.
Monday, March 19, 2018

7. Computational Efficiency
Our Contributions
A new complex activity recognition technique for detecting distracted pedestrian activities, called Dominant Frequency-based Activity Matching (DFAM).
Comparative evaluation of DFAM with well-known complex activity recognition approaches in the literature.
DFAM Goal:
A favorable balance between computational efficiency, detection accuracy, and energy consumption.
Detection Accuracy
Energy Consumption
Computational Efficiency
Monday, March 19, 2018

8. Dominant Frequency-based Activity Matching (DFAM)
Inspired from Shazam’s audio matching algorithm proposed by Avery Wang [E].
Key advantages over traditional activity matching:
Does not use computationally expensive time domain feature matching.
Streamlines multi-source and multi-sensor data fusion.
Wang, Avery. "An Industrial Strength Audio Search Algorithm." Ismir. Vol
Monday, March 19, 2018

9. DFAM Architecture
Monday, March 19, 2018

10. Traditional CAR Classifiers for Comparison
Naive Bayes (NB).
Decision Tree (DT).
Random Forests (RF).
Support Vector Machine (SVM).
k-Nearest Neighbors (k-NN).
Features (taken from the CAR literature):
Mean, minimum, maximum, standard deviation, variance, energy and entropy of discrete FFT components for each axis.
Root mean square.
Mean, median, and maximum of the instantaneous speed (only for the accelerometer data).
Mean, median, and maximum of roll velocity (only for the gyroscope data).
Monday, March 19, 2018

11. Experimental Setup
DFAM and traditional CAR classifiers implemented using Java on:
A 64-bit Debian Linux PC with an Intel Core i5 processor and 8GB RAM.
A Motorola Moto XT1096 Android smartphone paired with a Sony Smartwatch 3 and a LG Urbane W150 smartwatch.
Distracted pedestrian activities evaluated:
Monday, March 19, 2018

12. Experimental Setup
Four different smartphone and smartwatch placement combinations:
41 sets of activity data collected from 23 participants:
Monday, March 19, 2018

13. Results - DFAM Performance
Same-side Sony+Moto placement (LL+RR)
Monday, March 19, 2018

14. Results - DFAM Performance
Different-side Sony+Moto placement (RL+LR)
Monday, March 19, 2018

15. Results - DFAM Performance
Classification accuracy of DFAM for (a) combined
datasets, and (b) individual sensors.
Monday, March 19, 2018

16. Results - DFAM Performance Comparison
Classification accuracy of DFAM compared with
traditional classifiers.
Monday, March 19, 2018

17. Results - DFAM Performance Comparison
Average response time and resource utilization of DFAM compared with traditional classifiers.
DFAM has significantly lower response time, CPU/RAM utilization, and power consumption.
Favorable for continuously running real-time applications.
Minimal performance impact on users’ mobile/wearable device.
Monday, March 19, 2018

18. Future Improvements
Hierarchical CAR model to further reduce resource footprint.
Monday, March 19, 2018

19. Acknowledgement
This work was supported by the United States National Science Foundation(NSF) under the award number
Monday, March 19, 2018