0. Invisible Sensing of Vehicle Steering with Smartphones
Dongyao Chen
Kyong-Tak Cho, Sihui Han, Zhizhuo Jin, Kang G. Shin
May 19, 2015

1. Camera + Various CV algorithms … Affordable, easy to use
Camera helps detect lane markers for
Detecting steering maneuvers
Affordable, easy to use +
Various
CV algorithms …
Autonomous vehicles use cameras to detect
Lane markers
Objects on road …
Camera determines the functionalities of
Performance is closely relate to the functionaility 1

2. CarSafe[MobiSys’ 13] app
How’s Camera Doing?
iOnRoad app
Citroën DS5
CarSafe[MobiSys’ 13] app
Performance of camera depends on the environment
Smartphones with camera
Built-in devices also use …
Research works also mentioned …
Our observation is that: the camera-based approaches 2

3. CarSafe[MobiSys’ 13] app
How’s Camera Doing?
iOnRoad app
Citroën DS5
CarSafe[MobiSys’ 13] app
Performance of camera depends on the environment
Lighting
Weather
Pavement
Placement
However, this slides will show most of existing driving assistant systems still overly-rely on the usage of camera. Camera face down, if it cannot see, it definitely cannot work. 2

4. CarSafe[MobiSys’ 13] app
How’s Camera Doing?
iOnRoad app
Citroën DS5 LDW System
CarSafe[MobiSys’ 13] app
How well a camera can see
Bottleneck of application’s performance
Real road condition
Lighting
Weather
Pavement
Placement
Performance of driving assistant systems is undermined by a fundamental limitation of camera, which is. 2

5. Key Idea: Camera-free Sensing of Vehicle Steering
Reliability
-Resistant to changing environments
Accuracy
-Visual similarity will not effect the detection results
Lightweight
-Lower computational consumption
The challenge is to find the right signature
Which can directly gives us a boost of reliability, accuracy, and lightweight.
“the right signature”: what we detect, how we detect 3

6. Differentiates maneuvers
V-Sense
Utilizes gyroscope
Differentiates maneuvers
Base on this idea, we propose V-Sense.
V-Sense utilizes… To increase the accuracy… In the end…
Works as a middleware 4

7. Sensing Steering with Gyroscope
In left turns
In lane changes
Gyroscope in the smartphone
Here is a logic jump 5

8. Understanding Vehicle Steering
In left & right turns
Detect bumps in gyroscope reading
How to detect bumps?
How to relate bump detection to different steering maneuvers?
Lane changes
Lane changes
Gyroscope in the smartphone
Here is a logic jump 5

9. Bump Detection Three-state algorithm:
No bump → One bump→ Waiting for bump
To determine if the following bump is source from the same maneuver, we detect the delay of next bump.
T_bump = 1.5s;
T_next_delay=3;
Delta_s = 0.05;
Delta_h = 0.07
----- 会议笔记(4/1/15 16:11) -----
Peak detection. Spend more time on the analytical things.
Show that the result is analytical reliable.
----- 会议笔记(4/28/15 11:34) -----
Dont need hyphend 6

10. Increase Accuracy: Steering & Curvy Road
Lane change
S-shape road
Turn
L-shape road
We also need to increase the accuracy, because, the driving trajectory of steering and on curvy road could be similar.
Horizontal displacement:
WCURVY >WLANE , WCURVY >WTURN
Integrate horizontal displacement at each time step 7

11. Calculate Horizontal DisplacementW4 W3 W2 W1 Ts
2Ts
3Ts
4Ts
Velocity v1 v2 v3 v4 θ1 θ2 θ3 θ4
Time Ts
2Ts
3Ts
4Ts
Velocity v1 v2 v3 v4 θ1 θ2 θ3 θ4
Time Ts
2Ts
3Ts
4Ts
Velocity v1 v2 v3 v4
Time
The whole lane change process can be separated into multiple time slots;
At each time slot, combine the changing angle with the previous result; We
Figure: Horizontal displacement of lane change 8

12. Detect Horizontal Displacement on Curvy Road
Let’s see the performance of this algorithm in the real world 9
Route image adapted from Google map:

13. Horizontal Displacement on Curvy Road
Show the driving direction 10

14. Horizontal Displacement on Curvy Road
Show the driving direction 10

15. Evaluation Setups Tested V-Sense on Samsung Galaxy S4 and S3
5 volunteers, 2 cars
40+h test 11

16. Reliability and Accuracy
Resilient to different placements:
Success Rate
----- 会议笔记(4/1/15 16:11) -----
1. what could happen if misclassification happens.
----- 会议笔记(4/28/15 10:12) -----
success ratio for correctly detect the maneuvers 12

17. Computational Costs 13 ----- 会议笔记(4/1/15 15:57) -----
show bars. S4 is even better than S5. 13

18. Comparison with Other Applications
~1,000,000
installs
~50,000
installs
~50,000
installs
110
ratings
Success Rate
----- 会议笔记(4/1/15 15:57) -----
Divde with ground truth.
----- 会议笔记(4/1/15 16:00) -----
parameters are different from previous slides
----- 会议笔记(4/1/15 16:11) -----
mention the evaluation condition. Specify the evaluation scenario. Specify highway, and local road. If you only drive on highway, the performance could be
Success rate
Caption: compare of lane change detection
Figure: Success rate of detection of lane change in urban area 14

19. When Steering your Car... 16 Before we show the first application.
Image:

20. Application I: Detection of Careless Steering
----- 会议笔记(4/1/15 16:11) -----
1. Specify the detail;
2. Change to careless. 17
Car icon:

21. Turn Signal Detection
Matched filter results > ground truth? Explain
SAME Size!!!
----- 会议笔记(4/28/15 11:29) -----
Should have captions for all figures
Use real image
Add CAPTIONS! 18

22. Application I: Detection of Careless Steering！
----- 会议笔记(4/1/15 16:11) -----
1. Specify the detail;
2. Change to careless. 19

23. Application I: Detection of Careless Steering
----- 会议笔记(4/1/15 16:11) -----
1. Specify the detail;
2. Change to careless. 19

24. Application II: Fine-grained Lane Guidance
GPS is unstable and its accuracy varies with environment
GPS Accuracy 20

25. Application II: Fine-grained Lane Guidance
Lane tracking at intersection:
Left
to Right
Right to
Left
----- 会议笔记(4/28/15 11:50) -----
ADAPTIVE TO DIFFEREWNT CASES 21

26. Conclusion V Two proof-of-concept apps
A reliable, accurate and light weighted middleware for detecting steering maneuvers
Two proof-of-concept apps
Careless steering warning
Fine-grained lane guidance
----- 会议笔记(4/1/15 15:57) -----
ssss! 22

27. Q&A Thank you! 23 ----- 会议笔记(4/1/15 15:57) -----
include . 23