1. WiFinger: Talk to Your Smart Devices with Finger-grained Gesture
Hong Li, Wei Yang, Jiangxin Wang, Yang Xu, Liusheng Huang
University of Science and Technology of China

2. Gesture Recognition SoundWave LeapMotion ArmTrack
SoundWave: using the Doppler effect to sense gestures.
I am a smartwatch and I can track my user’s arm.
ArmTrack

3. WiFi signal is ubiquitous
People can access to WiFi signals almost everywhere.
WiFi can be effected by surrounding human activities.

4. Channel State Information(CSI)
Receiver
Sender
Channel H x y
𝑦=𝐻×𝑥+𝑛

5. Channel State Information(CSI)
CSI is an estimation of H Human Gesture and other movements can effect the propagation of multipath signals. H includes gesture information

6. What can we do with CSI?
WiFall

7. What can we do with CSI?
WiKey

8. Finger gesture recognition

9. Challenges
How to detect and capture the subtle signal changes caused by micro- movements?
How to extract distinguishable features?
How to classify these features of different finger movements?

10. WiFinger Overview
动态时间规整
离散小波变换

11. CSI Capture
Each packet corresponds to a CSI matrix H has a size of 30*1
CSI stream 𝐶=[ 𝐻 𝑡 1 , 𝐻 𝑡 2 ,⋯, 𝐻 𝑡 𝐿 ]

12. Preprocessing 带权重的滑动平均 Outlier Removal [𝜇−𝛾×𝜎, 𝜇+𝛾×𝜎]
𝜇是中值，𝜎是绝对中位差=𝑚𝑒𝑑𝑖𝑎𝑛( 𝑥 𝑖 −𝑚𝑒𝑑𝑖𝑎𝑛(𝑥)),𝛾是3
m=30，历史数据相关性
Outlier Removal
[𝜇−𝛾×𝜎, 𝜇+𝛾×𝜎]
Low-pass Filtering
Cut-off frequency: 60Hz
Weighted Moving Average

13. How to extract the gestures?
Find a sign indicator of CSI stream
Set a threshold of the sign indicator to detect the starting an finishing points

14. Gesture Extraction The sign indicator is defined as 𝔼 ℎ 2 2 / 𝛿 𝑞 2
Remove the DC component
Cut the CSI stream with a sliding window
Calculate the correlation matrix
The sign indicator is defined as 𝔼 ℎ 2 2 / 𝛿 𝑞 2
ℎ 2 is the principal component,
𝑞 2 is the second eigenvector
𝛿 𝑞 2 = 1 𝑁 𝑐 −1 𝑙=2 𝑁 𝑐 | 𝑞 2 𝑙 − 𝑞 2 (𝑙−1)|

15. Threshold of sign indicator
Guard interval 𝑇 𝑏
Smooth the amplitude with median filter
Select the value of the third quartile of the sign indicator as the threshold

16. Feature Extraction Finger gesture profile P (30 * L)
Combine P to Feature vector F (5 * L)
F=[ 𝑓 1~6 , 𝑓 7~12 , 𝑓 13~18 , 𝑓 19~24 , 𝑓 25~30 ]
𝑓 𝑘~𝑙 = 1 𝑙−𝑘+1 𝑖=𝑘 𝑙 𝑝 𝑖
Compress F with DWT(Discrete Wavelet Transformation)

17. Discrete Wavelet Transformation
低通滤波器，高通滤波器，降采样滤波器

18. Discrete Wavelet Transformation
Decompose the signal into a coarse approximation coefficients and detail coefficients
低通滤波器，高通滤波器，降采样滤波器
Compress the original signal while preserving both time and frequency domain information

19. Classification（kNN classifier）
The distance is calculated by DTW(Dynamic Time Warping)
动态时间规整

20. Implementation & Evalutaion
1 directional TX antenna and 3 omni-directional RX antenna

21. Data Collection
10 users perform ASL gesture No.1 to No.9

22. Automatic Gesture Extraction Accuracy
User 2 and user 7 separately have average gesture detect ratio of 76% and 81%. They cannot fully stretch their fingers when performing 3 and 9.

23. Gesture Recognition
The average recognition accuracies of gesture 3 and 9 are relatively lower. The two gesture and hard to perform.

24. Gesture Recognition
The recognition accuracy of user 5 is around 80%

25. Number Text Input Using WiFinger
WiFinger achieves average recognition accuracy of 82.67%

26. CSI patterns for different gestures
两把椅子和一个柜子

27. Devices Positioning
As the distance between transceivers increasing, the patterns nearly disappeared.

28. Frequency Band

29. Summary
Design a scheme enables WiFi signals to realize continuously number text input by recognizing finger-grained gestures from ASL.
Achieve high recognition accuracy with an average recognition accuracy 90.4% per user.
It is a non-intrusive and device-free solution to finger-grained gestures recognition.