CAT: High-PreCision Acoustic Motion Tracking Wenguang Mao, Jian He, Lili Qiu UT Austin MobiCom 2016

Why motion tracking? Motion-based Games Virtual Reality

Support motion-based interaction Why motion tracking? Smart Appliance Support motion-based interaction

Possible solutions Vision based approach Needs extra hardware Depends on lighting condition Computationally heavy

Possible solutions RF based approach WiFi : limited accuracy (e.g., 10 cm [Chronos16]) RFID: limited accuracy (e.g., 4 cm [RF-Idraw]) 60 GHz waves: extra hardware not widely available 60GHz Antenna

Acoustic Signal Slow propagation – helpful to achieve high accuracy Easily available speakers and mics – widely available Low sampling rate – feasible for SW processing

CAT 𝒗𝒆𝒍 𝒅𝒊𝒔𝒕 CAT

Optimization Framework Key Components CAT Distributed FMCW Distance Optimization Framework Doppler Shift Audio Samples Velocity Movement Trajectory

FMCW 𝑓 𝑡 𝑑 FMCW for propagation delay estimation Less bandwidth usage than using a sharp pulse Send a chirp whose freq. changes linearly over time Estimate the frequency difference 𝑓 𝑡 𝑑 =𝑘𝑓, and 𝑡 𝑑 ~ distance travelled by the chirp 𝑓 𝑡 𝑑

Distributed FMCW Speaker (sender) and microphone (receiver): Not known when the chirp is sent Two-step distance estimation Sampling rate offset Drift compensation Time Frequency Transmitted Received

Two-Step Distance Estimation Decompose distance 𝑅 𝑛 into two parts Pseudo-transmission time 𝑅 𝑛 = 𝑅 𝑛 − 𝑅 1 + 𝑅 1 Reference point

Two-Step Distance Estimation Decompose distance 𝑅 𝑛 into two parts Pseudo-transmission time 𝑅 𝑛 = 𝑅 𝑛 − 𝑅 1 + 𝑅 1 Reference point

Pseudo-Transmission Time 𝑅 1 𝑅 𝑛 𝑅 𝑛 − 𝑅 1 ~ 𝑓 𝑛 − 𝑓 1 Time Frequency Transmitted Received 𝑓 1 𝑓 𝑛 Pseudo-Transmitted

Two-Step Distance Estimation Decompose distance 𝑅 𝑛 into two parts Pseudo-transmission time 𝑅 𝑛 = 𝑅 𝑛 − 𝑅 1 + 𝑅 1 Reference point

Reference Point 𝐷 1 2 − 𝐷 2 2 = 𝐴 2 𝐷 1 − 𝐷 2 ~ ( 𝑓 1 − 𝑓 2 ) Doppler + Doppler - 𝐷 1 2 − 𝐷 2 2 = 𝐴 2 𝐷 1 − 𝐷 2 ~ ( 𝑓 1 − 𝑓 2 )

Drift Compensation Estimated distance drift over time

1764 samples at the receiver Drift Compensation Due to imperfect clocks, the sender and the receiver have different the sampling rates E.g., 44100.1 Hz (sender), 44099.9 Hz (receiver) 1764 samples at the receiver 1764 samples at the sender Prop. delay Chirp 1 Prop. Delay Chirp 2 Chirp diff.

Drift Compensation

Drift Compensation

Doppler Shift Measurement Measure frequency shift 𝑭 𝑺 between transmitted and received signals Velocity is given by 𝐅 𝐒 𝒗=𝒄 𝑭 𝒔

Optimization framework No error accumulation Smooth the estimated results Fusing distance and velocity measurements Find position 𝒛 that fits the measurements best Efficient algorithm for solving it Incorporate IMU measurements 𝒊 𝒋 𝜶 𝒛 𝒊 − 𝒄 𝒋 − 𝒛 𝟎 − 𝒄 𝒋 − 𝒅 𝑭𝑴𝑪𝑾 𝒊,𝒋 𝟐 + 𝒊 𝒋 𝜷( 𝒛 𝒊+𝟏 − 𝒄 𝒋 − 𝒛 𝒊 − 𝒄 𝒋 − 𝒗 𝒅𝒐𝒑𝒑𝒍𝒆𝒓 𝒊,𝒋 ⋅𝑻) Dist. measurement fitting error Vel. measurement fitting error Multiple tracking periods

Experiments 2D tracking with 2 speakers 2D tracking with 3 speakers

2D Tracking Accuracy 2 cm 8 cm 6mm (𝜶,𝜷) 𝒊 𝒋 𝜶 𝒛 𝒊 − 𝒄 𝒋 − 𝒛 𝟎 − 𝒄 𝒋 − 𝒅 𝑭𝑴𝑪𝑾 𝒊,𝒋 𝟐 + 𝒊 𝒋 𝜷( 𝒛 𝒊+𝟏 − 𝒄 𝒋 − 𝒛 𝒊 − 𝒄 𝒋 − 𝒗 𝒅𝒐𝒑𝒑𝒍𝒆𝒓 𝒊,𝒋 ⋅𝑻) CAT is accurate and fusing distance/velocity significantly improves the performance

3D Tracking 8-9 mm 3D tracking error

4mm trace error  easy to use User Study Red: reference Blue: traced by users 4mm trace error  easy to use (a) CAT (b) AAMouse (Doppler only)

Conclusion Distributed FMCW to support a separate sender and receiver Optimization framework and algorithm to fuse distance and velocity over time CAT tracking system Achieves mm-level accuracy on commodity devices Future work: develop new applications