MART: Music Assisted Running Trainer 張智星 CSIE Dept, National Taiwan University

Introduction Objective Challenges Develop a phone app that can adjust the music tempo according the runner’s step frequency and timing. Challenges How to find the runner’s step frequency correctly via tri-accelerometer and gyroscope only? How to find the step timing? How to adjust the tempo of music?

Proposed Methods – System Overview Tri-Accelerometer : ax , ay , az Gyroscope : yaw , roll , pitch Angular Velocity : wx , wy , wz Step frequency estimation Step timing estimation Time-scale modification of music Gyroscope Synchronized music

Real-time Identification of Next Step Timing

Proposed Methods - Signal for Step Detection Acceleration Rotation Rate

Proposed Methods - Step Period Estimation Method 1 Period Threshold Pl /4 Pa /4 (Pl+Pa)/8 0.35 sec 0.42 sec 0.525 sec 120 SPM 50 SPM 0.7 sec

Proposed Methods - Step Period Estimation Method 2 120 SPM 50 SPM (a) upper bound = average height * 2.4 lower bound = average low * 2.4 (b) Threshold= max * 0.7

Proposed Methods - Step Period Estimation Method 3

Proposed Methods - Onset Strength Curve Three basic types: Origin (O): signal Diff (D): First-order differential signal Smooth (S): Smoothed signal Four combination types: OD: O + D DS: D + S OS: O + S ODS: O + D + S f-measure of origin: 0.97 f-measure of OS: 0.967

Proposed Methods - Onset Strength Curve (cont.) moving average window: size: 35 range: -17 ~ + 17

Proposed Methods - Finding The Steps

Experiments - Data Collection Bluetooth Headset A Sounder ( Ground Truth ) Arm Hand Pocket iphone 4s Motion Sample Rate : 60 Hz Step Sound (Headset) Sample Rate : 44100 Hz Resolution : 16 bits Channel : mono 這頁也先不用改 Example recording

Experiments - Dataset # of runners: 20 # of recordings: 108 # of recordings on arm : 37 # of recordings in hand: 36 # of recordings in pocket: 21 # of recordings on arm under varying speed: 14 Distance: 100m (94), 200m (7), 400m (7) Running time: 35~45sec (94), 50~80 sec (7) , 60~100 sec (7) Track material: PU, asphalt road 這頁先不用改

Experiments - Smartphone Locations under Steady Speed Method 1 arm 0.99 hand 0.725 pocket 0.773 Method 2 arm 0.995 hand 0.726 pocket 0.775 Method 3 arm 0.964 hand 0.756 pocket 0.78

Experiments - System Parameters Signal & Strength Curve Signal Rot Strength Curve Origin Smoothing Size None Trend Removal 35, moving average Boundary size 4 Method 2 – Noise Removal Candidate Threshold 0.7 Bound Ratio 2.4 Frame Size 2.75 sec Method 3 - Ceps Zero Padding 10 Peak Threshold 0.1 Frame Size 3.85 sec Method 1 - ACF & Correction Smoothing 9, gaussian window Period Threshold Pl /4 Peak Threshold 0.12 Frame size 2.6 sec 應該先給 一個 預設的 參數表 實驗需要用 ten folder Step Frequency & Music Tempo weight 0.3 BpmTh 6

Experiments - Step Timing Prediction Tolerance : 0.1 sec Method 1 f-measure precision recall Steady 0.993 0.992 Varying 0.989 Methods Speed Method 2 f-measure precision recall Steady 0.991 0.99 Varying 0.989 Methods Speed Method 3 f-measure precision recall Steady 0.986 0.987 0.984 Varying 0.96 0.962 0.959 Methods Speed

Conclusions & Future Work Immediate work Implementation in phone app Future work Training mode which considers… Physiological Signals Heartbeat rates Breathing Sweating Environmental conditions Path: tilted, surface type, etc Weather: temperature, humidity, etc

Thanks for your listening