1. MART: Music Assisted Running Trainer
張智星
CSIE Dept, National Taiwan University

2. Introduction Objective Challenges
Develop a phone app that can adjust the music tempo according the runner’s step frequency and timing.
Challenges
How to find the runner’s step frequency correctly via tri-accelerometer and gyroscope only?
How to find the step timing?
How to adjust the tempo of music?

3. Proposed Methods – System Overview
Tri-Accelerometer : ax , ay , az
Gyroscope : yaw , roll , pitch
Angular Velocity : wx , wy , wz
Step frequency estimation
Step timing estimation
Time-scale modification of music
Gyroscope
Synchronized music

4. Real-time Identification of Next Step Timing

5. Proposed Methods - Signal for Step Detection
Acceleration
Rotation Rate

6. Proposed Methods - Step Period Estimation Method 1
Period Threshold
Pl /4
Pa /4
(Pl+Pa)/8
0.35 sec
0.42 sec
0.525 sec
120 SPM
50 SPM
0.7 sec

7. Proposed Methods - Step Period Estimation Method 2
120 SPM
50 SPM
(a)
upper bound = average height * 2.4
lower bound = average low * 2.4
(b)
Threshold= max * 0.7

8. Proposed Methods - Step Period Estimation Method 3

9. Proposed Methods - Onset Strength Curve
Three basic types:
Origin (O): signal Diff (D): First-order differential signal
Smooth (S): Smoothed signal
Four combination types:
OD: O + D DS: D + S
OS: O + S
ODS: O + D + S
f-measure of origin: 0.97
f-measure of OS: 0.967

10. Proposed Methods - Onset Strength Curve (cont.)
moving average window: size: 35
range: -17 ~ + 17

11. Proposed Methods - Finding The Steps

12. Experiments - Data Collection
Bluetooth Headset
A Sounder ( Ground Truth )
Arm
Hand
Pocket
iphone 4s
Motion
Sample Rate : 60 Hz
Step Sound (Headset) Sample Rate : Hz Resolution : 16 bits Channel : mono
這頁也先不用改
Example recording

13. Experiments - Dataset # of runners: 20 # of recordings: 108
# of recordings on arm : 37
# of recordings in hand: 36
# of recordings in pocket: 21
# of recordings on arm under varying speed: 14
Distance: 100m (94), 200m (7), 400m (7)
Running time: 35~45sec (94), 50~80 sec (7) , 60~100 sec (7)
Track material: PU, asphalt road
這頁先不用改

14. Experiments - Smartphone Locations under Steady Speed
Method 1
arm
0.99
hand
0.725
pocket
0.773
Method 2
arm
0.995
hand
0.726
pocket
0.775
Method 3
arm
0.964
hand
0.756
pocket
0.78

15. Experiments - System Parameters
Signal & Strength Curve
Signal
Rot
Strength Curve
Origin
Smoothing Size
None
Trend Removal
35, moving average
Boundary size 4
Method 2 – Noise Removal
Candidate Threshold
0.7
Bound Ratio
2.4
Frame Size
2.75 sec
Method 3 - Ceps
Zero Padding 10
Peak Threshold
0.1
Frame Size
3.85 sec
Method 1 - ACF & Correction
Smoothing
9, gaussian window
Period Threshold
Pl /4
Peak Threshold
0.12
Frame size
2.6 sec
應該先給 一個 預設的 參數表
實驗需要用 ten folder
Step Frequency & Music Tempo
weight
0.3
BpmTh 6

16. Experiments - Step Timing Prediction
Tolerance : 0.1 sec
Method 1
f-measure
precision
recall
Steady
0.993
0.992
Varying
0.989
Methods
Speed
Method 2
f-measure
precision
recall
Steady
0.991
0.99
Varying
0.989
Methods
Speed
Method 3
f-measure
precision
recall
Steady
0.986
0.987
0.984
Varying
0.96
0.962
0.959
Methods
Speed

17. Conclusions & Future Work
Immediate work
Implementation in phone app
Future work
Training mode which considers…
Physiological Signals
Heartbeat rates
Breathing
Sweating
Environmental conditions
Path: tilted, surface type, etc
Weather: temperature, humidity, etc

18. Thanks for your listening