1. D-Jogger Your body as music controller What is Djogger? System design
Bart Moens, Leon Van Noorden, Marc Leman
Wim De Wilde, Dirk Cambier, Patrick Santens
Bert Celie, Jan Boone, Dirk Declercq, Jan Bourgois
IPEM, UGent
Revaki, Neurologie, UZGent & Arteveldehogeschool
Dept. of Movement and Sports Sciences, UGent
What is Djogger?
D-Jogger is a music controller that makes use of body movement to dynamically select music and adapt its tempo to the walking or running frequency (steps per minute) of the user. It automates and optimizes the musical choice during workouts by analyzing the walking frequency. After a song is chosen, the playback speed adapts to the user without influencing the pitch. D-Jogger is a personal DJ mixing on the users movement: a beat with every step.
D-Jogger features several music alignment strategies: different ways to synchronize music to ones gait. Each strategy has its own specific purpose in different research domains.
Fixed tempo music: music is chosen based on the gait frequency, but not adapted. This can be used to see how people react to music close to their gait frequency: do they adapt to the music? Can we speed them up or slow them down just with adequately chosen music? Does it improve their gait?
Adaptive tempo music: music is continuously adapted in tempo to match the gait frequency. Both the system and the music are dynamical systems. This is used to study human-machine interaction and entrainment.
Phase synchronized music forces synchronization upon the user so that each footfall automatically coincides with a beat. This method is used to study physiological effects of music during workouts.
System design
Multi-sensor input (1)
Annotated music library (2)
Accelerometer
Gyroscope
Other sensors
Beat tracking software
Real-time input processing (DSP):
collection of step detection algorithms
Metadata
Song database
Steps per minute
Phase information
Tempo and beat information
Song Selection
Music processing (3)
Tempo and phase alignment strategies
Music Selection
Phase Vocoder
Tempo adjustments
Audio
Tempo matched audio
Fundamental research
Applied research
Human-machine interaction: are you the leader or the follower?
Goal:
D-Jogger allows us to test several different music alignment strategies. Which strategy works best and results in the most steps in synchrony with the beat? What is the underlying mechanism? Who leads the human machine interaction?
Results:
The total amount of steps in sync differ greatly depending on the chosen alignment strategy.
The starting phase (when the music starts) is very important: the chance of uninstructed synchronization almost doubles when the first beat coincides with a footfall.
Control of the phase is much more efficient than the control of the tempo and defines who leads the interaction: either the human (when phase is matched) or the machine (phase is ignored or fixed tempo)
In Sync
Parkinson Rehabilitation
Results:
Average step length increased 5 to 6 cm when using auditory cues (left picture)
Average gait speed increased up to 10cm/s when using auditory cues for female patients (right picture)
While any auditory tempo-based cueing system improves gait, music is experienced most pleasent
Goal:
Parkinson patients can have trouble walking fluently. A non-invase aid called cueing provides rhytmical auditory stimuli, improving the gait. Our goal is to obtain the same resuts with musical cues instead of classical (metronome) cues.
Manipulating walking and running speed with music
Enhancing sport performance
Results:
Synchronous and asynchronous music influence slightly positive perception of exercise
Positive and negative responders towards musical influence were recognized for physiological variables
During high-intensity exercises, users preferred asynchronous music
During moderate-intensitiy exercises, users preferred synchronous music.
Goal:
Can we influence spontanious walking and running tempo with only music? By providing music just a tiny bit faster or slower then the current pace, will the user adapt his or her tempo? What are the underlying dynamics for this mechanism?
Goal:
Music is a well known motivational tool that is used in recreational sports or athletical training. In this experiment, we try to determine the effect of synchronous and asynchronous music during prolonged exercises on physiological variables such as heart rate, oxygen intake and blood lactate. The effect of music was also measured on subjective factors: the perceived exertion rate and excercise enjoyment.
Results:
While running in synchrony with the music, unannounced tempo adjustments of up to 1.5% are generally followed without instruction. This means we can slightly manipulate performance with music tempo.
While running, adjusting to slower music happens more often and faster then speeding up.
Some people synchronize better then others. Our ability to synchronize seems to depend on several factors such as music education, gender and music appreciation.
Relative phase: more then tempo synchronisation
Beathealth: Health & Wellness on the Beat
Beathealth is an European project that aims to bring an intelligent technological architecture capable of delivering embodied, flexible and efficient rhythmical stimulation adapted to individuals' motor performance and skills to enhance and/or recovering movement activity. It involves:
Fundamental research for maximizing the beneficial effects of rhythmic stimulation
Technological development to achieve state-of-the-art system reliability, flexibility, and portability
A new IT service to collect and share kinematic data with others (e.g., medical doctors, coach, etc)
The beneficial effects of BeatHealth will be evaluated in patients with movement disorders (i.e., Parkinson's disease), and in healthy citizens of various ages with moderate physical activity.
IPEM develops laboratory prototypes for the fundamental research based build upon the D-Jogger technology and previous findings. More information: Partners:
Bart Moens Leon Van Noorden, Marc Leman
IPEM, Ghent University