1. Bradley W. Vines McGill University
Bradley Vines:
Good afternoon. I will be discussing (title here). This talk is intended as an introduction and an overview of FDA.
Functional Data Analysis: Techniques for Exploring Temporal Processes in Music
Bradley W. Vines
McGill University

2. Collaborators Daniel Levitin (McGill University)
Bradley Vines:
I have collaborated with the following researchers to develop applications of FDA techniques for use in music cognition researche
Daniel Levitin (McGill University)
Carol Krumhansl (Cornell University)
Jim Ramsay (McGill University)
Regina Nuzzo (McGill University)
Stephen McAdams (IRCAM)
12/2/2018
ICMPC8

3. Talk Outline What is Functional Data Analysis? Steps of a typical FDA
Demonstrate some of the major FDA tools
Smoothing
Registration
General Linear Modeling (significance testing)
12/2/2018
ICMPC8

4. An Example of Functional Data
The idea of “tension” has a shared meaning across participants, based upon extra-musical experiences like tension in physical objects, in social situations and in the body.
Solo clarinet performances
3 Treatment Groups
Auditory only
Visual only
Auditory + Visual
Continuous Tension Judgments
I will begin with an example of functional data. This data comes from a study that I presented earlier this week in which we explored the impact of seeing a musician perform.
I will be using these data to demonstrate the Functional Data Analysis tools throughout the talk
12/2/2018
ICMPC8

5. An example of functional data
The question is: How can this data be analyzed? Correlations are useful for identifying similarities between data sets as are multiple regressions, but they reduce all of the information to a summary statistic. Here we are also interested in how the relations between the different groups changes over time. This is where Functional Data Analysis is well suited.
With functional data analysis software tools and analysis techniques, it is possible to explore changes over time and to understand WHEN important changes or relations are occurring.
12/2/2018
ICMPC8

6. What is Functional Data Analysis? (Ramsay & Silverman, 1997)
The meaning of the music and its impact depends upon the relations between events over time and the way that those relations change.
What is Functional Data Analysis? (Ramsay & Silverman, 1997)
Bradley Vines:
Temporal dynamics are an important aspect of music and they are the focus of much music cognition research
Examples of time dependent measures in musical stimuli:
For data drawn from continuous processes
Growth curves, market value, movement, ERP’s
Model data as functions of time
Temporal dynamics in music
(Vines, Nuzzo, & Levitin, under review)
continuous measurements of emotion
expressive timing profiles
physiological measurements
movement tracking
Software tools available in Matlab and in S-Plus
Including measurements like growth curves…
To understand music, we need to know how changes in sound effect a listener and the performer -
Music necessarily occurs through time as the unfolding of related events.
Bradley Vines:
All of these data involve processes that evolve over time and therefore that may be intuitively thought of as functions of time, which makes FDA techniques a useful and meaningful way to analyze and explore such measures.
Because we are interested not only in the current moment in music but also how events are changing over time and even how changes are changing over time (as in expressive timing profiles), it is meaningful and intuitive to think of continuous measurements as functions of time and to model them as such.
It makes intuitive sense to think about this kind of data in terms of functions of time
I will give the ftp site later in the talk.
Tools for visualizing the data, revealing trends in variation and for significance testing
12/2/2018
ICMPC8

7. Modeling data as functions of time
Bradley Vines:
Using basis-fitting techniques
Take a number of basic functions and add them together in such a way as to create the desired function (Add together basic functions)
Same process as in sound synthesis
Fourier analysis does just the opposite
Basis functions
Element functions that can be added together to approximate the data.
W1*F1(t) + W2*F2(t) + W3*F3(t)…
A least squares algorithm is used to determine the weighting coefficients.
12/2/2018
ICMPC8

8. Two basis types Fourier B-spline Polynomial functions Knots
Bradley Vines:
Using basis-fitting techniques
Take a number of basic functions and add them together in such a way as to create the desired function (Add together basic functions)
Same process as in sound synthesis
Fourier analysis does just the opposite
Fourier
B-spline
Polynomial functions
Knots
Usually, however, data is messy, and non-periodic. B-spline bases are useful for data that do not have a simply periodicity.
I will be concentrating on the use of B-spline bases to model functional data
12/2/2018
ICMPC8

9. Visualizing B-spline Bases
12/2/2018
ICMPC8

10. Visualizing B-spline Bases
12/2/2018
ICMPC8

11. Steps in a typical FDA Representing the data in Matlab: Matrices
Each row: a sample point in time
Each column: an observation (participant/performer)
Third dimension for multivariate observations
As in Schubert’s multi-dimensional continuous interface
Valence
Arousal
(Schubert, 1999)
12/2/2018
ICMPC8

12. Steps in a typical FDA Modeling the data with functions
Two major considerations:
Order of the B-spline bases
The number of basis functions
12/2/2018
ICMPC8

13. Steps in a typical FDA Modeling the data with functions
Two major considerations:
Order of the B-spline bases
The number of basis functions
The order of B-spline bases
Determines how many derivatives will be smooth.
12/2/2018
ICMPC8

14. Steps in a typical FDA The number of basis functions Tradeoff:
Affects the quality of fit to the data
The more B-splines, the smaller the error
Tradeoff:
Modeling data accurately
Excluding unimportant noise in the data
12/2/2018
ICMPC8

15. Original Data Remember to mention that there were 800 samples.
“There are 800 samples shown here”
12/2/2018
ICMPC8

16. Modeled Data
Correlation = .9975
12/2/2018
ICMPC8

17. Modeled Data
The choice of # of B-splines really depends upon the assumptions that the researcher can make about the data. If it is known that there is a single objective event that lead to two peaks, then it might be ideal to treat those two peaks with a single curve.
Correlation = .97
12/2/2018
ICMPC8

18. Major FDA Tools Bradley Vines:
With the data all prepared and modeled with functions, it is possible to go on to use the tools available in FDA.
Major FDA Tools

19. Controlling Unwanted Variability
Curvature (high frequency noise)
Smoothing
Amplitude
Scaling
Phase
Registration
12/2/2018
ICMPC8

20. Nine Tension Judgments
12/2/2018
ICMPC8

21. 12/2/2018
ICMPC8

22. 12/2/2018
ICMPC8

23. Nine Tension Judgments
12/2/2018
ICMPC8

24. Nine Tension Judgments
12/2/2018
ICMPC8

25. Nine Tension Judgments
Bradley Vines:
Note that some of the judgments were shifted ahead or backwards at the two points.
Nine Tension Judgments
12/2/2018
ICMPC8

26. Time Warping
12/2/2018
ICMPC8

27. General Linear Modeling
Functional regression
Functional significance test (F-test)
12/2/2018
ICMPC8

28. The effect of adding video
12/2/2018
ICMPC8

29. Functional Linear Model
Y(t) = U(t) + B1(t) [if video is added]
12/2/2018
ICMPC8

30. Results The question is: When is the difference from zero significant?
12/2/2018
ICMPC8

31. Significance Testing Analogous components to traditional F-testing:
MSE(t) = SSE(t) / df(error)
-with df(error) = N participants - P parameters
MSR(t) = [SSY(t) – SSE(t)]/df(model)
-with df(model) = P parameters - 1
FRATIO(t) = MSR(t)/MSE(t)
12/2/2018
ICMPC8

32. Significance Testing An F-value that is itself a function of time.
12/2/2018
ICMPC8

33. Other FDA techniques that are available
Analysis of covariance
Functional correlation analysis
Canonical correlation analysis
Principal Components Analysis
12/2/2018
ICMPC8

34. Prof. James Ramsay’s ftp site:
FUNCTIONAL DATA ANALYSIS: TECHNIQUES FOR EXPLORING TEMPORAL PROCESSES IN MUSIC
Prof. James Ramsay’s ftp site:
12/2/2018
ICMPC8

35. Smoothing
The smoothing parameter, lambda, controls the curvature of a function.
Trade off between perfect fit to the original data and a best linear approximation for the data. Penalizes variance
12/2/2018
ICMPC8

36. Smoothing
Examples of curves before and after smoothing (try to find a good singly participant who is nice and dynamic for all of this, or a mean curve, I suppose)
12/2/2018
ICMPC8

37. Principal Components Analysis
Traditional statistics:
Identifying major modes of variation
Reducing the number of dimensions in the data
Determine which variables are related
Functional analogue:
Reveals major modes of variation
Can reveal trends in phase and in magnitude
12/2/2018
ICMPC8

38. Principal Components Analysis
Monthly temperature data
(available on the ftp website)
Weather stations across Canada
Exploring trends in the data and grouping weather stations
12/2/2018
ICMPC8

39. Monthly Weather Data Bradley Vines: 32 weather stations 12/2/2018
ICMPC8

40. Eigenvalues, VARIMAX PCA
12/2/2018
ICMPC8

41. VARIMAX Principal Components
12/2/2018
ICMPC8

42. VARIMAX Principal Components
12/2/2018
ICMPC8

43. VARIMAX Principal Components
12/2/2018
ICMPC8

44. VARIMAX Principal Components
12/2/2018
ICMPC8

45. Component Scores
12/2/2018
ICMPC8