Psychometrics, Dynamics, and Functional Data Analysis Jim Ramsay Jim Ramsay McGill University “The views “The views

Testing as Input/Output Analysis A test score is actually a derivative with respect to time. A test score is actually a derivative with respect to time. Consequently a differential equation model for testing data seems natural. Consequently a differential equation model for testing data seems natural. Dynamic testing data will be more and more important. Dynamic testing data will be more and more important. We have some new tools for working with dynamic data. We have some new tools for working with dynamic data. So let’s consider how to use time as a covariate. So let’s consider how to use time as a covariate.

Learning to Play Golf We buy some clubs. We play a few games, each being an 18 item test. It’s harder than it looks. We buy some clubs. We play a few games, each being an 18 item test. It’s harder than it looks. We take a lesson. We play a few games. Our score improves to a better level. We take a lesson. We play a few games. Our score improves to a better level. We take another lesson, play some games, and things improve again. We take another lesson, play some games, and things improve again. Key question: How quickly is a lesson reflected in an improvement in score? Key question: How quickly is a lesson reflected in an improvement in score?

Brainergy Energy is defined as “the capacity to do work.” Energy is defined as “the capacity to do work.” Kinetic energy E = Mv 2 /2, and involves mass, distance, and time. Kinetic energy E = Mv 2 /2, and involves mass, distance, and time. We are interested in “the capacity to solve problems.” We are interested in “the capacity to solve problems.” Problems involve difficulties (=mass), number of problems (=distance) and time. Problems involve difficulties (=mass), number of problems (=distance) and time. Let’s call mental energy brainergy. Let’s call mental energy brainergy.

Brain Power What counts is problem solving per unit time. What counts is problem solving per unit time. Power = energy expended per unit time. Power = energy expended per unit time. Brain power = maximum difficulty of problem solvable per unit time, or Brain power = maximum difficulty of problem solvable per unit time, or number of lighter problems solved per unit time. number of lighter problems solved per unit time. That is, brain power = d brainergy/dt. That is, brain power = d brainergy/dt.

Brain Power and Time Scales We need the concept of brain power when we consider intelligence on two time scales: 1. Long term: How much knowledge is available over large time intervals, like a school year 2. Short term: How much new knowledge can be acquired over a short time interval, like a single class.

Tests Measure Brain Power Mental tests and psychological scales are one of the greatest technological achievements of the 20 th century. Mental tests and psychological scales are one of the greatest technological achievements of the 20 th century. Tests work so well because they are time- limited. Tests work so well because they are time- limited. Test scores reflect brain power rather than brainergy. Test scores reflect brain power rather than brainergy.

Inputs to Brain Power Information about the structure of the problems. Information about the structure of the problems. A set of tools to solve them. A set of tools to solve them. Training in the use of these tools. Training in the use of these tools. All these require time. All these require time. Inputs to acquisition of brain power are functions of time. Inputs to acquisition of brain power are functions of time.

A Differential Equation in Time Links one or more time-derivatives, dx/dt, d 2 x/dt 2,…, to the function x(t) itself. Links one or more time-derivatives, dx/dt, d 2 x/dt 2,…, to the function x(t) itself. Is a model for system dynamics: change over time. Is a model for system dynamics: change over time. Can also include one or more input or covariate functions. Can also include one or more input or covariate functions. x(t) is a long-term description. x(t) is a long-term description. dx/dt is a short-term description. dx/dt is a short-term description.

A Simple Example E(t) is brainergy, dE/dt is brain power. E(t) is brainergy, dE/dt is brain power. f(t) is an input function of time, such as education. f(t) is an input function of time, such as education. α and β are constants, β > 0. α and β are constants, β > 0.

Most differential equations don’t have explicit solutions, but this one does. Most differential equations don’t have explicit solutions, but this one does. Let E 0 be brainergy at time t = 0, and which will often be 0. Let E 0 be brainergy at time t = 0, and which will often be 0. Let’s see what happens when α=1, β varies, and f(t) is a step function. Let’s see what happens when α=1, β varies, and f(t) is a step function.

The slope of E(t) when f(t) goes positive is β. The slope of E(t) when f(t) goes positive is β. β controls how fast the system responds to the input f(t). β controls how fast the system responds to the input f(t). If the system is a problem solver, then β indicates how quickly the person learns to solve a problem. If the system is a problem solver, then β indicates how quickly the person learns to solve a problem. After about 4/β time units, full capacity is reached, and the system is ready for more input. After about 4/β time units, full capacity is reached, and the system is ready for more input.

Fitting Differential Equations We have noisy discrete-time data, and want to use them to estimate a differential equation. We have noisy discrete-time data, and want to use them to estimate a differential equation. We want a solution E(t) to the equation to fit the data as well as possible. We want a solution E(t) to the equation to fit the data as well as possible. We need lots of flexibility in choosing a differential equation, and we can’t assume that there is an explicit solution to the equation. We need lots of flexibility in choosing a differential equation, and we can’t assume that there is an explicit solution to the equation.

Functional Data Analysis A collection of methods for analyzing curves or functions as data A collection of methods for analyzing curves or functions as data A common theme is using derivatives in various ways A common theme is using derivatives in various ways See Ramsay and Silverman (1997) Functional Data Analysis. Springer. See Ramsay and Silverman (1997) Functional Data Analysis. Springer. And Ramsay and Silverman (2002) Applied Functional Data Analysis. Springer. And Ramsay and Silverman (2002) Applied Functional Data Analysis. Springer.

Two Functional Data Analysis Techniques L-spline Smoothing: given noisy data and a differential equation, find a function E(t) that will smooth the data and at the same time be nearly a solution to the differential equation. L-spline Smoothing: given noisy data and a differential equation, find a function E(t) that will smooth the data and at the same time be nearly a solution to the differential equation. Principal Differential Analysis: given a function E(t), estimate a linear differential equation for which E(t) is a solution. Principal Differential Analysis: given a function E(t), estimate a linear differential equation for which E(t) is a solution.

Estimating a DIFE from noisy data We’ve recently combined these two methods into a technique for estimating a differential equation from noisy data. We’ve recently combined these two methods into a technique for estimating a differential equation from noisy data. In our simple example, this amounts to estimating parameters α and β. In our simple example, this amounts to estimating parameters α and β. But much more complex DIFE’s can be estimated as well, including linear or nonlinear, and single or multiple variable systems. But much more complex DIFE’s can be estimated as well, including linear or nonlinear, and single or multiple variable systems.

An Oil Refinery Here are some data from an oil refinery in Corpus Christi. Here are some data from an oil refinery in Corpus Christi. The input f(t) (reflux flow) is negatively coupled to the output E(t) (tray 47 level). The input f(t) (reflux flow) is negatively coupled to the output E(t) (tray 47 level). The smooth curve is a solution to the differential equation that best represents this relationship. The smooth curve is a solution to the differential equation that best represents this relationship.

Perhaps this oil refinery is not too smart!

Many situations will call for multiple outputs: Performance with a putter, a driver, and an iron, for example. Or in algebra and geometry. Many situations will call for multiple outputs: Performance with a putter, a driver, and an iron, for example. Or in algebra and geometry. And many situations will involve multiple inputs: Regular classes, tutoring sessions, labs and etc. And many situations will involve multiple inputs: Regular classes, tutoring sessions, labs and etc. The technology used in these illustrations can handle these situations, at least for linear differential equations. Nonlinear equations don’t pose any problem in principle. The technology used in these illustrations can handle these situations, at least for linear differential equations. Nonlinear equations don’t pose any problem in principle.

Some Simulated Data Imagine that the data are golf scores over successive games, and that the input is a set of three equally-spaced lessons from a golf pro. Imagine that the data are golf scores over successive games, and that the input is a set of three equally-spaced lessons from a golf pro. The following slides show three golfers. Which is a future Tiger Woods? The following slides show three golfers. Which is a future Tiger Woods?

These lessons are nicely timed.

This person needs to find another sport!

This person should get lessons more often!

Is this Model Good Enough? Specifying β to be constant is too simple. Allowing for fatigue, boredom, and other things requires a function β(t). Specifying β to be constant is too simple. Allowing for fatigue, boredom, and other things requires a function β(t). A first order equation can’t allow for sudden transient effects like insight. We may need a differential equation involving higher derivatives. A first order equation can’t allow for sudden transient effects like insight. We may need a differential equation involving higher derivatives. We may need nonlinear equations as well. We may need nonlinear equations as well.

A Nonlinear Differential Equation The summed output from these two equations will exhibit both the rapid learning and long-term retention required of human learners. See H. R. Wilson (1999) Spikes, Decisions and Actions, Oxford, for many more examples of differential equation models in neuroscience.

Control Theory Engineers who work with input/output systems have developed ways of designing feedback loops to optimize outputs. Engineers who work with input/output systems have developed ways of designing feedback loops to optimize outputs. We’re working with a team of chemical engineers at Queen’s University. We’re working with a team of chemical engineers at Queen’s University.

Where Would the Data Come From? Can we design customized learning situations, like golf, and track how a learner makes progress as a function of time and inputs? Can we design customized learning situations, like golf, and track how a learner makes progress as a function of time and inputs? Perhaps video and computer games are nearly what we need. Perhaps video and computer games are nearly what we need. We already know that people will pay big money to have these experiences. We already know that people will pay big money to have these experiences. Would corporations with deep pockets pay for this kind of testing? Would corporations with deep pockets pay for this kind of testing?

Conclusions Dynamic testing would generate performance data over time that depend on one or more functional covariates. Dynamic testing would generate performance data over time that depend on one or more functional covariates. New tools are available for these data that fit them with a differential equation. New tools are available for these data that fit them with a differential equation. Dynamic psychometrics looks promising! Dynamic psychometrics looks promising!