1. The General Linear Model (GLM): the marriage between linear systems and stats
FFA

2. fMRI Data Processing Stream
raw scanner data
preprocessing
identify task related activity

3. The General Linear Model (GLM)
Jargon/Terms to remember
General Linear Model (GLM)
HRF (HIRF): hemodynamic (impulse) response function
Design matrix
Predictors
Betas ()
Residual error
Deconvolution
How can we relate the hemodynamic response for a single event to the hemodynamic response during a blocked stimulation?
Will the rise time be the same? Duration of response be the same?
We need a model - we will spend a lecture explaining the current model for the linear systems approach which provides a mean to predict the expected BOLD response from the experimental paradigm. We will discuss both the appeal and constraints of this model.
But first we will give some intuition 3

4. How do I predict the BOLD response to any stimulus?
Convolution
How do I predict the BOLD response to any stimulus?

5. How do I predict the BOLD response to any stimulus?
Convolution
How do I predict the BOLD response to any stimulus?
According to the linear systems approach
Need to know the impulse response function
Predicted response is the convolution between the input and the impulse response function

6. How do I predict the BOLD response to any stimulus?
Convolution
How do I predict the BOLD response to any stimulus?
According to the linear systems approach
Need to know the impulse response function
Predicted response is the convolution between the input and the impulse response function

7. How do I predict the BOLD response to any stimulus?
Convolution
How do I predict the BOLD response to any stimulus?
According to the linear systems approach
Need to know the impulse response function
Predicted response is the convolution between the input and the impulse response function

8. Examples of model hemodynamic impulse response function (HRF) estimated from human fMRI measurements
No undershoot
No undershoot
Derived empirically in V1
Shortest stimulus is 3s
Mathematical model based on Turner balloon model of BOLD responses
Derived empirically in V1
Shortest stimulus is 1s
used decovolution

9. How do I predict the BOLD response to any stimulus?
Convolution or
How do I predict the BOLD response to any stimulus?
Stimulus
Predicted BOLD?

10. How do I predict the BOLD response to any stimulus?
Convolution or
How do I predict the BOLD response to any stimulus?
Stimulus
Predicted BOLD?
Turn the stimulus
into a series of impulses
Sum up the time shifted impulse response
In other words convolve your stimulus with the hrf

11. Convolve stimulus with hrf
Convolution
Stimulus
Predicted BOLD
Convolve stimulus with hrf
We do this numerically in matlab
with the conv function
Convolution_tutorial.m

12. How do I predict the BOLD response to any stimulus?
Convolution
How do I predict the BOLD response to any stimulus?
Convolution_tutorial.m

13. Hypothesis: there are neurons in the brain that respond more (increase firing) to moving than still visual stimuli

14. Awesome fMRI experiment
Hypothesis: there are neurons in the brain that respond more (increase firing) to moving than still visual stimuli
Awesome fMRI experiment
Scan subjects as they view 6 alternating blocks of moving vs. stationary stimuli
Moving dots 1
Still dots
time

15. Awesome fMRI experiment
Hypothesis: there are neurons in the brain that respond more (increase firing) to moving than still visual stimuli
Awesome fMRI experiment
Scan subjects as they view 6 alternating blocks of moving vs. stationary stimuli
Moving dots 1
Still dots
Prediction:
BOLD response in regions containing motion sensitive neurons will be stronger during blocks of moving dots than blocks of still dots
time

16. Awesome fMRI experiment
Hypothesis: there are neurons in the brain that respond more (increase firing) to moving than still visual stimuli
Awesome fMRI experiment
Scan subjects as they view 6 alternating blocks of moving vs. stationary stimuli
Note that I generated a vector of 1s and 0s indicating at
each timepoint what is the stimulus: 0=still; 1=moving;
Moving dots 1
Still dots
Prediction:
BOLD response in regions containing motion sensitive neurons will be stronger during blocks of moving dots than blocks of still dots
time

17. Awesome fMRI experiment
Hypothesis: there are neurons in the brain that respond more (increase firing) to moving than still visual stimuli
Awesome fMRI experiment
Scan subjects as they view 6 alternating blocks of moving vs. stationary stimuli
Note that I generated a vector of 1s and 0s indicating at
each timepoint what is the stimulus: 0=still; 1=moving;
This is called the design matrix.
Moving dots 1
Still dots
Prediction:
BOLD response in regions containing motion sensitive neurons will be stronger during blocks of moving dots than blocks of still dots
time

18. Based on our linear model we will generate a prediction
by convolving the design matrix with an HRF
g(t) predictor 1
HRF * 18

19. Based on our linear model we will generate a prediction
by convolving the design matrix with an HRF
g(t) predictor 1
HRF *
How well does my prediction match the data?
Time series from an example voxel
time course data 19

20. I want to write a mathematical notation relating the prediction to the data for 2 reasons: (1) to estimate the model parameters and (2) test how good the fit is
time course data
g(t) predictor

21. I want to write a mathematical notation relating the prediction to the data for 2 reasons: (1) to estimate the model parameters and (2) test how good the fit is
time course data
g(t) predictor

22. I want to write a mathematical notation relating the prediction to the data for 2 reasons: (1) to estimate the model parameters and (2) test how good the fit is
time course data
g(t) predictor

23. I want to write a mathematical notation relating the prediction to the data for 2 reasons: (1) to estimate the model parameters and (2) test how good the fit is
Because this is a linear model, I predict my time course y(t) is a scaled version of the predictor g(t), plus an offset term b0.
time course data
g(t) predictor

24. I want to write a mathematical notation relating the prediction to the data for 2 reasons: (1) to estimate the model parameters and (2) test how good the fit is
Because this is a linear model, I predict my time course y(t) is a scaled version of the predictor g(t), plus an offset term b0.
time course data
g(t) predictor
The model has 2
parameters:
b1 scales the predictor
b0 shifts it from baseline
and an error term
e(t), residual error: what the linear model doesn’t explain in the data

25. I want to write a mathematical notation relating the prediction to the data for 2 reasons: (1) to estimate the model parameters and (2) test how good the fit is
Because this is a linear model, I predict my time course y(t) is a scaled version of the predictor g(t), plus an offset term b0.
time course data
g(t) predictor
The model has 2
parameters:
b1 scales the predictor
b0 shifts it from baseline
and an error term
e(t), residual error: what the linear model doesn’t explain in the data

26. g(t) predictor We can solve this with a simple linear regression!
I want to write a mathematical notation relating the prediction to the data for 2 reasons: (1) to estimate the model parameters and (2) test how good the fit is
We can solve this with a simple linear regression!
time course data
g(t) predictor
The model has 2
parameters:
b1 scales the predictor
b0 shifts it from baseline
and an error term
e(t), residual error: what the linear model doesn’t explain in the data

27. * g(t) predictor Design matrix HRF Stimulus: used to
generate predictor
Offset term
(b0)
HRF * 27

28. is scaled to match the time course
Design matrix
Solution (b1):
how much the predictor
is scaled to match the time course

29. is scaled to match the time course
Design matrix
Solution (b1):
how much the predictor
is scaled to match the time course
Here the signal is around zero
so b0 is negligible.
Nevertheless researchers rarely report it because they
are interested about the effect of the stimulus and not the baseline brain signal

30. The residual error is the difference between the data and the model’s prediction
e(t)=residual error

31. The residual error is the difference between the data and the model’s prediction
The error term can be used to estimate how much
variance in the data is explained by the model
e(t)=residual error

32. How does the HRF affect results?* * *

33. How does the HRF affect results?* * *

34. How does the HRF affect results?

35. The General Linear Model: expand the regression model to have more than one predictor
y(t) time course of voxel
gi(t) i-th factor
bi coefficient of i-th factor
b shift from baseline
e(t) additive noise

36. General Linear Model: Matrix Notation
Y time course column vector (nx1)
n: number of time samples
G matrix of concatenated predictors (nxp)
p:number of predictors (number of experimental conditions)
b vector of factor coefficients (px1)
e additive noise
Least Squares Solution:

37. General Linear Model: Matrix Notation
Y time course column vector (nx1)
n: number of time samples
G matrix of concatenated predictors (nxp)
p:number of predictors (number of experimental conditions)
b vector of factor coefficients (px1)
e additive noise
Least Squares Solution:
As an experimenter you generate the design matrix, which then
gets convolved with the HRF to generator predictors gi

38. Example GLM with 4 predictors and 2 baselines for run1 and run 2

39. Example GLM with 4 predictors and 2 baselines for run1 and run 2
Scaled by bs 39

40. Example GLM with 4 predictors and 2 baselines for run1 and run 2
GLM explains
70.5% of the time course variance

41. Example GLM with 4 predictors and 2 baselines for run1 and run 2
GLM explains
46.7% of the time course variance

42. Correlated Predictors
Avoid predictors that are correlated with one another
This is why we NEVER include a baseline predictor
baseline predictor is almost completely correlated (r = -1) with the sum of other existing predictors
if we included a baseline predictor, the model would have problems assigning variance to stimulus predictors vs. baseline predictors
for example, the model could not distinguish between two possibilities (e.g., Beta1=1, Beta2=0 vs. Beta1=0, Beta2=-1)
Stimulus predictor
Baseline predictor

43. Correlated Predictors
Avoid predictors that are correlated with one another
This is why we NEVER include a baseline predictor
baseline predictor is almost completely correlated (r = -1) with the sum of other existing predictors
if we included a baseline predictor, the model would have problems assigning variance to stimulus predictors vs. baseline predictors
for example, the model could not distinguish between two possibilities (e.g., Beta1=1, Beta2=0 vs. Beta1=0, Beta2=-1)
Stimulus predictor
Baseline predictor

44. Correlated Predictors
Avoid predictors that are correlated with one another
This is why we NEVER include a baseline predictor
baseline predictor is almost completely correlated (r = -1) with the sum of other existing predictors
if we included a baseline predictor, the model would have problems assigning variance to stimulus predictors vs. baseline predictors
for example, the model could not distinguish between two possibilities (e.g., Beta1=1, Beta2=0 vs. Beta1=0, Beta2=-1)
Stimulus predictor
Baseline predictor

45. Deconvolution Can you solve the inverse problem?
If I measure the summed response of several impulses, can I recover the hemodynamic response to a single event?

46. Series of identical single events that are closely space in time
Series of identical single events that are closely space in time. Can I recover the hemodynamic response of a single event? h3 h4 h2 h5 h1 h6 h7 y3 y6 y5 y2 y4 y1
=y8
=y7

47. Series of identical single events that are closely space in time
Series of identical single events that are closely space in time. Can I recover the hemodynamic response of a single event? h3 h4 h2 h5 h1 h6 h7 y3 y6 y5 y2 y4 y1
=y8
=y7

48. Series of identical single events that are closely space in time
Series of identical single events that are closely space in time. Can I recover the hemodynamic response of a single event? h3 h4 h2 h5 h1 h6 h7 y3 y6 y5 y2 y4
Deconvolution: Compute the hemodynamic response hi in a time window by solving the GLM at each time point rather than assume an HRF; y1
=y8
=y7

49. Not if they are presented in a fixed interval
Series of identical single events that are closely space in time. Can I recover the hemodynamic response of a single event? h3 h4 h2 h5 h1 h6 h7 y3 y6 y5 y2 y4
Deconvolution: Compute the hemodynamic response hi in a time window by solving the GLM at each time point rather than assume an HRF; y1
=y8
=y7

50. Series of identical single events that are closely space in time
Series of identical single events that are closely space in time. Can I recover the hemodynamic response of a single event?
Yes, if they are displaced (jittered) in time h3 h4 h2 h5 h6 h1 h7 y3 y6 y5 y2 y4 y1
Deconvolution: Compute the hemodynamic response hi in a time window by solving the GLM at each time point rather than assume an HRF;
=y7
=y8

51. Estimating s Using Standard GLM

52. Deconvolution, estimating s in a time window without assuming a specific HRF

53. Back to nonlinearities- re-evaluating the GLM when it fails
Birn et al NeuroImage 2001
Linear model fails for brief and rapid stimuli:
(1) Responses are non-linear for durations shorter than 2s
(2) Model substantially underestimates responses for brief stimuli

54. Problem: GLM relates stimulus to BOLD not neural responses to BOLD
Gaussian noise
Black box
fMRI
response
Neural
response
MRI
Scanner
Hemo-dynamics
Stimulus +
If nonlinearities have a neural origin then incorporating a model of neural responses that accounts for these nonlinearities may better predict BOLD responses than the standard GLM

55. We sought to test nonlinearities by measuring brain responses to combinations of sustained and transient visual stimuli and comparing the predictions of the GLM to a new encoding model that takes into account nonlinear neural responses

56. We sought to test nonlinearities by measuring brain responses to combinations of sustained and transient visual stimuli and comparing the predictions of the GLM to a new encoding model that takes into account nonlinear neural responses

57. Predicted V1 responses from the standard GLM
Stigliani, Jeska & Grill-Spector, Biorxiv 2017

58. V1 responses to transient stimuli differ from the predictions of the standard GLM
Stigliani, Jeska & Grill-Spector, Biorxiv 2017

59. We developed a 2-temporal channel encoding model of neural responses in the visual system and tested if it better predicts BOLD responses than the standard GLM
Stigliani, Jeska & Grill-Spector, Biorxiv 2017

60. 2-temporal channel model better predicts V1 responses to time varying stimuli than the standard GLM
Stigliani, Jeska & Grill-Spector, Biorxiv 2017

61. 2-temporal channel model better predicts V1 responses to time varying stimuli than the standard GLM
Stigliani, Jeska & Grill-Spector, Biorxiv 2017

62. 2-temporal channel model also explains other data such as the data from the Birn paper
Supplementary Figure S4: The 2 temporal-channel model explains response nonlinearities for briefly presented stimuli. (a) Figure adapted from Birn et al. (y-axis values are unreported in original version). Top left: measured V1 responses to brief (250– 2000 ms) presentations of a checkerboard stimulus that was contrast inverted at 8 Hz in all trial durations; Top right: predicted V1 responses based on a standard linear model solved using responses to longer presentations of the checkerboard stimulus. Bottom: same data as above except the measured and predicted fMRI responses are superimposed for each trial duration. (b) Simulated V1 responses to the stimuli used by Birn et al. that are derived with the  weights solved using models fit to V1 data from Experiments 1 and 2 of the present study. Left: predictions of the 2 temporal-channel model for each trial duration; Right: predictions of the standard model for each trial duration. Bottom: same data as above except the predictions of the two models are superimposed for each trial duration. The simulations show that the standard model replicates Birn et al.’s linear model and underestimate responses. In contrast, the 2 temporal-channel model better explains the measured responses (a‑left) and predicts higher responses than the standard model in each duration (b‑bottom).