Karl Friston, Oliver Josephs Experimental Design and Optimisation Rik Henson With thanks to: Karl Friston, Oliver Josephs

Overview 1. A Taxonomy of Designs 2. Blocked vs Randomised Designs 3. Efficient Designs

A taxonomy of design Categorical designs Parametric designs Subtraction - Additive factors and pure insertion Conjunction - Testing multiple hypotheses Parametric designs Linear - Cognitive components and dimensions Nonlinear - Polynomial expansions Factorial designs Categorical - Interactions and pure insertion - Adaptation, modulation and dual-task inference Parametric - Linear and nonlinear interactions - Psychophysiological Interactions

A taxonomy of design Categorical designs Parametric designs Subtraction - Additive factors and pure insertion Conjunction - Testing multiple hypotheses Parametric designs Linear - Cognitive components and dimensions Nonlinear - Polynomial expansions Factorial designs Categorical - Interactions and pure insertion - Adaptation, modulation and dual-task inference Parametric - Linear and nonlinear interactions - Psychophysiological Interactions

A categorical analysis Experimental design Word generation G Word repetition R R G R G R G R G R G R G G - R = Intrinsic word generation …under assumption of pure insertion, ie, that G and R do not differ in other ways

A taxonomy of design Categorical designs Parametric designs Subtraction - Additive factors and pure insertion Conjunction - Testing multiple hypotheses Parametric designs Linear - Cognitive components and dimensions Nonlinear - Polynomial expansions Factorial designs Categorical - Interactions and pure insertion - Adaptation, modulation and dual-task inference Parametric - Linear and nonlinear interactions - Psychophysiological Interactions

Cognitive Conjunctions One way to minimise problem of pure insertion is to isolate same process in several different ways (ie, multiple subtractions of different conditions) Task (1/2) Viewing Naming Stimuli (A/B) Objects Colours A1 A2 B2 B1 Visual Processing V Object Recognition R Phonological Retrieval P Object viewing R,V Colour viewing V Object naming P,R,V Colour naming P,V (Object - Colour viewing) [1 -1 0 0] & (Object - Colour naming) [0 0 1 -1] [ R,V - V ] & [ P,R,V - P,V ] = R & R = R (assuming RxP = 0; see later) Common object recognition response (R) Price et al, 1997

Cognitive Conjunctions

A taxonomy of design Categorical designs Parametric designs Subtraction - Additive factors and pure insertion Conjunction - Testing multiple hypotheses Parametric designs Linear - Cognitive components and dimensions Nonlinear - Polynomial expansions Factorial designs Categorical - Interactions and pure insertion - Adaptation, modulation and dual-task inference Parametric - Linear and nonlinear interactions - Psychophysiological Interactions

A (linear) parametric contrast Linear effect of time

A taxonomy of design Categorical designs Parametric designs Subtraction - Additive factors and pure insertion Conjunction - Testing multiple hypotheses Parametric designs Linear - Cognitive components and dimensions Nonlinear - Polynomial expansions Factorial designs Categorical - Interactions and pure insertion - Adaptation, modulation and dual-task inference Parametric - Linear and nonlinear interactions - Psychophysiological Interactions

Nonlinear parametric design matrix Quadratic (2nd) (Constant) (0th) SPM{F} E.g, F-contrast [0 1 0] on Quadratic Parameter => Linear (1st) Inverted ‘U’ response to increasing word presentation rate in the DLPFC Polynomial expansion: f(x) ~ b1 x + b2 x2 + ... …(N-1)th order for N levels

A taxonomy of design Categorical designs Parametric designs Subtraction - Additive factors and pure insertion Conjunction - Testing multiple hypotheses Parametric designs Linear - Cognitive components and dimensions Nonlinear - Polynomial expansions Factorial designs Categorical - Interactions and pure insertion - Adaptation, modulation and dual-task inference Parametric - Linear and nonlinear interactions - Psychophysiological Interactions

Interactions and pure insertion Presence of an interaction can show a failure of pure insertion (using earlier example)… A1 A2 B2 B1 Task (1/2) Viewing Naming Stimuli (A/B) Objects Colours Visual Processing V Object Recognition R Phonological Retrieval P Object viewing R,V Colour viewing V Object naming P,R,V,RxP Colour naming P,V Naming-specific object recognition viewing naming Object - Colour (Object – Colour) x (Viewing – Naming) [1 -1 0 0] - [0 0 1 -1] = [1 -1]  [1 -1] = [1 -1 -1 1] [ R,V - V ] - [ P,R,V,RxP - P,V ] = R – R,RxP = RxP

Interactions and pure insertion

A taxonomy of design Categorical designs Parametric designs Subtraction - Additive factors and pure insertion Conjunction - Testing multiple hypotheses Parametric designs Linear - Cognitive components and dimensions Nonlinear - Polynomial expansions Factorial designs Categorical - Interactions and pure insertion - Adaptation, modulation and dual-task inference Parametric - Linear and nonlinear interactions - Psychophysiological Interactions

(Linear) Parametric Interaction A (Linear) Time-by-Condition Interaction (“Generation strategy”?) Contrast: [5 3 1 -1 -3 -5]  [-1 1]

Nonlinear Parametric Interaction F-contrast tests for nonlinear Generation-by-Time interaction (including both linear and Quadratic components) Factorial Design with 2 factors: Gen/Rep (Categorical, 2 levels) Time (Parametric, 6 levels) Time effects modelled with both linear and quadratic components… G-R Time Lin Time Quad G x T Lin G x T Quad

A taxonomy of design Categorical designs Parametric designs Subtraction - Additive factors and pure insertion Conjunction - Testing multiple hypotheses Parametric designs Linear - Cognitive components and dimensions Nonlinear - Polynomial expansions Factorial designs Categorical - Interactions and pure insertion - Adaptation, modulation and dual-task inference Parametric - Linear and nonlinear interactions - Psychophysiological Interactions

Psycho-physiological Interaction (PPI) Parametric, factorial design, in which one factor is psychological (eg attention) ...and other is physiological (viz. activity extracted from a brain region of interest) SPM{Z} V1 activity Attention time V1 attention V5 V5 activity no attention Attentional modulation of V1 - V5 contribution V1 activity

Psycho-physiological Interaction (PPI) 0 0 1 SPM{Z} V1 activity time attention V5 activity no attention V1 Att V1 x Att V1 activity V1xAtt

Overview 1. A Taxonomy of Designs 2. Blocked vs Randomised Designs 3. Efficient Designs

Epoch vs Events Sustained epoch Blocks of events => Epochs are periods of sustained stimulation (e.g, box-car functions) Events are impulses (delta-functions) In SPM99, epochs and events are distinct (eg, in choice of basis functions) In SPM2/5, all conditions are specified in terms of their 1) onsets and 2) durations… … events simply have zero duration Near-identical regressors can be created by: 1) sustained epochs, 2) rapid series of events (SOAs<~3s) i.e, designs can be blocked or randomised … models can be epoch or event-related Boxcar function Sustained epoch Blocks of events Delta functions Convolved with HRF =>

Advantages of Event-related Models 1. Randomised (intermixed) trial order c.f. confounds of blocked designs (Johnson et al 1997) 2. Post hoc / subjective classification of trials e.g, according to subsequent memory (Wagner et al 1998) 3. Some events can only be indicated by subject (in time) e.g, spontaneous perceptual changes (Kleinschmidt et al 1998) 4. Some trials cannot be blocked e.g, “oddball” designs (Clark et al., 2000) 5. More accurate models even for blocked designs? e.g, (Price et al, 1999)

Disadvantages of Randomised Designs 1. Less efficient for detecting effects than are blocked designs (see later…) 2. Some psychological processes may be better blocked (eg task-switching, attentional instructions)

Mixed Designs “Blocks” of trials with varying SOAs: Blocks are modelled as epochs (sustained or “state” effect) Trials are modelled as events (transient or “item” effects) (normally confounded in conventional blocked designs) Varying (some short, some long) SOAs between trials needed to decorrelate epoch and event-related covariates (see later) For example, Chawla et al (1999): Visual stimulus = dots periodically changing in colour or motion Epochs of attention to: 1) motion, or 2) colour Events are target stimuli differing in motion or colour

(Chawla et al 1999) V5 Motion change under attention to motion (red) or color (blue) Item Effect (Evoked) State Effect (Baseline) V4 Color change under attention to motion (red) or color (blue)

Mixed Designs “Blocks” of trials with varying SOAs: Blocks are modelled as epochs (sustained or “state” effect) Trials are modelled as events (transient or “item” effects) (normally confounded in conventional blocked designs) Varying (some short, some long) SOAs between trials needed to decorrelate epoch and event-related covariates (see later) Allows conclusion that selective attention modulates BOTH: 1) baseline activity (state-effect, additive) 2) evoked response (item-effect, multiplicative) (But note tension between maximising fMRI efficiency to separate item and state effects, and maximising efficiency for each effect alone, and between long SOAs and maintaining a “cognitive set”)

Overview 1. A Taxonomy of Designs 2. Blocked vs Randomised Designs 3. Efficient Designs

General Advice Scan as long as subjects can accommodate (eg 40-60mins); keep subjects as busy as possible! If a Group study, number of subjects more important than time per subject (though additional set-up time may encourage multiple experiments per subject) Do not contrast conditions that are far apart in time (because of low-freq noise) Randomize the order, or randomize the SOA, of conditions that are close in time http://www.mrc-cbu.cam.ac.uk/Imaging/Common/fMRI-efficiency.shtml

Expanded Overview 1. A Taxonomy of Designs 2. Blocked vs Randomised Designs 3. Efficient Designs 3.1 Response vs Baseline (signal-processing) 3.2 Response 1 - Response 2 (statistics) 3.3 Response 1 & Response 2 (correlations)

Fixed SOA = 16s  = Not particularly efficient… Stimulus (“Neural”) HRF Predicted Data  = Not particularly efficient…

Fixed SOA = 4s  = Very Inefficient… Stimulus (“Neural”) HRF Predicted Data  = Very Inefficient…

Randomised, SOAmin= 4s  = More Efficient… Stimulus (“Neural”) HRF Predicted Data  = More Efficient…

Blocked, SOAmin= 4s  = Even more Efficient… Stimulus (“Neural”) HRF Predicted Data  = Even more Efficient…

Blocked, epoch = 20s Stimulus (“Neural”) HRF Predicted Data  = =  Blocked-epoch (with small SOA) and Time-Freq equivalences

Sinusoidal modulation, f = 1/33s Stimulus (“Neural”) HRF Predicted Data  =  = The most efficient design of all!

High-pass Filtering power spectrum highpass filter power spectrum aliasing fMRI contains low frequency noise: Physical (scanner drifts) Physiological (aliased) cardiac (~1 Hz) respiratory (~0.25 Hz) power spectrum highpass filter power spectrum noise signal (eg infinite 30s on-off)

 =  = Blocked (80s), SOAmin=4s, highpass filter = 1/120s Stimulus (“Neural”) HRF Predicted Data  “Effective HRF” (after highpass filtering) (Josephs & Henson, 1999) =  = Don’t have long (>60s) blocks!

Randomised, SOAmin=4s, highpass filter = 1/120s Stimulus (“Neural”) HRF Predicted Data  =  = (Randomised design spreads power over frequencies)

2. How about multiple conditions? We have talked about detecting a basic response vs baseline, but how about detecting differences between two or more response-types (event-types)?

Design Efficiency T = cTb / std(cTb) std(cTb) = sqrt(2cT(XTX)-1c) For max. T, want min. contrast variability (Friston et al, 1999) If assume that noise variance (2) is unaffected by changes in X… …then want maximal efficiency, e: e(c,X) = { cT (XTX)-1 c }-1

Efficiency - Multiple Event-types Design parametrised by: SOAmin Minimum SOA pi(h) Probability of event-type i given history h of last m events With n event-types pi(h) is a nm  n Transition Matrix Example: Randomised AB A B A 0.5 0.5 B 0.5 0.5 => ABBBABAABABAAA... 4s smoothing; 1/60s highpass filtering Josephs & Henson (1999) Differential Effect (A-B) Common Effect (A+B)

Efficiency - Multiple Event-types Example: Alternating AB A B A 0 1 B 1 0 => ABABABABABAB... 4s smoothing; 1/60s highpass filtering Josephs & Henson (1999) Permuted (A-B) Alternating (A-B) Example: Permuted AB A B AA 0 1 AB 0.5 0.5 BA 0.5 0.5 BB 1 0 => ABBAABABABBA...

Efficiency - Multiple Event-types Example: Null events A B A 0.33 0.33 B 0.33 0.33 => AB-BAA--B---ABB... Efficient for differential and main effects at short SOA Equivalent to stochastic SOA (Null Event like third unmodelled event-type) Selective averaging of data (Dale & Buckner 1997) 4s smoothing; 1/60s highpass filtering Josephs & Henson (1999) Null Events (A-B) Null Events (A+B)

Interim Conclusions Optimal design for one contrast may not be optimal for another With randomised designs, optimal SOA for differential effect (A-B) is minimal SOA (assuming no saturation; see later), whereas optimal SOA for main effect (A+B) is 16-20s Inclusion of null events improves efficiency for main effect at short SOAs (at cost of efficiency for differential effects) If order constrained, intermediate SOAs (5-20s) can be optimal If SOA constrained, pseudorandomised designs can be optimal (but may introduce context-sensitivity)

3. How about separating responses? What if interested in both contrasts [1 0] and [0 1]? For example: 1) Mixed designs (item-state effects) 2) Working Memory trials (stimulus-response) In the efficiency of a contrast (see earlier): e(c,X) = { cT (XTX)-1 c }-1 XTX represents covariance of regressors in design matrix High covariance increases elements of (XTX)-1 So, when correlation between regressors, efficiency to detect effect of each one separately is reduced

Correlations between Regressors [1 -1] [1 1] Negative correlation between two regressors means separate (orthogonal) effect of each is estimated poorly, though difference between regressors estimated well

Eg 1: Item and State effects (see earlier) Blocks = 40s, Fixed SOA = 4s Design Matrix (X) Efficiency = 16 [1 0] (Item Effect) Correlation = .97 Not good…

Eg 1: Item and State effects (see earlier) Blocks = 40s, Randomised SOAmin= 2s Design Matrix (X) Efficiency = 54 [1 0] (Item Effect) Correlation = .78 Better…

Eg 2: Stimulus-Response Paradigms Each trial consists of 2 successive events: e.g, Stimulus - Response Each event every 4s (Stimulus every 8s) Stim Resp Design Matrix (X) Efficiency = 29 [1 0] (Stimulus) Correlation = -.65

Eg 2: Stimulus-Response Paradigms Each trial consists of 2 successive events: e.g, Stimulus - Response Solution 1: Time between Stim- Resp events jittered from 0-8 seconds... Stim Resp Design Matrix (X) Efficiency = 40 [1 0] (Stimulus) Correlation = +.33

Eg 2: Stimulus-Response Paradigms Each trial consists of 2 successive events: e.g, Stimulus - Response Solution 2: Stim event every 8s, but Resp event only occurs on 50% trials... Stim Resp Design Matrix (X) Efficiency = 47 [1 0] (Stimulus) Correlation = -.24

Underadditivity at short SOAs Nonlinear Effects Underadditivity at short SOAs Linear Prediction Volterra Implications for Efficiency

The End This talk appears as Chapter 15 in the SPM book: http://www.mrc-cbu.cam.ac.uk/~rh01/Henson_Design_SPMBook_2006_preprint.pdf For further info on how to design an efficient fMRI experiment, see: http://www.mrc-cbu.cam.ac.uk/Imaging/Common/fMRI-efficiency.shtml

Cognitive Conjunctions Original (SPM97) definition of conjunctions entailed sum of two simple effects (A1-A2 + B1-B2) plus exclusive masking with interaction (A1-A2) - (B1-B2) Ie, “effects significant and of similar size” (Difference between conjunctions and masking is that conjunction p-values reflect the conjoint probabilities of the contrasts) SPM2 defintion of conjunctions uses advances in Gaussian Field Theory (e.g, T2 fields), allowing corrected p-values However, the logic has changed slightly, in that voxels can survive a conjunction even though they show an interaction A1-A2 B1-B2 p((A1-A2)= (B1-B2))>P2 p(A1=A2+B1=B2)<P1 + p(A1=A2)<p A1-A2 B1-B2 p(B1=B2)<p

Psycho-physiological Interaction (PPI) PPIs tested by a GLM with form: y = (V1A).b1 + V1.b2 + A.b3 + e c = [1 0 0] However, the interaction term of interest, V1A, is the product of V1 activity and Attention block AFTER convolution with HRF We are really interested in interaction at neural level, but: (HRF  V1)  (HRF  A)  HRF  (V1  A) (unless A low frequency, eg, blocked; so problem for event-related PPIs) SPM2 can effect a deconvolution of physiological regressors (V1), before calculating interaction term and reconvolving with the HRF Deconvolution is ill-constrained, so regularised using smoothness priors (using ReML)

Note on Epoch Durations As duration of epochs increases from 0 to ~2s, shape of convolved response changes little (mainly amplitude of response changes) Since it is the “amplitude” that is effectively estimated by the GLM, the results for epochs of constant duration <2s will be very similar to those for events (at typical SNRs) If however the epochs vary in duration from trial-to-trial (e.g, to match RT), then epoch and event models will give different results However, while RT-related duration may be appropriate for “motor” regions, it may not be appropriate for all regions (e.g, “visual”) Thus a “parametric modulation” of events by RT may be a better model in such situations

Epoch vs Events Rate = 1/4s Rate = 1/2s Though blocks of trials can be modelled as either epochs (boxcars) or runs of events… … interpretation of parameters differs… Consider an experiment presenting words at different rates in different blocks: An “epoch” model will estimate parameter that increases with rate, because the parameter reflects response per block An “event” model may estimate parameter that decreases with rate, because the parameter reflects response per word b=3 b=5 b=11 b=9

Blocked Randomised Data Model O = Old Words N = New Words O1 O2 O3 N1

Event-Related ~4s R F R = Words Later Remembered F = Words Later Forgotten Event-Related ~4s Data Model

“Oddball” … Time

Blocked Design “Epoch” model “Event” model Data “Epoch” model Model O1 O2 O3 N1 N2 N3 N1 N2 N3 “Event” model O1 O2 O3

BOLD Response Latency (Iterative) Height Peak Delay Onset Delay Dispersion Different fits across subjects Four-parameter HRF, nonparametric Random Effects (SNPM99) Advantages of iterative vs linear: Height “independent” of shape Canonical “height” confounded by latency (e.g, different shapes across subjects); no slice-timing error 2. Distinction of onset/peak latency Allowing better neural inferences? Disadvantages of iterative: 1. Unreasonable fits (onset/peak tension) Priors on parameter distributions? (Bayesian estimation) 2. Local minima, failure of convergence? 3. CPU time (~3 days for above) FIR used to deconvolve data, before nonlinear fitting over PST Height p<.05 (cor) 1-2 SNPM

Efficiency – Detection vs Estimation “Detection power” vs “Estimation efficiency” (Liu et al, 2001) Detect response, or characterise shape of response? Maximal detection power in blocked designs; Maximal estimation efficiency in randomised designs => simply corresponds to choice of basis functions: detection = canonical HRF estimation = FIR

Efficiency - Single Event-type Design parametrised by: SOAmin Minimum SOA p(t) Probability of event at each SOAmin Deterministic p(t)=1 iff t=nT Stationary stochastic p(t)=constant Dynamic stochastic p(t) varies (eg blocked) Blocked designs most efficient! (with small SOAmin)

1. Basic Response vs Baseline To detect a basic event-related response versus baseline… Do not present stimuli at a fixed rate Varying the SOA (eg via null events), with a minimal shortest SOA, is more efficient Presenting stimuli rapidly within on/off blocks of ~20s is even more efficient (though psychological downsides, eg predictability?) Longer blocks (>60 seconds) can be confounded by low-frequency noise