Study Design and Efficiency Margarita Sarri Hugo Spiers

We will talk about: What kinds of designs are out there? - Blocked vs event-related designs What kinds of designs are out there? - Blocked vs event-related designs How can I order my events? How can I order my events? What is estimation efficiency? What is estimation efficiency? Which designs are more efficient? Which designs are more efficient? Spacing of events Spacing of events Sampling issues Sampling issues Filtering issues Filtering issues

Event related vs Blocked designs Blocked / Epoch/ Box design Blocked / Epoch/ Box design Types of trials are ‘blocked’ together e.g. AAAAA BBBBB AAAAA. Types of trials are ‘blocked’ together e.g. AAAAA BBBBB AAAAA. Event related design Event related design Types of trials are interleaved and each trial is modelled separately as an ‘event’ e.g. AABABBAB Types of trials are interleaved and each trial is modelled separately as an ‘event’ e.g. AABABBAB

In general 2 blocks more efficient than 4. In general 2 blocks more efficient than 4. Ideal modulation frequency being approximately 16sec Ideal modulation frequency being approximately 16sec but you may not be able to test certain things with such a design… So you may want to go for an event related design… Blocked design  typically used in experiments where the detection of activation is the primary goal. e.g localise a specific brain region showing a differential response to one type of stimulus (e.g. faces vs houses)  e.g localise a specific brain region showing a differential response to one type of stimulus (e.g. faces vs houses)

Why should I use efMRI ? Flexibility and randomization Flexibility and randomization eliminate predictability of block designs eliminate predictability of block designs avoid practice effects/strategy use avoid practice effects/strategy use Post hoc sorting Post hoc sorting e.g. classification of correct vs. incorrect, subjective perception: aware vs. unaware, remembered vs. forgotten items, parametric scores: e.g. fast vs. slow RTs e.g. classification of correct vs. incorrect, subjective perception: aware vs. unaware, remembered vs. forgotten items, parametric scores: e.g. fast vs. slow RTs Measuring novelty: Rare or unpredictable events Measuring novelty: Rare or unpredictable events e.g. oddball designs. e.g. oddball designs. Allows to look at events on a shorter time scale. Allows to look at events on a shorter time scale. P L H A K

But you can also combine block and efMRI… A block can be treated as a continuous train of event-trials E.g Otten, Henson & Rugg, Nature Neuroscience 2002 E.g Otten, Henson & Rugg, Nature Neuroscience 2002 ‘Subsequent memory’ experiment separating transient (events) and sustained (blocks) neural activity. At the beginning of each trial a cue instructed subjects to make an phonological or semantic judgement. 83secrest83sec

Hmmm I think I like efMRI. But how do I order my trials?

efMRI: Sequencing of events Deterministic designs: the occurrence of events is pre-determined e.g. a blocked design or alternating design (all the probabilities are zero or one ) Stochasticdesigns: the occurrence of an event depends on a a specified probability e.g. random or permuted design Stochastic designs can be stationary or dynamic Blocked Alternating Random Permuted

How do I do I create a permuted order of events? ensure mini-runs of same stimuli… ensure mini-runs of same stimuli… i.e. modulate the probability of different event-types over experimental time

Permutation methods continued…

So what is Efficiency?

Efficiency is… Efficiency is a numerical value Efficiency is a numerical value which reflects the ability of your design to detect the effect of interest General Linear Model: General Linear Model: Y = X. β + e Data Design Matrix Parameters error Efficiency is the ability to estimate β, given the design matrix X Efficiency is the ability to estimate β, given the design matrix X Efficiency can be calculated because the variance of β is proportional to the variance of X Efficiency can be calculated because the variance of β is proportional to the variance of X

What is variance? Standard Deviation Variance = Standard Deviation 2 Variance = Standard Deviation 2 High Variance Low Variance Standard Deviation

Testing a Hypothesis T- Test for the difference between 2 conditions Lower ability to detect a difference Higher ability to detect a difference Standard Deviation By reducing the variance in the design we can maximize our T values

How do we calculate it? Efficiency  Inverse( Var(β) ) Efficiency  Inverse( Var(β) ) Inverse( Var(β) )  Var(X) Inverse( Var(β) )  Var(X) Var(X)  Inverse( X T X ) Var(X)  Inverse( X T X )

A B C D A B C D XXTXT A B C D = A B C D A B C D A B C D XT XXT X Non- overlapping conditions Overlappingconditions

A B C D A B C D A B C D XT XXT X inverse (X T X) A B C D A B C D A B C D

The efficiency is related to the specific contrast you are interested in Efficiency = inverse(σ 2 c T Inverse(X T X) c) Where c = contrast σ 2 = noise variance But if we assume that noise variance σ 2 is constant then: Efficiency = inverse (c T Inverse (X T X) c)

When c is Simple Effect, e.g. main effect of A c = [ ] inverse(X T X) A B C D A B C D A B C D Efficiency = Inverse( c T Inverse(X T X) c) A, B:Efficiency = 1 / 0.2 = 5 C, D: Efficiency = 1 / 0.6 = CTCT C

When c is contrast difference, e.g. For A – B c = [ ] inverse(X T X) A B C D A B C D A B C D Efficiency = Inverse( c T Inverse(X T X) c) A-B: Efficiency = 1 / 0.4 = 2.5 C-D: Efficiency = 1 / 2 = CTCT C

Variable No. of Trials X inv(X T X) 4.2 Random: Events = Relative Efficiency Random: Events = 50

How does trial order effect Efficiency?

Example ORDER 1 Interleaved stimuli ORDER 2 Blocks of stimuli

A B C D E F A B C D E F Different Designs – Boxcar Events inv(X T X) A B C D E F A B C D E F A B C D E F X Blocked FixedInterleaved Random

1.5 Different Designs Blocked FixedInterleaved Random-Uniform Random-Sinusoidal inv(X T X) X Relative Efficiency

Different Designs 1.5 Blocked X inv(X T X) Relative Efficiency

Sequencing of events Stochastic designs: at each point at which an event could occur there is a specified probability of that event occurring. The timing of when the events occur is specified. Non-occurrence = null event. Deterministic designs: the occurrence of events is pre- determined. The variable deterministic design i.e. a blocked design, is the most efficient.

Joel’s example of different stimulus presentations Blocked design Fully randomised Dynamic stochastic A B C Tasks Efficiency calculation

different designs different designs { minimum SOA (inter-stimulus interval) probability of occurrence

How fast can I present my trials?

The absolute minimum… Early event-related fMRI studies used a long Stimulus Onset Asynchrony (SOA) to allow BOLD response to return to baseline (20-30s). Early event-related fMRI studies used a long Stimulus Onset Asynchrony (SOA) to allow BOLD response to return to baseline (20-30s). However, if the BOLD response is explicitly modelled, overlap between successive responses at short SOAs can be accommodated… (assuming that successive responses add up in a linear fashion) However, if the BOLD response is explicitly modelled, overlap between successive responses at short SOAs can be accommodated… (assuming that successive responses add up in a linear fashion) The lower limit on SOAs is dictated by nonlinear interactions among events that can be though of as saturation phenomena or ‘‘refractoriness’’ at a neuronal or hemodynamic level. The lower limit on SOAs is dictated by nonlinear interactions among events that can be though of as saturation phenomena or ‘‘refractoriness’’ at a neuronal or hemodynamic level. But, very short SOAs (< 1s) are not advisable as the predicted additive effects upon the HRF of two closely occurring stimuli break down. But, very short SOAs (< 1s) are not advisable as the predicted additive effects upon the HRF of two closely occurring stimuli break down. Brief Stimulus Undershoot Initial Undershoot Peak So you can have events occurring even every 1-2 sec! But think of psychological validity! max. oxygenation: 4- 6s post-stimulus

And how should my events be spaced? optimal SOA

Choosing the best SOA Optimal SOA depends on: Optimal SOA depends on: Probability of occurrence (design) Probability of occurrence (design) Whether one is looking for evoked responses per se or differences in evoked responses. Whether one is looking for evoked responses per se or differences in evoked responses. Generally SOAs that are small and randomly distributed are the most efficient. Rapid presentation rates allow for the maintenance of a particular cognitive or attentional set, decrease the latitude that the subject has for engaging alternative strategies, or incidental processing. Random SOAs ensure that preparatory or anticipatory factors do not confound event-related responses and ensure a uniform context in which events are presented.

Probability SOA ONE TRIAL TYPE TWO TRIAL TYPES Main effect Differential responses the most efficient SOA for differential responses is very small. the most efficient SOA for differential responses is very small. longer SOAs of around 16 s are necessary to estimate the responses themselves. longer SOAs of around 16 s are necessary to estimate the responses themselves. Stationary Stochastic designs

What should I do if I am interested in the main effects (‘evoked responses’)? You can use long SOA’s (around 16 secs!). But behaviourally this may be inefficient You can use long SOA’s (around 16 secs!). But behaviourally this may be inefficient So you can introduce ‘null’ events and keep your SOA short. So you can introduce ‘null’ events and keep your SOA short. These null events now provide a baseline against which the response to either trial type 1 or 2 can be estimated even using a very small SOA. (p= ) These null events now provide a baseline against which the response to either trial type 1 or 2 can be estimated even using a very small SOA. (p= ) to identify areas that are activated by both event types

Here is what happens when you add null events… Random Note that although null events increase efficiency for main effects (at short SOA’s), they slightly decrease efficiency for differential effects

What should I do if I am interested in the differential effects? For very short SOA’s use a randomised design But for medium SOA’s a permuted (4-6sec) or an alternating (8sec) design is better

To sum up: Remember that… Blocked designs generally more efficient Blocked designs generally more efficient Some random event-related designs are much better than others. Some random event-related designs are much better than others. Different design is appropriate depending on what you want to optimize. Different design is appropriate depending on what you want to optimize. Critical properties to optimize Critical properties to optimize Ordering of trials Ordering of trials spacing between stimuli spacing between stimuli

Timing of the SOAs in relation to the TR If the TR (Repetition Time of slice collection) is divisible by the SOA then data collected for each event will be from the same slices, at the same points along the HRF. If the TR (Repetition Time of slice collection) is divisible by the SOA then data collected for each event will be from the same slices, at the same points along the HRF. Therefore, either choose a TR and SOA that are not divisible or introduce a ‘jitter’ such that the SOA is randomly shifted. Therefore, either choose a TR and SOA that are not divisible or introduce a ‘jitter’ such that the SOA is randomly shifted. Scans TR = 4s Stimulus (synchronous) SOA=8s Stimulus (asynchronous) SOA=6s Stimulus (random jitter)

Temporal Filtering: The High Pass Filter A temporal filter is used in fMRI to get rid of noise, thus increasing the efficiency of the data. A temporal filter is used in fMRI to get rid of noise, thus increasing the efficiency of the data. Non-neuronal noise tends to be of low- frequency, including ‘scanner drift’ and physiological phenomenon. Non-neuronal noise tends to be of low- frequency, including ‘scanner drift’ and physiological phenomenon. Applying a high pass filter means that parameters that occur at a slow rate are removed from the analysis. Applying a high pass filter means that parameters that occur at a slow rate are removed from the analysis. The default high pass filter in SPM is 128s, thus if you have experimental events occurring less frequently than once every 128s then the associated signal will be removed by the filter!! The default high pass filter in SPM is 128s, thus if you have experimental events occurring less frequently than once every 128s then the associated signal will be removed by the filter!!

Sources

Summary Blocked designs are generally the most efficient, but blocked designs have restrictions. Blocked designs are generally the most efficient, but blocked designs have restrictions. For event-related designs, dynamic stochastic presentation of stimuli is most efficient. For event-related designs, dynamic stochastic presentation of stimuli is most efficient. However, the most optimal design for your data depends on the SOA that you use. The general rule is the smaller your SOA the better, but sometimes a small SOA may not be possible. However, the most optimal design for your data depends on the SOA that you use. The general rule is the smaller your SOA the better, but sometimes a small SOA may not be possible. Also, the most optimal design for one contrast may not be optimal for another e.g. the inclusion of null events improves the efficiency of main effects at short SOAs, at the cost of efficiency for differential effects. Also, the most optimal design for one contrast may not be optimal for another e.g. the inclusion of null events improves the efficiency of main effects at short SOAs, at the cost of efficiency for differential effects. Finally, there is no point scanning two tasks to look for differences between them if they are too different or too similar. Finally, there is no point scanning two tasks to look for differences between them if they are too different or too similar.