1. Dr. Matthew Keough August 8th, 2018 Summer School
(Brief) RECOMENDATIONS FOR designing your randomized controlled trial (rct)
Dr. Matthew Keough
August 8th, 2018
Summer School

2. What is an RCT?
A randomized controlled trial (RCT) is an experiment in which investigators randomly assign eligible human research participants or other units of study (e.g., classrooms, clinics, playgrounds) into groups to receive or not receive one or more interventions that are being compared. The results are analyzed by comparing outcomes in the groups. (CIHR, 2018)
A scientific experiment that tests a new treatment, while simultaneously tries to reduce bias (Kraemer & Wilson, 2002)

3. Psychological Perspective

4. Strongest Weakest Meta-Analysis RCTs Cohort studies
Case-control studies
Case series, case reports
Editorials, expert opinion
Strongest
Weakest

6. MAKE SURE THAT YOUR TREATMENT IS EMPIRICALLY-INFORMED
recommendation #1
MAKE SURE THAT YOUR TREATMENT IS EMPIRICALLY-INFORMED
Are you the first one to tackle this problem?
What has been done already?
Is there a “best practice” treatment that you can improve on?

7. Pre-register your trial protocol Do what you say you’ll do
RECOMMENDATION #2
2. BE TRANSPARENT
Pre-register your trial protocol
Do what you say you’ll do
Do not deviate from your original plans

9. Question: How do Null results relate to registration?
These too have the potential to inform clinical decisions
Can identify flaws in the research design
Week 2 readings: problem with psychology is studies that don’t find significant results aren’t published!!!!

10. 3. DESIGN WELL AND MEASURE Who is in your target population?
RECOMMENDATION #3
3. DESIGN WELL AND MEASURE
Who is in your target population?
Who will you allow in?
What groups will you have?
What symptoms/variables will you measure and how often?

11. Who is in your target population and who will you let in?
Traditional method: recruit ”purest” groups possible (i.e., maximize internal validity)
Easier said than done (especially in psychology)
Recent method: allow for more “complex groups” but just measure and model it statistically (i.e., maximize external validity)

12. Question: why do you think having a heterogeneous sample is beneficial?
Question: want to improve generalizability, results can have more impact
Solves methodological issues with recruitment

13. SIDENOTE: efficacy vs. effectiveness
Efficacy: testing if your treatment works “in the lab” or under ideal, controlled conditions
Specific interventions
Internal validity
Strict methodology
The next major category of decision points would be the actual purpose of the study. As I mentioned, RCTs are typically comparing one thing to another, but there this really boils down to 2 types of RCTs: efficacy and effectiveness.
Efficacy to me is what most people think of when they hear RCT. This is looking at the effects of specific interventions
Hunsley, Elliot, & Therrien, 2013; Nathan, Stuart, & Dolan, 2000

14. SIDENOTE: efficacy vs. effectiveness
Effectiveness: testing if your treatment works “in the real world” or under less controlled conditions
Feasibility, real-world situations
External validity
More open methodology
The next major category of decision points would be the actual purpose of the study. As I mentioned, RCTs are typically comparing one thing to another, but there this really boils down to 2 types of RCTs: efficacy and effectiveness.
Efficacy to me is what most people think of when they hear RCT. This is looking at the effects of specific interventions
Hunsley, Elliot, & Therrien, 2013; Nathan, Stuart, & Dolan, 2000

15. What groups will you have?
Clinical trials examine how something performs compared to another (Nathan, Stuart, & Dolan, 2000; Parloff, 1986)
Waitlist control: = passive
‘Standard practice’ control = receiving commonly accepted treatment (i.e., TAU)
Placebo control group
it is not surprising to us that methodology is probably one of the most time-consuming parts of research, but also the most important. This is essentially determining if you are able to find what you are looking for (e.g., if you don’t run a power analysis and have the right number of people, you can’t expect to detect a result even if it is there)

16. Question: How does the type of control group impact interpretation?
Another question: what control group do you think is most relevant to psychotherapy??
Question:
You are comparing an active intervention to a group that is getting nothing. Depending on your resaerch question, this may not be addressing what you were hoping/wanting
This impacts interpretation because it limits what you can be concluding. So can you say that your new intervention is better than people that are getting nothing (which in reality, may not be the best or most accurate comparison you can be making) or can you say that you intervention does not have that much difference between the current, standard practice out there.
It also depends on your research question. You want to make sure you have a method that allows you to actually measure what you are wanting!!
Some argue it is sufficient to compare to a group of individuals who are not receiving any type of intervention or treatment (passive), while others argue it is more ethical to have the control group be “standard practice” for that group

17. Methodology: blinding
Blinding is when the patient/researcher or both are unaware of the condition that they are in
Psychotherapy research most often has patient single-blinding
Pro: Eliminates bias
Con: Very difficult to have double-blinding (Nathan, Stuart, & Dolan, 2000)

18. QUESTION: WHY DO YOU THINK IT IS DIFFICULT TO HAVE COMPLETE BLINDING?
Or phrased differently? What poses as a challenge of blinding in psychotherapy research?
Very challenging for therapist to not know the intent
SO what do you thnk is best practice?

19. What symptoms/variables will you measure and how often?
Need to specify a primary outcome
Specify secondary outcomes (and you should!)
Make sure all measures are reliable (and ideally have been validated in your target population)
Measure moderators and mediators as well!!

20. Evaluating outcomes: Moderation and mediation
Moderators specify for whom and/or under what conditions treatment works (Kraemer & Wilson, 2002)
Identify subpopulations with possibly different causal mechanisms and who may benefit most
Meditators identify possible mechanisms through which a treatment might achieve its effects
The so-called “why”

21. Question: what is the relevance of moderators and mediators for evaluating outcomes?
as alluded to above, these methods can be used in a way to further identify the benefit of the intervention, and as a way to combat a high number of inclusion/exclusion criteria

22. 4. DECIDE HOW YOU WILL HANDLE MISSING DATA
RECOMMENDATION #4
4. DECIDE HOW YOU WILL HANDLE MISSING DATA
How will you handle drop-outs?
Should you include them? Or exclude them?

24. Question: Why do drop-outs pose a problem for an RCT?

25. DECIDE HOW YOU WILL HANDLE MISSING DATA
INTENTION TO TREAT
Include all randomized participants
Impute data OR use full information methods
Depends on “pattern of missingness” or lack thereof
COMPLETE CASE ANALYSIS
Not the best method BUT, can be unbiased if missingness is random