1. NC3Rs resources to improve the design of animal experiments
Dr Nathalie Percie du Sert
BBSRC STARS Course
Tuesday 11 April 2017

2. The NC3Rs (UK National Centre for the 3Rs)
Lead on the discovery and application of new technologies and approaches to replace, reduce and refine the use of animals for scientific purposes (the 3Rs)
Visit our website:
@NC3Rs
National Centre for the 3Rs
Work with research funders, journals, academia, industry and regulators
Activities divided between:
Research funding
Centre-led science programmes

3. NC3Rs resources The ARRIVE guidelines and the Experimental Design Assistant

4. Background Quality of published animal research
Experimental design
Only 12% of publications report randomisation and 14% report blinding
Sample size justification – missing in 100%
Reporting of studies
Animal characteristics – missing in 25%
Only 59% stated the study hypothesis, number and characteristics of animals used
Statistical analysis
Only 70% of publications fully described the statistical methods and presented the results with a measure of variability
Survey reviewed 271 publications and identified key areas for improvement
Kilkenny C, Parsons N, Kadyszewski E, Festing MF, Cuthill IC, Fry D, et al. (2009). Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLoS One 4(11): e7824.

5. Experimental design – internal validity
Consider threats which might compromise the validity of the experiment, any opportunities for the investigator to influence:
animal selection
conduct of the experiment
assessment of outcome
which outcomes are reported
Measures used to reduce validity threats include:
Random allocation to treatment groups
Allocation concealment
Blinding during outcome assessment
Inclusion/exclusion criteria

6. Experimental design – randomisation
Method is important – haphazard is not random
Use a validated procedure (e.g. computer generated, throw a dice, flip a coin)
Randomisation is crucial for two reasons:
Minimise selection bias
e.g. haphazard selection may results in slowest mice allocated to the same group
Key assumption of the statistical analysis
Different groups should be drawn from the same background population using random sampling

7. Experimental design – randomisation
347 responses
Is 17 the “most random” number?
Pick a number between 1 and 20

8. Experimental design – blinding
12 students
Maze-bright and maze-dull rats
Elevated T-maze, dark arm reinforced
Rats had been labelled bright or dull randomly
Only difference was in the minds of the investigators!
Rosenthal R, Fode KL (1963). The effect of experimenter bias on the performance of the albino rat. Behavioral Science 8(3):

9. Improving the reporting of in vivo research The ARRIVE guidelines
The ARRIVE guidelines were developed to improve the reporting of biomedical research using animals.
Checklist of 20 items, containing key information necessary to describe a study comprehensively and transparently.
The guidelines include:
Information which relates to internal validity
Information which would allow a study to be repeated
Information about the context and scientific relevance of the study

10. The EDA Experimental Design Assistant
The EDA was developed to improve the design of animal experiments
Web-based tool
Aimed at in vivo researchers
Developed as a collaboration between:
In vivo researchers
Statisticians
Academia and industry
Software designers specialised in artificial intelligence
Road tested by researchers and statisticians

11. The EDA offers:
The ability to build a stepwise visual representation of an experiment – the EDA diagram
Effect of drug A on plasma glucose levels
Animals characteristics:
diabetic mice
Experimental unit:: mouse
Vehicle
Drug
Measurement:
Plasma glucose
Outcome
Measure:
Glucose levels
Analysis
Group 2
Group 1
Pool of animals
Allocation:
Complete randomisation
Pharmacological intervention 1
Pharmacological intervention 2
Independent variable of interest : Drug A

12. The EDA offers:
The ability to build a stepwise visual representation of an experiment – the EDA diagram
Practical steps
Analysis
Experiment
Effect of drug A on plasma glucose levels
Animals characteristics:
diabetic mice
Experimental unit:: mouse
Vehicle
Drug
Measurement:
Plasma glucose
Outcome
Measure:
Glucose levels
Analysis
Group 2
Group 1
Pool of animals
Allocation:
Complete randomisation
Pharmacological intervention 1
Pharmacological intervention 2
Independent variable of interest : Drug A

13. The EDA diagram
Examples
Templates
Feedback from the critique

18. The EDA offers:
The ability to build a stepwise visual representation of an experiment – the EDA diagram
Feedback and advice on your experimental plan
Dedicated support for randomisation, blinding and sample size calculation
Practical information to improve knowledge of experimental design
Improved transparency of the experimental plan, allowing more efficient communication

19. Feedback and advice from the EDA
Dataset of rules triggers prompts based on the EDA diagram
Feedback provided:
Diagram structure
Ask to provide more information
Point out inconsistencies
Prompt you to consider things that are not on the diagram
Highlight implications of some of the choices made
Provide recommendation for analysis
Rule set to be expanded over time
Provide more feedback
Enable the system to detect more subtle issues
Increasingly specific feedback

20. The EDA offers:
The ability to build a stepwise visual representation of an experiment – the EDA diagram
Feedback and advice on your experimental plan
Dedicated support for randomisation, blinding and sample size calculation
Practical information to improve knowledge of experimental design
Improves transparency of the experimental plan and helps communication

21. The EDA offers:
The ability to build a stepwise visual representation of an experiment – the EDA diagram
Feedback and advice on your experimental plan
Dedicated support for randomisation, blinding and sample size calculation
Practical information to improve knowledge of experimental design
Improved transparency of the experimental plan, allowing more efficient communication

22. The EDA offers:
The ability to build a stepwise visual representation of an experiment – the EDA diagram
Feedback and advice on your experimental plan
Dedicated support for randomisation, blinding and sample size calculation
Practical information to improve knowledge of experimental design
Improved transparency of the experimental plan, allowing more efficient communication

23. The EDA workflow

24. Objectives Improve the reliability of published results
Promote better understanding of experimental design, raise awareness about issues
Facilitate peer review/assessment of the experimental plans with an explicit description
Transparency
Pre-registration
Promote more careful consideration of the experimental plans
Spend time planning
Diagram facilitate discussion

25. EDA uptake
~2500 accounts on the system with 30 diagrams created per week Recommended by UK funders

26. Acknowledgments https://eda.nc3rs.org.uk www.nc3rs.org.uk/ARRIVE
NC3RS working group
Prof Clare Stanford (Chair), UCL
Dr Simon Bate, GlaxoSmithKline
Dr Manuel Berdoy, University of Oxford
Dr Robin Clark, Huntingdon Life Sciences
Prof Innes Cuthill, University of Bristol
Dr Derek Fry, University of Manchester
Dr Natasha Karp, Wellcome Trust Sanger Institute
Prof Malcolm Macleod, University of Edinburgh
Dr Lawrence Moon, King’s College London
Dr Richard Preziosi, University of Manchester
Certus Technology
Dr Brian Lings
Mr Ian Bamsey
Alpha testers
Beta testers