1. Nicola Totton Bangor University
Review of pilot and feasibility studies from a registry website; an overview of recent practice.
Nicola Totton
Bangor University

2. Introduction
Pilot and feasibility studies are small studies used as a precursor to completing a full clinical trial
Method for evaluating the trial suitability and testing study components in practice
Aim to assess recruitment and retention rates
Pilot and feasibility work recommended by the MRC1 and several researchers2-6
Launch of the Journal of Pilot and Feasibility Studies in January 2015
Developing and evaluating complex interventions: new guidance. MRC, 2006.
A tutorial on pilot studies: the what, why and how. Thabane L et al (2010). BMC Medical Research Methodology.
Design and analysis of pilot studies: recommendation for good practice. Lancaster GA, Dodd S & Williamson PR. (2002). Journal of Evaluation in Clinical Practice.
From pilot studies to confirmatory studies. N. Duan, Shanghai Archives of Psychiatry.
Pilot and feasibility studies come of age! G Lancaster, Pilot and Feasibility Studies.
The importance of pilot studies. ER van Teijlingen & V Hundley. Social Research Updated, University of Surrey.

3. What are the differences?
The terms are often used interchangeably
A framework8 was published in 2016 to define
Feasibility studies are exploratory and an overarching term when assessing whether a future study can be done
Pilot studies are a mini replication of a future study
Pilot studies are a specific subset of feasibility studies
Feasibility Studies
Pilot Studies
In practice, these terms are often used interchangeably to describe a small scale trial and there is confusion with the difference between the two.
In 2016, a framework was published which aims to clearly define what is a pilot and what is a feasibility study. This suggested that the terms were not mutually exclusive and that a pilot study can be considered as a specific type of feasibility study.
8. Defining Feasibility and Pilot Studies in Preparation for Randomised Controlled Trials: Development of a Conceptual Framework. Eldridge SM, Lancaster GA et al, 2016.

4. Aims
Prevalence: Evaluate the prevalence of pilot and feasibility studies over time
Characteristics: Assess key characteristics and whether these differ between pilot and feasibility studies
Issues: Identify issues currently present within pilot and feasibility research

5. Methods
The ISRCTN website is an international clinical trials registry
A web scraping methodology was adopted using R
Search terms “pilot” and “feasibility” were used on the public title field
Pilot term returned 474 records and feasibility returned 215 records
In total 664 unique studies were identified
Andrew Brand (NWORTH) has used a similar methodology to create an online tool for easily evaluating the characteristics of registered pilot and feasibility studies. A poster describing the system (An Online Tool for Exploring Recruitment Achievability for a Feasibility and Pilot Studies in the UK) can be found on poster board 12, Hall 2A.

6. Methods
Prevalence: The proportion of pilot and feasibility trials from all registered studies assessed per year
Characteristics:
Data such as target sample size, type of study, start and end date etc. were extracted into an Excel file
Subset of trials evaluated to in more detail and compared to review to assess changes over time
Issues: Identify any issues arising from previous studies found from the subset

7. Results - Prevalence 69% pilot studies 30% feasibility studies
1% both a pilot and feasibility study

8. Results - Prevalence
The overall proportion remains consistent but number of feasibility studies has increased to match pilot studies
Whilst there is some fluctuation, the overall proportion of trials labelled as pilot or feasibility generally remains consistent over time, possibly with a slightly increase as shown by the dark red line.
The red line shows pilot studies were initially more common, however in recent years feasibility studies, shown by the orange line, have increased in popularity and are now on a par with pilot studies.

9. Results - Characteristics
Median target participants per trial:
Overall - 50 (range: 5-130,000)
Pilot studies - 44 (range: 5-130,000)
Feasibility studies - 60 (range: 6-1,600)
Median time period for the trial:
Overall - 18 months (range: 0-126)
Pilot studies - 17 months (range: 0-126)
Feasibility studies months (range: 0-107)

10. Results - Characteristics
Condition category:
Mental and behavioural disorder studies (20%)
Cancer studies (13%)
Study type:
Interventional (96%)
Randomised controlled trials (85%)
Setting:
Hospital (45%)
Recruitment:
Single country (97%)

11. Results – Characteristics
Trial Component Included?
ISRCTN Subset (2011)
Overall
(n=31)
Pilot
(n=22)
Feasibility
(n=9)
Sample size calculation
32%
25%
Randomisation
74%
73%
75%
Control group
77%
63%
Blinding
26%
36% 0%
Hypothesis testing
33%
38%
Further study suggested
78%
83%
67%
Review13
(n=23)
35%
62%
69%
19%
81%
13. What is a pilot or feasibility study? A review of current practice and editorial policy. Arian et al, BMC Medical Research Methodology.

12. Results – Issues Publication bias
Pilot and feasibility terms assigned incorrectly in 29% of cases
Unclear motivation
Unclear aims of the study
Inappropriate efficacy testing

13. Conclusions
Pilot and feasibility studies are shorter than full trials with less participants
Slight differences within the characteristics between pilot and feasibility studies
Studies sometimes mislabelled and have unclear motivation
Aims do not line up with the intended design
Results often not being published

14. Thank you
Any Questions?