1. Conducting a Pilot Study
Osama A Samarkandi, PhD, RN
BSc, GMD, BSN, MSN, NIAC
EMS 423; EMS Research and Evidence Based Practice

2. Pilot Study
The pilot study is: an opportunity for the researcher to test his or her study design,
Incorporates every step of the research process,
The major thrust of the pilot study is to test the experimental protocol,
The pilot study might demonstrate that an intervention technique.
After deciding what to study, designing the study, and taking care of all of the interactions with other people, the researcher still has one more task before actually implementing the study and collecting data: conducting a pilot study. This step may be the most undervalued and most often omitted piece of any research project. In fact, it is at least as important as any other step in the process, if not more so. Conducting a pilot study ultimately results in a better research project. The pilot study allows the researcher to practice the implementation of the study, to identify weaknesses and problems in the study design and to make appropriate revisions to the research project before valuable time and resources have been wasted. It is, effectively, a dress rehearsal.
A pilot study incorporates every step of the research process, but many or those steps will have already been completed in the planning of the actual study. The topic will have been decided, the literature search completed, and the question and hypothesis formulated. The type of study will have been chosen, the methods developed, and a power calculation performed. Ethics approval will have been obtained, and the researcher will have already arranged for the assistance of colleagues.
The major thrust of the pilot study is to test the experimental protocol. The data collection process, and the planned analysis-steps that are yet to be completed. During the pilot study, subjects are enrolled and data are collected in the manner outlined by the study protocol. The primary difference between the pilot and actual study is that only a small number of subjects are enrolled. How many subjects are enrolled, or how long the pilot study goes on, depends on how long it takes to thoroughly test the experimental protocol. For a study that will involve reviewing 200 charts, the pilot might only include 15 or 20 charts. For a survey that will be distributed to thousands or local residents. The pilot study might include hundreds of people. It is up to the investigator to decide how much practice is needed before moving on to the actual study.
As the pilot study proceeds, the investigator can examine the process and determine what things, if any, about the study design are not working or not turning out as expected. Because this is only a pilot trial, these problems will not fatally affect the actual study. Indeed, the investigator wants to encounter as many problems and as many shortcomings as possible during the pilot study. By identifying these issues during a pilot study, the investigator can tweak the research protocol without corrupting the actual study.
The pilot study can identify shortcomings in any step of the study, including those that have already been completed and those that have yet to be instituted. Throughout the pilot study process, the researcher should be considering how the results of the pilot project might affect each of those steps.
The pilot study might demonstrate that an intervention technique, as described in the literature for emergency department patients, can't be performed in the pre-hospital environment. The researcher can use the findings of the pilot study to modify the technique in a way that makes it work. Or, if the researcher is using a medication dose recommended in the literature, but the number of side effects seems high, the researcher can go back to the literature for more information and consider reducing the dose or using a different medication.
If the pilot study suggests it will be extremely difficult to collect enough data to test a certain null hypothesis, for example one using "survival as an outcome, the researcher may choose to revise the hypothesis and study "return of pulse" as an out-come. Or, if it appears that the effect of an intervention will be much larger than originally anticipated, the investigator might be able to revise the hypothesis and at the same time reduce the sample size required to test the hypothesis. Such changes are not inappropriate as long as they are made before the actual data collection process is started. Making such changes in a hypothesis after a study has been implemented and true data collection has started would be inappropriate. That's why the pilot study is essential.
The pilot study might show that the data needed can’t be obtained by a retrospective chart review, and the researcher could then revise the study to be prospective in nature. Or the pilot trial might illustrate that, while the data are available retrospectively, the researcher must abstract those data from the subjects inpatient records because the EMS reports are frequently, incomplete. The protocol would have to be revised to include review of the hospital record.
Any revision or the protocol would also require that the I RB review and approve those changes. Other IRB concerns that the pilot trial might demonstrate include problems with the consent process, with the ability of the investigator to maintain confidentiality, or with other ethical issues. By addressing these issues before the actual study begins, the researcher would be protecting both the il1legrilY of the study and the privacy of the subjects.
One of the most likely areas in which a pilot trial might identify problems is with the cooperation of the investigator's colleagues in conducting the study. If field EMS providers don't complete the data collection forms during a pilot trial, there's no reason to believe they'll do it during the actual study. The researcher will have to find a way to encourage data collection. Listening to those procedures is an important part of this process. During the pilot trial, tile researcher should seek feedback and advice from those colleagues. It might turn out that something as simple as changing the layout of a data collection form or the sequence of the data collection process can make the implementation of the study protocol much more palatable. Whether the), are paramedics, nurses, physicians, or eyen bench scientists, nothing will discourage the participation of colleagues like in convenience. Making the research process as user-friendly as possible will go a long way toward the success or a project. The pilot study can be invaluable in streamlining the data collection process.
The data collected during the pilot project should also be subjected to the same statistical analysis as planned for the actual study. Through this process the researcher will be able to determine whether the tests that have been chosen are appropriate and/or practical. If the analysis was originally planned anticipating normally distributed data, but the pilot suggests that the data arc skewed, the Statistical tests might have to be changed. If the original plan was to report descriptive statistics as percentages, the pi lot analysis might suggest that raw numbers would be more meaningful to the reader. The researcher should not use these trial analyses to draw conclusions about the study question; it 's unlikely that any pilot study would have sufficient power for that. The pilot analysis is only intended to determine whether the analytical approach is sound.
As always, a change in anyone step or the research process will probably affect other steps. If the pilot study suggests a change in the original question, that will likely result in a change in the methodology or the analysis. If the pilot study results in a change in the statistical analysis, the researcher will also have to reconsider the sample size calculation, and perhaps the manner in which data are recorded. Any change in the experimental protocol will also need 10 be reported to and reviewed by the IRB. All of the pieces of the research process arc intertwined.
There are some negative points associated with doing a pi lot trial. For example, if the study population is small-perhaps a specific paramedic class-then conducting the pilot trial might introduce some bias if the subjects for the pilot trial and the actual study are the same people. In a single-system clinical trial, paramedic behaviors could be biased in the same way. There are ways to limit these effects, the easiest being to ask another researcher from a different setting-perhaps a neighboring system--to conduct the pilot study. Unfortunately, that will leave open the possibility that issues about the setting will go undiscovered. The researcher will have to decide which is greater, the risk or introducing bias or the risk or not conducting the pilot in the actual study setting.
Another reason researchers often omit a pilot study is lime. Conducting a pilot study almost always means that data collection for the actual stud), will be delayed. Often researchers are faced with deadlines. Deadlines might be imposed by funding agencies, or might be necessary because of an upcoming change in patient care protocols. Time can also be an issue if the researcher is hoping to finish the study in time to submit the results to a particular meeting. More often, though, issues about time are simply related 10 the anticipation and impatience or the researcher.
Whatever reasons (or excuses) one has for not doing a pilot study, they are rarely substantial enough to outweigh .he risks or proceeding to data collection without doing one. Almost every study will encounter problems. The failure to identify those problems ahead of time and to rectify as many or them as possible is often catastrophic for research projects. Things that, when identified prospectively, are mild irritations and easy to fix can destroy a project when they arise unexpectedly.
The amount or time, energy, and money spent on a pilot project is usually minuscule compared to the prospect of losing all of the work that went into, and the data produced by a full-scale study that falls apart halfway through the process.
Some researchers who have a wealth of experience and a proven track record in a given topic area have enough knowledge about research in a particular realm that they are able to forgo a pilot trial. Those people are few and far between, and even they proceed with the understanding that they are taking some risks. Most experienced researchers understand the importance of a good pilot stud)" and would never proceed without doing one.

3. Summary Test to run study protocol, Examine research process steps.
The pilot trail is a test run of the study protocol. It gives the investigator a chance to examine each step of the research process; those already completed and those yet to come. By conducting a pilot trial, the researcher can identify problems with the study before implementing full-scale data collection. The pilot study is the last chance the investigator will have to go back and refine previous steps in the research process. While conducting a pilot trial can be challenging and can delay study implementation, the benefits of conducting a pilot stud) far outweigh the risks of not conducting one.