1. Reliability Measurement of the PIOMI Premature Infant Oral Motor Intervention Brenda S. Lessen, PhD, RN Illinois Wesleyan University Clare A. Morello, BSN, RN Children’s Hospital of Illinois
Reliability Rating Tool
Response to Study Data
Abstract
Training Prior to Reliability Study
Amateur video using term infant to demonstrate steps
PIOMI tool
One on one instructions
Hand model practice
Training Revisions Based on Study
Professional video using preterm infant; added correct timing
PIOMI tool with pictures/less text
Kept one on one instructions
Kept hand model practice
A reliability study was conducted in a Level III NICU to determine the reliability of the Premature Infant Oral Motor Intervention (PIOMI). The PIOMI is a five minute, pre-feeding intervention developed to improve feeding skills in the premature infant. Three registered nurses (RNs) were recruited and trained to perform the PIOMI. A specific training plan was established, and a reliability rating tool was developed. Two observers rated three RNs performing the PIOMI twice each on premature infants. The reliabilities among the observers (interobserver), among different RNs (interuser), and of the same RN performing the PIOMI twice (test-retest) were calculated. The PIOMI demonstrated high interobserver (97.57%), interuser (97.59%), and test-retest (97.58%) reliabilities.
CORRECT ORDER OF STEPS
CORRECT TECHNIQUE IN EACH STEP
CORRECT TIME TO PERFORM EACH STEP
Interobserver Reliability: Consistency in which more than one observer can correctly rate the person performing the PIOMI.
Interuser Reliability: Consistency in which the PIOMI can be performed correctly between multiple persons.
Test-retest Reliability: Ability of each person to perform the PIOMI consistently more than once.
Discussion
The PIOMI Reliability Rating Tool was found to be accurate and usable for measuring correct time, technique, and order and among more than one rater. It is also recommended for use in training.
Strong reliability is essential to assure that an intervention can be fully and properly implemented before translating evidence-based interventions into practice.
Effective training is imperative to achieve high levels of reliability, and the specific type of training must be measured along with any reliability data.
Subject Demographics
Age
Gender
Ethnic/Racial Background
Experience in the NICU
Number of Times Performing Oral Motor at Study Site
Number of Times Performing the PIOMI Prior to This Study
RN A 54
Female
Caucasian 34
>100 25
RN B 29 7
>50
RN C 37 13
20-30
Reliability Results
Correct Order
Correct Technique
Correct Timing
TOTAL RELIABILITY
Interobserver
100%
97.20%
95.52%
97.57%
Interuser
97.59%
RN A and RN B
95.83%
93.33%
96.39%
RN A and RN C
97.87%
98.58%
RN B and RN C
97.92%
95.45%
97.79%
Test-Retest
97.58%
RN A
95.65%
98.55%
RN B
95.35%
98.45%
RN C
87.23%
95.74%
IRB Approval
This research protocol was approved by the Illinois Wesleyan University Institutional Review Board, Peoria Institutional Review Board, OSF Saint Francis Nursing Review Board, and OSF Clinical Research Office. Both authors also completed the Collaborative Institutional Training Initiative (CITI) curriculum for Biomedical Researchers on Human Research. Written consent was obtained from all subjects.
Acknowledgements
The PIOMI demonstrated high interobserver (97.57%), interuser (97.59%), and test-retest (97.58%) reliabilities.
This study was partially funded by Illinois Wesleyan University. Illinois Wesleyan University had no role in the study design, implementation or publication. We would like to thank EmmaLee McMullen, Cheryl Lopeman, and Tina Neisler for their assistance in data collection
Weakest Reliabilities Targeted for Training
Primary References
Lessen, B.S. & Morello, C.A. (2012) Reliability Measurement of the Premature Infant Oral Motor Intervention. (Under Review) Journal of Neonatal Nursing
Lessen, B.S. (2011) Effect of the Premature Infant Oral Motor Intervention on Feeding Progression and Length of Stay in Preterm Infants, Advances in Neonatal Care, 11 (2),
Interobserver Reliability
Interuser Reliability
Test-Retest
Correct Technique
Cheek C-Stretch
94.44%
91.67% %
83.33% %
Lip Curl
94.12%
75.00% %
80.00% %
Midblade of Tongue
88.89%
100.00%
Correct Time
86.67% %
66.67% %
60.00% %
3 steps were found more difficult to consistently perform within the correct time frame and/or using the correct technique.