1. Self-Report of Waking-State Oral Parafunctional Behaviors
in the Natural Environment
Sarah E. F. Kaplan, Richard Ohrbach
School of Dental Medicine, University at Buffalo, Buffalo, NY, USA
Presentation #3907
ABSTRACT
RESULTS
CONCLUSIONS
Objectives: To determine if monitoring normally non-conscious oral parafunctional behaviors changes the rate by which subjects report the behaviors over repeat Ecological Momentary Assessment (EMA) periods.
Methods: Study I: Field Study. RDC/TMD Validation Project subjects with high or low frequency Oral Behavior Checklist (OBC) behaviors were identified (n=22). Two different paper versions (21-item, 11-item) of the OBC were administered before and after EMA field data collection. Field data consisted of 11 selected OBC behaviors which were sampled ~8 times/day for 3-9 days via PalmPilot. The remaining 10 OBC items were controls. Study II: OBC Test-Retest (TRT) Reliability Study. The 21-item OBC was administered twice at 14-day interval to n=74 Validation Project subjects without intervening EMA data collection.3 Data Analysis: Reactivity was evaluated using intraclass correlation coefficients (ICC), weighted kappas, and graphical depictions using STATA.
Results: Total score reliability from 11-item (ICC=0.65, 95%CL 0.28, 0.85) and 21-item (ICC=0.80, 95%CL 0.56, 0.92) OBC was substantial. 4 Subscore of EMA items in the 21-item OBC exhibited equivalent ICC=0.65 to the 11-item OBC, indicating item response was independent of context. Total score from the control items exhibited ICC=0.73. Study II: Total score reliability from 21-item OBC was almost perfect (ICC=0.88, 95%CL 0.82, 0.92). Both Study I and II showed little or no change in score reporting between test administrations. Point estimates of 21-item version within each Study did not differ.
Conclusion: EMA monitoring of normally non-conscious oral parafunctional behaviors in the natural environment does not significantly change the rate of retrospectively reported behaviors, highlighting opportunities for research as well as the difficultly for patients to change their overuse behaviors.
Figure 1. EMA 11-Item Total Score.
Figure 4. Sample EMA Field Data Item Scores. Weighted kappa for selected items.
Study I
EMA field data collection did not change retrospective report of behaviors.
EMA item order and context did not affect EMA item reliability. Response to individual items was not affected by the order in which the items were asked, and between the 11-item and 21-item questionnaires.
Experimental design is key to understanding reliability.
Study II
Total score reliability from 21-item OBC TRT was almost perfect (ICC=0.88, 95%CL 0.82, 0.92).
Both Study I and Study II
There was little or no change in score reporting between test administrations.
Point estimates did not differ, therefore items showed no reactivity.
Overall
EMA monitoring of normally non-conscious oral parafunctional behaviors in the natural environment does not significantly change the rate of retrospectively reported behaviors, highlighting opportunities for research as well as the difficultly for patients to change their overuse behaviors. 1A 1B
ICC = 0.65
(95%CI 0.28, 0.85)
n = 15
n = 15
INTRODUCTION
Figure 1A: All, but 2, observed values lay within 1 SD from the Line of Unity.
Background: Jaw overuse behaviors during waking hours occur outside conscious awareness, making their detection and quantification difficult, but Ecological Momentary Assessment (EMA) is a valid field data collection method for such measurement.1, 2
Why study this topic? Hypothesized that these behaviors in the waking state contribute to temporomandibular disorder pain. 2
The Oral Behaviors Checklist (OBC) is the paper self-report instrument used to identify and quantify the frequency of jaw overuse behaviors.
Objectives
To validate the retrospective self-report of waking-state oral parafunctional behaviors.
To determine if monitoring normally non-conscious oral parafunctional behaviors changes the rate by which subjects report the behaviors over repeat EMA periods.
Figure 1B: No visible trends in EMA total score.
Figure 2. OBC 21-Item Total Score.
Figure 5. Sample Non-EMA Field Data Item Scores. Weighted kappa for selected items. 2A 2B
ICC = 0.80
(95%CI 0.56, 0.92)
FUTURE EXPLORATIONS
Analysis of modes of self-report: Retrospective versus Actual Self-Report of TMD Behavior.
Objective: Comparison of the retrospectively reported frequency of oral parafunctional behaviors to the actual reports of oral parafunctional behaviors in the natural environment during field testing.
n = 15
n = 15
MATERIALS and METHODS
Figure 2A: All, but 1, observed values lay within 1 SD from the Line of Unity.
Figure 2B: No visible trends in OBC total score.
REFERENCES
Study I: Field Study
RDC/TMD Validation Project subjects with high or low frequency Oral Behavior Checklist (OBC) behaviors were identified (n=22).
Field data consisted of 11 behaviors sampled ~8 times/day for 3-9 days via PalmPilot. The remaining 10 OBC items were controls.
Study II: OBC Test-Retest (TRT) Reliability Study
The 21-item OBC was administered twice at 14-day interval to n=74 Validation Project subjects without intervening EMA data collection.3
Ohrbach, R. Assessment and Further Development of RDC ⁄TMD Axis II Biobehavioural Instruments: A Research Programme Progress Report. Journal of Oral Rehabilitation. 2010; 37:
Ohrbach, R., Markiewicz, M.R., McCall Jr, W.D. Waking-State Oral Parafunctional Behaviors: Specificity and Validity as Assessed by Electromyography. European Journal of Oral Science. 2008; 116:
Ohrbach, R., et al. The Research Diagnostic Criteria for Temporomandibular Disorders. IV: Evaluation of Psychometric Properties of the Axis II Measures. Journal of Orofacial Pain. 2010; 24:48-62.
Landis, J.R. Koch, G.G. The Measurement of Observer Agreement for Categorical Data. Biometrics. 1977; 1:
Stone, A. A. et al. The Science of Self-Report: Implications for Research and Practice. Lawrence Erlbaum Associates, New Jersey, 2000, p
Anastasi, A. Psychological Testing. 6th ed. Macmillan Publishing Company, New York,
Shrout, P.E., Fleiss, J.L. Intraclass Correlations: Uses in Assessing Rater Reliability. Psychological Bulletin. 1979; 86:
Campbell, D.T., Stanley, J.C. Experimental and Quasi-Experimental Designs for Research. Houghton Mifflin Company, USA, 1963.
Figure 3. OBC 21-Item TRT Total Score. 3A 3B
Figures 4, 5. Individual Item Scores.
11-Item and 21-Item show substantial item reliability and wide item confidence limits; Kappa was underpowered.
21-Item and 21-Item TRT items show no reactivity.
ICC = 0.88
(95%CI 0.82, 0.92)
Eleven EMA Field Data Items:
Ten Control Non-EMA Field Data
Clench or press
Items:
Touch or hold w/ contact
Sleep brux
Hold, tight or tense w/o contact
Sleep strained position
Jaw forward or side
Waking brux
Press tongue
Hold jaw rigid/tense jaw
Bite/chew/play soft tissue
Tongue between teeth
Bite/Hold objects between teeth
Play music instruments
Chew gum
Chew one side
Lean on jaw
Eat between meals
Yawn
Sustain talk
Cradle telephone
Sing
DESIGN
Stage 1
Stage 2
Stage 3
Study I
Paper OBC1
& EMA1
EMA Field Data Collection (3-9d)
Paper OBC2
& EMA2
Study II
No EMA Field Data Collection
n = 74
n = 74
Figure 3B: No visible trends in OBC T-RT total score.
Figure 3A: All observed values lay within 1 SD from the Line of Unity.
Data Management
Reactivity was evaluated using intraclass correlation coefficient (ICC), weighted kappa, and graphical depictions using STATA.
Table 1. Total Item Score.
11-Item
21-Item
21-Item Test-Retest
ICC (3,1)
95.0% CL
EMA Items Total Item Score
0.65
0.28, 0.85
0.29, 0.85
0.83
0.75, 0.89
Non-EMA Items Total Item Score
0.73
0.41, 0.89
0.86
0.78, 0.91
ACKNOWLEDGEMENTS
Weighted kappa or ICC (3,1):4
= Almost Perfect
= Substantial
Table I. Total Item Score.11-Item and 21-Item had same EMA item reliability.
This study was supported in part by the Dental Student Research Fellowship for Summer Research and the data collection by the NIH/NIDCR U01-DE