1. A Conversation: November 5, 2014
CMTO Registered Massage Therapist Program Transition to the Federal Inter-jurisdictional Competency Standards
A Conversation:
November 5, 2014

2. A Brief History
CMTO Assessments (MCQ and OSCE) have traditionally been built upon job analysis research performed in Ontario
The results were a legally-defensible link between what competent RMTs do on-the-job and successful performance on the OSCE and MCQ registration examinations
The research results provide the content for the examinations

3. Registration Examination Cycle
Job Analysis Research
Defining the knowledge elements that define competent practice
Content Outline
(drives exam content)
Question(item) and test development
Examination Assembly and psychometric analysis of results
Establishment of Cut Scores
(Passing Standards)
Award of Registration
Recertification
(RMTs demonstrate continued competency)

4. Inter-jurisdictional Competency Standards (IJC Standards)
Created in response to federal direction that the RMT credential be more transportable
Developed by a consortium of representatives/peers from regulated provinces
Canada-wide Job Analysis Study performed
Results used to establish IJC Standards

5. IJC Standards
Represent the common body of knowledge representing MT practice in Canada
Questions for the CMTO:
Do the IJC Standards represent MT practice in Ontario?
Can a link between the IJC Standards and the CMTO assessment instruments (OSCE and MCQ) be established?

6. 2014 CMTO IJC Standards Transition Research Study
Performed by CMTO RMTs (Subject Matter Experts), psychometric experts, representatives from the other regulated, and soon to be regulated province of Manitoba.
Study performed in Spring/Summer of 2014
What procedures were followed?
What were the results?
How do the results relate to the CMTO Exams?

7. IJC Standards Transition Procedures – Review of IJC Standards
Review of the IJC document:
Practice Competency
Domain & Level
Performance Indicator
Assessment Vehicle
Cognitive
Psychomotor
Affective
Indicator #
Reg Exam MC
Reg Exam OP
Ed Academic
Ed Simulated
Ed Clinical
1. Professional Practice
Communication
Utilize effective written communication. 2 1
Document complete and accurate information. x
Ensure legibility of information. 3
Employ clear, concise and profession-specific language. 4
Write in a manner that corresponds to the needs of the reader. 5
Employ appropriate formatting, grammar and spelling.

8. Tasks Identified for Inclusion in Study Based on Assessment Vehicle
Only tasks identified as assessable via Reg Exam MC (multiple choice) and/or Reg Exam OP (performance/OSCE) were considered during the study.
Tasks meeting study criteria would then be compared with current MCQ and OSCE content.

9. Tasks/Knowledge Elements: The 12 Considerations
Is the task important to test?
How difficult is it to test the task?
Are there references supporting the task?
Where does the task fall on the continuum of priority?
What level of professional experience is required to perform the task?
Is the task on the current MCQ/OSCE?

10. Task/Knowledge Elements: The 12 Considerations (continued)
Are there any tasks left off?
Is the task appropriate for testing in Ontario?
Does the task belong on MCQ/OSCE, both?
Are there non-assessable/duplicated tasks?
Based on final MCQ and OSCE outlines, do the examination specifications need to change (e.g., length, timing, #of stations, etc.,)?
Content area weightings for OSCE/MCQ?

11. Results based on SME opinions:
Assessment Question posed to SMEs
Assessment Decision
Is the task/concept important to test?
Yes/No
How difficult is the task/concept to test?
Very Difficult/Difficult/Moderate/Easy
What is the priority to test this task/concept?
Not Important/Somewhat Important/Important/Extremely Important
Is there a reference available to support assessment of this task/concept?
What level of professional experience is required to demonstrate competence in this task/concept?
Entry-level, Experienced, Advanced

12. Decision Criterion Applied:
Assessment/Question posed to SMEs
Decision Required for Task/Concept Inclusion
1. Is the task/concept important to test?
Yes
2. How difficult is the task/concept to test?
Difficult/Moderate/Easy
3. What is the priority to test this task/ concept?
Somewhat Important/Important/Extremely Important
4. Is there a reference available to support assessment of this task/ concept?
5. What level of professional experience is required to demonstrate competence in this task/concept?
Entry-level

13. Questions 6-12
15 Tasks on the IJC Standards not on OSCE/MCQ. 2 Tasks on OSCE/MCQ not on IJC
IJC deemed comprehensive – no tasks left off
All tasks appropriate for Ontario (universal)
For tasks that passed review–adopted IJC recommendations for placement on exams
Tasks assessable – duplicates eliminated, some editing for clarity

14. Questions 6-12 (continued)
Test Specifications: MCQ stays 150, 125 scored, 4-option multiple-choice, three hours. OSCE stations remained the same, some editing of graded assessments/ competencies to insure fairness to candidates.
Content area weightings established.

15. 2015 MCQ Content Outline Professional Practice:
26 Tasks, 17% of exam, 22 questions (of 125)
1.1 Communication
1.2 Professional Conduct
1.3 Therapeutic Relationship
II. Assessment and Treatment Planning:
63 Tasks, 37% of exam, 46 questions
2.1 Assessment
2.2 Treatment Planning
2.3 Adjunctive Therapeutic Modalities

16. 2015 MCQ Outline (continued)
Treatment and Patient Self-Care
79 tasks, 46% of Exam, 57 Questions
3.1 Treatment Principles
3.2 Modalities and Techniques
3.2.a Massage Skills
3.2.b Myofascial Techniques
3.2.c Therapeutic Exercise
3.2.d. Joint Mobilization
3.2.e Thermal Therapy

17. 2015 OSCE Content Outline Professional Practice:
17 Tasks, 10% of exam
1.1 Communication
1.2 Professional Conduct
1.3 Therapeutic Relationship
II. Assessment and Treatment Planning:
68 Tasks, 35% of exam
2.1 Assessment
2.2 Treatment Planning

18. 2015 OSCE Content Outline (continued)
Treatment and Patient Self-Care:
101 tasks, 55% of Exam
3.1 Treatment Principles
3.2 Modalities and Techniques
3.2.a Massage Skills
3.2.b Myofascial Techniques
3.2.c Therapeutic Exercise
3.2.d. Joint Mobilization
3.2.e Thermal Therapy

19.
Becoming an RMT: Certification Examinations/Introduction

20. MCQ Questions/OSCE Stations Updated
MCQ items linked to new IJC Standards-based content outline
OSCE stations/tasks scenarios linked to IJC Standards-based content outline
Some questions/tasks eliminated from MCQ bank - new questions written
New competencies/assessments created for the OSCE

21. Who develops the OSCE and MCQ?
In accordance with International Standards all MCQ question writers and OSCE developers, trainers and raters are RMTs in good standing.
SMEs Undergo rigorous training
Development is supported by research
Supervised by psychometric experts
Question/OSCE stations undergo psychometric analysis prior to live scoring

22. Information Dissemination
Notice provided to stakeholders
New exams launched in Spring/Summer 2015
All psychometric standards will be followed for: item/examination/station/competency analysis, cut score establishment, score scaling and reporting, ensuring that all subsequent forms are equivalent in content, difficulty and defensibility
Cycle shown in slide 3 remains the same

23. Transition Study results confirm that the 2015 IJC Standards-based CMTO MCQ and OSCE Examinations are linked to competent practice as an RMT in Ontario (and Canada)

24. CMTO continues to ensure that the RMT Examinations meet standards related to :
Validity (questions are supportable with approved references)
Reliability (psychometric analyses)
Fairness (all candidates treated equitably)
Standardization (all candidates receive comparable examination experiences)