Chapter 6 Test Construction

Test Construction Standardized testing tools Measure specific, limited aspects of function or attitude Test (assessment) developed by therapist Matches scope of services within the therapist’s setting Does not usually go through the rigorous testing for reliability and validity

Development of In-House Forms “In-House forms” could include Intake assessments forms Attendance /participation forms Functional skills checklist Satisfaction surveys Interest checklists No rigorous development process, but they must be based on technical knowledge of what is normal

Characteristics of Most In-House Assessments What do YOU think would be on it? Page 112

Determining the Scope of the Assessment Three things that have a MAJOR impact on what is contained in the in-house assessment The typical needs and strengths of the client populations being assessed The scope of practice of the recreational therapists on that specific unit The potential discharge destination of the client population being served (or the typical prognosis)

What you should be looking for... Pages 113-114 As you develop the in-house assessment, YOU should ask yourself... As you coordinate the scope of the in-house assessment with the other members of the treatment team, you AND the rest of the treatment team should ask... Lastly, as you take D/C patterns into consideration, You should ask yourself...

Measuring functional tasks when standards don’t exist... Many functional tasks that the CTRS is expected to evaluate do not have readily available standardized tests. When you develop an assessment to help define a client’s functional skills, at a minimum, you should review journals or text books to see how other professionals have defined the functional skill, including any method of breaking it into parts.

Criterion-Referenced Test You will need to set up the testing situation to allow the actual measurement of the skills instead of just asking the client if they can do it. Ask AND observe Health care standards require that the measurement be taken using a criterion-referenced test Therapist will need to set up the testing scenario to allow the therapist to observe whether initiation actually takes place

Actually Developing the form Important things to remember: Form should make finding critical information easy Order on the form should reflect the order of questions in the interview Information can be recorded easily and accurately Depending on facility, form may need to be reviewed by Records Review Committee If so, develop according to those policies from the start

Self Report You may have some predetermined scope of information that you want to know about your clients. In some cases, you may have the clients fill out the testing score themselves. Four methods of self-report Rating scales Checklists Q-sorts Free response

Self Report Methods Rating scales Checklists Q-sorts Free-response Usually an ordinal scale, (like a likert scale) Allows therapist to obtain a numerical score that reflects how much Checklists Provides therapist with a binary count (yes/no) Q-sorts Multiple cards, each containing an element. Client is asked to sort the cards into predetermined sets of piles Free-response Therapist provides client with stimulus, expects client to respond “On my days off my favorite activity is___________”

Checklists Helpful to ensure nothing has been overlooked It’s all listed there Used to categorize or select from a list of possibilities Possible activities Used to list critical tasks that need to be completed

Behaviors Behavior A generic term covering acts, activities, responses, reactions, movements, processes, operations, etc., Any measureable response of an organism

Observing Behaviors vs. Behavioral Observation Observing client behavior Used during interviews, therapy sessions, and informal interactions Behavioral Observation A formal protocol that has specific rules that must be followed

Observing client’s behavior Competencies that the therapist should hold related to observing client’s behavior Page 118 Therapist should be able to filter out all the things that could be observed and focus on what should be observed. What is relevant to the process Verbal expression (p. 120) Physical movement (p. 120)

Sampling Techniques Techniques defining what will be included in the observation Ad libitum (ad lib) Sampling Point Sampling AKA Scan sampling AKA instantaneous sampling

Behaviors with Important Implications Tardive dyskinesia Seizures Suicide

Behavioral Observation The formal identification and analysis of behavior that relies on a well-defined method of collecting data for examination and interpretation Three phase process Training Practice

Developing Questionnaires Use of questionnaires in clinical setting Quality assurance reviews Client satisfaction measures Client preference inquiries Ask yourself if a questionnaire is the type of data collection device that is most appropriate for gathering the data desired

Ten Steps of Questionnaire Development 1. Determine the characteristics and needs of the group of individuals to be surveyed Briefly outline the characteristics of the population to be evaluated Outline any special needs of the respondents If population is heterogeneous, explore possibility of separate questionnaires for different subsets Obtain copies of relevant earlier questionnaires

Ten Steps of Questionnaire Development 2. List main topics to be covered and decide the appropriate order for them to be presented Decide on topics and scope Determine most appropriate order 3. Select the method(s) by which the questionnaire will be analyzed and clarify the availability of resources for this. What do you want to do with the information you receive?

Ten Steps of Questionnaire Development 4. Draft and write the questions Select appropriate vocabulary Ensure ease of readability Avoid leading questions Strive for clarity of intent for each question Limit length of time you are asking the individual to remember events Avoid multiple topic questions Include an adequate range of answers

Ten Steps of Questionnaire Development 5. Decide on the question order and number the questions 6. Pay close attention to the overall design/format of the questionnaire Determine if a separate cover sheet is needed Select appropriate paper color Determine how and where instructions need to be arranged Arrange a reader friendly layout of the questionnaire

Ten Steps of Questionnaire Development 7. Draft either a letter of information sheet to be included with or to form part of the questionnaire Explain purpose List sponsorships State why they are being asked Confirm confidentiality of responses Provide contact information Give due date

Ten Steps of Questionnaire Development 8. Pilot or pre-test 9. Prepare copies for dissemination Apply confidential individualized identifiers to each form 10. Distribute the questionnaires

Done! Now HOP on out of here...