1. University of Houston Graduate College of Social Work

2. The Signature Pedagogy: Defining, Measuring and Operationalizing Competencies in Field Education Virginia Cooke Robbins, MSW, LMSW, AP Susan P. Robbins, Ph.D., LCSW Sharon Alpert, MSW, LCSW Sharona Perel Jacobs, MSW, LCSW © 2009, Susan P. Robbins and Virginia Cooke Robbins

3. Competency-based Field Education  is now mandated by the new EPAS  This requires clear and measureable operational definitions of the practice behaviors that demonstrate each competency

4. Competency-based Field Education  There is scant literature on how to define and measure field competencies  There are no “shared standards” among Field Instructors  Evaluation of student performance has been idiosyncratic

5. Competency-based Field Education It is important to achieve consistency in the way that competencies and practice behaviors are:  operationally defined  measured

6. Process & Method  We examined the 10 core competencies for each semester of field  These competencies in EPAS apply to both the foundation and advanced curriculum

7. Process & Method  We ensured that the associated practice behaviors were developmentally progressive across the four required semesters

8. Process & Method  In conjunction with the Field Practicum Advisory Committee (comprised of field instructors, faculty, and students)  Practice behaviors were delineated using the suggestions in EPAS  These were further operationalized as specific practice tasks for each competency in each field course

9. Process & Method  Demonstration of the competency could be evaluated by the student’s successful completion of the practice tasks  Corresponding field evaluations for each semester were directly linked to the EPAS competencies

10. Field Evaluations  Unacceptable progress - Never demonstrates awareness, knowledge & skills  Insufficient progress - Rarely demonstrates awareness, knowledge & skills  Emerging competence - Inconsistently demonstrates awareness, knowledge & skills  Competence -Consistently demonstrates awareness, knowledge & skills  Advanced Competence - Expertly demonstrates awareness, knowledge & skills

11. Field Evaluations  Field instructors were also required to provide narrative evidence to support their evaluative rankings  This allowed us to determine similarities and differences in the way that various field instructors evaluate students to arrive at their rankings

12. Process & Method  a content analysis was then preformed on the narrative evidence  This confirmed our suspicion that there is little uniformity in field evaluations

13. Process & Method This raised critical questions:  What does competent practice in each core area actually look like?  If we cannot define competence, how do we measure its achievement?

14. Process & Method To answer these questions, we:  aligned narrative evidence of behaviors with their Likert Scale rankings and noted discrepancies  shared the discrepancies with the Field Practicum Advisory Committee (FPAC)

15. Analysis of Evidence Supporting the Ratings for FP I Number of Comments at the Rating Level Given by Field Instructors IPECCACComments not Relevant Partial Info Emerging Competency (21)2 4 7044 Competency (64)01 30 01320 Advanced Competency (13)0011 1 10 TOTAL (98)254811824

16. Process & Method  the FPAC subsequently decided which behaviors should describe each level of competency  this created consistency in operationalizing and describing practice behaviors at different levels of competency

17. Recommendations Based on Data Analysis  Educate Field Instructors that practice competencies for each semester stand alone and are not a progressive expectation from one semester to another  Review the definitions of levels of performance

18. Recommendations Based on Data Analysis  Request more specificity from Field Instructors about ratings that are not at the competence level  Reliability may be more consistent with sample behaviors that would meet the Competency level

19. Next Steps  Train Field Instructors to recognize the behaviors associated with each level of competency

20. Next Steps  Content analysis of narrative evidence in field evaluations will continue periodically  This ensures that operational definitions will continue to evolve and reflect best practices as they emerge in the practice world