University of Houston Graduate College of Social Work

The Signature Pedagogy: Defining, Measuring and Operationalizing Competencies in Field Education Virginia Cooke Robbins, MSW, LMSW, AP Susan P. Robbins, Ph.D., LCSW Sharon Alpert, MSW, LCSW Sharona Perel Jacobs, MSW, LCSW © 2009, Susan P. Robbins and Virginia Cooke Robbins

Competency-based Field Education  is now mandated by the new EPAS  This requires clear and measureable operational definitions of the practice behaviors that demonstrate each competency

Competency-based Field Education  There is scant literature on how to define and measure field competencies  There are no “shared standards” among Field Instructors  Evaluation of student performance has been idiosyncratic

Competency-based Field Education It is important to achieve consistency in the way that competencies and practice behaviors are:  operationally defined  measured

Process & Method  We examined the 10 core competencies for each semester of field  These competencies in EPAS apply to both the foundation and advanced curriculum

Process & Method  We ensured that the associated practice behaviors were developmentally progressive across the four required semesters

Process & Method  In conjunction with the Field Practicum Advisory Committee (comprised of field instructors, faculty, and students)  Practice behaviors were delineated using the suggestions in EPAS  These were further operationalized as specific practice tasks for each competency in each field course

Process & Method  Demonstration of the competency could be evaluated by the student’s successful completion of the practice tasks  Corresponding field evaluations for each semester were directly linked to the EPAS competencies

Field Evaluations  Unacceptable progress - Never demonstrates awareness, knowledge & skills  Insufficient progress - Rarely demonstrates awareness, knowledge & skills  Emerging competence - Inconsistently demonstrates awareness, knowledge & skills  Competence -Consistently demonstrates awareness, knowledge & skills  Advanced Competence - Expertly demonstrates awareness, knowledge & skills

Field Evaluations  Field instructors were also required to provide narrative evidence to support their evaluative rankings  This allowed us to determine similarities and differences in the way that various field instructors evaluate students to arrive at their rankings

Process & Method  a content analysis was then preformed on the narrative evidence  This confirmed our suspicion that there is little uniformity in field evaluations

Process & Method This raised critical questions:  What does competent practice in each core area actually look like?  If we cannot define competence, how do we measure its achievement?

Process & Method To answer these questions, we:  aligned narrative evidence of behaviors with their Likert Scale rankings and noted discrepancies  shared the discrepancies with the Field Practicum Advisory Committee (FPAC)

Analysis of Evidence Supporting the Ratings for FP I Number of Comments at the Rating Level Given by Field Instructors IPECCACComments not Relevant Partial Info Emerging Competency (21) Competency (64) Advanced Competency (13) TOTAL (98)

Process & Method  the FPAC subsequently decided which behaviors should describe each level of competency  this created consistency in operationalizing and describing practice behaviors at different levels of competency

Recommendations Based on Data Analysis  Educate Field Instructors that practice competencies for each semester stand alone and are not a progressive expectation from one semester to another  Review the definitions of levels of performance

Recommendations Based on Data Analysis  Request more specificity from Field Instructors about ratings that are not at the competence level  Reliability may be more consistent with sample behaviors that would meet the Competency level

Next Steps  Train Field Instructors to recognize the behaviors associated with each level of competency

Next Steps  Content analysis of narrative evidence in field evaluations will continue periodically  This ensures that operational definitions will continue to evolve and reflect best practices as they emerge in the practice world