Building an assessment framework: Processes, measures, training, and challenges Wendy Smith, Ph.D., LCSW Shelley Levin, Ph.D., LCSW Betsy Phillips, Ph.D. LCSW

Assessment Outcomes & 5. 5. 5. Program Improvement: 1. Framework Development: Goals: reaffirmation & school 2015 EPAS Holistic assessment Use of field model 2. Creation and Use of Assessment Tool: Competencies & Behaviors Specialized practice Mapping across curriculum Training materials & videos for faculty 3. Data Collection: Manual data entry Electronic data entry 4. Analysis of Results: independendent contractor SLO Workgroup (representatiive faculty) Outcomes & 5. 5. 5. Program Improvement: Present findings to faculty Explicit and implicit Curriculum/program changes Changes to assessment framework

Framework development CSWE reaffirmation and 2015 EPAS School of social work goals: evaluation of new curriculum; program improvement Need for new assessment tool Large, complex program, including virtual program option

Key principles underlying assessment framework Assessment over time. Field assessment instrument should enable assessment over time, from first semester through fourth semester, with expectations for level of competency increasing. Curriculum assessment should be holistic. Faculty assessment of student competency as demonstrated in classroom should be decoupled from exclusive dependence on grades. Faculty should be engaged in assessment process. Faculty members participate in development of framework and instruments, receive training and support to complete assessments, and are involved in discussion of findings and implications for program improvement.  

Assessment tool: explicit curriculum Development of specialized competencies & behaviors concentrations and departments Competency mapping process Faculty involvement Development of rubric, based on field model

Faculty training Department meetings Written guidelines Online video In person training/discussion

Data collection Faculty rate behaviors for every student in each class Manual data entry Excel Electronic data entry Salesforce Faculty support during data collection Faculty support during data collection

Analysis of results Independent contractor provides data analysis Student Learning Outcomes Workgroup Standing committee of Curriculum Council Representative faculty – lines and departments Discuss findings and implications Implicit curriculum assessment Employer survey Student survey on diversity

2015 EPAS Field Practicum Assessment Semesters 1 and 2 2 3 4 5 6 7 8 9 10 2  Red = Does not meet expectations Green = Meets expectations Blue = Exceeds expectations Competency 2: Examples of Generalist Practice Behaviors 2a. Communicates their understanding of the importance of diversity and differences in shaping life experiences as learners. (Example: The student respects and encourages involvement from family members to develop a case plan that reflects their cultural values.) 2b. Engages clients and constituencies as experts of their own experiences. Constituencies include individuals, families, groups, organizations, and/or communities. (Example: The client has a mental illness. The student respectfully asks her todescribe her experience in the home environment where mental illness is stigmatized.)  

2015 EPAS Field Practicum Assessment Specialized Practice Sem 3 1 2 4 5 6 7 8 9 10 Sem 4 Red = Does not meet expectations Green = Meets expectations Blue = Exceeds expectations Competency 2: Examples of Specialist Practice Behaviors (Adults and Healthy Aging) 2a. Recognizes and communicates understanding of how diversity and difference shape human experience and identity for adults of all ages when addressing their mental and physical health and well-being. 2b. Consistently applies an intersectional framework with individuals, groups and families that considers multiple factors, including age, class, color, culture, disability and ability, ethnicity, gender, gender identity, immigration status, marital status, political ideology, race, religion/spirituality, sexual orientation, and tribal sovereign status when addressing the mental and physical health of adults of all ages and their families.

Challenges in field assessment Grade inflation Solution: comments required for entries in blue zone Field instructors have difficulty understanding item(s) Solution: Examples provided for each item Training provided to faculty and field instructors on use of assessment tool.

Challenges in curriculum assessment New EPAS New curriculum On ground (local and regional) and virtual (national) programs Existing assessment framework was unsatisfactory Creating specialized competencies simultaneously with creating assessment framework

Program Improvement Presentation of findings to faculty and administration Plans for curriculum, field, and program improvement Modifications to assessment framework for next round.

Contacts for further information Wendy Smith, Associate Dean, Curriculum Planning & Assessment. wsmith@usc.edu Betsy Phillips, Assistant Dean, Faculty Development. betsy.phillips@usc.edu Rona Smith, Assistant Dean, Academic Affairs. ronasmit@usc.edu