Evaluating a rating scale for assessing trainee clinical psychologists’ clinical skills in-vivo. Dr. Alison Tweed* and Ms. Rebecca Graber Introduction and Objectives Developing clinical skills is seen as a core competency within clinical psychology training. Yet, there are no published, formalised or standard means of assessing trainees’ acquisition of clinical skill in- vivo, despite a call for this from the training community (Gallichan & Mitchell, 2008). This poster reports on reliability and validity evaluation research of the Clinical Skills Assessment Rating Form (CSA-RF), a seven- domain rating scale developed at the University of Leicester to assess trainees’ clinical skills in-vivo. Design and methodology The evaluation incorporated a mixed-methods design, undertaken in two phases: Phase 1 (Quantitative) used correlations and factor analysis to establish validity and reliability estimates and determine the factor structure of the scale. Phase 2 (Qualitative) investigated how the scale was used by paired raters as intended for Programme assessments. Tape-recorded rating sessions were analysed using Thematic Analysis (Braun & Clarke, 2006). Results Phase 1 (Quantitative) Statistical analysis revealed acceptable levels of internal consistency (construct validity) of the scale. Chronbach’s alpha = Inter-rater reliability analysis is still ongoing. Factor analysis revealed 5 factors accounting for 50.7% of the variance: Demonstrating Professional Therapeutic Engagement, Creating a Secure Base, Formulation, Facilitating Mutual Understanding and Session Structure. Phase 2 (Qualitative) Thematic analysis is still underway. A preliminary model has been developed (see below). Core themes relate to the negotiation of raters’ expectations according to principles or guidelines of “good enough” performance. Negotiation between raters is undertaken within personal and inter-personal frameworks, using what happens on the video as a starting point for discussion. Discussion and Conclusions The CSA-RF appears to be a valid in-vivo measure of the generic clinical skills of trainee clinical psychologists. Whilst the research has been undertaken using simulated patients, it is expected that the findings can be generalised to genuine clinical populations. Findings from Phase 2 will inform the development of training materials for new raters. The study has gone some way to answering a need for a robust measure of clinical skill that can be applied to diverse therapeutic orientations. The CSA-RF has applicability for Clinical Psychology Programme staff on a national level who are interested in providing more formalised means of assessing trainees’ generic clinical skills. Participants Eighteen qualified clinical psychologists took part in Phase 1, each rating eight different 15-minute interactions of trainees undertaking a clinical session with a simulated patient on video. In Phase 2, ten clinical psychologists (five pairs) rated one 30-minute interaction, first individually and then jointly to reach a consensus. Expectations NEGOTIATION Outcome: Is the trainee good enough? Personal Inter-personal What happens References and Contact Information Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2), Gallichan, D. & Mitchell, A. (2008). Making the case for in-vivo assessment s of clinical competence. Clinical Psychology Forum, 184, *University of Leicester, School of Psychology, Clinical Section, 104 Regent Road, Leicester, LE1 7LT.