Assessing EM registrars’ leadership and non-technical skills

Definitions Non-technical skills “the cognitive, social and personal resource skills that complement technical skills, and contribute to safe and efficient task performance” (Flin, 2003)

Leadership “directing and coordinating the activities of others, assessing team performance, assigning tasks, motivating team members and organising and executing management plans” Salas (2004)

Scenario 3

Why assess? Focus group training efforts Focus individual development Encourages self reflection Assessment drives learning (e.g. motivation) Observational assessment is beneficial for the observer!

The process Behavioural marker system 1.Choose an appropriate time 2.Observe for min 3.Fill out assessment form (examples / rating) 4.Reflection 5.Feedback 6.Improvement plan 7.REPEAT

Observation Framework CategoryElement Leadership & Management Maintenance of standards Workload management Supervision and feedback Teamwork & Cooperation Team building Exchanging information Authority and assertiveness Decision Making Option generation Selecting and comm. options Outcome review Situation Awareness Gathering information Anticipating Understanding and comm.

1. Management & Supervision Maintenance of Standards Subscribes to clinical and safety standards as well as considering performance targets. Monitors compliance. Workload Management Manages own and others’ workload to avoid both under and over-activity. Includes prioritising, delegating, asking for help and offering assistance. Supervision and Feedback Assesses capabilities and identifies knowledge gaps. Provides opportunities for teaching and constructive feedback.

2. Teamwork & Cooperation Team BuildingProvides motivation and support for the team. Appears friendly and approachable. Exchanging Information Gives verbal and written information concisely and effectively. Listens, acknowledges receipt of information and clarifies when necessary. Authority & Assertiveness Behaves in an appropriately forceful manner and speaks up when necessary. Resolves conflict effectively and remains calm when under pressure.

3. Decision Making Option GenerationUses all resources (written and verbal) to gather information and generate appropriate options for a given problem or task. Involves team members in the decision making process. Selecting & Communicating Options Considers risks of various options and discusses this with the team. Involves clearly stating decisions and explaining reasons, if necessary. Outcome ReviewOnce a decision has been made, reviews suitability in light of new information or change in circumstances and considers new options. Confirms tasks have been done.

4. Situational Awareness Gathering InformationSurveys the environment to pick up cues that may need action as well as requesting reports from others. AnticipatingAnticipates potential issues such as staffing or cubicle availability in the department and discusses contingencies. Informing the TeamCross-checks information to ensure it is reliable. Communicates situation to keep team ‘in the picture’ rather than just expecting action.

The Rating Scale Unacceptable Standard Several examples of poor behaviour or behaviour that directly compromises patient safety Acceptable Standard Performance was of a satisfactory standard with mostly good behaviour observed. Standard expected of a competent trainee. Exemplary Standard Performance was of a consistently high standard. A model for other team members

Problems with assessment Time consuming Inter-rater reliability Hawthorne effect Bias (halo/ horn effect, leniency, severity) Cognitive processes not observable Cultural barriers within the profession

Pilot Study Assessors “It made me reflect on the robustness of ‘follow up on actions taken’ (D.M - Outcome Review) in my daily practice” “Made me think about (verbal)medication orders given before prescribed” “How I get points across and ensure people understood what I wanted”

Trainees “Very beneficial. Did reflect on my habits. Got some points highlighted which I would never have thought of” “We (the registrars) are not aware of what we are doing and it is useful for someone to observe and tell us” “Ask for help, delegate and try not to do things alone”

? Hawthorne effect - no evidence “may make you nervous” and “adds more pressure but it’s worth it” All agreed assessments should be completed by other trainees as well as consultants All agreed it should be implemented more widely (“optional”)