1. Types of assessments and reflective practice
STP Induction Day
Types of assessments and reflective practice
10th September :10 to 11:35
Types of assessments and reflective practice
Jane Lynch M.Sc and Lisa Ayers PhD, Training Programme Directors
Dr Michael Thomas
National School of Healthcare Science

2. STP Induction Day Types of assessments and reflective practice
10th September :10 to 11:35
Aim of the session:
Learn about the different types of work based assessment methods.
Understand how they are submitted and how you will be assessed.
Learn how to be a better reflective scientist.
Dr Michael Thomas
National School of Healthcare Science

3. Why do we do assessments?
STP Induction Day
Types of assessments and reflective practice
10th September :10 to 11:35
Why do we do assessments?
KNOWS ABOUT
HEARD OF
Based on work by Miller, GE, The Assessment of Clinical Skills/Competence/Performance; Acad. Med. 1990; 65(9); 63-67
Adapted by Drs. R. Mehay & R. Burns, UK (Jan 2009)
Not to make you look stupid.
Ensure your assessor has the evidence they require to say that you’re competent
Millers Pyramid: A framework proposed by psychologist George Miller for assessing levels of clinical competence back in 1990
Competency assessment needs to match the level of knowledge.
Think of it like a driving test – you can do it, but you have to demonstrate that you can do it in order to pass
Assessments aren’t there to make you look stupid. They are there to ensure that your assessor has the necessary evidence to sign you off as competent.
The formal assessment of competency is often related to Miller’s pyramid, shown here with levels from “knows” to “does” as cognition moves into behaviour.
Traditionally this has been shown only in 4 stages but recent discussions have seen suggestions that we include 2 underpinning awareness levels that usually occur before we "know".
The first two stages, “knows” and “knows how”, can be assessed using the traditional assessment tools of written and oral tests including MCQs, case presentations and essays. However, these stages, “knowing” and “knowing how” (cognition) don’t necessarily extrapolate to the application of knowledge in the workplace (ie behaviour).
To demonstrate clinical competence, assessment at levels 3 and 4 becomes more important, but also more challenging. Level 3, “shows how”, can currently be assessed by practical examinations, observed long or short cases, or OSCE or OSFA style examinations.
However, the only way to assess level 4, “does”, is to observe the practitioner at work in the real world. In other words this measures our ability to perform.
It’s a bit like learning to drive!
You first learn the theory, then maybe give it a go in a supervised environment before showing how through the driving test itself.
Once you have passed the driving test you are competent
Dr Michael Thomas
National School of Healthcare Science

4. What is competence & competency
STP Induction Day
Types of assessments and reflective practice
10th September :10 to 11:35
What is competence & competency
Being
Competent
means
Understanding
why it should be
done that way
Being able
to apply
knowledge and skills consistently
to transfer skills
to different situation
to do different
tasks at the
same time
Dealing with every
day problems
Fitting in with others
In the
workplace
policies &
procedures
Knowing how
to do the job
Competence:
the ability to do something, the job, successfully or efficiently
Competency:
a set of knowledge, skills, experience and other attributes necessary to do the job properly
Competence is the outcome and competencies the inputs to do achieve the task
Competence: can be said to be the ability (or qualification) of an individual to do a job properly
whilst
Competency: is about the demonstration of a set of knowledge, skills, experience and other attributes necessary to do the job properly
Competence is the outcome and competencies the inputs to achieve the task
Being competent means all the things shown in this diagram.
Dr Michael Thomas
National School of Healthcare Science

5. Where do I find the curriculum?
STP Induction Day
Types of assessments and reflective practice
10th September :10 to 11:35
Where do I find the curriculum?
The Curriculum Library, contains all the information about modules.
The STp curriculum is housed on the website – currently being updated to provide a fresh, easily navigated platform for trainees and training officers to find information and answers to a whole host of frequently asked questions
Within the curriculum library there are sections for each specialty and these include Core, rotation and specialist modules, with the required competencies and assessments listed.
Dr Michael Thomas
National School of Healthcare Science

6. Understanding the level of competency
STP Induction Day
Types of assessments and reflective practice
10th September :10 to 11:35
Understanding the level of competency
Rotations
Vs.
Specialisms
“the skills appropriate for that level of training”
e.g. basic/competent/advanced /expert
Level 1
Awareness
The trainee has been introduced to the process/procedure associated with the competency:
Level 2
Performance
The trainee has repeatedly performed the process/procedure (supervised) with increasing confidence:
Level 3
Proficient
The trainee has demonstrated repeated successful performance of the process/procedure (indirect supervision):
Level 4
Competent
The trainee performs the task(s) referring infrequently to their supervisor as required:
For each assessment, It is possible to identify different levels of competency
And for the STP programme the level of competency required will differ between the competencies required for Rotations and those for Specialisms
For ‘Competency’, we should read
‘The skills appropriate for that level of training’
e.g. level or an awareness, ability to perform, actually proficient, fully competent
We can take Miller’s pyramid concept and consider the assessment of competence as a four part hierarchy:
Having Awareness
Demonstrating Performance
Being Proficient
Being Competent
Dr Michael Thomas
National School of Healthcare Science

7. What are the types of assessments?
STP Induction Day
Types of assessments and reflective practice
10th September :10 to 11:35
What are the types of assessments?
Competencies
Direct Observation of Practical Skills (DOPS)
Observed Clinical Event (OCE)
Case-Based Discussion (CBD)
Direct Observation of Practical Skills (DOPS)
DOPS assess the performance of a skill or procedure. Feedback is generated, learning needs are identified and an action plan is agreed. Each specialism will
have a core list of skills with documentation of what is expected at the relevant stage of training.
DOPS are designed to provide feedback on procedural skills essential to the provision of good clinical care. Trainees are asked to undertake practical procedures with a different observer for each encounter. The trainee chooses the timing, procedure and the observers who are competent in the procedure assessed.
The assessment involves an assessor observing the trainee perform a practical procedure within the workplace; and a structured checklist is designed to give guidance for the assessors. Most procedures take no longer than minutes. Feedback would normally take about 5 minutes.
Observed Clinical Event (OCE)
A clinical event is any occasion when the trainee is present with a patient as part of the clinical team. This is true for all patient-facing occasions whether the trainee only observes, or communicates with, touches, positions or examines a patient. OCEs are used to assess the trainee’s professional attitudes and behaviours, communication and clinical skills as relevant.
These assessments usually take place in a clinical setting with a patient present and review your interaction, history taking, communication skills, physical and clinical examinations skills, clinical judgement, organisation and efficiency, professionalism and overall clinical care.
Case based discussion
take place after a clinical event and assess the trainees skills in discussing, explaining and justifying aspects of the outcome or output from a workplace activity ie the report / record / result.  
Dr Michael Thomas
National School of Healthcare Science

8. MSF - Multi Source Feedback
STP Induction Day
Types of assessments and reflective practice
10th September :10 to 11:35
MSF - Multi Source Feedback
Two must be completed during the programme but they are not an assessment
Anonymous feedback from a sample of colleagues on your attitudes and their opinions of your performance and professional attitude.
Provides data for reflection on your performance and gives useful feedback for self evaluation.
Provides an opportunity for self reflection on your performance against perceived performance by your colleagues.
Identifies areas of development which you may not be aware of.
Identifies areas of strength and good practice which you can build on..
The Multi-Source Feedback (MSF) tool is used to collect your colleagues’ opinions on your clinical performance and professional behaviour. It provides data for reflection on your performance and self-evaluation.
The feedback you receive can be helpful for self-reflection and can identify areas of development which you may not have thought about.
Feedback through MSF can also be useful to inform your annual appraisal and help in developing your training plan.
MSF is not integral to OneFile but is a separate product- details of how to access it are on the website.
You are required to undertake two MSFs during the three years of the STP programme and it is suggested the first is done at 18 months into the programme and the second towards the end of the programme.
Undertaken at 18 months in and towards the end of the programme
Dr Michael Thomas
National School of Healthcare Science

9. How much evidence is good enough?
STP Induction Day
Types of assessments and reflective practice
10th September :10 to 11:35
How much evidence is good enough?
Evidence needs to demonstrate competence
Evidence is likely to improve over time
Don’t expect competencies to be signed off first time
Use assessor feedback positively
If you have surpassed the competence level, don’t “dumb down”.
Evidence needs to demonstrate competence in a particular task or skill – Technical, Clinical or Professional
In the NHS you need to be 100% competent to be safe, however, you are not expected to be the expert at this point in your career, but you must be able to demonstrate that you can practice that task or skill safely.
Exceeding at a skill or task and surpassing the competence level is optimal, but not essential for every competency
Dr Michael Thomas
National School of Healthcare Science

10. What does good evidence look like?
STP Induction Day
Types of assessments and reflective practice
10th September :10 to 11:35
What does good evidence look like?
Demonstrate that the evidence has met the learning outcome.
Apply your academic learning in a clinical work based context linking evidence to frameworks, literature and best practice.
Evidence should be critically reflective - what happened? What did you learn? How will what you have learned affect future practice?
Use patient feedback through comments or satisfaction surveys. Your focus on patient care should always be included when relevant.
Make evidence visual e.g. photos, videos etc. rather than a bank of text, add spider diagrams, flow charts with meaningful annotations, describe the process.
Keep it brief and to the point.
Don’t just an upload of a policy or a guideline document.
• Don’t do a 10,000-word essay or copy and paste documents – Hard work for you and your assessor
DO IT AS YOU GO ALONG AND NOT ALL AT THE END, Constantly upload evidence to evidence library, even if competency isn’t complete
Do consider patient and public confidentiality
Compendium of everything you will learn and can be referred back to
Dr Michael Thomas
National School of Healthcare Science

11. What is reflective practice ?
STP Induction Day
Types of assessments and reflective practice
10th September :10 to 11:35
What is reflective practice ?
Writing reflective evidence is important to show what you have learned.
Reflect on what you would do if something goes wrong, rather than ‘it went perfectly’. What would you do differently next time?
Are the lessons you learned useful for other or future activities?
What did you find was the greatest challenge in doing this activity?
Reflect on your university work with evidence of the activity e.g. PowerPoint presentation, photos of peer to peer feedback forms is good evidence.
Reflection on a consultation, you could draw on application of literature.
A reflective account of a visit to A&E e.g. patient history taking, consent etc.
What did you learn about yourself from this activity?
What was challenging?
Make sure your reflections are not just Descriptive:
What were my thoughts before and after the event?
How did I feel about the event?
What choices did I make and why?
What could I have done differently?
How might my approach change in the future?
But get better at reflection by thinking more about:
What would I have done differently?
Are the lessons I learned useful for other or future activities?
What did I find the greatest challenge in doing this activity?
Why was this a challenge?
What did I learn about myself from this activity?
Reflections is a way :
You can add additional information and reflect on your personal development.
To study your own experiences to improve the way you work.
A great way to increase confidence and become a more proactive and qualified professional.
Required at all stages of your career and life
Dr Michael Thomas
National School of Healthcare Science

12. STP Induction Day Types of assessments and reflective practice
10th September :10 to 11:35
Ideas for Evidence
“General competency”
Examples could be:
Description of the problem or clinical issue being considered
Case report, treatment plans,
Analytic results
Use references to academic papers or guidance documents about the condition.
Evidence prepared for other purposes, e.g. routine validations, calibrations, audits can be used
“Clinical competency”
Upload anonymised information such as:
Annotated test results
A management plan
Case study
Evidence you understand the impact on the patient
“Professional competency”
Show your engagement through upload of evidence of:
Raising awareness
Being inspirational to others
Getting involved
Becoming an ambassador
Spreading the word
Various types of evidence lend themselves to particular competencies:
Dr Michael Thomas
National School of Healthcare Science

13. STP Induction Day Types of assessments and reflective practice
10th September :10 to 11:35
Top Tips for Evidence
Previous work can be used as evidence
Project work or reports can be very motivational and an efficient way to complete competencies.
Use real life events
Observations of your interactions with patients and colleagues
Presentations to your peers and others signed of by your training officer
Witness Statements
Use a variety of formats – Text, flow diagrams, photos, videos
Keep it interesting and relevant
Dr Michael Thomas
National School of Healthcare Science