Transcript Postgraduate Diploma in General Pharmacy Practice

Postgraduate Diploma in General
Pharmacy Practice
Induction & Refresher session for
Educational and Practice Supervisors
Online presentation 4:
Workplace-based assessment
In this presentation we will cover
1. Philosophy of workplace-based assessment
(WPBA)
2. Controversies around WPBA
3. Principles of WPBA in the diploma
4. Signposting to WPBA tools used in the diploma
Note that assessments like MCQ/OSCE will be
covered in a separate presentation
What is WPBA?
• Work-based learning inevitably leads to
workplace-based assessment
• Example: a practitioner meets a diploma
learning outcome and a competency related
to providing information to patients
• They record evidence of this, e.g. a mini-CEX
• Do they pass? Or fail? Or neither?
What do we assess in the Diploma
programme
• In the diploma programme, we may assess
knowledge or skills related to:
– The General Level Framework competency
framework
– The curriculum guide/DAP learning outcomes
Formative assessment and summative
assessment
• Formative assessment:
– Assessment FOR learning
• Summative assessment:
– Assessment OF learning
• The two are often confused, which causes
anxiety and compromises the effectiveness of
WPBA
Controversies around WPBA
•
•
•
•
•
Confusion between formative vs summative
Fear about the word ‘assessment’
Correct use of WPBA tools
Adequate training to use the tools
Progress decisions made on the basis of WPBA
Formative vs. Summative in practice
Compare driving lessons and driving tests (acknowledgement: General Medical
Council and their partners)
DRIVING LESSONS:
•Learner drivers would not be happy if they did not receive feedback from their
instructor (trainer) during and at the end of a driving lesson
•It is not about pass/fail
•Feedback matters more than the result (within reason, since there is a patient
at the end of it)
•This is assessment FOR learning
DRIVING TEST:
•When learners feel ready, they take a driving test
•This is a pass/fail test
•The result matters more than the feedback
•This is assessment OF learning
Evidence for WPBA
• AMEE Guide No. 31 - NORCINI & BURCH (2007)
• The research literature on work-based formative
assessment and feedback suggests that it is a
powerful means for changing the behaviour of
learner
• Several formative assessment methods have
been developed for use in the workplace and
there is preliminary evidence of their reliability
and validity
What about our practitioner who
met a learning
outcome/competency earlier?
•Their mini-CEX was assessment FOR their
learning. Academically, they won’t have
‘failed’
•Of course, an employing organisation can
use formative assessments to identify poor
practice
What about competence and
competency frameworks like the GLF?
Competency frameworks such as the General Level Framework
used in the diploma get mixed reviews:
Favourable view
“.. training programmes are required that both allow trainees to
develop appropriate competences required to do their job. It
was also accepted that, to learn, trainees need to be
challenged in their use of knowledge, develop their skills and
acquire the necessary competence.” Sir John Temple, 2010
Challenges
‘Tickbox’ mentality, misuse of frameworks (e.g inappropriate
summative use), misunderstandings, seen as tedious
General Level Framework
New General Level Framework
Delivery of Patient
Care
Personal
Problem Solving
Management and
Organisation
Patient consultation
Organisation
Gathering
information
Clinical governance
Need for the drug
Effective
communication skills
Knowledge
Service Provision
Analysing
information
Budget setting and
reimbursement
Providing
information
Organisations
Selection of the drug
Teamwork
Drug specific issues
Professionalism
Provision of drug
product
Training
Follow up
Medicines
information
Monitoring drug
therapy
Evaluation of
outcomes
Staff management
Procurement
GLF Assessment Rating
Consistently
Demonstrates the expected standard
of practice with very rare relapses
85-100%
Mostly
Implies standard practice with
occasional lapses
51-84%
Sometimes
Much more haphazard than “mostly”
25-50%
Rarely
Very rarely meets the standard
expected. No logical thought process
appears to apply
0-24%
12
How should the GLF be used?
•Formatively, to identify competence and need to develop
•Use diploma WPBA tools, which are based on GLF
competencies, as evidence against the GLF
•Observational evidence is fine
•Avoid seeking to just ‘tick the box’ of competencies met –
look beyond this to the development associated with the
competencies
•Make it ‘aspirational’ rather than ‘impositional’ – it
contains competencies expected of a band 6 pharmacist so
it can be a valuable tool
•As more competencies are met this will identify other
areas to concentrate on
•Use it over the 3 years of the programme – don’t rush it
Evidence to support GLF


JPB assessment tools can provide evidence of meeting GLF
competencies, e.g.. mini-PAT, mini-CEX, CbD, MRCF
Example: Delivery of patient care competencies:
Competence:
Need for drug
Suggested assessment tools,
areas(s) to collect evidence
Relevant patient background
Mini-CEX, CbD, MI enquiries
From: Assessment Handbook Appendix 4
14
Principles of Diploma WPBA
•
•
•
•
•
•
•
•
Formative
Based on what you need them to be able to
do – captured in the GLF and learning
outcomes
Practitioner-led with your support
Aid practitioner development
Help to identify “performance issues” early on
Some subjectivity in assessment
No ‘mark’ but good feedback essential
Look for trends
15
What diploma practitioners
want from their WPBA
assessors








Encourages and motivates
Well informed about the assessment
tools
Impartial
Listens
Approachable
Clear purpose
Promotes reflection (Feedback) and
action planning
Provides direction when needed
What WPBA assessors want
from diploma practitioners








Organised
Knowledgeable
Does not expect to be taught or
directed
Appreciates assessors’ time & input
Uses assessors’ time efficiently
Personable
Visibly learns and develops
Responds appropriately to feedback
16
Competency hierarchy
(David Webb, Director of Specialist Pharmacy Services, East and South East England)
Assessment Strategies
Deployed
Performance
mini-CEX, mini-PAT, CbD, MRCF,DOPS
Experience
Bundled
Competencies
OSCE, simulation
Integration
Developed
Knowledge, Skills and Abilities
Education and training
Innate
Characteristics and Traits
MCQ, MEQ
DipGPP Assessment
Extended
intervention
Formative
WPBA mapped to
GLF
Mini-PAT
MCQ
Mini-CEX
OSCE
CBD
? Change to
Supervised
Learning
Experiences
Summative
? and
MRCF
Viva
DOPS
Assessments
of
performance
Skills
Timeframe
Assessment/Activity
Minimum Numbers
2 (4 & 8 months)
12 months
RITA
Meeting with Educational Supervisor.
Mini-PAT
2*
Sign off 12 month competency
Mini-CEX
2
CbD
2
MRCF
2
DOPS
1
Log of Significant Interventions
Extended Interventions
illustrating the practice
of the practitioner
3
Social Care
1
GPhC CPD records
12
GLF Evidence Summary Matrix
GLF
1
1 signed off by ES at 12 months
Learning Set Work (as described at Learning Sets)
18 months
All of the above plus
RITA
1 (15 months)
Mini-PAT
1*
Mini-CEX
3
CbD
3
MRCF
3
DOPS
1
Log of Significant Interventions
Illustrating the practice of the practitioner
7
Extended Interventions
1
Social Care
6
GPhC CPD records
GLF Evidence Summary Matrix
GLF
Learning Set Work (as described at Learning Sets)
updated from 12 month portfolio
1 signed off by ES at 18 months
19
The Diploma Portfolio
•
•
•
•
Assessment handbook says what should be in
the portfolio, e.g. numbers of each
assessment tool
Don’t clutter with underpinning evidence
Contents signed-off by Educational Supervisor
and reviewed on assessment days at UCL
Remember to use the assessment handbook
20
Diploma portfolio tension
• There is a tension between needing to ‘pass’ a portfolio
review when the contents of it are formative
• Remember that the portfolio review involves
assessment through questioning the practitioner on
the day regarding their knowledge and skills
• A practitioner can only proceed to a portfolio review if:
– They have the minimum number of WPBA in it
– The Trust is satisfied with work-based performance
• We need to be honest about this tension
• Medicine is looking at renaming some WPBA to try and
resolve this. We may do the same
Summary
• We have looked at WPBA including what it is, some
controversies, and how it is used for the diploma
• We have outlined how the GLF should be used and
how it might be misused
• We have outlined the competency hierarchy and the
range of WPBA in the diploma programme
• We have looked at the portfolio review and the
inherent tension to acknowledge
• Please see if there are other online presentations that
will help you and remember to refer back to the
‘Diploma Supervisor induction’ document