Transcript National Workplace Assessment for Licensing

Workplace based assessment
for the nMRCGP
nMRCGP
Integrated assessment package
comprising:
Applied knowledge test (AKT)
Clinical skills assessment (CSA)
Workplace based assessment (WPBA)
Workplace based assessment
Definition
“The evaluation of a doctor’s progress
over time in their performance in those
areas of professional practice best tested
in the workplace”
Some principles of assessment
Utility
Validity
Reliability
Educational impact
Acceptability
Feasibility
Why workplace-based assessment?
 Tests something important and different from
other components
“Does do versus can do”
 Reconnects assessment with learning
 Has high educational impact
 Valid and reliable
 In keeping with PMETB guidance
The WPBA framework
The components of the framework
An integrated package comprising:
A competency-based training record and…
Two externally marked work-based tools
That applies over an entire training envelope
(3 years from August 2007)
The educational model of
WPBA for nMRCGP
The competency-based
training record
Key features
 Competency-based
 Developmental
 Evidential
 Locally assessed
 Triangulated
Competency-based
12 competency areas
Best tested in the workplace setting
Developmental progression for each
competency area
Competency demonstrated “when ready”
Process is learner led
The 12 competency areas
1. Communication and
consulting skills
2. Practising holistically
3. Data gathering and
interpretation
4. Making a diagnosis/
making decisions
5. Clinical management
6. Managing complexity
and promoting health
7. Primary care
administration and IMT
8. Working with colleagues
and in teams
9. Community orientation
10. Maintaining
performance, learning
and teaching
11. Maintaining an ethical
approach to practice
12. Fitness to practice
Developmental progression
“a process of monitoring student’s progress
through an area of learning so that
decisions can be made about the best way
to facilitate future learning”
Evidential
Notion of multiple sampling
From multiple perspectives
Tool-box of “approved” methods (locally
assessed and national external tools)
Sufficiency of evidence defined
Locally assessed
 Assessed by clinical supervisor in hospital or
general practice setting
 Regular reviews at 6 month intervals by
educational supervisor
Review all the assessment information gathered
Judge progress against competency areas
Provide developmental feedback
Triangulated
Internally:
Different raters
Local tools (e.g. CBD, COT, mini CEX, DOPS)
Different settings (hospital and general practice)
Externally:
Multiple raters
Externally marked tools (MSF and PSQ)
Different settings (hospital and general practice)
Gathering the evidence about the
learner’s developmental progress
Evidence from
Locally assessed tools
External tools, and…
Naturally occurring information
Local tools
CBD (case based discussion)
COT (consultation observation tool)
mini-CEX (clinical evaluation
exercise)
DOPS (direct observation of
procedural skills)
Case based discussion
Structured oral interview
Designed to assess professional
judgement
Across a range of competency areas
Starting point is the written record of cases
selected by the trainee
Will be used in general practice and
hospital settings
COT
Tool to assess consultation skills
Based on MRCGP consulting skills criteria
Can be assessed using video or direct
observation during general practice
settings
Mini CEX
Used instead of COT in hospital settings
DOPS
For assessing relevant technical skills
during GP training:
Cervical cytology
Complex or intimate examinations (e.g. rectal,
pelvic, breast)
Minor surgical skills
Similar to F2 DOPS
External tools
MSF (multi-source feedback)
PSQ (patient satisfaction survey)
MSF
Assessment of clinical ability and
professional behaviour
Rated by 5 clinical and 5 non clinical
colleagues on 2 occasions in GP practice
Simple web based tool
Is able to discriminate between doctors
BUT
Needs skill of trainer in giving feedback
PSQ
Measures consultation and relational
empathy (CARE)
30 consecutive consultations
Central optical scanning and generation of
results
Can differentiate between doctors
BUT
Needs skill of trainer in giving feedback
Naturally occurring evidence
From direct observation during training
“tagged” against appropriate competency
headings
Other practice-based activities
Monitoring progress and final
decisions
During training
 Meeting at end of ST1 and ST2
 A Deanery based panel will:
Review the ETR of any trainee who:
 “Fails “ in any of the external tools during training
 Where requested by the educational supervisor i.e. where
there may be a problem with the trainee’s progress in
performance against the 12 competency areas
Ensure the trainee is making satisfactory progress
The Final Judgement
The trainer makes a recommendation as
to whether the registrar has achieved
competence in all 12 ETR areas at the end
of training
This together with achievement in the
external workplace assessments will be
reviewed by a Deanery panel
4 possible outcomes:
 Pass in ETR and external assessments – clear
pass.
 Fail in ETR and fail in external assessments –
clear fail. Referred to Deanery panel but no
change in decision
 Pass in ETR but fail in external assessments –
referred to Deanery panel for decision
 Fail in ETR but pass in external assessments –
referred to Deanery panel for decision
Workplace-based assessment ST1
6M
Interim review
Interim review
Based on evidence:
Based on evidence:
*3 x mini-CEX
3 x mini-CEX
3 x CBD
3 x CBD
**DOPS
**DOPS
**Clinical
supervisors report
**Clinical
supervisors report
* COT if GP post
** if appropriate
Deanery panel
if unsatisfactory
6M
MSF x 2
PSQ
Workplace-based assessment ST2
6M
Deanery panel
if unsatisfactory
6M
Interim review
Interim review
Based on evidence:
Based on evidence:
*3 x mini-CEX
*3 x mini-CEX
3 x CBD
3 x CBD
**DOPS
**Clinical
supervisors report
**Clinical
supervisors report
* or COT if GP post
** if appropriate
**DOPS
Workplace-based assessment ST3
6M
6M
Interim review
Final review
Based on evidence:
Based on evidence:
*6 x COT
6 x COT
6 x CBD
6 x CBD
**DOPS
MSF x 2
**Clinical
supervisors report
PSQ
* mini-CEX if hospital post
** if appropriate
Deanery sign
off or panel
review if
unsatisfactory
**DOPS