Transcript nMRCGP Clinical Skills Assessment: an evolving process

nMRCGP
Workplace-based Assessment
March 2007
Promoting Excellence in Family Medicine
nMRCGP
Integrated assessment package comprising:
Applied knowledge test (AKT)
Clinical skills assessment (CSA)
Workplace-based assessment (WPBA)
Promoting
Excellence in
Family
Medicine
Workplace-based assessment
“The evaluation of a doctor’s progress over
time in their performance in those areas of
professional practice best tested in the
workplace.”
Promoting
Excellence in
Family
Medicine
Some principles of assessment
Validity
Reliability
Educational impact
Acceptability
Feasibility
Promoting
Excellence in
Family
Medicine
Why workplace-based assessment?
Tests something important and different from
other components
“Does do versus can do”
Reconnects assessment with learning
Has high educational impact
Valid and reliable
In keeping with PMETB guidance
Promoting
Excellence in
Family
Medicine
The WPBA framework
An integrated package comprising a
competency-based training record that applies
over an entire training envelope (3 years from
August 2007)
Promoting
Excellence in
Family
Medicine
The educational model of
WPBA for nMRCGP
Promoting
Excellence in
Family
Medicine
The competency-based training record
Competency-based
Developmental
Evidential
Locally assessed
Triangulated
Promoting
Excellence in
Family
Medicine
Competency-based
12 competency areas
Best tested in the workplace setting
Developmental progression for each
competency area
Competency demonstrated “when ready”
Process is learner led
Promoting
Excellence in
Family
Medicine
The 12 competency areas
1. Communication and
7. Primary care administration
consulting skills
and IMT
2. Practising holistically
8. Working with colleagues and
3. Data gathering and
in teams
interpretation
9. Community orientation
4. Making a diagnosis/ making 10. Maintaining performance,
decisions
learning and teaching
5. Clinical management
11. Maintaining an ethical
6. Managing complexity and approach to practice
promoting health
12.Fitness to practice
Promoting
Excellence in
Family
Medicine
Developmental progression
“a process of monitoring a student’s
progress through an area of learning
so that decisions can be made about
the best way to facilitate future learning”
Promoting
Excellence in
Family
Medicine
Evidential
Notion of multiple sampling
From multiple perspectives
Tool-box of “approved” methods (locally
assessed and national complementary tools)
Sufficiency of evidence defined
Promoting
Excellence in
Family
Medicine
Locally assessed
Assessed by clinical supervisor in hospital or
general practice setting
Regular reviews at 6 month intervals by
trainer/educational supervisor
Review all the assessment information
gathered
Judge progress against competency areas
Provide developmental feedback
Promoting
Excellence in
Family
Medicine
Triangulated
Different raters
Many tools (e.g. CBD, COT, mini CEX,
DOPS, MSF and PSQ)
Different settings (hospital and general
practice)
Promoting
Excellence in
Family
Medicine
Gathering the evidence
about the learner’s
developmental progress
Promoting Excellence in Family Medicine
Evidence from
Locally assessed tools
Complementary tools and…
Naturally occurring information
Promoting
Excellence in
Family
Medicine
Tools for Evidence
CBD (case based discussion)
COT (consultation observation tool)
mini-CEX (clinical evaluation exercise)
DOPS (direct observation of procedural skills)
MSF (multi-source feedback)
PSQ (patient satisfaction questionnaire)
Promoting
Excellence in
Family
Medicine
Case-based discussion
Structured oral interview
Designed to assess professional judgement
Across a range of competency areas
Starting point is the written record of cases
selected by the trainee
Will be used in general practice and hospital
settings
Promoting
Excellence in
Family
Medicine
COT
Tool to assess consultation skills
Based on MRCGP consulting skills criteria
Can be assessed using video or direct
observation during general practice settings
Promoting
Excellence in
Family
Medicine
Mini CEX
Used instead of COT in hospital settings
Promoting
Excellence in
Family
Medicine
DOPS
For assessing relevant technical skills during
GP training:
Cervical cytology
Complex or intimate examinations
(e.g. rectal, pelvic, breast)
Minor surgical skills
Similar to F2 DOPS
Promoting
Excellence in
Family
Medicine
MSF
Assessment of clinical ability and professional
behaviour
ST1 Rated by 5 clinical colleagues, 2 occasions
ST3 Rated by 5 clinical and 5 non-clinical
colleagues on 2 occasions
Simple web based tool
Is able to discriminate between doctors
Needs skill of trainer in giving feedback
Promoting
Excellence in
Family
Medicine
PSQ
Measures consultation and relational empathy
(CARE)
30 consecutive consultations in GP setting
Central optical scanning and generation of results
Can differentiate between doctors
Needs skill of trainer in giving feedback
Promoting
Excellence in
Family
Medicine
Naturally occurring evidence
From direct observation during training
“tagged” against appropriate competency
headings
Other practice-based activities
Promoting
Excellence in
Family
Medicine
Monitoring Progress
Interim reviews with trainer
6 month intervals
ensure the trainee is making satisfactory
progress
agree training needs
Promoting
Excellence in
Family
Medicine
Monitoring Progress
Deanery Panel meeting at end of ST1
and ST2
reviews the training records of every
trainee
face to face review with trainees when
unsatisfactory achievement in either of
the complementary tools
or when requested by the educational
supervisor
Promoting
Excellence in
Family
Medicine
Workplace-based assessment ST1
6 month
Interim review
Promoting
Excellence in
Family
Medicine
12 month
Deanery panel
if unsatisfactory
Interim review
Based on evidence:
Based on evidence:
3 x COT or mini-CEX
3 x COT or mini-CEX
3 x CBD
3 x CBD
1 x MSF
1 x MSF
1 x PSQ *
1 x PSQ *
DOPS **
DOPS **
Clinical supervisors’
report **
Clinical supervisors’
report **
* if GP post
** if appropriate
Workplace-based assessment ST2
18 month
Promoting
Excellence in
Family
Medicine
24 month
Interim review
Interim review
Based on evidence:
Based on evidence:
3 x COT or mini-CEX
3 x COT or mini-CEX
3 x CBD
3 x CBD
1 x PSQ *
1 x PSQ *
DOPS **
DOPS **
Clinical
supervisors’ report **
Clinical supervisors’
report **
Deanery panel
if unsatisfactory
* if GP post
** if appropriate
Workplace-based assessment ST3
30 month
34 month
Interim review
Final review
Based on evidence:
Based on evidence:
6 x COT
6 x COT
6 x CBD
6 x CBD
1 x MSF
1 x MSF
DOPS **
DOPS **
Deanery sign off or
panel review if
unsatisfactory
PSQ
Promoting
Excellence in
Family
Medicine
** if appropriate
The final judgement
The trainer makes a recommendation as to
whether the trainee has achieved competence
in all 12 areas at the end of training
Promoting
Excellence in
Family
Medicine
Review by Deanery Panel
Review of e-portfolio if satisfactory
level achieved in training record
Review of e-portfolio and face-toface meeting with trainee, if
satisfactory level not achieved
Promoting
Excellence in
Family
Medicine