Transcript The Royal College of Pathologists

Introduction to Workplacebased Assessments
Dr Hani Zakhour
Director of Training and Educational
Standards
The Royal College of Pathologists
19 October 2007
History I

PMETB




Standards for postgraduate training and
assessment were published in 2005
Training and assessment should be much
more integrated – an assessment system
Clear view of PMETB and GMC (joint paper
2005) that these should include continuous
monitoring not just periodic College
examinations
Workplace-based assessment” PMETB Jan
2005
RCPath 07
History II

RCPath





College has been fully engaged with reforms
in Medical Education
January 2005, College Council witnessed a
presentation of RCP pilot of WBA, tools were
commended by Council
Director of Examinations and Assessment
May 2005
WBA Manager appointed October 2005
WBA Working group March 2006
RCPath 07
PMETB principle documents

Standards for Curricula
Janet Grant Chair Curriculum Sub-Committee Sue Fox, Namita
Kumar, Ewen Sim, Members Curriculum Sub-Committee
Revised March 2005

Principles for an assessment system for
postgraduate medical Training
Lesley Southgate Chair Assessment Committee Janet Grant
Chair Curriculum Sub-Committee 14 September 2004
RCPath 07
Test methods as part of the
blueprint

Test methods for assessment programmes can
only be selected once the purpose and content
are established




What is the best trade-off between fidelity and
efficiency.
What is the programme for writing and refining test
material?
How are marking schedules developed and reviewed
where relevant?
Is new test material piloted before it is used?
RCPath 07
How will the assessments be
collated?



Assessment tools will build up a profile as the
year progresses
Evidence of a response to feedback will be
shown by subsequent assessment of the same
area
Overall record will form the basis for a
submission to the deanery as evidence of
satisfactory completion of the year ARCP
RCPath 07
Standard Setting



It is only possible to set a sensible standard if
the purpose of the assessment is clearly set out.
The most important thing about a standard is
who sets it
It is the standard expected of trainee at the end
of a particular stage


Understanding principles of the specialty and
practices under direct supervision (Stage A)
In-depth knowledge and understanding of principles
and practices of the specialty (stage D)
RCPath 07
Standardisation



Commonality in the standards that are
required of trainees.
Grade descriptor, intended as a guide
Grade scale




1,2 - Below expectation
3 - Borderline
4 - Meets expectations
5,6 - Above expectations
RCPath 07
Tools

Tools taken from F2 programme

CbD Case-based Discussion (Dame Lesley Southgate)


DOPS Directly Observed Procedural Skills (RCP)


used to enable the documenting of conversations about,
and presentations of, cases by trainees
a method, similar to the Mini-CEX that has been designed
specifically for the assessment of practical skills.
MiniCEX Mini Clinical Evaluation Exercise (John Norcini)

designed to provide feedback on skills essential to the
provision of good clinical care by observing an actual
clinical encounter
RCPath 07
Tools

Tools taken from F2 programme

MSF Multi Source Feedback - ePath SPRAT (Dr Helena
Davies)


provides feedback from a range of co-workers across the
domains of Good Medical Practice. These can be mapped
to the core objectives of the curriculum
ECE Evaluation of Clinical Events (WBA Group
Histopathology)

It replaces MiniCEX for Histopathologists
RCPath 07
Purpose of Assessment
Tools

To assess what cannot be assessed by an
examination.

CbD
 Indicates trainees’ capability, potential and behaviour
 Provides immediate feedback on trainee’s learning
needs highlighting strengths and identifying
weaknesses

DOPs



Indicates trainees’ acquisition of practical skills and
understanding of equipment
Indicates trainee interacts appropriately with patients
MiniCEX

Indicates trainees’ acquisition of clinical skills
RCPath 07
Pitfalls of WBA

To avoid time-consuming “busy” assessment
forms,




the assessments are often simplified if not simplistic
computerised forms can over-simplify complex
judgements
A robust rating can only occur if there are a
number of independent observers – which can
be difficult in small specialties
They have only been validated in a number of
clinical settings but not in all pathology settings
RCPath 07
Beware!

Test methods can only be selected once the
purpose and content are established




What is the best trade-off between fidelity and
efficiency.
What is the programme for writing and refining
test material?
How are marking schedules developed and
reviewed where relevant?
Is new test material piloted before it is used?
RCPath 07
Principles of conduct for
WBA




Assessment based on performance in the
workplace and measured there
Observations can be combined from a
number of assessors to provide a robust
rating for the candidate
Assessment occurs during normal work
immediately followed by feedback on their
routine performance
WBA is not cast in stone – work in
progress and subject to improvement
following evaluation
RCPath 07
Who can be an assessor?

Assessors can be consultants who are
supervisors as well as staff of all grades in
contact with the trainee







Clinical Scientists
Nurses
BMS
Senior trainees
Autopsy room staff and others
No need for prior approval by RCPath
Need to follow WBA guidelines and are willing to
engage in the process
RCPath 07
Who is it for?

Mandatory for all StRs (Specialty
Registrars) appointed from August 2007
to one of the following:



Specialty training programme with a National Training
Number (NTN)
Fixed Term Specialty Training Appointment (FTSTA)
Locum Appointment for Training (LAT)
RCPath 07
Feedback



One of the most important aspects of any
assessment programme is the effect it has on
learning and preparation for individuals and the
profession.
How is feedback given to individuals and
groups about the outcomes of an assessment?
Finally the mechanisms for appeal about the
outcome should be documented and reported.
RCPath 07
RCPath 07
Present and future




2007 – Launch of tools and paper-based
assessment forms.
2008 – Launch of web-based system
Use data generated by all assessments
including MSF and OSPE for validation
Ensure compliance with MMC Gold Guide
and planned ARCP
RCPath 07
Acknowledgments



Sandra Dewar
Dr Trevor Gray
Dr Mary Thompson
RCPath 07
Tribute
Professor
Dame Lesley
Professor
Janet Grant
Southgate
RCPath 07
RCPath 07
RCPath 07