Transcript PBA presentation

Workplace-Based Assessment
Procedure Based (PBA)
These slides have been prepared to facilitate discussion on the use of the PBA.
The suggested practical group exercise at the end is designed to be used with a training video.
Overview
•
•
•
•
•
•
•
•
•
•
What is the PBA?
Purposes of the PBA
PBA Form
PBA Worksheet
PBA Domains
Completing the form
PBA Global Summary
PBA vs elogbook levels
How does the PBA work?
Who should use the PBA?
•
•
•
•
•
•
•
•
•
PBA Blueprint
How many PBAs?
PBA validity and reliability
Preparation for the PBA
During the PBA
After the PBA
Group exercise
Further Reading
Acknowledgements
What is the PBA?
• Mandatory workplace assessment for learning
• An observation of the trainee performing a practical
procedure usually in the operating theatre
• Available for advanced specialty index procedures
• Covers patient safety, consent, pre-operative planning and
preparation, intraoperative technique, communication with
the team and post-operative management
PBA vs DOPS
DOPS:
• The counterpart of the PBA
• For the simpler index procedures for Core Training
• Set in Outpatients or Treatment Room
• Adaptable as a portion of a more complex procedure
• Should be developed alongside the PBA
Purposes of the PBA
• To provide feedback to help the trainee improve their
technical and procedural skills, decision making, leadership
and professional skills
• To encourage a reflective approach to learning
• To help the trainee build evidence of their progress
• PBAs form one of the indicators that inform the AES’s end
of placement report for the ARCP
• Individual PBAs are not designed to sign off independent
practice.
PBA Form
PBA Worksheet
Provides examples
of positive and
negative behaviours
that are associated
with each
competency
Serves as an
internal check of
the validity of the
competency
Provides a useful
guide for assessors
and trainees
PBA Domains
Six core domains:
1. Consent
2. Pre-operation planning
3. Pre operative preparation
4. Exposure and closure
5. Intra operative technique
6. Post operative management
Completing the form
• Rated to the standard of Certificate of Completion of Training (CCT)
• Duration of the procedure allows analysis of progression
• Duration of the assessment includes feedback element
• Operation more difficult highlights unusual problems
• Simulated setting allows PBAs to be used in skill labs etc.
• Competencies marked G for generic allow cross specialty analysis
• Competencies marked T are procedure-specific
• 3-point rating scale plus Not assessed for areas outside remit or not
observed
• Area for written feedback
• Trainee can add reflection when PBA is uploaded
• Trainee can link to syllabus topics when PBA is uploaded
PBA Global Summary
• Four levels describing of the trainee’s overall competence on
that occasion
• Maps to surgical logbook levels
• Level 4 is equivalent to a specialist in NHS practice
• Level 4 in any one procedure does not permit a trainee to
perform a procedure unsupervised
PBA vs elogbook levels
PBA LEVELS
0
Insufficient evidence observed
1
Unable to perform the procedure under supervision
2
Able to perform under supervision
3
Able to perform with minimum supervision
4
Competent to perform unsupervised
ELOGBOOK LEVELS
https://www.elogbook.org/
Observed (O)
Assisting (A)
Supervised - trainer scrubbed (S-TS)
Supervised - trainer unscrubbed (S-TU)
Performed (P)
Training more junior trainee (T)
How does the PBA work?
• The trainee selects an advanced procedure, representative of
their normal workload
• The setting is in the operative theatre, scrubbed
• The assessor observes the trainee carrying out the procedure
OR parts of procedures
• The assessor and trainee discuss the procedure using the PBA
form as a guide
• The assessor provides feedback on progress, covering strengths,
exploring development needs and agreeing action plans within a
feasible timescale.
• The assessor records their feedback on the PBA form and
provides a rating against competencies where applicable
• Takes as long as the procedure with 10-15 minutes feedback
Who should use the PBA?
Trainee
• The PBA is for higher specialty training - ST3 and above
Assessor
• Consultants
• Trainee’s current AES
• Senior trainees depending on training level and complexity
of procedure
• Senior SAS grade surgeons
Assessors must have received training in the tool and be
expert in the clinical problem/task
PBA Blueprint
Good Medical Practice
Assessment blueprints
help to ensure that
assessment methods are
mapped to the domains of
the curriculum and Good
Medical Practice.
A single method will
cover some domains and
the range of assessments
should cover all domains
Knowledge
Skills and
Performance
*
Indirectly assessed
b) Apply knowledge and experience
to practice
c) Keep clear, accurate and legible
records
Safety and
Quality
Communication
Partnership &
Teamwork
** Directly assessed
a) Maintain your professional
performance
Maintaining
Trust
a) Put into effect systems to protect
patients and improve care
b) Respond to risks to safety
c) Protect patients and colleagues
from any risk posed by your
health
a) Communicate effectively
b) Work constructively with
colleagues and delegate
effectively
c) Establish and maintain
partnerships with patients
a) Show respect for patients
b) Treat patients and colleagues
fairly and without discrimination
c) Act with honesty and integrity
Curriculum
Knowledge
Judgement
Technique
Professional
*
**
**
**
**
**
**
**
**
**
**
*
*
*
*
**
**
**
**
**
**
**
*
**
**
*
**
**
**
**
**
**
**
*
**
**
**
**
**
**
**
**
**
**
**
**
**
*
How many PBAs?
• To be used throughout surgical training
• Ideally performed every time the trainee is supervised
undertaking an index procedure
• Should be of a sufficient number at level 4 but will vary
according to the complexity of the procedure and
experience of the trainee
• Should be agreed with the trainee’s AES
• The JCST Quality Indicators state that training posts
should offer at least 40 WBAs per year, of which the PBA
should be of an equal proportion to other methods. This is
checked at the ARCP
PBA validity and reliability
Validity and reliability is enhanced by:
•
•
•
•
•
•
•
Assessor training and trainee induction
Linking to Learning Agreement objectives
Use as an assessment for learning
Using frequently throughout training
Using a range of assessors who are qualified and trained
Use across index procedures
Increasing the complexity according to the trainee’s
competence and progression
• Feedback addressing trainee development needs
• Triangulation with the surgical logbook
• Trainee reflection on feedback
Preparation for the PBA
Trainee:
• Arrange the case and PBA with the assessor well in
advance
• Arrange to have the form to hand, either printed or online
• Reflect on the procedure beforehand
Assessor:
• Receive training in the tool
• Ensure it is a procedure in which you have expertise
Both:
• Ensure the patient has given consent
• Make suitable time for the procedure and feedback
• Agree the appropriate level of supervision
During the PBA
Trainee:
• Carry out the procedure as you would normally
• Explain what you intend to do throughout
• Ensure the patient suffers no increased risk or discomfort
Trainer:
• Observe the encounter
• Intervene if patient safety is at risk
• Immediately afterwards and in a suitable environment, discuss with
the trainee, giving the trainee the opportunity to give their view
• Summarise the feedback including what was done well and what
needs to be improved
• Expand on areas that were less than satisfactory, including
suggestions for remedial steps and actions
• Ensure the form captures feedback and areas observed
After the PBA
Trainee:
• Upload feedback to the portfolio accurately in good time
• Reflect on performance and feedback received
• Follow up action plans
Trainer:
• Validate the assessment in good time
• If necessary, follow up on any actions agreed
• Discuss the trainee’s needs with the AES if necessary
Group exercise
Purpose: To practice rating PBA, discuss best practice and enhance consistency
Technique: With a group of new and experienced trainers:
1.
2.
Group discusses the meaning of the PBA competencies
Group views a PBA video (see ISCP PBA example videos)
https://www.iscp.ac.uk/surgical/assessment_observation.aspx (pause at point of feedback in order
to discuss the trainee’s performance)
3.
4.
5.
Each trainer completes the PBA Form ratings and writes feedback comments
based on the video
Group compares ratings and discusses written feedback
Each trainer reflects on whether they would adjust their ratings/comments in
light of discussion
Repeat the exercise with another video (alternatively a role play).
Further discussion:
• Use of scenarios in a simulated setting to help trainees practice skills
Further reading
1.
ISCP Guidance notes on using the PBA
https://www.iscp.ac.uk/static/public/cex_guidance.pdf
2.
ISCP Tips for using PBA
https://www.iscp.ac.uk/static/public/cex_tips.pdf
3.
PBA Form
https://www.iscp.ac.uk/static/public/cex_form.pdf
Acknowledgements
The PBA was derived from OSATS by the Surgical Education
Research Group, Department of Surgery, University of Toronto.
The use of the PBA in the UK was pioneered in Orthopaedics by
the Orthopaedic Curriculum and Assessment Project (OCAP)
The OpComp Project in General Surgery led to further updates
The surgical Specialty Advisory Committees (SACs) developed
PBAs for the surgical specialties within for the Intercollegiate
Surgical Curriculum Programme (ISCP)
The ISCP PBA was validated by Professor Jonathan Beard
http://www.journalslibrary.nihr.ac.uk/__data/assets/pdf_file/0003/64749/FullReporthta15010.pdf