Transcript Implementing Work-Based Assessments

Implementing Work-Based
Assessments
Professor T.Masud
Nottingham University Hospitals
NHS Trust
Overview- Curricula and Assessments for Training
Selection
F1
F2
FP
Curriculum
ST1
CCT in Geriatric Medicine
JRCPTB Cert. in GIM L2
Allocation
ST2
ST3
ST4
ST5
ST6
Core Training
Speciality Curriculum
GIM (Acute) L1
Curriculum
GIM (Acute) L2 Curriculum
Generic C. L1
Generic Curriculum L2
MRCP Parts 1, 2 (written), PACES
KBA (specialist exam)
WORK BASED ASSESSMENTS
*Tooke report recommends separating F1 and F2 & joining F2 to ST1+2
ST7
Generic Curriculum - builds on Foundation Curriculum
(competencies categorised into Knowledge, Skills, Attitudes & Behaviour)
Level 1 (Mandatory CT Competencies)
– 1.1 History taking, examination, record keeping
– 1.2 Time management and Decision Making skills
– 1.3 Good Quality Care and Patient Safety
– 1.4 Infection Control
– 1.6 Valid Consent
– 3.1 Communication wih patients within a consultation
– Focus Area 4 Working with Colleagues
– Focus Area 6 Professional Behaviour
Generic Curriculum - builds on Foundation Curriculum
(competencies categorised into Knowledge, Skills, Attitudes & Behaviour)
Level 2 Competency Areas
–
1.5
Health Promotion and Public Health
–
1.6i
Medical Ethics and Public Health
–
1.6iii Legal framework for Practice
–
1.7
Ethical research
–
1.8
Managing Long Term Conditions and Promoting Patient Self-Care
–
Focus Area 2 Governance + Maintaining Good Clinical Practice
–
3.2
Breaking Bad News
–
3.3
Complaints and Medical Error
–
Focus Area 5 Teaching and Training
–
Focus Area 7 Management and NHS Structure
Curriculum for GIM (Acute Medicine)– Levels1+2
(Categorised into levels 1 & 2 by Knowledge, Skills , Attitudes+Behaviour)
Emergency Presentations
–
–
–
–
Cardio-respiratory arrest
Shocked patient
Unconscious patient
Anaphylaxis
Top 20 Common Medical Presentations
Abdominal Pain
Acute back pain
Blackout/Collapse
Breathlessness
Chest pain
Confusion- acute
Cough
Diarrhoea
Falls
Fever-
Fits/seizures
Haematemesis/Melaena
Headache
Jaundice
Limb pain/swelling
Palpitations
Poisoning
Rash
Vomiting /nausea
Weakness/Paralysis
Curriculum for GIM (Acute Medicine)– Levels1+2
(Categorised into levels 1 & 2 by Knowledge, Skills , Attitudes+Behaviour)
Other important presentations
Abdo mass/hepatosplenomegaly
Abdo swelling/constipation
Abnormal sensation
Aggressive/disturbed behaviour
Alcohol/substance dependence
Anxiety / panic behaviour
Bruising
Chance findings
Dialysis
Dyspepsia
Dysuria
Genital discharge & ulceration
Haematuria
Haemoptysis
Head Injury
Hoarseness and stridor
Hypothermia
Immobility
Involuntary movements
Joint swelling
Lymphadenopathy
Loin pain
Medical complics of acute
illness/surgery
Medical problems in pregnancy
Memory loss
Micturition (difficult)
Neck pain
Non-organic physical symptoms
Polydipsia
Polyuria
Pruritis
Rectal bleeding
Skin and mouth ulcers
Speech disturbances
Suicidal ideation
Swallowing difficulties
Syncope and presyncope
Unsteadiness /balance problems
Visual disturbance
Weight loss
Investigation Competencies
SYSTEM SPECIFIC
COMPETENCIES
Allergy
Cancer & palliative care
Cardiovascular medicine
Clinical pharmacology
Dermatology
Diabetes and endocrinology
Gastroenterology / hepatology
Haematology
Immunology
Infectious diseases
Medicine in the Elderly
Musculoskeletal system
Neurology
Psychiatry
Public Health / H.Promotion
Renal medicine
Respiratory medicine
Procedural Competencies
Geriatric Medicine ST Curriculum (Jan 2007) 1
Primary learning objectives (achieve competencies in:)
[various settings]
1.
Perform Comprehensive Geriatric Assessment
2.
Diagnose and manage acute illness in old age
3.
Diagnose and manage those with chronic disease and disability
4.
Provide rehabilitation with the multidisciplinary team to an older patient
5.
Plan the transfer of care of frail older patients from hospital
6.
Assess a patient’s suitability for and provide appropriate care to those in
long term (continuing) care in the NHS or community
7.
Apply knowledge and skills of a competent geriatrician in an
intermediate care and /or community setting
Geriatric Medicine ST Curriculum (Jan 2007) 2
8.
Assess and manage older patients presenting with the common geriatric
problems (syndromes):
a. Falls- with and wihout fracture
b. Delirium
c. Incontinence
d. Poor mobility
9.
Demonstrate competence in following subspecialities:
a. Palliative care
b. Orthogeriatrics
c. Old Age Psychiatry
d. Specialist Stroke Care
10.
Be competent in
a. Research methodology (basic)
b. Ethical principles of research
c. Critical appraisal of medical literature
d. Preferably to have personal experience of research
[basic science or clinical (health service)]
Assessment Plan in Geriatric Medicine - Overview
MRCP PACES
KBA (SE)
Work Based Assessments
– Mini-CEX, DOPS, ACAT, CbD, MSF, PS
Assessment of Generic Areas
– Research (portfolio, supervisor reports, publications)
– Audit (portfolio, audit reports, supervisors reports)
– Clinical Governance (portfolio, supervisor reports)
– Teaching (portfolio, assessed teaching)
Regular Appraisal by Educational Supervisor
(including Educational Supervisor Reports)
Training Record
– educational supervisor + consulatant trainer reports
– completed + signed work based assessments
– reflective learning
– course certificates, audit reports, publications etc
Work Based Assessment Methods
Mini-Clinical Evaluation Exercise (mini-CEX)
Direct Observation of Procedural Skills (DOPS)
Multi-Source Feedback (MSF)
Case-Based Discussions (CbD)
Patient Survey (PS)
Acute Care Assessment Tool (ACAT)
Mini-CEX (ST3-ST7) [n=25]
4 Acute (1 ward round)
1 Delirium / Depression
2 Rehab (1 ward round)
1 Old Age Psychiatry (HV / Ward referral)
1 Pre-op Orthogeriatric
1 Osteoporosis / metabolic bone disease
1 Post-op Orthogeriatric
1 Comprehensive Geriatric Assessment
2 Chronic disease (clinic eg DM, OA)
1 Intermediate care / Home visit
1 MDT chair (discharge)
1 Continuing care
1 Continence
1 Day Hospital
1 Falls
3 Stroke (acute WR, Rehab WR, TIA clinic
1 Movement disorder
1 Palliative care (break bad news)
CbD (ST3-ST7) [n=26]
4 Acute (diagnosis, Mx, prescribing)
1 Depression / dementia
2 Rehab
1 Falls
2 Continuing care (≥ 1 non-NHS)
1 Continence
1 Evidence Based Medicine
1 Orthogeriatric – acute
1 Ethics / Law
1 Orthogeriatric - rehab
1 Health Promotion
1 Acute Stroke
1 Complaint
1 Rehab Stroke
1 Intermediate Care
1 Neurovascular investigation (TIA)
1 Transfer of Care problem
1 Palliative care
1 Delirium
1 Tissue Viability / Hypothermia
1 Old Age Psychiatry
Mid-Trent Experience of MSF
(360 degree appraisals for SPRS)
Overview of Earlier Pilot
• Early pilot June-August 2004
• 18 SPRs in 6 NHS Trusts
• Then 13 SPRs (1 abroad, 1 maternity leave)
• 11 SPRs performed 360 degree appraisals
• Minimal guidance
• Experience discussed at RITAs Sept 2004 (subjective)
• Questionnaire sent to SPRs in Jan 2005 (objective)
Who decided which people the form should
be sent to?
6
5
4
No.
3
2
Ward
Manager
1
0
SPR
SEC
ED SUP
OTHER
Did you find the exercise useful?
Yes- extremely useful
1
Yes- quite useful
4
Not sure
3
No- not that useful
2
No- waste of time
1
Comments: Those who found process
useful
• Gave useful feedback on how I am performing on the ward
• I think it is a great way of assessing clinicians
• Getting positive feedback improved my confidence.
• The process informed the appraisal meeting with my
educational supervisor
Comments: Those who found process not
useful
• Other people should distribute the forms so the SPR does
not know who has been asked to complete them (x2)
• Meetings with the educational supervisor more important
than the 360 degree appraisal
Further analysis
• Of the 5 SPRs who found the process useful, for 3
of them “others” decided who to send the forms to.
• Of the 6 SPRS who who did not find the process
useful or were not sure, all 6 SPRs chose the people
to send the forms to
Conclusion of the Early pilot
Main hypothesis generated:
The process is more useful if educational supervisor / consultant
trainer decides to whom the questionnaires are sent to.
Randomisation- by Trust
Randomised to
3 Trusts
SPRs decides
(n=8)
3 Trusts
ES/Con decides
(n=7)
Questionnaire sent to SPRs and ESs
after the RITAs in Oct 2005
Mean no of Qs sent 17.8 (range 10-20)
Mean returns 15.1 (range 10-20)
Mean response rate 85% (range 55%100%)
No difference between the 2 groups
Did you find the process useful?
Total
SPR
Decides
ES/Con
Decides
Yes- extremely
2
0
2
Yes- quite
9
4
5
Not sure
2
2
0
No- not that
useful
No- waste of
time
2
2
0
0
0
0
“50% useful”
“100% useful”
[Chi sq 4.77, df 1, p=0.029; Fishers p=0.051]
Mini-CEX
12 SPRs had performed at least 1 Mini-CEX
9 found them useful, 3 not
Egs - Ward round (x4); MDM (x2); Tilt (x1)
- Rest not stated
DOPS + Patient Satisfaction Qs
4 SPRs had performed at least 1 DOPS
3 found them useful (Temp pacing, OGD, not stated)
1 not useful (LP)
1 SPR performed a patient satisfaction Q- useful
Feedback from Educational supervisors
(5 returns from 7)
2 found the 360 degree process extremely
useful
2 found it quite useful
1 not sure
Feedback from Educational supervisors
Was the process time-consuming?
No- 4
Yes–slightly 1
Yes – very 0
Assessment Plan For Speciality Training in
Geriatric Medicine
Summative
MRCP
MiniCEX
CbD
6
6
Acute
Acute
6
6
6
6
ST6
6
6
ST7
1
2
TOT
25
26
ST3
KBE
(SE)
Formative
*
ST4
ST5
*
MSF
ACAT
PS
ES+CT
Reports
1
*
1
*
*
*
1
4
Teach
Asses
*
Acute
Audit
*
Audit
(eg rehab)
*
PYA
*
2
Cert
ALS
*
1
2
Audit /
Research
ALS
Res.
Meth.
*
ALS
Teach
Audit (eg IC)
Res. present.
ALS
Publication
ALS
Manag.
Role of the ARCP
1. Learning agreement:
aims & intended learning outcomes
based on specialty curriculum
2. Advice on portfolio
3. Regular feedback (2 way)
4. Personal Development Plan
5. Trainer’s structured report
6. Workplace based (NHS) appraisal
Educational
Appraisal
Workplace
based (NHS)
appraisal
ARCP
A. Evidence
1 Assessment of performance e.g.
workplace based assessments and
observational methods
E.g. mini-CEX, DOPS, video,
CBD
examinations
structured report
2. Assessment of experience, e.g.
portfolio/log book
audit
research
critical incidents
Annual
Review of
Competence
Progression
Outcome
B. Annual Competence Review
Appropriately constituted panel
considers evidence
Outcome of review
Educational supervisor and/or TPD meet
with trainee to
review competence outcome with
trainee
plan next part of training
Annual
Planning
Based on a paper from PMETB’s Workplace Assessment Group (2005)
Challenges
Overlap between SpRs and StRs
StRs – proactive
Educational supervisors – proactive
Consultant Trainers – proactive
New documentation – structured
Time in Job plans
Useful process or tick-box exercise?
Some Suggestions
• StRs and CTs to have forms for CbDs, Mini-CEX
handy
• 2 CbDs and 2 Mini-CEX per 4 month attachment
• 3 CbDs and 3 Mini-CEX per 6 month attachment
• Importance of Meeting with Education Supervisor
about 1 month before ARCP
• Regular half day / full day teaching sessions
mapped to curricula
StR
Educational
Supervisor
Consultant
Trainer
BGS ETC
SAC
JRCPTB
PMETB
TPDs
Partnership
Regional
Advisor
Specialist
Training
Committee
NHS
Trust
Deanery
Assessments! Assessments!
More Work Based Assessments