Transcript CSV Module 4 - Association for Academic Psychiatry

Clinical Skills Verification Rater Training
MODULE 4
Strategies for Clinical Skills Assessment:
Models and Best Practices
Michael Jibson, M.D., Ph.D.
David Kaye , M.D.
Sandra Sexson , M.D.
Module 4 Pre-Test
1.
The strength of a Clinical Skills Verification
Examination stems from its ability to serve as a
forum for all of the following except:
A.
A high stakes examination
B.
direct observation of clinical skills
C.
provision of immediate feedback on
performance
D.
repeated administrations during residency
Module 4 Pre-Test
2.
Which of the following models for CSV exam
is the least standardized?
A.
Workshop Model
B.
Mock Board Model
C.
Clinical Evaluation Model
Module 4 Pre-Test
3.
Which of the following models for the CSV
exam makes the greatest use of resident peer
feedback?
A.
Workshop Model
B.
Mock Board Model
C.
Clinical Evaluation Model
Module 4 Pre-Test
4. Within a program, consistency of faculty
expectations and rating of residents on the CSV
exam can be maximized by:
A. All raters using the same evaluation model
B. Limiting assessment to the three competency
areas outlined by the ABPN
C. Using standardized patients
D. Monitoring inter-rater reliability
Clinical Skill Verification (CSV)
Strengths of Clinical Skills Verification
• Direct observation of clinical skill
• Multiple observations and observers
• Defined areas for evaluation
• Specific anchors for performance
• Immediate feedback
• Ample opportunity for remediation
Evaluation Model
We recommend that the evaluation:
• Be done early in residency and repeated
frequently
• Include immediate feedback
• Avoid the “high-stakes exam” model
Evaluation Model
Mock Board Model
• Use the CSV for the current practice of
“Mock Board” exams
• Prearranged patients
• Scheduled interviews
Mock Board Model
Advantages
Disadvantages
• Familiar format
• Retains high-stakes
exam format
• Highly standardized
• Narrow patient
selection
• Difficult to
schedule
Evaluation Model
Workshop Model
• Use the CSV form for a workshop or class
on interviewing
• Involve peer resident observers
• Videotape the encounter
Workshop Model
Advantages
• Multiple residents
involved
• Highly
standardized
• Excellent
feedback (if
videotaped)
Disadvantages
• Few opportunities
for each resident to
interview
• Narrow patient
selection
Evaluation Model
Clinical Model
• Use the CSV form for any initial clinical
encounter with supervising faculty
• Inpatient admission
• Emergency room evaluation
• Outpatient assessment
Clinical Model
Advantages
• Broad range of
patients
• Many
opportunities to
interview
• Easy to schedule
Disadvantages
• Moderately
standardized
• Faculty buy-in
essential
Evaluation Model
We recommend that the evaluation:
• Occur during regularly scheduled clinical
rotations
• Involve the standard interview used in that
clinical setting
• Involve randomly selected patients
• Be done as part of the routines of clinical
care
Additional Issues
Programs may want to consider:
• Using the exam to test additional
competencies for their own purposes
• Requiring residents to pass the examination
before advancing through training
Additional Competencies
CSV requirements may be combined with
other clinical competencies:
• Case formulation
• Diagnostic assessment
• Treatment planning
Faculty Training
We recommend that programs:
• Offer training to faculty in how to conduct
and score the examination
• Make an effort to standardize scoring on the
examination
• Monitor inter-rater reliability internally
• Consider credentialing faculty as evaluators
Post-Test Module 4
1.
The strength of a Clinical Skills Verification
Examination stems from its ability to serve as a
forum for all of the following except:
A.
A high stakes examination
B.
direct observation of clinical skills
C.
provision of immediate feedback on
performance
D.
repeated administrations during residency
Post-Test Module 4
1.
The strength of a Clinical Skills Verification
Examination stems from its ability to serve as a
forum for all of the following except:
A.
A high stakes examination
The in –residency CSV exam allows programs to
administer repeatedly and avoid the high-stakes
aspect.
Post-Test Module 4
2.
Which of the following models for CSV exam
is the least standardized?
A.
Workshop Model
B.
Mock Board Model
C.
Clinical Evaluation Model
Post-Test Module 4
2.
Which of the following models for CSV exam
is the least standardized?
C.
Clinical Model
The clinical model utilizes patients seen on regular
clinical services without pre-selection of patients
Post-Test Module 4
3.
Which of the following models for the CSV
exam makes the greatest use of resident peer
feedback?
A.
Workshop Model
B.
Mock Board Model
C.
Clinical Evaluation Model
Post-Test Module 4
3.
Which of the following models for the CSV
exam makes the greatest use of resident peer
feedback?
A.
Workshop Model
Post-Test Module 4
4. Within a program, consistency of faculty
expectations and rating of residents on the CSV
exam can be maximized by:
A.
All raters using the same evaluation model
B.
Limiting assessment to the three competency
areas outlined by the ABPN
C.
Using standardized patients
D.
Monitoring inter-rater reliability
Post-Test Module 4
4. Within a program, consistency of faculty
expectations and rating of residents on the CSV
exam can be maximized by:
D.
Monitoring inter-rater reliability
Along with faculty training on how to conduct and score the
examination, tracking inter-rater reliability among faculty after
observing the same performance and giving feedback to faculty
regarding their ratings compared to others is a very effective way of
standardizing ratings.