Transcript Slide 1
Simulation-based Endovascular Skills Assessment: The Future of Credentialing?
Maureen M. Tedesco, Jimmy J. Pak, E. John Harris, Jr, Thomas M. Krummel, Ronald L. Dalman, Jason T. Lee 22nd Annual Meeting Western Vascular Society September 10, 2007 Stanford Vascular Surgery
Disclosures
Jason T. Lee-
educational grant from Cordis Endovascular to study simulation technology.
Drs. Dalman, Krummel and Lee
: technical grant from Cordis Endovascular in the form of 2 simulators.
Vascular Surgery
Background
• High-fidelity simulation has become important in surgical education.
– Laparoscopy – Endoscopy – Cystoscopy • Training on simulation improves operating room performance of surgical residents.* * Seymour et al. Annals of Surgery, 2002 Vascular Surgery
Background
• Simulation required during physician training for carotid angioplasty and stenting. • Recent applications: – Skills assessment – Technical competency – Board certification • American Board of Vascular Medicine • American Board of Surgery Vascular Surgery
Purpose
Does global performance assessment during endovascular simulation correlate well with self reported procedural skill and prior experience level?
Vascular Surgery
Vascular Surgery
Methods
• 17 general surgery residents interviewing for vascular fellowship training • Pre-test questionnaire: – # of major index vascular procedures – # of specific endovascular procedures • Diagnostic arteriograms • Aortic stent-grafts • Peripheral angioplasty/stenting • Renal stenting • Carotid stenting Vascular Surgery
Methods
Procedicus Vascular Intervention System Trainer (VIST ® ) simulator: Right Renal angioplasty and stenting (RAS) module Vascular Surgery
Methods
Subjects were evaluated by an experienced interventionalist using a global rating scale.
Vascular Surgery
Global Rating Scale (1-5)
Angiogram advance wire into suprarenal aorta without forming a J or pushing against obstruction place pigtail catheter into renal angiogram position/wire manipulation knowledge of renal anatomy/perform angiogram Wire Access Intervention select proper catheter/wire for renal canalization safely traverse lesion select guiding catheter select appropriate renal stent deploy renal stent select proper balloon for renal angioplasty post stent perform completion angiogram Vascular Surgery
Methods
• VIST ® provided objective measurements: – total procedure time – fluoroscopy time – volume of contrast used (mL) – % of lesion covered – placement accuracy – presence of residual stenosis – # of cine loops used Vascular Surgery
Methods
• Post test questionnaire: – Grade his/her own performance – Opinion about optimal number of cases Vascular Surgery
Results
Low Experience (LE,<20) Moderate Experience (ME, 20-100) TOTAL Subjects 8 Endovascular Cases (range) Open Cases (range) 11.1 ± (4-20) 6.8 78.8 ± 38.0
(40-150) 9 17 46.6 ± 22.6 (25-89) 29.9 ± 24.6 (4-89) 75.0 ± 41.1
(40-150) 76.9 ± 38.2 (40-150) Vascular Surgery
Global assessment Total procedure time (sec) Fluoroscopy Time Contrast used (mL) % lesion covered *Placement accuracy (mm) No residual stenosis (% of group) Number of Cine loops Vascular Surgery Low Experience (n = 8) 2.69
895.6
459.6
15.6 96.8
4.85
75% 5.5
Moderate Experience (n = 9) 3.55
947 412 19.2
94.9
6.64
89% 4.7
p value 0.04
NS NS NS NS NS NS NS
Global assessment Total procedure time (sec) Fluoroscopy Time Contrast used (mL) % lesion covered Placement accuracy (mm) Low Experience (n = 8) 2.69
895.6
459.6
15.6 96.8
4.85
No residual stenosis (% of group) Number of Cine loops Vascular Surgery 75% 5.5
Moderate Experience (n = 9) 3.55
947 412 19.2
94.9
6.64
89% 4.7
p value NS 0.04
NS NS NS NS NS NS
Results
Post-test questionnaire: poor correlation between the global assessment score and subjects ’ self assessment score.
5 4.5
4 3.5
3 2.5
2 1.5
1 0.5
0 0 R 2 = 0.1645
1 2 3 4
self assessment score
5 6 Vascular Surgery
Results
• Post-test questionnaire: – vascular surgeons = 19.2 ± 14.4 cases – interventional cardiologists = 14.7 ± cases 14.8 – interventional radiologists = 12.3 ± cases 12.0 • p = NS Vascular Surgery
Summary
• Significant difference in a global assessment score between two groups of surgical residents with varying levels of self-reported endovascular experience. • Global rating scale was able to discern even minimal differences in experience. • No difference between the two study groups with respect to the VIST objective measurements.
Vascular Surgery
Limitations
• Only one “ expert ” observer variability.
observer, no inter • Each subject underwent only one session, without the opportunity to practice or learn the equipment. • Stress may have played a role in this testing situation.
Vascular Surgery
Conclusion
• Correlation between self-reported case completion and global rating score by an observer.
• Objective measures provided by the simulator may not be valid to determine endovascular skills.
• More meaningful criteria to determine how to integrate simulation into skill assessment. • Future research is required to determine if simulator-based testing should be incorporated into the credentialing of vascular specialists. Vascular Surgery
Vascular Surgery