Does success on the American Board of Surgery Qualifying Examination guarantee Certifying Examination success? Thomas W.

Transcript Does success on the American Board of Surgery Qualifying Examination guarantee Certifying Examination success? Thomas W.

Does success on the American Board of Surgery Qualifying Examination guarantee Certifying Examination success?

Thomas W. Biester MS, Jonathan D. Rubright MS, Andrew T. Jones PhD, Mark A. Malangoni MD, FACS The American Board of Surgery, Philadelphia, PA

Board Certification Matters

•

Important benchmark to achieve following completion of residency (Nahrwold DL, Surg, 1999)

•

Associated with better outcomes in patient care (Wenthofer E, Med Educ 2009; Reid RO, Arch Intern Med, 2010)

•

Often required to obtain or maintain hospital privileges

Mission of the American Board of Surgery

•

Serves as an agent for protection of the public and for quality surgical care

•

Certifies surgeons who have met education and training standards

•

Sets standards for maintenance of certification

Process of Obtaining Board Certification

• • • • •

Appropriate undergraduate medical education Successfully complete an ACGME approved surgery residency program Obtain favorable summary evaluation from the program director Pass the ABS Qualifying Examination (QE) Pass the ABS Certifying Examination (CE)

Oral Exams

“Bridge the gap” between the cognitive domain and behavioral evaluation of performance in practice 15/24 ABMS boards have an oral exam

Purposes

•

Determine whether there is a correlation between passing the QE and passing the CE

•

Ascertain whether there is a discrete QE score that would predict passing the CE

Design

• • • • • •

Examinees who passed the ABS QE from 2006-10 QE equated scores matched to 1 st CE pass/fail decisions Contingency table analysis Phi coefficient and point bi-serial for correlation ROC curve constructed p < 0.01

Results

• • • •

4,385 examinees who passed the QE Mean QE score = 82.1 + 5.8 (range, 58-99) 92.3% passed the QE on the 1 st attempt 82.8% passed the CE on the 1 st attempt

Relationship between 1

outcome and 1

CE QE

CE fail CE pass Total QE pass 639 (16%) 3407 (84%) 4046 (92%) QE fail Total 115 (34%) 224 (66%) 339 (8%) 754 (17%) 3631 (83%) 4385 Φ = 0.13, p < 0.001

QE 1 st QE scaled scores and CE pass/fail decisions CE Scale Score Fail Pass Total 55-64 65-69 70-74 75-79 80-84 85-99 Total 4 (67%) 2 (33%) 6 37 (41%) 74 (30%) 53 (59%) 169 (70%) 90 243 286 (26%) 820 (74%) 1106 243 (17%) 1196 (83%) 1439 110 (7%) 1391 (93%) 1501 754 (17%) 3631 (83%) 4385 r pb = 0.229, p < 0.001

1 st QE scaled scores and successful certification (2001-05) Scale Score 50-64 Not certified Certified 23 (96%) 1 (4%) Total 24 65-69 70-74 75-79 80-84 85-99 Total 26 (54%) 64 (33%) 93 (17%) 57 (5%) 38 (2%) 313 (6%) 22 (46%) 133 (68%) 48 197 442 (83%) 535 1054 (95%) 1111 1487 (98%) 1525 4776 (94%) 5089 r pb = 0.339, p < 0.001

Summary

• • •

The correlation between the 1 st QE score and CE pass/fail rate is low Failure on the 1 st risk of CE failure QE doubles the Excellent correlation between 1 score and eventual board certification st QE

Conclusions