NAS -

(and other new program requirements)

Overview of what you will need to do!

J. L. Marsh, MD Chair - Orthopaedic RRC Director of the ABOS

Carroll B. Larson Chair Residency Program Director University of Iowa Hospitals and Clinics Iowa City Iowa

We are about to experience tremendous changes to the oversight of GME

 

Next Accreditation System (NAS) Milestones

    

Procedural minimums Continuous data assessment Surgical skills training Greater institutional oversight No PIF, less and different site visits

The financial implications of these changes are uncertain at this time

Traditional Program Review

   

Program review scheduled PIF prepared and sent to ACGME and SV SV – 1-2 days RRC review



PIF and SVR

SV and PIF are key portions of program review 

Board pass rates, Resident Survey, Case log data



RRC actions



Initial or continued accreditation with citations

 

1-5 year cycle Progress report



Propose probation

What is NAS???

• Program accreditation system

without

: – Mandated site visits – PIF’s – 100% mandatory PR’s – Direct resident interviews • Program accreditation system

with

: – Annual or semi annual data review – Performance metrics within the data elements – Emphasis on outcomes – Focused site visits driven by data metrics – More opportunity for program innovation – Data charts, graphs and data flows!!!

Notice the “PIF and site visit” are going away Notice the word “data” is used 5 times!!

NAS is not all about Milestones

 Milestones will be a work in progress but will not be a part of program accreditation for several years  But Milestones are one of the new requirements where increased effort and cost for programs maybe necessary and they need to start now!!

NAS represents a substantial c

h

ange in program oversight

 Change in focus / function of RRC   More educational Less regulatory  PD’s empowered to innovate & create an excellent program  Core vs. detail requirements  Improved tools for program review without a PIF  Focused reviews triggered by parameters set by the RRC

The data for Program Review by RRC

 Trended & weighted performance metrics – Program data – Resident and faculty scholarship (new template) – Clinical experience (enhanced case logs) – Resident Survey – Core Faculty Survey – Semi-annual Resident Evaluation • Milestones (new) (new questions) (new) • Clinical Competency Committee (new) – Rolling Board pass rates (Parts I & II) – Program Self-Study (new) Many of these are outcomes, many of them are new!

Site Visit (every 10 years)

Performance thresholds based on data elements

• Weighting of data elements will provide screening criteria • RRC annual review and action only if necessary • Potential actions include:  Initial or Continued Accreditation  Request more information from program  Request Site visit (focused or full) on short timeline  Continued Accreditation with Warning  Probation • “If there is a problem get in there and fix it” • “If the data is good…….leave them alone…innovation”

Absent – SVR and PIF document

Program Requirement Changes

• • • • Common & Specialty Specific (no change)

Core:

requirement must be met as specified; if not, program can be cited

Detail

: programs will not be assessed for compliance with these requirements if they demonstrate good educational outcomes. These are mandatory for new programs & those that failed to meet outcomes expectations (on Probation or Continued Accreditation with Warning)

Outcome

– Some data elements are based on these

NAS Program Review



Each

program reviewed

at least

annually  NAS is a continuous accreditation process – Review of annually submitted data – Supplemented by: • Reports of self-study visits every ten years • Progress reports (when requested) • Reports of site visits (as necessary)

Proposed workflow prior to RRC meeting



Annual Review of Data (Oct. – Nov.)



Options Available Prior to January RRC Meeting

• Focused Site Visit • Full Site Visit • Request Clarification or Progress Report • Send Material out to RRC for Review Highlighting the Problem(s) for Peer Decision • Move to Consent Agenda

NAS: Focused Site Visit

• Minimal notification given • Minimal document preparation expected • Team of site visitors • Specific program area(s) investigated as instructed by the RRC

NAS: Full Site Visit

• Application for new program • At the end of the initial accreditation period • RRC identifies broad issues / concerns • Other serious conditions or situations identified by the RRC

Theoretical Work Flow – Consent agenda (90%)



Programs (CA) meet all established performance indicator thresholds (40%) –

letter from ED Continued accreditation with no RRC review 

Programs (CA) fail to meet 1 established performance indicator (30%) -

letter from ED Continued Accreditation but notes need for improvement - indicates the deficiency 

Programs that fail to meet 2 established performance indicator thresholds but not “High Stakes” indicators (20%)

– ED reviews program for trends – if first time event letter from ED Continued Accreditation notes problems no further RRC review

Theoretical Work Flow – RRC review

 Programs fail to meet 3 -5 established performance indicator thresholds (7%) – Two RRC reviewers assigned  Programs fail 6-9 performance indicator thresholds (3%) – Assigned a focused or full site visit

For NAS you need to do two things!

•

Do well on the performance metrics

•

Appoint a new committee to oversee Milestones and develop a plan to evaluate them

The data for Program Review by RRC

 Trended & weighted performance metrics – Program data – Resident and faculty scholarship (new template) – Clinical experience (enhanced case logs)

Maintain a consistent solid performance on all

– Resident Survey (new questions) – Core Faculty Survey (new)

of these!!

– Semi-annual Resident Evaluation • Milestones (new) • Clinical Competency Committee (new) Many of these are outcomes, many of them are new!

– Rolling Board pass rates (Parts I & II) – Program Self-Study (new) Site Visit (every 10 years)

Clinical Competency Committee Semi

Annual Review of Data to assign Milestones

•

CCC - Faculty time and input necessary for these individual resident evaluations which are the Milestones

•

There may also be a PEC committee. – program evaluation

Milestones

5 level assessments of resident knowledge, skills, attitudes, and other attributes of performance in the six competencies in a developmental framework from less to more advanced. They are designed to demonstrate program outcomes by assessing resident progress through the competencies measured in the milestone framework !

Milestones : Medical Knowledge &

• ACL • Ankle Arthritis • Ankle Fracture • Carpal Tunnel • Degenerative Spine • Diabetic Foot • Diaphyseal Femur & Tibia Fracture • Distal Radius Fracture • Adult Elbow Fracture • Hip & Knee Osteoarthritis

Patient Care

• Hip Fracture • Metastatic Bone Lesion • Meniscal Tear • Pediatric Septic Hip • Rotator Cuff Injury • Pediatric Supracondylar Humerus Fracture Small slices of clinical care – a biopsy of resident performance!

Milestones: Medical knowledge (example)

Operationalizing Milestone reporting?

• • • The faculty, PD and PC time and effort to accomplish this remain uncertain Therefore the tradeoff for absence of SV’s and PIF’s remains uncertain In my opinion they are good assessments which will make a more uniform national standard to assess resident competence

There are other non NAS requirements that will have financial implications for your department

•

6 months of PGY 1 ortho

•

Mandated surgical skills training through simulation

PG-1 Year Changes 2013-2014

• ABOS certification rules developed from results of a CORD survey • ACGME/RRC accreditation rules developed from ABOS Good news – they are the same!

 6 months of orthopaedic surgery  Basic surgical skills training

PG-1 Year Changes 2013-2014 More time on orthopaedics!

 So orthopaedic PGY 1’s will be on ortho for 6 months instead of 3 months   In our program we have 6 PGY 1’s so effectively this is a junior level 1.5 FTE 6 months of orthopaedics  How much of a cost advantage for a department is this?

Basic surgical skills requirements (core)

 A curriculum with goals and objectives  Assessment metrics  A dedicated space for the skills training

This is what is required!

What will that cost?

 Training in basic skills required of residents for emergency care and to prepare residents for future participation in surgical procedures

Results of a 2011 National Orthopaedic Program Director and Resident Survey –

Karam and Marsh JBJS 2012

 Only 50% of residency programs have a skills lab and program.

 There is high interest among PD’s in a skills curriculum.

 Most PD’s have little knowledge of the budget for skills training or the cost of a skills lab  Cost is a challenge to expansion of skills programs

Interest in a curriculum?

Barrier to skills program

Lack of funding *Percentages may not total 100% because respondents were allowed to choose more than one answer.

Members of the ABOS (AOA/CORD and AAOS ) Surgical Skills Task Force

• • • • • • • • •

J. Lawrence Marsh, MD – Chair (ABOS) James E. Carpenter, MD (ABOS) Shepard R. Hurwitz, MD (ABOS) Michelle A. James, MD (ABOS) Joel T. Jeffries, MD (AOA/CORD) David F. Martin, MD (ABOS) Peter M. Murray, MD (ABOS) Bradford O. Parsons, MD (AAOS) Robert A. Pedowitz, MD, Ph.D. Co Chair (AAOS)

• • •

Brian C. Toolan, MD (AAOS) Ann E. Van Heest, MD (AOA/CORD) M. Daniel Wongworawat, MD (AAOS)

1.

Modules

(ABOS skills taskforce modules) Sterile technique and operating room set up 11. Basic techniques in ORIF 2.

Knot tying & suturing 12. Principles and techniques of fracture reduction 3.

Microsurgical suturing 13. External fixation 4.

Soft tissue handling techniques 14. Basic Arthroscopy skills 5.

Casting and splinting 15. Basics of Arthoplasty 6.

Traction 16. Joint injection 7.

Compartment syndrome 17. Patient Safety 8.

Bone handling techniques 9.

Fluoroscopy 10. K-wire techniques

Modules should include:

 Low cost low tech options

Modules should include:

Evaluation and assessment strategies

• Guided practice until performance within time standards • Video of performance with blinded review by expert faculty with “ pass ” or “ needs more practice ” • OR performance ONLY after verification

January 2013 All 6 PGY 1’s Some call on weekend no other clinical work

Summary and Conclusions

 Resident satisfaction was high.  A dedicated month of surgical simulation has potential to change the paradigm of skills training for junior residents.  Considerable time invested in the planning and execution but faculty members were eager to contribute.  The greatest expense was for cadaveric specimens. With better planning more cost effective simulations, this expense could be reduced.

Overal Summary and Conclusions

 NAS will take more time/effort and more yearly cost  Less demanding in the year of a site visit  The balance is hard to know  Other changes to PR at PGY 1 will affect finances  Less availability for all night call  More ortho time for PGY 1’s (3 vs 6 months)  Surgical simulation will cost money  More or less depending….

 This investment may be worthwhile