Transcript PSYCHOMOTOR SKILLS IN RESIDENCY TRAINING

Jose Y. Cueto Jr., MD, MHPEd
Chairman, 2012
Professional Regulatory Board of Medicine
Professional Regulation Commission
Objectives for the session:
1. Discuss the educational principles behind the acquisition
and development of psychomotor skills
2. Examine required number of operations for OB-GYN
residents
3. Discuss research studies relevant to training and how to
utilize research data
4. Formulate a system of determining and validating
requirements in number of operations in residency
training (both for graduation and for certification)
5. Formulate a system of documentation and reporting
Objectives for the session:
6. Determine how the number of operations performed
during residency will affect credentialing and
privileging in hospitals
7. Discuss performance-based evaluation, specifically
the Objective Structured Clinical Exam (OSCE)
Main Objective of Training
NOVICE
COMPETENT
SURGEON
EXPERT
I. Educational Principles
 Fitt and Posner (1967): 3 phases of skills acquisition
 Miller’s Pyramid
Fitt and Posner: 3 Phases
1. Cognitive phase
2. Associative phase
3. Autonomous or Fixation phase
Fitt and Posner: 3 Phases
PHASES
1. COGNITIVE PHASE
(the mental part)
COMPONENTS / FEATURES
•Understanding the principle behind the procedure
•Knowing the indications and contraindications
•Recognizing the risks and complications
•Identifying and analyzing the steps and their proper
sequence
•Identifying the critical parts of the procedure
• Internalizing what are observed during assists and
what are discussed during rounds and conferences
Fitt and Posner: 3 Phases
PHASES
COMPONENTS / FEATURES
•Converting the mental picture into actual
2. ASSOCIATIVE PHASE
action
(the action or activity part) •Performing the procedure according to
determined sequence
•Ensuring guidance and supervision by
Consultant
•Obtaining feedback on what was done right
and what needs to be corrected
•Allowing adequate practice to polish rough
movements
•Developing ability to concentrate on the
procedure
Fitt and Posner: 3 Phases
PHASES
3. AUTONOMOUS or
FIXATION PHASE
(the refined, smooth part)
COMPONENTS / FEATURES
•Mastering the steps and correct sequence
•Developing smooth movements with minimal
wasted moves
•Making the skill become “automatic”
•Modifying the procedure when conditions require
it
•Precaution: If uncorrected during the 2nd phase,
there is danger of incorporating erroneous habits
which will be difficult to unlearn
Miller’s Pyramid
Does
Shows how
Knows how
Knows
Progression of Skills Acquisition
FITT and POSNER: 3 PHASES
MILLER’S PYRAMID
Cognitive Phase
Knows
Knows how
Associative Phase
Shows how
Autonomous or Fixation Phase
Does
Progression of Skills Acquisition
and Development
2nd Assist (mainly exposure)
1st Assist (limited participation in procedure)
Surgeon (under direct supervision)
Surgeon (independently performing)
Tracking Progression and Scheme
of Reporting: 3 columns
Operation / Procedure
1. Total abdominal hysterectomy
2. Cesarian hysterectomy
3. Adnexal surgery
4. Cesarian section
5. Vaginal Hysterectomy
Number
of First
Assists
Number of
supervised
operations /
procedures
Number of
independently
performed
operations /
procedures
II. Handbook of PBOGS, 2006
PROCEDURE
1st
2nd
NSD
50
20
70
Episiorrhaphy
25
10
35
Outlet forceps / vacuum extraction
15
or 15
15
Dilatation and Curettage
20
20
40
3
3
Manual extraction of placenta
Partial breech extraction
Cesarian section
7
10
Abdominal hysterectomy
Adnexal surgery
3rd
or 8
20
5
5
4th
TOTAL
8
30
5
5
8
10
Vaginal hysterectomy
1
or 1
1
Cesarian or postpartum
hysterectomy
1
or 1
1
Evacuation of H. mole
1
or 1
1
My Observation
 Description of The Residency Training Programs
should include a section on Psychomotor Skills
 Include an explanation on the required number of
operations using statistics
 Clarify the progression of operative cases handled by
residents and provide the basis
 Documentation and reporting found in “Basic
Requirements for Examination”, page 42; but not
discussed in the curriculum (same requirements for
graduation and certification)
General Principles
 Progression: Trainees first learn simple tasks, before
they progress to procedures of medium difficulty, and
finally to complicated procedures
 Transfer of learning / transfer of skills: what trainees
learn from simple skills are utilized in the performance
of medium and complicated skills
 Trainees need to learn to pay attention to details,
follow proper sequence, observe meticulous technique
Application of Principles
1. The simpler the procedure, the shorter it takes to
learn, and the less number of cases needed to master it
2. The more complicated the procedure, the longer it
takes to learn, and the more number of cases needed
to master it
3. The more complicated the procedure, the higher the
need for supervision and guidance
Implementation
Residency Training
Committee
1st yr
1st yr
2nd yr
3rd yr
4th yr
Roles according to year level
Operation
1st yr
2nd yr
Requirement
for Abdominal
hysterectomy
3rd yr
5
Role 1
2nd assist
1st assist
surgeon
Role 2
2nd assist
1st assist
(12)
Surgeon,
supervised
(3)
4th yr
5
surgeon
Surgeon,
independent
(7)
Evaluating Phases of Learning
Fitt’s 3 Phases
Miller’s
Pyramid
Method of Evaluation/Setting
1. Cognitive Phase
Knows
Knows how
Discussion
Q and A
Record review
Pre-op conference
OR
Ward / office
2. Associative Phase
Shows how
Direct observation
(Supervised by
Consultant)
3. Autonomous Phase
Does
Direct observation
Record review
Outcome evaluation
Skills lab
Simulations
OR
OR
Ward
Rounds
Conference
III. Researches
1.Does Residency Training Improve Cognitive
Competence in Obstetric and Gynaecologic
Surgery?, Balayla, Abenhaim and Martin, McGill
University, Montreal, Can, J Obstet Gynaecol Can
2012;34(2):190–196
2. Competency-based Residency Training: The Next
Advance in Graduate Medical Education, Donlin
Long, MD, PhD, Johns Hopkins University School of
Medicine, Academic Medicine, Dec 2000
Researches
3. Factors Associated with a Successful Outcome in
the PBS Certifying Examinations, Crisostomo and
Marfori, Philippine Journal of Surgical Specialties,
Oct-Dec, 2010
Does Residency Training Improve
Cognitive Competence in Obstetric and
Gynaecologic Surgery? (JOGC,2012)
Objectives:
1.To develop an operative knowledge assessment tool to
evaluate the cognitive competence of trainees in
obstetric and gynaecologic surgery
2.To determine the rate of change in competence
during a five-year residency program .
J Obstet Gynaecol Can
Methods:
 Twenty-eight participants in five training groups
(PGY-1 to PGY-5) in McGill University’s residency
program in obstetrics and gynaecology
 Evaluation based on surgical cognitive competence
(SCC) assessment tools
 Three different obstetric and gynaecologic operations:
open total abdominal hysterectomy (TAH), Caesarean
section, and laparoscopic bilateral tubal sterilization
(BTL)
J Obstet Gynaecol Can
Performance of an operation/procedure
Three fundamental components:
1. cognitive factor
2. technical element
3. judgment component
J Obstet Gynaecol Can
COMPONENT
1. cognitive factor
2. technical element
3. judgement component
FEATURE
the knowledge of the theoretical steps
of the procedure
takes the theoretical steps into account
and translates them into the
performance of the operation
comes from surgical experience and
allows a surgeon to rely on his or her
own intuition to determine the
appropriate operative course of action
on a case by case basis
J Obstet Gynaecol Can
Summary of Findings:
OPERATION
1. Total Abdominal
Hysterectomy
2. Cesarian Section
3. Laparoscopic BTL
PERCENTAGE INCREASE
IN COGNITIVE
COMPETENCE PER YEAR
15.73%
8.06%
16.31%
J Obstet Gynaecol Can
Findings:
 At level of PGY-5, residents had 100% surgical
cognitive competence
 This type of information may be helpful in
ascertaining how long a residency program
should be
Competency-based Residency Training:
The Next Advance in Graduate Medical
Education (AM, 2000)
 Donlin Long, MD, PhD
 Professor of Neurosurgery, Johns Hopkins University
 Studied NSS residents
 Introduced competency-based program
Academic Medicine
Traditional Program
Competency-based Program
1. Fixed number of years
2. Residents have to learn all
specified knowledge and
skills in the allotted time
3. Problem of evaluating
competence of every
resident
4. Graduate may not be
competent to perform
required procedure or
manage particular patients.
1. Specifies maximum duration
2. Time taken to acquire
knowledge and skills is based
on the abilities of individual
trainees
3. Evaluation of every resident
in every procedure
4. Resident is evaluated and
certified to have acquired
competence and confidence
to practice independently
Factors Associated with a Successful
Outcome in the PBS Certifying
Examinations (PJSS, 2010)
Objective: To determine the factors associated with a
successful outcome in the PBS certifying exams
(written and oral)
Method: Retrospective, cross-sectional study utilizing
370 candidates from 2006-2009, with 137 (37.0%)
successful outcomes
PJSS
Significant Factors:
1. Younger age of examinees
2. Previous performance in the RITE
3. Taking the exam within a year of completion of
residency
4. Training in a university-based program
5. Undertaking subspecialty fellowship during the
examination year
PJSS
Other Factors (Not Significant)
1. Sex (M-F)
2. Marital Status
3. Case volume
4. Continuous/interrupted program
5. Location of training program (MM vs. outside)
PJSS
CASE VOLUME performed during residency (major
operative procedures):
1. High volume: 299 0r more cases (upper 3rd)
2. Medium volume: 171-298 cases (middle 3rd)
3. Low volume: less than 171 cases (lower 3rd)
NOTE: did NOT influence performance in written and
oral exams
Open for further research
 Relationship of case volume to:
1. surgical cognitive competence
2. technical element
3. judgment component
Utilizing Research Data (PJSS)
FAVORABLE FACTORS
(passing the PBS Certifying Exam)
Accreditation Committee
Residency Training Committee
DECISIONS / ACTIONS
1. Graduation from a university-based
training program
Closely monitor the residents and
graduates from gov’t and private
institutions
2. Satisfactory performance in the
Residency In-training Exam
Institute remedial measures for
residents with low scores in the RITE
(identify topic areas)
3. Taking the exam within one year after Encourage / require graduates to take
end of residency
the certifying exam within one year
after end of training
Utilizing Research Data (JOGC)
FACTORS IDENTIFIED
DECISIONS / ACTIONS
1. Percentage increase in surgical
cognitive competence per year; reaches
100% at level PGY5
Policy: convert all 4-year programs to 5year programs
2. Ability of the resident to identify the
most critical steps (given 24-32 total
steps in certain procedures)
Provide in-depth discussions and
adequate exposure and practice prior to
allowing resident to perform actual
operation
3. The cognitive factor and judgment
component are more important than
the technical factor in the performance
of procedures
Evaluate the development of cognitive
factor and judgment component of
operative skills
Utilizing Research Data (AM)
FACTOR IDENTIFIED
Abilities of residents and pace of
acquiring knowledge and learning
operative skills differ.
DECISION / ACTION
Provide flexibility in the duration of a
training program but place a limit or
maximum duration.
Evaluate competencies in each
procedure before certifying for
promotion to higher level or for
graduation.
When in doubt, provide extension for
additional operative cases
The essential messages
1. The process of acquiring and developing operative
skills is more important than the output (number of
operations performed).
2. Quality is more important than quantity.
3. Supervision and feedback are critical. There may be
institutions where residents get to perform so many
operations by themselves, but they never get to know
which steps are done correctly or incorrectly.
The Essential Messages
4. The cognitive factor and the judgment component
are very critical in the performance of procedures
5. The fixed duration of residency training may not be
appropriate for a number of residents
Determining Requirements
Factors to consider:
1. Degree of difficulty: simplicity / complexity of
procedure /low-risk / high-risk
*The simpler the procedure, the lower the number
required to acquire competence
*The more complex / complicated the procedure, the
higher the number required to acquire competence
Determining Requirements
Factors to consider:
2. Trainee factor: fast / slow learner
dexterity with procedures
*The fast learner and the trainee with dexterity /
adeptness at performing procedures will require lower
number of cases
*The slow learner and the trainee with
“clumsiness” in performing procedures will need a
higher number of cases
Determining Requirements
Factors to consider:
3. Institutional factor: high volume vs. low volume
private vs. charity/service patients
*The resident belonging to a high-volume hospital
will require a lower number of cases (reinforcement)
*The resident belonging to a low-volume hospital
will require a higher number of cases (too few and far
in-between)
Determining required number of
operations
1. Use of the Delphi technique
 12-30 Experts
 List of operations
 3 rounds
 Questions to answer
Questions
Based on your experience and expert opinion:
 How many times should a resident assist in the
following procedure before he can be given his first
case?
 How many times should a resident perform this
procedure under direct supervision before he is
allowed to perform it independently?
 How many times should a resident perform this
procedure before he can acquire the competence and
the confidence to perform it safely on his own?
Three Rounds
Round 1: 12-30 experts give their proposed number of
operations, based on the questions; without them
communicating with each other
Round 2: the experts are given feedback on how their
colleagues answered the questions (tabulation of
results); afterwards they are asked for their modified
list of proposed number of operations
Round 3: the experts are gathered and they are asked to
arrive at a consensus regarding required number of
operations
Validation Stage
1. Identify institutions:
 Government
 University-based
 Private
2. Conduct a parallel research study: longitudinal
tracking of residents until they get to the certifying
exams
3. Based on results, modify/maintain the requirements
Detailed Documentation of
Operative Experience (35 cases)
List of Operations
Number
of
Assists
Number of Number of
supervised independently
operations performed
operations
Total abdominal hysterectomy
15
Cesarian/postpartum hysterectomy
1
Vaginal hysterectomy
1
Adnexal procedures
10
Vaginal extraction of H. mole
1
Indicated manual extraction of
placenta
1
Breech deliveries
1
Outlet forceps or vacuum
5
My Observation
 Notation: Starting 2006, only 70% of cases with
complete transfer of technical responsibility is allowed
 What is the basis for the policy?
 What problem does it solve? Lack of residents’ cases?
 Consultants’ cases given to residents: fall under 2nd
column (supervised cases)
 Private cases: no complete transfer of responsibility
demo cases to show residents how
procedures should be done
Residents’ Responsibilities
1. provide preoperative evaluation, assessment of risks
2.identify indications/contraindications to planned
procedure, possible complications
3. perform the procedure, modifying it in presence of
unforeseen conditions
4. providing immediate and long-term care.
Note: For private patients, decisions will always be made
by the Attending Consultant (The judgment
component is lacking)
Deficiencies and Actions
What happens if residents do not meet requirements?
 Extension of rotation in a particular service where
deficiencies occurred.
 Reduce the number of residents or admit residents
every other year.
 Terminate program after adequate opportunities for
correcting deficiencies have been given
Implications for Future
Credentialing and Privileging
Operations / Number Performed / Required
Allowed
Not
Allowed
1. Dilatation and curettage
50
(40)
/
2. Cesarian section
50
(30)
/
3. Adnexal surgery
15
(10)
/
4. Abdominal hysterectomy
10
(10)
/
5. Vaginal hysterectomy
1
(1)
x
6. Cesarian hysterectomy
1
(1)
x
Question
 Will the departments accept deficiencies, and still
allow residents to graduate?
 Will the board accept deficiencies and still allow
graduates to take the certifying exams?
Recommendations
1. Shift FOCUS to process of acquiring and developing
operative skills rather than the number performed
Recommendations
1. Shift FOCUS to process of acquiring and developing
operative skills rather than the number performed
2. Aim for qualitative improvement by providing
guidance, supervision and feedback.
Recommendations
1. Shift FOCUS to process of acquiring and developing
operative skills rather than the number performed
2. Aim for qualitative improvement by providing
guidance, supervision and feedback
3. Pay attention to the cognitive factor and the judgment
component of performance.
Recommendations
1. Shift FOCUS to process of acquiring and developing
operative skills rather than the number performed
2. Aim for qualitative improvement by providing guidance,
supervision and feedbac
3. Pay attention to the cognitive factor and the judgment
component of performance
4. Conduct researches on different components or aspects of
residency training
Performance-based Evaluation
The Objective Structured Clinical Exam (OSCE)
 Multiple stations
 Time allotted: 5-25 mins
 Well-defined clinical task
 Use of real or standardized patient (SP)
 Use of raters
 Rating scales and checklists
OSCE
MILLER’S PYRAMID
LEVEL OF PERFORMANCE
MOST EFFECTIVELY MEASURED
BY OSCE
KNOWS
KNOWS HOW
SHOWS HOW
DOES
+++
+++++
OSCE
COMPETENCIES
KNOWLEDGE
PROBLEM-SOLVING
CLINICAL DECISION-MAKING
SKILLS, HISTORY-TAKING
SKILLS, PHYSICAL EXAM
SKILLS, PROCEDURES
SKILLS, INTERPERSONAL
ATTITUDES
LEVEL OF PERFORMANCE
EFFECTIVELY MEASURED BY OSCE
++
+++++
+++++
+++++
+++++
+++++
+++
+++
OSCE Development
 Planning/preparing needed resources
 Identification of competencies (test blueprint)
 Identification/recruitment of raters
 Training of standardized patients and raters
 Conducting workshops on standard-setting
 Constructing rating scales/checklists
 Gathering diagnostic materials
 Selecting venue
 Pilot-testing
Summary
 Educational basis for the acquisition of skills
 Re-examined PBOGS requirements
 Research studies on training
 System of determining requirements and validation
 System of documentation and reporting
 Discussion on policies
 Future implications
 OSCE
END
Workshop Activity
1.Determining requirements in 3 columns (Delphi
method)
Operative
Procedure
Round 1
Round 2
Round 3
Number
of
1st Assists
Number of
Supervised
operations
Number of
independently
performed
operations
Workshop Output
1.Determining requirements in 3 columns (Delphi
method)
Caesarian
section
Number
of
1st Assists
Number of
Supervised
operations
Number of
independently
performed
operations
Group 1
8
7
15
Group 2
5
5
5
Group 3
10
3
12
Workshop Output
1.Determining requirements in 3 columns (Delphi
method)
TAHBSO
Number
of
1st Assists
Number of
Supervised
operations
Number of
independently
performed
operations
Group 1
12
8
18
Group 2
10
5
5
Group 3
10
5
8
Workshop Output
1.Determining requirements in 3 columns (Delphi
method)
TAHBSO
Group 1
Group 2
Group 3
Number
of
1st Assists
Number of
Supervised
operations
Number of
independently
performed
operations
Workshop Activity
2. Discussion: graduates who cannot pass the certifying
exams: reasons / courses of action
PBOGS Passing: 65-67% (2001-2006)
Philhealth: CS done by Diplomates/Fellows (60%)
CS done by GP’s with training (30%)
CS done by MD’s w/o training (7.4%)
Physician Act of 2012: Art. V. Sec.28 (k): Performing…
an area of specialization without fulfilling specialization
requirements prescribed by the AIPO and the Board of
Medicine