Transcript Slide 1

Program Planning Committees: Tools for Building Capacity
4th National Accreditation Conference
Toronto, Ontario
September 21, 2012
Elizabeth A. Lindsay MS, PhD
Disclosure
I have no actual or potential conflict of interest in relation
to this workshop.
Learning Objectives
After active engagement in this workshop, participants will
be better able to:
• Identify recommended competencies for planning
committee members
• Discuss planning committee roles and responsibilities
with a focus on building capacity and meeting standards
• Identify and develop practice planning committee
solutions for use in faculty development
Agenda
A.
•
•
•
Background for the workshop – 20 min
Why are planning committees important?
Is there currently a problem with planning processes?
What do committee members need to be able to do – what is the scope of their
roles?
• Overview of the “Gatekeeper Study” results
B. Small group discussion – 20 min
Objectives - to discuss specific competencies and tools directed at selected aspects
of the planning process for large or small group learning
C. Full group discussion - 35 min
• Presentation of examples of tools to support planning processes
• Examples of tools recommended in small group discussion
D. Evaluation/Feedback – 5 min.
What does a high functioning program planning
committee look like?
Is our present system working well?
What do they presently do?
Usual Practice for Planning One-day Programs
• 2-3 face-to- face meetings before event
• E-mail communication to finalize faculty, titles &
objectives
• Occasionally, 1 debriefing meeting
Impact of Group Learning on
Clinical Practices
•
Multiple reviews show us the challenges of demonstrating impact of
educational programs on competency and performance and there are
pressures to improve the impact of our programs as well as to expand the
type of activities we offer.
•
However, group learning continues to be the predominant CME/CPD
activity for most physicians, as reported by the MainCert and MainPro
programs in Canada
•
Recent study of family physicians in the Ottawa region demonstrated that
they are quite satisfied with the CME/CPD options available to them and
believe that group learning does impact their clinical practices….little
pressure to change.
•
Expectations for the planning processes for accredited group learning and
on-line learning are clearly defined – how well are we doing them? Those
who evaluate applications for accreditation can advise us.
Seems appropriate that we try to
improve impact
How?
 Identify ingredients that can increase impact
 Transfer this knowledge and skill to those who build
or lead the programs
Step 5 Execute sessions
according to plan and evaluate
sessions, report to planning
committee
Step 4 Invite faculty and share
learning objectives,
recommendations for learning
formats
Step 3 Outline content
area to reflect learning
objectives and choose
format/methods/faculty
Step 2 Identify subject area for
content and write learning
objectives
Steps for
Building
an
Accreditable
Program
Step 6 Feedback and
other learning from past
meetings Synthesize
results
Step 1 Clarify target
audience and carry out
needs assessment process
What do we know about impact of group learning?
Group learning ( lectures, workshops, small groups, rounds)
Effect size average - 6-10% *
Key factors that determine impact - amount and type of interaction;
– less complex behaviours and more serious outcomes*
Previously noted factors that enhance impact:
 needs based;
 relevance to practice
 reflection before, during and after meetings
 commitment to change at educational events
 combined with other interventions
 sequential
 repetition
*Forsetlund et al 2009
What do planning committees need to be able to do? From the
AAMC/SACME Harrison survey*….
Toward a more effective product: Enhancing didactic activities
•
Practice enablers (patient material, flow-sheets, algorithms etc at or after course)
•
Follow-up method post-course to reinforce learning
Examples used in the survey
•
Pre-course planning ( needs assessment, objectives, choose methods, evaluate)
•
In-course enhancement
– Meaningful interactivity
– Simulations
– Tools for patient management
•
•
*
Follow-up methods post-course to reinforce learning
Sequential learning
Association of American Medical Colleges. Academic CME in the US and Canada: The 2010 AAMC/SACME Harrison survey p. 13
Basic Assumptions for the Gatekeeper Study
Investigators:
Elizabeth Lindsay, Paul Hendry, Michael Allen, Heather Armson , Jatinder Takhar
• Health care providers who serve on planning
committees and as presenters are the “gatekeepers” to
improving effectiveness of group learning programs
• There is a relationship between program design,
implementation processes and potential impact of group
learning programs on participants
We Applied Behavior Change Frameworks* to
Guide our Study Questions
Are physicians who serve on planning committees
 Aware of best practices in program design?
 Willing to apply new planning processes?
Do physicians who serve on planning committees
believe that selected planning processes are:
 Important?
 Feasible?
 Will make affect impact of the program?
*PRECEDE (Green) and Theory of Planned Behavior (Azjen)
Study Participants
• 52 physicians who had participated in planning committees at
the University of Ottawa over an 2.5 yr period.
Respondents
• Family physicians - 17
• Other specialists - 15
Total 32/62%
Competency areas selected for study**
•
Assessed information that describes the learning needs of potential participants in the program eg. gaps
between current and recommended clinical practices
•
Created learning objectives for the program that state what the learner will be able to do as a result of
participating in the specific sessions.
•
Recommended types of learning formats that work best for different types of objectives eg for knowledge based
or skills based outcomes
•
Discuss objectives and recommended learning formats with faculty/teachers to ensure mutual understanding
•
Encourage faculty/teachers to present the level of evidence supporting educational messages (eg. Metaanalysis, RCT, expert opinion)
•
Identify strategies that will facilitate practice integration of recommended best practices or address barriers to
implementation
•
Audit programs to assess existence of bias or compliance with accreditation standards
**Selected from Alliance for CME list of competencies for CME/CPD professionals
Study
participants
Frequency
Willingness
Importance
Feasibility
Impact
Scale 1-5
Scale 1-5
Scale 1-5
Scale 1-5
Scale 1-5
Mean Responses
Mean Responses
Mean Responses
Mean Responses
Mean Responses
4 or 5
4 or 5
4 or 5
4 or 5
4 or 5
Assess
needs
4.0
78%
4.2
81%
4.9
100%
4.1
77%
4.6
90%
Create
learning
objectives
4.2
78%
4.4
81%
4.6
100%
4.4
90%
4.2
87%
Recommend
learning
formats
4.2
81%
4.3
81%
4.8
99%
4.4
90%
4.5
93%
Discuss obj
&form with
presenters
3.4
4.1
78%
4.6
97%
4.1
4.3
87%
Encourage
careful data
presentation
2.9
31%
4.1
75%
4.5
87%
4.1
70%
3.8
58%
Identify
integration
strategies
3.3
41%
4.1
78%
4.5
3.9
69%
4.1
76%
Audit for
bias and
standards
2.0
16%
3.5
53%
4.3
3.6
54%
3.4
43%
56%
90%
90%
80%
Conclusions
•
•
•
•
High levels of willingness to improve
Strong belief in the importance of all factors but
Not as strong regarding belief of potential impact
Lots of variability regarding feasibility
Purpose and process of small group
discussion
 Focus on one element of the planning process
 Considering our expectations regarding planning
committees – what can we do to support them?
 Describe and create a list of specific processes and tools
you use or think we should use to support planning
committees
Topics for small group discussion
Composition of the Planning Committee – The Chair and Preparation
Composition of the Planning Committee – The Committee Members
Competencies and Pre-work
Structure and Process of Planning Meetings – During meetings
Structure and Process of Planning Meetings – Between Meetings
Choose effective presenters and helping them prepare
Choose educational strategies that go beyond didactic presentations with
Q&A
Full Group Discussion
With the package of resources as a basis --- add the
ideas raised by each of the small groups
Topics for small group discussion
Composition of the Planning Committee – The Chair and Preparation
Qualities of a high functioning chair
Preparation Ideas – evidence for importance of improving impact
Composition of the Planning Committee – The Committee Members
Competencies and Pre-work
Content Experts
Audience Representatives
Education Designer
Event Planner
Topics for small group discussion
Structure and Process of Planning Meetings – During meetings
Agenda should reflect time for different steps in the process in order to set
up expectations for the meeting process eg. Time for looking at needs
assessment information
Use a planning template that can be completed during discussion at the
meeting, columns for Gap and Objective, Title, Presenters – event
planner can do some of this with help from the chair or another
committee member
Structure and Process of Planning Meetings – Between Meetings
The event planner, education designer and chair can work on planning
template between meetings to ensure the process moves forward in a
timely manner
Individual members of the committee may have tasks that they have
agreed to complete eg. Check on availability of presenter or find a
particular resource – ideally meeting planner should follow-up to gather
this information and apply it to the planning template.
Topics for small group discussion
Choose effective presenters and helping them prepare
Presenters strengths – ideas for how to find them
–
–
–
–
–
Knowledgeable and creditable regarding content
Demonstrates good understanding of the clinical setting in which learner
works and the challenges faced there
Clarity regarding content and presentation format
Sufficient interaction to ensure understanding eg. Able to answer questions
appropriately
Create template letters of invitation to potential presenters to ensure
draft objectives and title are included and invite feedback
Choose educational strategies that go beyond didactic presentations with
Q&A
Match objective with learning format – see Michie
What makes a workshop, a workshop?
Improving interaction – Tip sheets
Resources
•
•
•
•
•
•
Strauss S, Tetroe, J, Graham ID. Knowledge Translation in Health Care: Moving from evidence to
practice. 2009. Wiley-Blackwell. Oxford.
Green LW, & Kreuter MW. Health Program Planning: An educational and ecological.
4th edition 2005. McGraw Hill. Toronto.
Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes,
1991;50:179-211
Michie S, Johnson M, Francis J, Hardeman W, Eccles M. From theory to intervention: Mapping
theoretically derived behavioural determinants to behaviour change techniques. Applied Psychology:An
International Review, 2008, 57(4):660-680.
Michie S, Johnson M, Abrahm C, Lawton R. Parker D, Walker A, on behalf of the “Psychological Theory”
Group. Making psychological theory useful for implementing evidence based practice: a consensus
approach. Qual Saf Health Care 2005;14:26-33.
Moore, DE. How physicians learn and how to design learning experiences for them: An approach based
on an interpretive review of evidence. In:Hager, M, Russell S & Fletcher SW (Eds.). Continuing
education in the health professions: Proceedings of a conference. A Report for the Josiah Macy
Foundation, 2008; http://www.josiahmacyfoundation.org/index.
Contents of Handout
1.
2.
3.
4.
5.
6.
7.
Title page
-
University of Ottawa Office of Continuing Medical Education (OCME)
Handbook for Planning Committees A How‐to‐Guide for the development of an
education program that may be accredited for Family Physicians (MainPro 1) and
Specialists (MOC)
Examples of slides for discussion of KT, evidence of impact of group learning.
Excel planning template for listing of topics, gaps and objectives
Tip Sheets – Example – Interactivity
Invitation template to participate on a planning committee, including roles,
responsibilities, time commitment, timelines and expectations regarding response
to communication.
K to A framework
Michie matching of objectives and methods