Transcript Slide 1

Development of Logic Model &
Performance Measures
AHEC GRANTEE PRESENTATION
APRIL 14, 2011
HRSA/ Division of Workforce &
Performance Management
Steps in BHPr Performance Process
• Develop Program Logic Models
• Develop Cluster Measures and Tables
• Program and Grantee Reviews of
Measures and Tables
• Develop Draft Grantee Guidance
• Review and Revision to Guidance
• Obtain Input on Guidance from Grantees
• Finalize Guidance
• Prepare Final OMB Package
• Obtain Approval from Advisory Groups
(Feb. 15 – March 31)
(March 1 – April 29)
(April 14 -- May 16)
(May 1 – 15)
(May 16 – 30)
(May 1– 30)
(May 30)
(June 1 – July 15)
(November 1 – 30)
OMB Categories
• Quantity- supply
• Quality- competency, knowledge
gained
• Diversity – racial/ethnic diversity
• Distribution – placement in
underserved areas
Program Description
and Outcomes
• Program level
• Activity Level
• Individual level
Elements of the proposed individual-level data
collection
• PII and contact information – collected only for individuals targeted
for longitudinal study and only at graduation/program completion
(MD/Residency, PA, APRN(PC NP?), Diversity program participants
in structured programs, RN?, faculty development?)
• Grant-based unique ID (organization-based unique-id; i.e., by
university?)
• SSN
• Name
• Current Address
• Expected Address in one year (if different)
• Phone numbers (home, cell)
• e-mail, facebook,
• Name and phone number for someone who will know where you
can be contacted
Elements of the proposed individual-level data
collection
• Data that rarely changes – collected on first
contact/”enrollment”
• Grant-based Unique ID
• Birthdate
• Gender
• Race (check all that apply)
• Ethnicity
• Parent’s income (broad categories)
• Address where individual grew up (prior to 21st birth
date) (“rural”)
Elements of the proposed individuallevel data collection
• Annual data – activities – one record for each defined
activity
• Grant-based Unique ID
• Activity (defined at the program/cluster level and specifics
reported at the grant level)
• Activity-related individual/immediate outcome
• Levels of activities
• Year-long training without separate activities – e.g., year 1
in medical or nursing school
• Structured, time-limited activities – e.g., clinical rotations,
CE courses, structured activities in diversity programs, etc.
Elements of the proposed individuallevel data collection
• Activity categories/data
• Clinical rotations – focus, location/type of
facility, length, contact hours, etc.
• Post-baccalaureate program
• Saturday Academy
• Formal mentoring
• Etc.
Program Name: Area Health Education Centers Program
Need(s): There is a shortage of high quality primary health care to meet growing demand in the U.S.
Goal(s): Train a greater number of competent health care providers to better meet the growing demand for primary
health care.
KEY STRATEGIES
Provide primary care
training to medical and
other health
professions students.
(c)(1)(B)
(c)(1)(C)
(c)(1)(F)
OUTPUTS
# and type of health
professions students
placed in communitybased primary care
rotations.
SHORT-TERM
OUTCOMES
Increased number of
prepared health
professions students
(those successfully
completing communitybased primary care
Total # of rotation sites rotations).
% rotation sites
located in:
-Rural areas
-Federally
designated HPSAs
and MUAs
-Facilities serving a
significant proportion
of vulnerable
patients in non
Federally designated
areas
INTERMEDIATE
OUTCOMES
Increased number
health professions
program graduates
who provide primary
care within one year
after completing
training, including in
rural/ underserved
areas.
LONG-TERM
OUTCOMES
Increased number and
diversity of health
training program
graduates who provide
primary care within five
years after completing
training, including in
rural/ underserved
areas.
KEY STRATEGIES
Provide continuing
education (CE) on
key primary care
topics (determined by
national priorities
and/or local needs
assessments) to
health professionals,
particularly those
providing primary
care in underserved
areas .
(c)(1)(E)
(c)(1)(F)
OUTPUTS
# and type of health
professionals (by
discipline) receiving
CE.
SHORT-TERM
OUTCOMES
INTERMEDIATE
OUTCOMES
LONG-TERM
OUTCOMES
Increased number of
prepared health
professionals (those
successfully earning CE
credits) .
Increased number of
health professionals
who integrate CE
course content into
their clinical practice
(within one year after
completing training).
Increased number of
clinical practice
changes as a result of
health professional’s
integration of CE
course content into
clinical practice
(within a five-year
period).
# of CE courses
offered by topic area. Increased knowledge of
health professionals on
% of CE offered inkey CE topics (as
person and via the
determined from preWeb.
post knowledge tests).
Employment location
of health
professionals
participating in CE
activities.
# of partners/
collaborators for CE
program offerings
Findings from
pre/post knowledge
tests.
KEY STRATEGIES
OUTPUTS
Provide health
careers outreach to
youth in grades K-12,
including careers in
public health.
- # and type of
activities conducted
to raise awareness
of health careers
among K-12 youth,
including public
health.
- # and type of
partner organizations
sponsoring health
career awareness
activities
- # of K-12 youth
reached by health
career awareness
activities
- % of K-12 youth
reached who are
URM,
disadvantaged, and
live in rural areas
- # and type of
activities conducted
to raise awareness
of health careers
among displaced
workers or adult
learners
- # of partnerships
with DOL/WIB
entities
Provide health
careers outreach to
displaced workers or
individuals/ adult
learners from
underrepresented
minority populations
or from
disadvantaged or
rural backgrounds.
(c)(1)(A)
(c)(1)(G)
(c)(1)(F)
SHORT-TERM
OUTCOMES
Increased number and
diversity of K-12 youth
that are aware of and
intend to pursue health
careers training
programs.
Increased number of
displaced workers or
adult learners receiving
health careers outreach
and training through
partnerships with
DOL/WIB entities.
INTERMEDIATE
OUTCOMES
LONG-TERM
OUTCOMES
Increased number
and diversity of K-12
youth that are
prepared and
qualified to enter
health professions
training programs,
including training
programs in public
health.
Increased number
and diversity of K-12
youth applying to and
being accepted at
health professions
training programs.
Increased number of
displaced workers or
adult learners who
obtain entry level
jobs in the health
professions.
Increased number of
displaced workers or
adult learners who
establish careers in
the health
professions.
KEY STRATEGIES
Provide
interdisciplinary/
interprofessional
education and training
opportunities to
medical and other
health professional
students, and
practicing health
professionals.
Develop partnerships
with community-based,
academic, and
healthcare workforce
entities that promote
interdisciplinary
approaches to primary
care.
(c)(1)(D)
(c)(1)(F)
OUTPUTS
SHORT-TERM
OUTCOMES
Total # of rotation or
training sites offering
interdisciplinary/
interprofessional
education.
Increased number of
rotation or training sites
offering interdisciplinary/
interprofessional
education and/or training.
# of disciplines
participating in
interdisciplinary
/interprofessional
education or training.
Increased number of
interdisciplinary/
interprofessional CE
offerings.
% of rotation sites
offering IPE located in
rural areas.
# of interdisciplinary/
interprofessional CE
offerings.
INTERMEDIATE
OUTCOMES
Increased number of
health professions
program graduates
that are trained in
interdisciplinary/
interprofessional
education.
Increased number of
practicing health
professionals that
Increased number of
receive
community-based,
interdisciplinary
academic, and healthcare /interprofessional CE.
workforce partnerships
promoting
interdisciplinary
approaches to primary
care.
LONG-TERM
OUTCOMES
Increased number of
health professionals
integrating
interdisciplinary/
interprofessional
education into clinical
practice.
Increased number of
interdisciplinary teams
delivering primary care,
including in
rural/underserved
areas.
Breakout Groups
• Best means to describe program activities and
outcomes
What is missing?
• Categorize major elements
• Brief summary report-out
Get to Work!
Proposed Common Quality Measures
– The number of program participants demonstrating PC
competencies
– Proportion of BHPr supported trainees who receive
training in medically underserved communities.
– Proportion of participants who receive a portion of their
clinical training in primary care.
– Proportion of participants receiving training in PC focus
areas
– Proportion of participants with increased knowledge gain
at the end of CE as reflected in pre-post testing scores
– Proportion of participants receiving multiple modes of PC
activities
– Overall retention of participants in programs
Proposed Common Quantity Measures
– The number and percent of participants in career development /
career enhancement/career advancement programs
– The number and percent of program participants completing training
who indicate their intent to practice as a HP
– The number and percent of program participants completing training
who indicate their intent to practice in primary care
– The number and percent of program participants completing training
who indicate their intent to practice in underserved areas.
– The number of CE offerings per topic/mode of training
– The number of new trainees/slots/units
Proposed Common Diversity Measures
– The number/type/proportion of graduates/completers
who are URM and/or disadvantaged.
– The number/type/percent of URM and disadvantaged
participants/faculty
– # and % of URM accepted into HP training program
– Increased retention rate of URM in BHPR programs
– Increased retention rate of URM/faculty in HP school
– The number and percent of URM and disadvantaged
participants who indicate their intent to work in primary
care and/or underserved areas.
– The number and percent of URM and disadvantaged
participants receiving training in primary care and/or
underserved areas.
Other Proposed Common
Measures/Reporting
• Distribution
– Proportion of BHPR supported HP who enter practice in underserved areas
• Infrastructure
– # participants completing faculty development training
– # PC AAU
• Progress Report (describe accomplishments)
– Describe evaluation activities
– Explain how partnerships/leveraging activities have influenced how you
conduct training activities (e.g. curriculum, enrollment, placements, etc.)?
– Educational innovations (e.g. Innovative curricula)
– Best practices
– Dissemination of knowledge/strategies