Transcript Program Review

Program Review
Steve Wilson
Director of Academic Assessment
UTB Definition
A program review provides the program an
opportunity to review, evaluate, and plan. It
forms the basis for evaluating the alignment of
the program with institutional and college
missions, goals and strategic directions..
Program reviews can assist in determining the
adequacy of program resources and identifying
requirements for improvements in program
quality and effectiveness
Program reviews are done every 5 years, based
on a schedule prescribed by The THECB
This year BS Biology, BS Math, BS Chemistry, BS
Physics, BAT Health Services Tech, BSN Nursing,
and BA Government are required to conduct
program reviews
• Chairs should appoint an Assessment Liaison
(preferably the same Liaison selected for the
Program SLO Analysis). Given the large
amount of work required to complete a
program review, it is recommended that the
Assessment Liaison also chair a departmental
level assessment committee so that the work
is manageable.
Committees Related to Program
Review
• School/College Assessment Committee: Each
department Assessment Liaison will be part of a
college level assessment committee. The Associate
Dean will chair the committee. The committee will
meet once a month to discuss assessment related
activities, including program reviews, to ensure
sufficient progress is being made to meet the
appropriate deadlines, and to report any issues or
problems to the relevant officers, like the Dean of the
college, Director of Academic Assessment or the
Executive Director of the Office of Institutional
Research, Planning and Effectiveness (OIRPE).
The Report
The Review is divided into several sections
1. The Review
2. Introduction
3. THECB Measures
4. The Curriculum
5. Faculty and Support Staff
6. Facilities and Budget
7. Quality and Effectiveness
8. Summary and Recommendation
The Review
The Review Fields
 The Programs Mission Statement
 Report all the programs options or tracks
 Include an organizational chart of the
department, school or college
Introduction
Description
The Introduction should provide a strong
description of how the program mission is
aligned to the goals of the school/college and/or
the institution.
Do not use minimum or vague reflective
language when describing the alignment.
Fields
 Provide a description of the programs history
or background to establish the review context
 Describe how the mission of the program
supports the mission of the school or college
and the mission of the institution
THECB Measures
Description
• The Texas Higher Education Coordinating Board keeps track
of data that measure institutional performance, like enrollment
counts, fall course completion rates, number of graduates,
graduation rates, fall-to-fall persistence rates, and average course
size by department. This section should thoroughly describe the
trends in the data over the past five years, the program’s barriers
and/or limitations that contributed to the performance trends
(inclines or declines) and plans to improve THECB performance
outcomes. This section should also outline the program’s
THECB performance targets for the next five years was well as
plans to meet those targets.
Do not use minimum or vague reflective analysis
of THECB measures or statements that do not
adequately describe plans for improvement
and/or targets for success.
Fields
• Reflect on the data: Describe strengths and
accomplishments of the program in meeting THECB
measures revealed by the data. Also, identify and describe
any contributing program limitations or weaknesses where
THECB performance outcomes did not meet proper
expectations or showed decline.
• Compare THECB measures to peer institutions outcomes.
• Plan: Describe how the program of study intends to
improve program effectiveness to meet those THECB
performance targets over the next five years.
• Benchmarks: Describe the performance targets (or
benchmarks) the program will strive to obtain for each
THECB measure for the next five years.
The Curriculum
Description
• Program syllabi and degree requirements should
be well defined, share a consistent design, and
published. The program should demonstrate
opportunities for student engagement in
scholarly and service activities. Strategies for
helping students achieve on-time graduation
should be clearly articulated; including
expectations for faculty advising roles. The
curriculum should be updated and responsive to
student regional needs, trends in the discipline,
and/or in response to the knowledge and skill
sets required by graduates.
• Do not use minimum or vague reflective
analysis of the curriculum or statements that
do not adequately describe plans for
improvement. Support your analysis with data
and examples if possible.
Fields
• Provide a copy of one syllabus per course
offered for the last two years
• Provide links or electronic file for student
handbooks, websites, etc., that publicize
program options and requirements
• Describe any changes or improvements made
to program curriculum in the last five years
and the rationale for these changes
• Reflect: Evaluate how well semester-bysemester plan are helping achieve on-time
graduation
• Plan: Identify what improvements are planned
for the program over the next five years
(innovation, use of technology, online courses,
expansion to offsite campus, etc.).
Faculty and Support Staff
Description
• Thoroughly describe how faculty course evaluations
are used for systematic, formal review of instructional
effectiveness of all faculty and how they are integrated
into professional plans and course level improvements.
This section should also clearly describe how all faculty
in the program have required credential of SACS COC,
the distribution of faculty workloads is equitable and
sufficient for ensuring program quality, the number,
and the number of support staff is adequate, sufficient
full-time faculty are available to support the program’s
effectiveness and meeting the needs of program
students.
Do not use minimum or vague reflective analysis
of the faculty course evaluations use, faculty
credentials, distribution of faculty workloads,
the number of faculty and support staff.
Support your analysis with data and examples if
possible.
Fields
 Identify the program coordinator
 Describe what qualifies this person to be the
program coordinator
 Identify the roles and responsibilities of the
program coordinator
 Describe how has the program coordinator has
contributed to the success of the program
• List the courseloads of every faculty member.
Courseloads are available through the Office of
Institutional Research, Planning, and
Effectiveness
• Provide a copy of the most current CV for all
faculty listed in the courseloads.
• Summarize the extent to which the program has
historically used part-time faculty and how this
may or may not contribute to the quality of the
program
• Justify the extent to which the program has sufficient
full-time faculty to accomplish its mission
• Provide a list of department support staff along with
his or her position, title, and brief description of duties.
• Summarize how the program staff contribute to the
quality and effectiveness of the program
• Describe the distribution of faculty workload including
any additional responsibilities such as support services,
research, special assignments, etc.
• Identify how the distribution of faculty
workload is equitable and how it contributes
to the quality of the program
• Give specific examples of how faculty have
used course evaluation for improvement at
the individual course level for the program as
a whole.
Facilities and Budget
Description
Thoroughly describe how the financial, space, and
other faculty resources are appropriate to meeting
the needs of the program, how the library
resources are current, diverse, and sufficient for
program students and how the program plans to
maintain library resources over the next five year.
Also describe how financial support and
scholarships that are available to students is
sufficient. Demonstrate evidence of positive gains
over time to increase financial support for students.
Do not use minimum or vague reflective analysis
of the budget, library resources, and financial
support for students. Support your analysis with
data and examples if possible.
Fields
• Summarize the program’s financial budget.
The budget and other financial resources
(documents) are available from the
Department Chairs. It is recommended that
questions pertaining to the sufficiency of the
budget and other financial resources be
completed in collaboration with the chairs –
as this needs to be considered and reflected
upon at the program (not department) level.
• Describe the extent to which financial resources are
sufficient to support the program
• Describe available work space for all program
employees (for example, space dedicated to the
program, like office space and labs).
• Explain the extent to which dedicated space for the
program is or is not sufficient for maximum
effectiveness and efficiency.
• Describe classroom, labs, workrooms, conference
rooms, and other institutional facilities assigned to the
program and whether the space is or is not sufficient
for maximum effectiveness and efficiency.
• Describe whether current library resources are or
are not adequate to meet program needs and the
efforts to keep them current. Information about
the library resources is available from the library.
It is recommended that questions pertaining to
library recourses be completed in collaboration
with the library.
• Describe efforts to keep dedicated program
library resources current over the past five years.
• Describe plans for the next five years to maintain
current library resources.
• Summarize the distribution of scholarships
and financial support given to students in the
program. Data on scholarship and financial aid
available through the Office of Institutional
Research, Planning, and Effectiveness.
• Describe the efforts the program has made to
increase financial support to students and the
impact these efforts had on program
completion rates.
• Identify any opportunities or challenges the
program faces with regard to financial,
physical, staffing, library, or other resources
needed to effectively operate the program.
• Reflect on how the program facilities and
budget does or does not support operational
efficiency or other learning goals.
Program Quality and Effectiveness
Description
Quality:
Thoroughly provide evidence and
reflective analysis of the success
of current students and
graduates, as well as rates of
satisfaction with collegiate
experiences. Identify implications
the data have on program
excellence and equality
Effectiveness
Provide evidence of systematic
and intentional assessment of
student learning, including
curriculum maps that
demonstrate developmental
sequences of competency
building across the curriculum,
clear and sound methodologies
for assessing student learning
outcomes, proficiency
benchmarks for success, results,
and improvement plans, and
closing the loop.
Do not use minimum or vague reflective analysis
of program quality and assessment. Support
your analysis with data and examples if possible.
Fields
• Summarize student characteristics from
enrollment data provided by the Office of
Institutional Research, Planning, and
Effectiveness
• Describe admission requirements of the program
of study, if applicable
• Describe trends over the last 5 years of
prospective students and the actual number
admitted to the program from data provided by
the Office of Institutional Research, Planning, and
Effectiveness
• Summarize student honors, awards, and
scholarships over the past 3-5 years, including
presentations, publications, exhibits,
performances, etc.
• Describe other student related activities and
achievements specific to program objectives
(service learning, educational experiences, etc.).
• Reflect on the Data, identify implications the data
and student characteristics described above have
on program excellence and equality.
• Summarize the program SLO assessment reports
from the past five years
• Describe whether and to what degree SLOs were
achieved and the criteria used to evaluate and
score the data. If any SLO data are missing,
provide justification where appropriate.
• Identify how the program utilized results specific
to planning and implementing improvement to
achieve objectives
• Provide curriculum maps and grading rubrics
Summary and Recommendations
Description
Outline priorities and a comprehensive plan for
enhancing the quality and effectiveness of the
program. Articulate the programs limitations
and concerns. Establish well-defined
expectations and priorities for the next five
years and link those expectations to findings and
lessons learned from the program review.
AY 2014-2015 Exception
Because the Merger will take place in 8/2015,
developing an action plan is not necessary.
However, to wrap UTB assessment up into a nice
package, we want departments to write a “lessons
learned” section, based on reflections of the 20142015 program review findings.
The fields will be available on SharePoint Site soon.
What happens when the report is
finished
The Internal Program Review Committee: Once
completed, the program review reports shall be
forwarded to a Review Committee of faculty
members external to the college or school. The
Review Committee shall review the reports and
then meet with individual department Chairs
and Assessment Liaisons to report their finding
and any recommendations they have to improve
the report.
Reponses to the Review Committee: After
meeting with the Internal Review Committee,
departments will write their responses, which
will address the review committees’
recommendations by either changing the report
appropriately, or writing a rebuttal.
Administrative Review: The Associate Vice
President for Academic Affairs and/or the Dean
of Graduate Studies, the college/school dean,
and the chair of the Review Committee shall
meet together with the Department Chair and
Liaison to review the program review and
finalize the report.
Done!!
See Calendar Handout