Transcript PowerPoint Presentation

Program Assessment Plans
Step by Step
MUS ASSESSMENT WORKSHOPS
R. W. Larsen, 2014
This presentation and handouts are available at:
http://www.montana.edu/provost/assessment.html
Overview
1.
2.
3.
4.
5.
6.
7.
Create program learning outcomes
Identify where each outcome is included in your
curriculum
Determine where you can gather evidence of student
performance for each outcome
Set an expected performance threshold for each
outcome
Create a schedule for assessing each outcome
Describe your process for using the assessment data
Submit your assessment plan
1. Create Program Learning Outcomes
What do you want your students to know, to be able to
do, and (sometimes) to be when they graduate from
your program?
The answers to this question are the list of learning
outcomes for your degree program.
1. Create Program Learning Outcomes
Program learning outcomes are generally written in
the form of statements starting with “Our graduates
will…”
Note: Many professional organizations have
developed lists of expected competencies or student
leaning outcomes.
Typical Program Learning Outcomes
Our graduates will have the knowledge required
to be successful in their field.
2. Our graduates will be able to function in a
professional manner in their field.
3. Our graduates will be able to analyze problems in
their field and develop solutions or strategies to
solve those problems.
4. Our graduates will be able to communicate
effectively.
1.
1. Create Program Learning Outcomes
Develop a list of learning outcomes for your degree
program.
2. Assign every outcome a number or letter so that it
can be easily identified.
1.
2. Find Each Outcome in Your Curriculum
1.
2.
3.
4.
5.
List all required courses
Remove courses outside of the faculty’s control
(optional)
Create a grid showing remaining courses (rows)
and learning outcomes (columns)
Use a highlighter to indicate where in the courses
each outcome is addressed
Add a code indicating cognitive skill level to the
highlighted cells, for example:
I = introductory , D = developing, M = mastery
2. Find Each Outcome in Your Curriculum
1.
2.
3.
4.
5.
List all required courses
Remove courses outside of the faculty’s control
(optional)
You are creating a
Create a grid showing remaining courses (rows)
curriculum
map.
and learning outcomes
(columns)
Use a highlighter to indicate where in the courses
each outcome is addressed
Add a code indicating cognitive skill level to the
highlighted cells, for example:
I = introductory , D = developing, M = mastery
2.1 List All Required Courses













ARCH 151--Design Fundamentals I
BIOB 170IN--Principles of Biological
Diversity
CHMY 121IN--Intro to General
Chemistry
WRIT 101W--College Writing I
M 145--Math for Liberal Arts
BIOB 110CS--Introduction to Plant
Biology
HORT 131--Landscape Design,
Hist/Theory
HORT 105--Miracle Growing
ENSC 245IN--Soils
HORT 231--Woody Ornamentals
HORT 232--Herbaceous Ornamentals
EGEN 115--Engineering Graphics
EGEN 116-- Engineering Graphics Lab













HORT 225--Landscape Graphics I
HORT 226--Landscape Graphics II
Communication Electives
HORT 310--Turfgrass Management
HORT 331--Planting Design
HORT 335--Site Development
HORT 336--Landscape Construction
Business Electives
HORT 431--Tough Plants in Tough
Places
HORT 432--Advanced Landscape
Design
Technical Electives
Horticulture Electives
Core Electives
Courses outside of department
shown in color
2.3 Create a Grid of Courses and Outcomes
Outcomes
1
BIOB 170IN
BIOB 110CS
HORT 131
HORT 105
ENSC 245IN
HORT 231
HORT 232
HORT 225
HORT 226
HORT 310
HORT 331
HORT 335
HORT 336
HORT 431
HORT 432
2
3
4
2.4 Highlight Courses that Include Outcome
Outcomes
1
BIOB 170IN
BIOB 110CS
HORT 131
HORT 105
ENSC 245IN
HORT 231
HORT 232
HORT 225
HORT 226
HORT 310
HORT 331
HORT 335
HORT 336
HORT 431
HORT 432
2
3
4
Note: Example
shown here is
pure fiction!
2.5 Add Code for Cognitive Skill Level
Outcomes
BIOB 170IN
BIOB 110CS
HORT 131
HORT 105
ENSC 245IN
HORT 231
HORT 232
HORT 225
HORT 226
HORT 310
HORT 331
HORT 335
HORT 336
HORT 431
HORT 432
1
I
I
I
I
I
I
I
I
D
D
D
D
I
2
3
4
M
D
D
D
D
I
I
M
2.5 Add Code for Cognitive Skill Level
Outcomes
1
This outcome
is
BIOB 170IN
I
not supported by
BIOB 110CS
I
the curriculum.
HORT 131
HORT 105
ENSC 245IN
HORT 231
HORT 232
HORT 225
HORT 226
HORT 310
HORT 331
HORT 335
HORT 336
HORT 431
HORT 432
I
I
I
I
I
I
D
D
D
D
I
2
3
4
M
D
D
D
D
I
I
M
2.5 Add Code for Cognitive Skill Level
Outcomes
BIOB 170IN
BIOB 110CS
HORT 131
HORT 105
ENSC 245IN
HORT 231
HORT 232
HORT 225
HORT 226
HORT 310
HORT 331
HORT 335
HORT 336
HORT 431
HORT 432
1
I
I
I
I
I
I
I
I
D
D
D
D
I
2
M
D
D
3
4
HORT 431 is not
asking enough
from the students,
and students are
never asked to
demonstrate
mastery in
outcome 1.
D
D
I
I
M
2.5 Add Code for Cognitive Skill Level
Outcomes
BIOB 170IN
BIOB 110CS
HORT 131
HORT 105
ENSC 245IN
HORT 231
HORT 232
HORT 225
HORT 226
HORT 310
HORT 331
HORT 335
HORT 336
HORT 431
HORT 432
1
I
I
I
I
I
I
I
I
D
D
D
D
I
2
M
3
Students are going
4 of
to have a lot
difficulty in this
outcome because
of the inverted
cognitive skill
development.
D
D
D
D
I
I
M
2.5 Add Code for Cognitive Skill Level
Outcomes
BIOB 170IN
BIOB 110CS
HORT 131
HORT 105
ENSC 245IN
HORT 231
HORT 232
HORT 225
HORT 226
HORT 310
HORT 331
HORT 335
HORT 336
HORT 431
HORT 432
1
I
I
I
I
I
I
I
I
D
D
D
D
I
2
3
4
M
We expect mastery in
communications, but
D
only have the
D
students write in one
course!
D
D
I
I
M
Two Possibilities:
2.5 Add Code for Cognitive Skill
Level
Outcomes
BIOB 170IN
BIOB 110CS
HORT 131
HORT 105
ENSC 245IN
HORT 231
HORT 232
HORT 225
HORT 226
HORT 310
HORT 331
HORT 335
HORT 336
HORT 431
HORT 432
1
I
I
I
I
I
I
I
I
D
D
D
D
I
2
This course
contributes nothing
towards the program
3 learning outcomes,
4
- or -
M
This course is
contributing towards
a learning outcome
than has not been
identified.
D
D
D
D
I
I
M
3. Where Can You Gather Data?
 The grid (curriculum map) shows where each
outcome is covered.
 The courses with higher cognitive skill levels are
typically good sources for assessment data.
Notes:
 You can save faculty time by gathering data from courses
taken by students in multiple programs.
 Knowledge of the discipline can also be demonstrated using
a discipline-based proficiency exam.
3. Data Source Identification
Outcomes
BIOB 170IN
BIOB 110CS
HORT 131
HORT 105
ENSC 245IN
HORT 231
HORT 232
HORT 225
HORT 226
HORT 310
HORT 331
HORT 335
HORT 336
HORT 431
HORT 432
1
I
I
I
I
I
I
I
I
D
2
D
D
D
M
D
D
M
3
4
I
I
I
I
I
D
M
D
D
M
4. Set Performance Thresholds
As long as students can “squeak by” and graduate with
an overall GPA of 2.001, it is unreasonable to expect
every student to demonstrate mastery in every
outcome.
Faculty need to decide what constitutes an acceptable
performance threshold for each outcome.
Example: At least 80% of students will be rated
“Acceptable” or higher on every category of
the scoring rubrics.
Example Scoring Rubric
Graduates will have an ability to design a system that meets stated needs.
Outcome Category
Clearly articulated
need that is to be
addressed by the
design.
Does the proposed
design meet the
need?
Unacceptable
No need is
indicated.
Marginal
Need stated
incompletely or
ambiguously.
No evidence that
the proposed
design will meet
the stated need.
Reader must infer
how the design
will meet the
need.
Has the group
assembled a logical
and practical
sequence of
integrated unit
operations?
The proposed
design cannot
achieve the
intended result.
The proposed
design will likely
work, but design
has significant
shortcomings.
Acceptable
Statement of need
is made, but some
improvement is
possible.
It is apparent that
the design will meet
the stated need, but
some improvements are possible.
The proposed
design appears to
be a reasonable
approach to
accomplishing the
intended task.
Exceptional
Clearly articulated
statement of need.
The utility of the
design in meeting
the stated need is
well communicated
as part of the
presentation.
The proposed
design exhibits a
high degree of
innovation.
5. Schedule for Assessing Each Outcome
Year
Outcome 2014 2015
2016
2017
1
X
X
2
X
X
3
4
X
2018
2019
X
X
X
6. Process for Using Assessment Data
1. Data is collected from identified courses.
2. Samples are scored by two faculty members using
3.
4.
5.
6.
prepared scoring rubrics.
The assessment coordinator tabulates the scores.
The scores are presented to the faculty for
assessment.
The faculty reviews the assessment results, and
makes decisions on how to respond.
A report summarizing the year’s assessment activities
and faculty decisions is submitted to the Provost’s
Office.
7. Submit Assessment Plan
Once you have an assessment plan with essential
elements:
 Program learning outcomes
 Identified data sources
 Schedule for assessing each outcome
 Process for using assessment data
then submit your completed plan to the Provost’s
Office – and start working the plan.