Transcript What Are Outcomes and Why Should I Want to Assess Them?

What Are Outcomes
and Why Should I Want
to Assess Them?
Dr. Jim Wilson, AVP Academic Affairs
Wilmington University
Dana Santoro, Data Analyst – Institutional Research
Wilmington University
1
Class of 2011








What Berlin wall?
They never “rolled down” a car window
They have grown up with bottled water
Wal-mart has always been a larger retailer than Sears and has
always employed more workers than GM
High definition televisions have always been available
China has always been more interested in making money than
in re-education
They never saw Johnny Carson live on television
Food packaging has always included nutritional labeling
Courtesy of Beloit College Public Affairs
2
Things Have Changed!

Higher Education Research Institute (HERI)
 UCLA
40yr Trends Presentation
3
Things Have Changed!
Item
1970
1995
Today
Postage Stamp
$0.06
$0.32
$0.39
Bread
$0.24
$1.40
$1.39
Milk
$1.32
$2.59
$4.29
Gas
$0.40
$1.35
$2.60
Car
$2,500.00
$12,800.00
$27,800.00
Income
$6,933.00
$19,717.00
$43,200.00
House
$40,000.00
$111,000.00
$264,000.00
4
Education: Costs
Wilmington University Tuition
30 credits, Undergraduate
1971
Full year tuition: $1,500
Today
Full year tuition: $8,400 (New Castle, UG)
5
Education: Some Changes

Today
 Student-centered learning
 The “Net Generation”
 Distance Learning, Online databases

1967
 Teacher-centered learning
 Classroom, Books, Library card catalogs
 Al Gore was only 20 years old
Courtesy of Beloit College Public Affairs
6
Education: Student Learning

A new “paradigm for higher education”
(Suskie, 2006.)
 Learning
Centered
 Assessment feedback helps faculty understand
What is and is not working effectively
 How to improve the curriculum as well as teaching and
learning strategies

7
Wilmington University Mission
8
What is Assessment of Student Learning?

“Good assessment must begin in the classroom
and end there.” (Wolvord & Anderson, 1998, p. 149)
9
What is Assessment of Student Learning?

“Systematically gathering, analyzing, and
interpreting evidence to determine how well
student learning matches our expectations.”
(Suskie, 2006, p.3.)
10
What is Assessment of Student Learning?

“Assessment is first and foremost a tool for
faculty members to use as they do their very
best to teach their students well.” (Middle States, 2003)
11
Assessment Is A Friend Who…
Honestly tells us how we are doing
 Helps to clarify what we are doing and why
 Helps us to make informed decisions
 Improves student learning
 Helps us to improve
 Brags about how well we are doing

12
Assessment Could Be a Foe!
Federal requirements
 Regional Accreditation
 Legislators
 Parents and Community groups
 Students
 Professional organizations

13
Summary
What is Assessment of Student Learning?

Process of gathering & evaluating information
 Quantitative and
qualitative
 From multiple sources

To enhance student learning
14
Student Outcomes Assessment Framework
15
Three Assessment Areas

Student Learning

Teaching Effectiveness

Student satisfaction with their academic
experience
16
Wilmington University
ACADEMIC AFFAIRS ASSESSMENT PLAN
COLLEGE MISSION
Assessment of
Teaching Effectiveness
Assessment of Student
Learning Outcomes
Graduation Competencies
Benchmarks
Benchmarks
Division / Program Mission
Professional Standards --------
PROGRAM COMPETENCIES
• IDEA Results
• Faculty Evaluations
• GPA Reports
• Enrollment Data
•ACT Survey Results
Benchmarks
License & Certification Exams
College Outcomes
Course Work
Major Field Exams
CECRAM
Rubrics, Test Embedded Questions, Clinical Evaluations
Data Collection, Analysis, Reporting, Decisions,Actions
17
Program Improvement Cycle
18
Language of Assessment

Direct evidence
 Direct
evidence of student learning indicates
whether or not a student has a command of
specific content

Examples: Homework assignments, research papers,
rubrics
 CECRAM
(Course-Embedded CriterionReferenced Assessment Measures)
19
Language of Assessment

Indirect Evidence
 Indirect
evidence of student learning is
correlational – data exists which indicates that
students are probably learning.
 Examples: Course grades, time spent on service
learning or homework
20
Language of Assessment

Formative assessment
 Ongoing
assessment that is intended to improve an
individual student’s performance.
 Purpose: to improve course content, provide
feedback to faculty.
21
Language of Assessment

Summative assessment
 Provides
a true gauge of “outcomes” of student
learning.
 Data is typically gathered near the end of program
completion
 Examples: course-embedded project/test/portfolio,
clinical evaluations, standardized exams
22
WU Language

“Good enough”
 The
process is accurate and truthful
(Suskie, 2004, p. 302)

“Closing the loop”
 Outcomes
data has been analyzed and appropriate
changes have been made.
23
Elements of Assessing Learning
1.
2.
3.
4.
Articulated expectation of student learning at
various levels.
A plan that describes student learning
assessment activities.
Evidence that student learning assessment
information is used.
Documented use of student learning assessment
as a part of institutional assessment.
24
Background Information

Academic Affairs Assessment Plan adopted in
2003

Plan called for evaluation in 2006
25
MSCHE
Outcomes Assessment Guidelines
Assessment should be:
 Useful
 Cost-effective,
simple
 Reasonably accurate and
truthful
 Organized, systematized and
sustained.
Suskie, L. (2005)
26
Major Changes
Related to Outcomes Assessment
Five Year Data Collection and
Implementation Cycle
 Revision of Graduation Competencies
 Representative Sampling
 Formal Reporting Cycle

 Five
Year Program Review
27
How to maintain momentum?

A 5 year cycle will be utilized
 Year
1 – Implement findings from program
review
 Year
2 – Data collection
 Year
3 – Reflection, take appropriate action and
collect “other kinds” of data
 Year
4 – Data collection
 Year
5 – Program Review Report
28
What are Examples?
(of closing the loop)
Program: B.S., Finance



Weakness: Undergraduate Finance majors were
missing some important finance skills
Source: ETS Major Field Exam
Action
 Adjust
the content of FIN 305
 Add a new requirement, FIN 306

Competency
 Disciplined Inquiry
 Finance
program competencies
29
What are Examples?
(of closing the loop)
Program: All UG Education programs



Weakness: Students were unclear of required skills
Source: Portfolio review
Actions:
 Curriculum
changes
 Provide candidates with exemplars
 Ensure understanding of required skills

Competency
 Disciplined inquiry
30
What are Examples?
(of closing the loop)
Program: M.S., Community Counseling

Weakness: Appraisal Techniques

Source: CECRAM, CPCE*, NCE* & surveys

Response: Integration of assessment with diagnosis
and treatment planning.

Psychopathology course will precede the appraisal course
*Standardized exams
31
What are Examples?
(of closing the loop)
Program: B.S., Nursing



Weakness: Student teaching projects - measurable
objectives were not reliably clear
Source: CECRAM
Actions:
content and a teaching video to NUR 425 –
Community Health Practitioner
 Added teaching content to NUR 330
 Revised curriculum and developed a new course:
NUR 323 - Nurse as Teacher
 Add
32
Conclusion
What is outcomes assessment of
student learning?
Why should I support the assessment
process?
33
References




Caffo, Betty, (2003). Striving for Excellence. Faculty Senate
Presentation, Wilmington College.
Middle States Commission on Higher Education. (2006).
Characteristics of excellence in higher education. Philadelphia:
Author.
Suskie, L. (2004). Assessing student learning: A common
sense guide. Bolton, MA: Anker.
Walvoord, B.E., and Anderson, V.J. (1998). Effective grading:
A tool for learning and assessment. San Francisco, CA:
Jossey-Bass.
34