Transcript Slide 1

“Just Do It” Assessment:
A Five Step Plan for Busy Faculty
Assessment Bites #4
March 1, 2007
Presenter: Mimi Harris Steadman Ed.D.
Director of Institutional Assessment
(716) 839-8567 [email protected]
1
Agenda
•
•
•
•
•
•
Assessment at Daemen
Focused Five Step Assessment Plan
Developing Learning Objectives
Collecting Information
Using Results
Good Assessment
2
Workshop Learning Objectives
Participants will be able to:
• Identify different types of assessment at
Daemen College
• List the steps of a basic assessment plan
• Articulate learning objectives for a course
or program
• Describe a variety of direct and indirect
assessment methods
3
Assessment Round Up
What kinds of assessment activities are
happening at Daemen?
What information about student learning and
institutional effectiveness do we already
collect?
4
Assessment at Daemen
•
Institutional Assessment
– The overarching plan for collecting and reporting information about institutional
effectiveness and student learning across the institution.
– Information collected at an institution-wide level, such as enrollment and
retention data, and surveys like the NSSE, CIRP and YFCY.
•
Core Curriculum Assessment
– Assessing student achievement of the core competencies, through direct
methods such as capstone courses, senior projects, and other student work
collected via course instructors or e-portfolios, and through indirect methods
such as student surveys or focus groups.
•
Departmental Assessment
– Assessing student learning in majors or programs through direct methods such
as written student work or performances, capstone courses, and success rates
on licensure exams or national exams, and through indirect measures such as
program reviews, job placement rates, and employer, alumni or student surveys.
•
Classroom or Course Assessment
– Assessment that determines whether students are learning what we think we are
teaching, and assessment that provides feedback to learners. Direct methods
for classroom assessment include assignments, tests, papers, CATs and projects
or presentations.
5
5 Step Assessment: ACRRU
BRAINSTORM the basic steps of assessment:
1.
2.
3.
4.
5.
A is for…
C is for…
R is for…
R is for…
U is for…
6
Focused Five Step
Assessment Plan (ACRRU)
1. Articulate learning objectives
-As a result of this program/course, students should be able to…
(*Offer instruction or other learning experiences*)
2. Collect information/evidence
-Using direct and indirect methods
3. Review/analyze data and reflect on findings
4. Report results
5. Use information for decision making and improvement
7
1. Articulate Learning Objectives
• Before we can plan for assessment, we must start by
defining learning objectives.
• Learning objectives describe what a student knows,
believes, or is able to do.
• Consider the alignment of objectives with your program
or institutional mission.
• Learning objectives (or goals) are broader at the
institution or general education level and get narrower at
the program, department or course level.
• Learning objectives guide course design and instruction.
• A few will do. Stagger your assessment and focus on
different objectives each year.
8
Why learning goals and not
teaching goals?
The true test of teaching effectiveness lies
not in what was taught, but in what was
learned. Teaching without learning is just
talking. -Tom Angelo and Pat Cross
• In other words, just because I “covered” the
material, doesn’t mean my students learned it.
Articulate Learning Objectives
9
Useful learning objectives:
• Describe what we expect our students to know
and be able to do
• Use verbs that describe observable action
• Describe learner rather than teacher actions
• Do not describe course content coverage
• Describe an acceptable performance level
• Can be assessed by one or more methods
• Make sense to colleagues and are clear to
outsiders
Articulate Learning Objectives
10
Sample learning objectives
• Program: General Education
– Program Goal: Recognize and appreciate artistic and
literary contributions of diverse cultures
• Course in Program: Caribbean Literature
– Course Goal: Demonstrate familiarity with themes
and genres of classic and contemporary Caribbean
literature
(taken from Student Learning Assessment: Options and Resources,
MSCHE 1993, p. 14)
Articulate Learning Objectives
11
More sample objectives
• Program: History
– Program Level Goal: Communicate orally and in
writing about historical topics
• Course in program: Medieval History
– Course Goal: Present a cogently-written, critical
analysis of gender and class roles in Medieval
England
(taken from Student Learning Assessment: Options and Resources,
MSCHE, 1993, p. 14)
Articulate Learning Objectives
12
Developing learning objectives
Start with:
• Existing departmental goal statements (including those that just
describe teaching goals rather than learning goals)
• General education goal statements (e.g., critical thinking, lifelong
learning)
• Disciplinary organizations
• Professional accrediting organizations
• Assessment plans and reports from other colleges
(see http://www2.acs.ncsu.edu/UPA/assmt/resource.htm)
Articulate Learning Objectives
13
Specifying learning objectives
Instructions: With one or two others, role play student and teacher and
generate a response to the following:
“If I’m your student, what do I have to do to convince you
that I’m where you want me to be at the end of this
lesson, unit or course?”
• _____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________
Reference: R. M. Diamond (1998). Designing and Assessing Courses and Curriculum.
San Francisco: Jossey-Bass, pages 134-135.
Articulate Learning Objectives
14
2. Collect Information/Evidence
Use at least two measures, one direct and one indirect.
Direct Measures: direct review of student work on an
exam, a project, or performance (with set criteria for
evaluation), licensure or certification exams, etc.
Indirect Measures: student reflections on learning,
student, alumni, or employer surveys, focus groups,
placement rates, etc.
Data can be new or existing, qualitative or quantitative,
collected at the course, program, or institutional level.
You don’t need lots of data, just the right data.
If we knew what it was we were doing, it would not be called research,
would it? –Albert Einstein
15
Data Collection Resources
See handouts:
• Examples of Direct and Indirect Measures
of Student Learning
• Quick Tip 6.0
– Data Gathering: Methods
• Quick Tip 6.02
– Choosing a Data Gathering Method
Collect Information
16
Do not let the perfect be the
enemy of the good.
“To avoid paralysis, a campus is well advised to remember that
assessment is an ongoing, flexible, naturally evolving process
offering many opportunities to fine-tune or change direction.
Meanwhile, even an imperfect process can yield useful results”
(Leskes and Wright, 2005, p. 23).
Collect Information
17
3. Review and Reflect on Findings
• Display and discuss findings in a group (ideally
in an annual retreat meeting format)
• Start by looking for general trends, then analyze
key issues more closely
• Identify areas for action or improvement
• Identify additional questions raised and future
focus areas for assessment
The outcome of any serious research can only be to make two
questions grown where only one grew before. -Thorstein Veblen
18
4. Report Results
• In a concise format, report on findings, plans for
action, plans for future assessment, and
resource issues.
• Who are the important audiences for your report,
and how will information reach them?
• Are there other venues for sharing assessment
findings?
• Don’t forget to celebrate successes.
How important it is for us to recognize and celebrate our heroes and
she-roes! -Maya Angelou
19
5. Use Assessment Information for
Decision Making & Improvement
“The aim of assessment is primarily to
educate and improve student
performance, not merely to audit it.”
-Grant Wiggins (1998). Educative Assessment,
San Francisco: Jossey-Bass, p. 7.
20
Suskie’s Five Dimensions of
Good Assessment
•
•
•
•
•
Used
Cost-Effective
Reasonably accurate and truthful results
Valued
Focus on and flow from important goals
Linda Suskie, Middle States Commission on Higher
Education, October 10, 2006 (presented at the IUPUI
2006 Assessment Institute in Indianapolis).
21
Aim for an assessment process that is
scholarly and professionally engaging,
rather than tedious and time consuming.
Keep your plans lean, focused, and useful
so that the assessment process is
sustainable and supports, rather than
detracts from, the work that matters most.
22
Resources
Marilee J. Bresciani, Carrie L. Zelna and James A. Anderson. (2004).
Assessing Student Learning and Development. A Handbook for
Practitioners. The National Association of Student Personnel
Administrators.
Andrea Leskes and Barbara D. Wright (2005). The Art & Science of
Assessing General Education Outcomes. Washington, D.C.: The
Association of American Colleges and Universities.
Middle States Commission on Higher Education (2003). Student
Learning Assessment: Options and Resources. Philadelphia:
MSCHE.
Barbara E. Walvoord (2004). Assessment Clear and Simple. A
Practical Guide for Institutions, Departments, and General
Education. San Francisco: Jossey-Bass.
23
•
Instructional Assessment Resources from The University of Texas at
Austin, Division of Instructional Innovation and Assessment, (including
“Quick Tip” sheets on topics such as rubrics, surveys, qualitative analysis
and more). http://www.utexas.edu/academic/diia/assessment/iar/index.php
•
Internet Resources for Higher Education Outcomes Assessment from
the North Carolina State University Office of University Planning and
Analysis. Links to everything related to assessment in higher education,
including articles and associations, organized by technique, by discipline, by
institution, and more. NCSU hosts the 2007 NC State Undergraduate
Assessment Symposium in the Raleigh area April 13 - 15, 2007.
http://www2.acs.ncsu.edu/UPA/assmt/resource.htm
•
National Assessment Institute sponsored by IUPUI (Indiana UniversityPurdue University Indianapolis) is a comprehensive assessment conference
each fall. http://www.planning.iupui.edu/51.html
•
***COMING SOON: The Daemen College Assessment Web Page***
24