Transcript Outcomes Assessment 101
Assessment 101
November 27, 2001
An Introduction to Assessment in Student Affairs
1
Overview
Topics Covered in Today’s Session
Definitions of Assessment and Related Terms Key Stakeholders in Assessment Benefits of Assessment in Student Affairs Assessment Myths and Misconceptions A Comprehensive Model of Assessment Guidelines for Good Assessment Practice Basic Steps in Implementing Assessment Assessment Resources 2
What is Assessment?
A Background Knowledge Probe
What does the term “assessment” mean to you? What synonyms come to mind when you hear the term ‘assessment’?
What questions, concerns, or misgivings do you have about assessment?
What benefits might there be as a result of assessing your program and/or students? 3
Definitions of Assessment
Schuh & Upcraft (2001)
Assessment is any effort to gather, analyze, and interpret evidence which describes institutional, divisional, (program,) or agency effectiveness It includes...
assessment of student learning outcomes assessment of cost effectiveness assessment of “client” satisfaction assessment of compliance with professional standards assessment of comparisons to other institutions It is not restricted to assessment of students, but includes assessment of faculty, staff, administrators, parents, employers, graduates ….
4
Definitions of Assessment
(AAHE, 1995)
Assessment is an ongoing process aimed at understanding and improving
student learning
It involves ...
making expectations RE learning explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the results to document, explain, and improve learning and performance 5
Definitions of Assessment
(the key underlying questions)
What do we expect our students to know, to be able to do, and to value as a result of completing our program?
Do our students know, do, and value these things?
Are they learning what we want them to learn?
How do we know? If they do
NOT
know, do, and value these things, how could our program be changed to improve their learning?
6
Definitions of Assessment
A Glossary of Related Terms
Evaluation any effort to use assessment evidence to improve program, departmental, divisional, or institutional effectiveness (Upcraft and Schuh, 1996) Measurement a reference to the particular methods gathering assessment evidence involved in including interviews, focus groups, surveys, portfolios, tests sometimes divided into qualitative vs. qualitative, or indirect vs. direct methods 7
Assessment &Related Terms:
A Program Evaluation Model
Inputs Activities Outputs Outcomes Resources and Constraints Resources Staff Funding Facilities Supplies Constraints Regulations Policies What is done with inputs (ie, services) Direct products of the activities Changes in or benefits to participants Workshops Immersions Mock Interviews Programs Internships Clinics Etc # of participants # of classes taught # of brochures distributed # of hours of service provided # of counseling sessions held new knowledge new skills attitudes values awareness modified behavior altered status 8
Why Assess?
Stakeholders in Assessment
Accreditation commissions & state legislators… Administrators and Staff...
Faculty members... Graduate training programs and employers...
Parents… Students… (prospective, current, exiting, alumni) 9
Why Assessment in Student Affairs?
“…without assessment, student affairs is left only to logic, intuition, moral imperatives, goodwill, or serendipity in justifying its existence…” – Upcraft & Schuh, 1996 10
Why Assessment in Student Affairs?
Relationship to the Mission of the College Does our program help to improve students’ intellectual, personal, and spiritual development?
Role in Strategic Planning, Budget and Policy Development, Decision-Making Survival Quality Affordability Accreditation 11
Why Assess Now?
Concerns about accountability Perceptions about learning and higher education Tight budgets and markets Savvy consumers Contemporary business practices (TQM, etc) Educational reform 12
Assessment Myths and Misconceptions
It’s a Passing Fad It’s Not My Job or Concern You Can’t Assess What We Do I Don’t Have the Time to Do Assessment I Don’t Have the Expertise to Do Assessment Assessment Might Reveal “Bad” News 13
A Comprehensive Model of Assessment Includes...
Tracking Who Uses Services, Programs, Facilities Assessing Student (Client) Needs Assessing Satisfaction with Services, Programs, Etc Assessing Campus Climate, Student Culture...
Assessing Program and Service Outcomes Benchmarking: Comparisons w Other Institutions Assessing Effectiveness Against Standards Assessing Cost Effectiveness 14
Guidelines for Good Practice:
What Accreditors Expect
Links to Mission, Goals, Objectives Involvement of All Segments of Campus Multiple Measures; Assessment of Cognitive, Behavioral, Affective Domains of Learning Evidence of Use in Improving Programs Ongoing Assessment & Re-Evaluation of Assessment Program Linkages Between Assessment and Planning/Budgeting Process 15
Good Practice:
Begin with a Plan
Essential Components of Assessment Plans
Goals, Objectives, Intended Outcomes Methods,Techniques,Target Groups Time Line Provisions for Administration of Plan Provisions for Use/Sharing of Findings Evaluation of the Assessment Program 16
Basic Steps in Assessment
(What to do and when to do it)
Identify your program goals and objectives Define the problem; identify information necessary to help solve it Determine where to get the necessary information Determine whom to study and when to study them Determine the best assessment methods, what instruments will be used, how the data will be collected and who will collect it Determine how the data will be coded and analyzed; analyze the data Report the results effectively Revise assessment objectives, criteria, and methods, etc., as appropriate 17
Assessment in Student Affairs
“…good practice in student affairs occurs when student affairs personnel ask, ‘what are students learning from our programs and services, and how can their learning be enhanced?’…” ACPA and NASPA Principles of Good Practice for Student Affairs, 1997
Guidelines for Good Practice
Some Final Words of Wisdom
Tie assessment to a few key objectives intended learning outcomes, key problems, OR primary strengths Choose assessment methods wisely Pilot test if possible Assess what you intend to use Assess at key decision points Make appropriate use of existing resources Keep it simple 19
More Final Words of Wisdom
“…assessment in student affairs is no longer a ‘luxury’ that is done only after all other priorities have been taken care of. Assessment is now a necessity that demands our highest priority…” Upcraft & Schuh, 1996
Assessment Resources
Books • • • Schuh, J.H, & Upcraft, M.L. (2001). Assessment practice in student affairs: An applications manual, San Francisco: Jossey-Bass Palomba, C.A. & Banta, T.W. (1999). Assessment Essentials , Nichols, J. O. (1995). book for student outcomes assessment and institutional effectiveness The departmental guide and record . New York: Agathon Press.
Web Sites http://www2.acs.ncsu.edu/UPA/survey/resource.htm
Local Resources Office of Institutional Effectiveness; Assessment Committees 21
Need Help?
On Which Topic(s)?: Check all that apply
Involving/Motivating Staff, Students in Assessment Identifying or Developing Program Goals, Objectives, and Intended Learning Outcomes Methods of Assessment and Criteria for Success General Guidelines For Selecting Measures Strengths and Weaknesses of Various Methods Choice & Implementation of a Specific Method Analysis and Interpretation of Assessment Findings Means of Reporting and Using Assessment Information Other (Development of Timelines, Administrative Provisions, including record keeping, etc.) Optional (Name or Program: ) 22
Concluding Questions:
A One-Minute paper
What is the most important thing you learned today about assessment?
What remains most unclear or confusing to you about assessment at this point?
23