Transcript A State-Level Student Learning Outcomes Assessment Process

Designing Valid Assessments
of Student Learning
Rodolfo Rincones
CETaL Workshop # 1
Enhancing Student Learning
UTEP Library - Blumberg Auditorium
September 3, 2009
Outline
Context of assessment in higher education
Purpose of assessment
Linking learning and assessment
Types of assessment
Activity
What we know ( 5 minutes)
What we want to know ( 5 minutes)
What we learned (at the end 5 minutes)
Context of Higher Education
Decreasing public funding sources
Tight budgets
Expanding pressures to do more with less
Escalating tuition
Public trust
Increase productivity
Accountability
Context of Higher Education
Factory-production model based on
semester credit hours, certification, transfer,
articulation, and “student success”
Limited records that reflect actual student
learning outcomes
Paradigm shift from teaching to learning
Donohue-Lynch, B. Assessment with 21st Century Tools
http://www.carnegiefoundation.org/perspectives/sub.asp?key=244&subkey=2140
Are we assessing student learning to appease a
growing tide of legislators and government
executives who increasingly question the full impact
of colleges and universities on student learning?
or
Are we assessing student learning to improve our
own educational practices, curriculum choices, and
instruction?
McKitrick, Sean , Assistant Provost for Curriculum, Instruction, & Assessment at Binghamton University (SUNY)
Undergraduate education in research universities requires
renewed emphasis on a point strongly made by John Dewey
almost a century ago: Learning is based on discovery
guided by mentoring rather than on the transmission of
information.
Inherent in inquiry-based learning is an element of
reciprocity: Faculty can learn from students as students are
learning from faculty
The Boyer Commission on Educating Undergraduates in the Research University, 1998
Today’s Learning
Today Learning in the 21st Century
Curriculum
“Curriculum is the formal and informal content and
process by which learners gain knowledge and
understanding, develop skills, and alter attitudes,
appreciations, and values under the auspices of that
school.” (Keller, 2004)
Creating the learning environment
Although shared knowledge is an important
component of a university education, no
simple formula of courses can serve all
students in our time.
Integrated educational experience
Collaborative learning experiences
Promote analysis, synthesis, and evaluation
Teaching
Involves developing the knowledge, skill, mind,
character, or ability of others. It “means not only
transmitting knowledge, but transforming and
extending it as well.”
“Dynamic endeavor involving all the analogies,
metaphors, and images that build bridges between
the teacher’s understanding and the student’s
learning. Pedagogical procedures must be
carefully planned, continuously examined, and
relate directly to the subject taught.” (Boyer 1990)
Learner-Centered Teaching
How to tie teaching, curriculum, and objectives of
learning rather than to the content delivery alone.
Focuses attention on what the student is learning,
how the student is learning, the conditions under
which the student is learning, whether the student
is retaining and applying the learning, and how
current learning positions the student for future
learning. (Weimer, 2002)
Purposes of educational assessment
Assessment is a comparison of achieved
results to intended goals (Ewell, 1985)
Traditional
Diagnose students’ strengths and weaknesses
Monitor students’ progress
Assign a grade
Determine program and institutional
effectiveness
Purposes of educational assessment
Formative - to aid learning
Summative - for review, transfer and
certification
Summative - for accountability to the public
Reasons for Assessing
Shupe, D. (2006). Toward a higher standard: the changing organizational context of accountability for educational results, ON THE HORIZON, Vol.
16, No. 2 2008, pp. 72-96.
Student Learning Outcome
Student learning outcomes should refer
normally to competencies or attainment
levels reached by students on completion of
an academic program
Courses must define two specific sets of
learning objectives
Traditional knowledge and skills associated
with the course
Abilities addressed in the course.
(Keller, 2004)
Abilities
Integrated Developmental Transferable
Communication
Analysis
Problem Solving
Valuing in DecisionMaking
Social Interaction
Global Perspectives
Effective
Citizenship
Aesthetic
Responsiveness
Learning Assessment
What is meant by “assessment” often varies
greatly—embracing everything from job
placement, through student satisfaction, to
self-reported gains in skill and knowledge
on the part of students and former students.
(Ewell, 2001)
Types of Measures
Direct Measures- provide clear and
compelling evidence of what students are
learning
Course-embedded assessments, including written work and
presentations scored using a rubric
Scores on locally designed tests and competency exams
accompanied by test “blueprints” describing what is being assessed
Score gains between entry and exit on tests, competency exams
and writing samples
Ratings of student skills in the context of class activities, projects
and discussions
Portfolios of student work
Scores on nationally- norm instruments (Ewell, 2001)
Types of Measures
Indirect Measures-Provide signs that students are
probably learning, but it is less clear exactly what they are
learning.
Student grades
Student evaluations and ratings of the knowledge and
skills they have gained
Student or graduate satisfaction with their learning in
general education competencies
Results of nationally-norm surveys (Ewell, 2001)
Assessment Evaluation Criteria
Reliability
Consistency with which assessment produce
measures whatever it is measuring
Validity
Reflects the defensibility of score-based
inferences made on the basis of an educational
assessment procedure
Assessment Evaluation Criteria
Absence-of-Bias
Degree to which assessments are free of
elements that would offend or penalize
examinees on the basis of examinees’ gender,
ethnicity or other characteristics (Popham, 2002)
Learning Objectives
Define a course in terms of the outcomes
the instructor expects students to achieve
Components
A description of what the student will be able to
do (verb)
The conditions under which the student will
perform the task
The criteria for evaluating student performance
Importance of Learning Objectives
Selection of content
Development of an instructional strategy
Development and selection of instructional
materials
Construction of tests and other instruments for
assessing and then evaluating student learning
outcomes
Efficient use of instructional time
Ongoing improvement and student feedback
Example
Objective: Given a set of data the student
will be able to compute the standard
deviation.
Condition - Given a set of data
Behavior - the student will be able to
compute the standard deviation.
Criterion - the number computed will be
correct
Course Assessment and
Enhancement Model
Combs, K., et al.(2008). Enhancing curriculum and delivery: linking assessment to learning objectives. Assessment & Evaluation in
Higher Education, Vol. 33, No. 1, pp. 87–102.
Your turn….
Connecting curriculum elements
Thank you!!!