Transcript Assessment Made Easy - Union County College

Assessment Made Easy
Maris Lown, EdD
Assessment answers this question:
How do we know students are
learning what we think we are
teaching?
Assessment “belongs to the
scholarship of teaching.” It is a
study of “situated teaching
practices . . . using particular forms
of research and knowledge.”
Litterest & Tompkins, (p. 10)
Types of Outcomes
Learning
Describes what we
want students to do
with the content they
are learning.
Success
Identifies whether or
not the course or
program is effective.
A learning outcome reflects what
students can do with the content.
A Success Outcome reflects
a standard of judgment that a
discipline, a department or an
institution chooses to apply to
students work.
Learning Outcomes
•
•
•
•
•
•
Are measurable
Begin with an action verb
Stress higher-order thinking skills
Are a learner oriented essential ability or skill
Lead to the identification of assessment tools
Identify “what a student is able to do with the
content”
• Provide the foundation for success outcomes
Success Outcomes
•
•
•
•
•
Are measurable
Begin with a metric
Stress a standard of judgment
Are learner oriented
Reported in terms of numerical scores that
place them in an achievement level based on
developed and tested criteria
• Build on learning outcomes
Outcomes
Learning
• Describes what we want
students to do wth the
content.
• Includes a performance
indicator
• Give a speech that is
designed to convince the
audience of a specified
point of view.
Success
• Identifies whether or not
the course is effective.
• Includes a metric
• 85% of students will
achieve a grade of C or
better on the course final
exam.
Comparison
Objectives
Outcomes
• Yields behaviors which are
predictable.
• Primarily concerned with
analysis.
• Usually evaluates discrete
specific behaviors
• Evaluates behaviors that
are most readily assessed
and therefore may ignore
behaviors that are more
important.
• Yields behaviors which are
unpredictable.
• Primarily concerned with
synthesis.
• Usually evaluates abilities
or skills.
• Evaluates outcomes of
education which are more
difficult to assess and are
often ignored.
10
Assessment: Direct Methods
Integrated Learning
•
•
•
•
•
•
•
•
Exams
Collaborative projects
Research papers
Videotaped interactions
Journaling
Case studies
Performances
Standardized exams
•
•
•
•
•
•
•
•
Portfolios
Creations
Projects
Internships
Capstone projects
Oral examinations
Concept Maps
Service learning
Assessment: Indirect Methods
Attitudes, Perceptions, Experiences
•
•
•
•
•
Student surveys
Focus groups
Course evaluations
Faculty surveys
CCSSE
•
•
•
•
•
Interviews
Graduate surveys
Employer surveys
Alumni surveys
Student Satisfaction Inventory
“Culture of Assessment”
Hierarchy of
Assessment of
Student Learning
What evidence have
you gathered
documenting that
these changes have
Improved Student
Learning?
LEVEL 5
How have you Changed
instruction/the curriculum to Improve
Teaching and Learning based on
these Data?
LEVEL 4
LEVEL 3
What Data have you Gathered Regarding
Achievement of PLOs/CLOs?
LEVEL 2
What Tools are you using to Assess Achievement of these
PLOs?
What are the Program/Course Learning Outcomes?
LEVEL 1
CURRICULUM DEVELOPMENT
Institutional Learning Outcomes
Program Learning Outcomes
Course Learning Outcomes
Unit Learning Outcomes
Learning Activities
Evaluation Tools
Middle States
• Level 5 requires evidence of changes made
due to assessment – departmental minutes.
• We will be required to show a portfolio of
assessment projects/artifacts.
• Must be faculty driven.
• How can we embed program assessment into
Five Year Program Assessments?
• Will we be required to connect learning
outcomes to success outcomes?
Program Review Process
• Connect to PLOs
• Can include direct assessment
– Papers,
– Exams
• Can include indirect assessment
– Student surveys
– Exit interviews
– Job placement
• Provide samples of program reviews
Institutional Learning Outcomes
• Skills/abilities with which every student
graduates.
– Critical thinking
– Reading
– Writing
Program Learning Outcomes
• Connect to institutional learning outcomes.
• Includes three or more general education
skills/abilities.
• Includes additional discipline skills/abilities.
• General education skills/abilities can be
customized to discipline competencies.
• Students in programs take ENG 101 and 102.
• ETS Proficiency Profile.
Course Learning Outcomes
• Connect to program learning outcomes.
• Includes general education skills/abilities
tailored to discipline.
• Focuses on discipline skills/abilities.
• How many?
Mapping Institutional Learning Outcomes to
Program Courses
Program
College Wide Learning Outcomes
Critical
Thinking
Communication
Teamwork
Information
Literacy
Biology AS
Biology 101
Biology 102
English 101
Speech 115
English 102
Biology 102
History 101
English 101
Nursing AAS
Nursing 101
Nursing 102
Nursing 203
Nursing 204
English 101
English 102 or
Speech 115
English 102
Nursing 203
Nursing 204
English 101
Liberal Studies
AA
History 105
English 101
Speech 115
English 102
English 101
Program Name
Mapping Program Learning Outcomes to Program Courses
Name of Program _______BIOLOGY________________________________________ Degree ____AA_______
Program Level Student Learning Outcomes
Biology
Program
Courses
BIO 111
BIO 112
BIO/CHE/PHY
electives (2)
CHE-111
CHE-112
ENG-101
ENG-102
MAT 143 or 171
MAT 144 or 172
Explain basic
biological
principles and
defend
scientific
perspectives
Apply the
Evaluate
scientific
the
method to relationship
hypothetical
between
or
mankind
laboratory
and the
problems
natural
world
Utilize
critical
thinking
skills to
solve
biologic
problems.
Communicate
scientific
material
effectively in
oral and
written
formats.
Demonstrate
information
literacy skills
and technology
skills when
gathering
scientific
information
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Mapping PLOs to CLOs, Political Science
PLOs
CLOs
POLI 101
POLI 102
POLI 220
POLI 221
Explore political
science
methodology
Explain basic
political science
concepts
Examine the
Analyze how
three branches of states gain and
government and lose power
how they check
and balance each
other
Investigate the
origins of
environmental
problems
Compare and
contrast political
ideologies and
theories of
governance
Interpret political
ideologies
including
conservatism,
liberalism,
socialism,
communism, and
fascism
Evaluate the
electoral process
and how
candidates are
selected to
represent us at
the federal level
Compare and
contrast the
state system
and the
international
system
Analyze the
goals of
environmental
interest groups
Analyze political
concepts,
theories, and
ideologies in
spoken and
written form
Discuss the
strengths and
weaknesses of
the political
process
Compare and
contrast the
ideological
differences of
the two major
parties
Analyze the
contributions
of international
institutions
Evaluate
environmental
legislation
including the
EPS and the
CWA
Learning outcomes drive the
selection of learning activities and
assessment tools. They lay the
foundation for success outcomes.
Mapping CLOs to Learning Activities &
Assessment Tools
Course
CLO
Learning Activity
Assessment
NURS I
Implement care using the
Critical Thinking Model and
the Nursing Process
Scaffolded patient
case study
Nursing Care Plan
NURS II
Assess human needs across
the life span
Simulation of a
newborn in distress
Newborn
Assessment
NURS III
Analyze therapeutic
communication when
interacting with patients
YouTube examples of Video taping patient
communication
interaction
approaches
NURS IV
Implement care focusing on
complex pathophysiology
Simulation of a
patient in
multisystem failure
and illness trajectory
Concept Map
NURS IV
Collaborate with members
of the health care team
when delivering care
Management Case
Studies
Journal
Collecting Evidence
•
•
•
•
•
Record data
Identify changes made
Collect additional data
Discuss whether changes made a difference
Can be memorialized in department minutes,
annual meeting, or a faculty in-service
Institutional Effectiveness Model
• Strategic Plans
– AMP
– Technology Master Plan
– Facilities Master Plan
– Matrix
– Non-credit
Institutional Effectiveness Model
• Surveys
– Community Needs Assessments
– Community College Survey of Student
Engagement
– Student Satisfaction Inventory
Institutional Effectiveness Model
• Academic Assessment
–
–
–
–
–
–
–
–
–
Proficiency Profile (e.g.)
Program Assessment
Program Review
Departmental Plans
Data Books
Graduate Surveys
Employer Surveys
Graduation Rates
Transfer Rates
Institutional Effectiveness Model
• Non Credit Assessment
– Targeted outcomes assessment
• Administrative Services Assessment
– Targeted outcomes assessment
– Planning, assessment and budget
• Student Services Assessment
– Targeted outcomes assessment
No assertions without evidence
Middle States
And now the easy
part!
Learning Management Systems
• Applications that ease assessment
• Use technology where applicable
• Grade books that provide assessment
information
Portfolio Assessment
•
•
•
•
•
•
Samples of students’ work over time
Uses a rubric
Offers a deep but narrower approach
Gives a richer texture
Can be resource intensive
Can use a sample rather than the entire
population
• Can be used for general education assessment
• Can be used for prospective employers
Capstone Assessment
•
•
•
•
•
•
Paralegal Program – Trial Notebook
Incorporates all Program Learning Outcomes
Used at the end of Litigation II
Juried Review
Uses a rubric related to outcomes
Report that includes level 5 and trended data
Assessment of Majors Courses
• Select 2 to 4 courses in the discipline (majors
courses, popular courses)
• Select assessment method
• Biology
– Assess BIO 111
– Assess BIO 112
– Assess BIO 105
– Assess Bio 106
Embedded Exam Questions
• Department develops questions that relate to
student learning outcomes.
• Questions are different for each course but
similar in that they assess the same behaviors,
e.g.
– Analyze an organic compound, analyze an
inorganic compound
– Identify normal lung sounds, identify abnormal
lung sounds
Blue Print Assessment Tools
Remembering
Understanding
Applying
Analyzing
1
3
23
14
7, 30
2
15
6
20
21
7, 27
8, 24
13, 25
1
22
2
26
19
10
17, 28
4
11
18
5, 29
12
CLO
3
4
5
9
6
7
8
15
16
Evaluating
Creating
Everything that counts is not
countable and everything that
is countable does not count.
A. Einstein
What is not countable
•
•
•
•
•
FYE
Trips
Clubs
Speakers
Study abroad
• Athletics
• Work study
• Service learning
Middle States, 2017
• Preparation, 2011-2014
– Assessment process
– Budget, planning, assessment
– Department plans
– Catalog
– Syllabi
• Preparation of SSR, 2014-2017
What gets measured get improved!
The central value of
learning outcomes
assessment is using
evidence and data to make
teaching more effective.
Whenever things sound easy,
it turns out there’s one part
you didn’t hear.
D. Westlake