Transcript Slide 1

Writing and Using Assessment
Plans to Enhance Teaching,
Learning, & Curriculum
November 10, 2006
Today’s Objectives
• Understand some reasons why we need to assess student learning
• Identify and discuss each component of an assessment plan
• Brainstorm about some ways to make assessment useful and
meaningful
• Brainstorm about appropriate methods to use when assessing
student learning
• Introduce ways that the assessment process can impact teaching,
learning, curriculum, and other things important to us in the
university community
External Pressure to Assess
•
Federal government pressures to
assess—NCLB Act, Spellings
Commission Reports, etc.
•
State pressures will not dissipate,
given budget constraints
•
Regional accreditors (for us, the
Middle States Association) require it
•
Local constituencies want it—they
want to know how well universities and
colleges are “adding value” to a
student’s education
•
The public increasingly expects it
•
Even grant funders want it—
assessment systems submitted with
grant applications, assessment results
included with grant reports, etc.
Example of Regional Accreditor’s Review:
Middle States Association Suggestion for Binghamton
University
Despite External Pressures, There Are Real
Advantages to Program Assessment
•
Enables programs to answer
external requests for information
•
Empowers faculty, not
bureaucracies to make decisions
about curriculum, instruction, and
learning
•
Enhances grant application
process
•
Empowers faculty to reflect on
student teaching and learning in a
way that is non-threatening
•
Enhances faculty’s ability to
publish
•
Improves student learning
Why An Assessment Plan?
• Facilitates periodic, not episodic assessment of student learning
and program outcomes
• Serves as template for conversations about student learning
• Leads to improvement in student learning
• Enables faculty to play central role in managing student learning
• Facilitates information gathering for program review process
• Makes it easier to do something about what assessments have
to say
Whose Needs Might We Consider When
We Develop Assessment Plans?
Speaking Different Languages:
Assessment Communities In Practice
University
Community
Regulatory
Community
•Mission
Statement
•State
Guidelines
•General
Education
Objectives
•Dispositional
Expectations
•Constituent
Expectations
and Norms
•Faculty
Expectations
Assessment
Plan
•Federal
Guidelines
•Professional
Standards
Program
Community
•Advisory Board
•Expectations
•Professional
Organization
Norms and
Expectations
•Employer
Expectations
•NorthCentral
Association
Guidelines
•Public
Expectations
and Norms
Reflect on some of the different audiences that might
be interested in or that might require assessment
information about student learning. What are some of
those audiences?
University Community:
Example: Faculty
Regulatory Community:
Example: Federal Government
Program (or Discipline) Community:
Example: National Council on Accrediting Teacher Education (program
accreditor)
Faculty Involvement Is Crucial
• The focus of assessment is not measurement,
but how assessment information is used by
faculty
• Only faculty are qualified to make decisions
about how to use assessment information for
enhancing teaching, learning, curriculum, etc.
• This is an opportunity, not an obstacle
Assessment As A Faculty-Based
Process
Defined Student Learning
Objectives
Evidence-Based Program
Enhancements—
(Curriculum, Learning,
Teaching, etc.)
Faculty
Focused Reflection/Discussion
Meaningful Measurement
(Qualitative, Quantitative,
Triangulated)
Components of An Assessment
Plan
• Student Learning Objectives
• Assessments
• Plan for Faculty Discussion About Assessments and
How they Lead to Recommendations to Further Enhance
Student Learning Objectives
• Plan to Implement Recommendations in Respect to
Curriculum, Instruction, etc.
Identifying Student Learning
Objectives
Defining Learning Objectives
• Knowledge—what do we want students to know
when they graduate (i.e., content knowledge, etc.)
• Skills—ability to perform specific tasks, think in certain
ways, etc.; what should a graduating student be able
to do?
• Competencies—ability to perform specific tasks “in real
time,” or “authentically” [knowledge + skills 
competencies]; also, what are some values, attitudes,
behaviors we feel are important for graduates to
have?
Listing Student Learning Outcomes By
Knowledge, Skill, and Competency
1. Think about some of the knowledge, skills, and
competencies, you would like a student from your program
to graduate with
2. List them
3. Discuss with a partner at your table.
Name of Program:
What Are Some Things You Want
Them to Know?
What Are Some Things
You Want Them To Do?
What Are Some Ways You
Would Like Them To Use
Their Newly-Acquired
Knowledge and Skills?
Assigning Relevant Assessments
Selecting Appropriate and
Meaningful Assessments
• At least one should be a “direct assessment” of student learning,
meaning that they should involve observations of actual student
performance
• Indirect assessments —those that include student opinions about a
programs’ ability to deliver on the student learning objectives, etc.
can also be very helpful
• Using a combination of these—two or three—might provide
meaningful information; “triangulation” is increasingly being required
by evaluators
• It is rarely a surprise to find out that departments and programs
initially feel that they never directly assess student learning, only to
find out after discussion that they have been doing so, only
informally
“Not everything that can be
counted counts, and not
everything that counts can be
counted.”
-Albert Einstein
“Data is the plural of
anecdote”
-Ronald Coase
Helpful Tips
•
The process of selecting appropriate assessments is usually dependent on a number
of factors —experience in process, resource limitations, etc.
•
While the preference is to move toward a comprehensive assessment system, reality
suggests that the development of an assessment system is very much
developmental; it takes time to develop such a system.
•
Although the prior point is important to keep in mind, most program and regional
accreditation organizations are beginning to expect 3-5 years of periodically collected
assessment information
•
Therefore, begin incrementally. Start with assessments that create a balance
between need to conserve resources and need to maximize the meaning gleaned
from the process
•
The most important question to ask at this point: “How will this information provide
faculty with legitimate information that will affect learning, teaching, and curriculum?”
•
A “shameless plug:” Ask the Assistant Provost for Curriculum, Instruction, &
Assessment for assistance!
Outcomes Assessments from the Perspective of
Different Communities
Assessment Plan
University
Community
Examples:
Student
Satisfaction
Surveys
Examples:
Regulatory
Community
General
Education
Syllabi
Reviews
Portfolio
Assessments
Graduate
Record Exam
(GRE) Math
Scores
Focus Group
Interviews
Unstructured
Student
Interviews
State
Licensure
Exams
(Passage
Rates)
Examples:
Disciplinary
Standards
Community
Capstone
Course
Case Study
Reviews
Advisory
Board Input
GRE Subject
Exam
Scores
Encouraging Faculty
Discussion
Defining When Faculty Will Discuss
Information
• Faculty (or an assessment committee) should be
given information in advance to consider
• Focus of meeting is to discuss what assessments
say about student learning objectives and any
recommendations that might stem from such
discussion; it should not be a “rubber stamping” of
findings or solely an opportunity to complain—
instead, the focus should be upon recommendations
• It is important to focus on communicating results to
faculty, and tracking ways that information is used
Tip on Producing An Effective
Meeting:
Try to Put Assessment Results
on One Page, If Possible
Assessment: Graduate Alumni Surveys
Assessment: Graduate Employer
Surveys
Assessment: Expert
Panel
NBPTS 1: Teachers Are Committed to
Students and Their Learning
1.1: Teachers recognize individual
differences in their students and adjust
their practice accordingly
1.2: Teachers have an understanding of
how students develop and learn
q8: "Using a variety of methods, strategies, and
materials to promote development, learning and
cooperation" (3.29--satisfactory);
q5: "Using a variety of methods, strategies, and
materials to promote development, learning and
cooperation" (4.14--good);
Graduate students know about
a wide range of researchbased pedagogies--expert
panel average was 3.7
(satisfactory/well prepared)
q4: "Using knowledge of child/adolescent
development and relationships with students and
families to plan instruction based on abilities,
interests and needs" (3.14--satisfactory)
q1: "Using knowledge of child/adolescent
development and relationships with students and
families to plan instruction based on abilities,
interests and needs" (4--well prepared)
Graduate students know how
to teach based upon what is
developmentally appropriate-expert panel average was 3.8
(satisfactory/well prepared)
q6: "Establishing caring, inclusive, stimulating and
safe environments that facilitate learning,
collaboration, independence, and intellectual risk
taking" (3.0--satisfactory)
q3: "Establishing caring, inclusive, stimulating and
safe environments that facilitate learning,
collaboration, independence, and intellectual risk
taking" (3.0--satisfactory)
Graduate students have
developed a recognition of
professional ethics of their
discipline--expert panel
average was 4 (well prepared)
q19: "Helping students explore important life issues,
and world issues" (2.57--satisfactory)
q16: "Helping students explore important life issues,
and world issues" (3.43--satisfactory)
Graduate students appreciate
how knowledge is linked to
other disciplines--expert panel
average was 3.4 (satisfactory).
1.3: Teachers treat students equitably
1.4: Teachers' mission extends beyond
developing the cognitive capacity of their
students
Implementing Faculty
Recommendations, and Affecting
Learning and Teaching
Ideas for Using Assessment
•
Use information to pursue department or program-level initiatives or projects
on teaching
•
Incorporate assessment information into curriculum discussions and
recommendations
•
Have periodic discussions or reflective discussions on teaching using
assessment information
•
Use for grant applications
•
Use in budget requests
•
Use in fundraising letters or alumni functions
•
Use to gather further information about student learning from instructors on
a periodic basis
Ways to Track Effect of Faculty
Discussions/Recommendations
• Encourage inclusion in annual report (section under teaching
effectiveness)
• Encourage submissions of faculty narratives —how were faculty
discussions and recommendations used in course design, selection
of courses, etc.?
• Track how recommendations were enacted —curriculum process,
department/program initiatives, course sequencing, equipment
requests, etc.
• Not so important to make an academic study out of this, or to make
this too huge of a process, but documentation is helpful when writing
annual reports on assessment, curriculum, & instruction
Future Directions and Vision
• Will ask for report on four questions at end of each
academic year
• Focus is not upon “reporting for reporting’s sake,” but
instead upon how we can assist
• Focus will be upon serving faculty—in helping them in
accomplishing the last two of the four core questions
• Focus primarily upon impact—how has process
impacted learning, teaching, and curriculum processes
Conclusion
• Four foci:
– Objectives
– Assessments
– Faculty Usage
– Impact
All assessment is a perpetual work
in progress. -- Linda Suske,
Unknown , May 3, 2005
In assessment, "the perfect is the
enemy of the good." Let's keep
striving for the good. -- Tom Angelo,
Unknown , Unknown
Questions? Comments?