Transcript Modeling E-M Fields in Complex Domains

ABET Assessment
One Department’s Experience
Paul H. Schimpf
penguin.ewu.edu/~pschimpf
Eastern Washington University
Assessment Colloquium
April 26, 2013
Caveats
 This represents just one person’s experiences
and opinions
– I am not an assessment specialist, nor do I wish to be
– feel free to disagree
– debate assessment strategy and methods in your
units
 We are all assessment experts
– most of us have probably spent more time thinking
formally about student assessment, as opposed to
course or program assessment
– but most of us DO perform course and program
assessment informally all the time
 There are many ways to successfully go about
assessment
Some Guiding Principles
 Make it sensical and meaningful
– I personally would rather see a refusal than a fake
 Make it flexible
– a valid outcome of assessment is making changes to
assessment (it’s a meta process and can be applied to itself)
 Minimize the workload
– try to use data that you are already collecting
– data analysis will add some work, so keep it flexible
– don’t forget that we have that little item called “Service”
 Empower the faculty
– they are likely assessing their coursework already
– don’t force them to change what they do, just ask them to
document it with as little impact on them as possible
What is Assessment?
 Tests and Grades constitute assessment, right?
– yes, but they focus on assessment of students
– do not provide the data that we are looking for in this context
 Surely all faculty members, while grading an
exam, have encountered this situation:
– “Hmmm, very few have answered this particular question
correctly. Even many that I think are my better students got
it wrong, but this is an important concept.”
– “Perhaps I didn’t cover this clearly? Maybe I should try
covering this again, with a quick review of a pre-req, or
perhaps from a different angle, or with different examples
that start with something they may be more familiar with.”
 Or maybe this situation:
– “Everyone got this correct. Maybe that means I can trim out
a little time spent on it in order to cover this other concept
I’ve been wanting to add to this class.”
What is Assessment?
 Whenever a faculty member is refining the content
or delivery of a course, assessment is happening
– It is about whether the conduct of the course could or should
change. This creates one possible avenue for continual
improvement.
– It is about specific objective(s) of the course. Grade statistics for
the entire course do not provide this information, nor do grade
statistics for an entire exam (unless perhaps the exam covers
only a single objective of the course).
– It is about whether students are adequately mastering or
understanding some objective. An average numerical score
across students may obscure that a few did very well, and a
whole lot more performed inadequately.
– It is more powerful to know what percentage of students scored
“adequately” than to know the average score.
What do you expect ME to do?
 “Most of my students failed this. What am I
supposed to do about it? I just got a bunch of
crappy students.”
– That is one possibility, but is it the only possibility?
– Is this a one-time occurrence or a recurring theme?
– Is this with regards to something extremely fundamental to
the course, or something esoteric?
– Is this a basic concept, or an advanced concept?
– Are you assuming any pre-requisite knowledge that isn’t
explicit in the course pre-reqs?
– Is it possible that these students came out of a pre-req
without the knowledge you assumed?
– Have you tried taking another approach?
Waste of Time?
 “I don’t believe anyone reads this and I don’t get
any feedback”
– “I thus conclude that it is a waste of time”
 First and foremost, the reports DO need to be read
by someone in your unit
– if the department is not interested in continual improvement
against its own objectives, then it is indeed meaningless
 Don’t expect detailed feedback from an outside
assessment enforcer
– unless you aren’t doing any at all
– or are not demonstrating that you are using assessment as part
of a process for continual improvement
– or are making claims in your assessment report that look
suspicious
Who Closes the Loop?
 Be careful what you wish for!
 Where do the domain experts reside?
 Who are the most important evaluators of
assessment reports?
 Who should recommend changes to programs,
course structure, or assessment practices, based
on assessment data?
 Do you want to put that into the hands of a nonexpert in your domain?
 Close the loop yourselves, and document it so
others can see that you are doing so.
How to Get Started
 I describe a “top down” approach
– but a bottom up analysis of program outcomes that existing
course objectives lead to might also be useful
 Start by defining some meaningful outcomes for
your program
– things that you want your students to understand, or be able to
do, or be prepared for, by the time they complete
– you will need to think about measurability, but at the program
level you need not worry too much about that
 Exam the various events that students go through
to identify possible points of assessment
– many assessment opportunities will naturally fall into the
classroom, but those are not your only opportunities
Some Assessment Opportunities
 Professional or Other Terminal Exams
 Advancement Exams
 Senior Capstone Presentations
– you can have observers rate various aspects of
performance using a rubric




Internship Reports
…?
and yes, coursework
Your objective is to assess each program
outcome in at least one place
– two places would be better …
A Suggestion for Courses
 With the help of the course instructor(s)
– create some specific learning objectives that are appropriate
for the course, and …
– that support (are demonstrations of) some program outcome,
– and are stated as something measurable
 IMPORTANT
– make sure it is understood that these are not necessarily the
only objectives of the course
– an instructor can add other objectives, and can choose to
assess them for personal use or not
 Example learning objectives
– students will know how to titrate an appropriate amount of
ethanol, …
– students will know how to program responses to various GUI
events, such as button presses, menu events, …
Empower the Instructor
 Create a simple AND flexible report form
 State the agreed-upon learning objective AND
the program outcome that it supports
 Allow the instructor to explain how they assess
that and what the results are in whatever form
they wish
 Allow the instructor to explain any conclusions
they drew from their assessments and any
actions they took or suggest to the department
CS Course Assessment Form
…
IMPORTANT
 It is NOT a bad thing to report…
 That an objective was not met
– hopefully some thoughts and suggestions on the matter
are offered
 That an objective was not measured because
the instructor feels it is an inappropriate
objective for the course
– hopefully this will lead to a discussion amongst the
course instructors as to appropriate objectives
 That all objectives were met
– this is the end goal after all
Other Suggestions
 Create an Assessment Plan, and Follow it
– It should list your program outcomes
– It should explain where your assessment points are,
which outcomes they assess, and what manner of
assessment is used
– It should explain the process you follow, including:
•
•
•
•
how often you perform each assessment
how you process the data
what documents are produced and how they are processed
how long it takes to cycle through assessment of all of your
program outcomes
– and perhaps more
• the CS plan also covers “Educational Goals” and how they tie to
“Program Outcomes”
• the CS plan also explains how our Program Outcomes relate to
ABET “attributes” of program outcomes
CS Assessment Process