Transcript A Review of the Phenomenon of Social Transformation in

Presented at the Adult Higher Education
Alliance 2010 Annual Conference
Saratoga Springs, NY
Cindy Stevens
Amy D. Rose
Northern Illinois University



Assessment Discussions
Accountability
The Meaning of College Level Learning

Testing

Portfolio Development


The Credit Exchange Model
also called Course Match Model
The Developmental Model




CAEL guidelines
Literature on the mechanics of PLA
Institutional procedures
Actual discipline specific understanding of
knowledge



Faculty members or adjuncts
Specialists in subject matter or have experience
in assessment
“Accuracy and fairness of evaluation including,
validity, reliability, and predictiveness is
largely dependent upon training” of assessors
and on their “ability to remain objective”
(Arnold, 1998, p.65).



Prior Learning Assessment (PLA) is a growing
area
Involves assessment of learning and gauging of
whether this learning is college-level
No clear idea of how different faculty members
think about this process
How do faculty assessors describe their
PLA process?
In what ways do faculty assessors
conceptualize college-level learning?
Basic Qualitative Research
Sample – interviewed 20 PLA assessors
who also were teaching in higher
education
Semi-structured interviews




Average of 20.5 years teaching at the college level
5 hold a masters degree, 1 ABD, and 14 hold a
doctorate degree
6 are adjunct, 4 non-tenured, 1 tenure tracked, 8
tenured, and 1 emeritus
PLA assessment in the areas of social science,
communications, religious studies, science, music,
dance, technology, business, art, and real estate.

There are many ways in which assessors learn PLA
process and assessment.





Formal
Informal
Nonformal
Ambivalent feelings of competence
Participants used the following approaches to
assessment:



External Outcome/Competency Based
Explicit Theory–Based
Implicit Guidelines
“We have always had training and we are responsible for much of the training
given at other institutions. Without this training, there would be even more
confusion about what to do and how to do it.”
“Meetings is an understatement , we have training, we do training, we sometimes
do more of that than we d anything else.”
A few participants did mention attending some kind of CAEL training or
workshop. One mentioned ACE books.
“I spent a lot of time shadowing my colleagues in the beginning.”
“There were like 4 of us maybe so, we would look at them in the
same room. We would ask questions of each other and it helped.”
“We found each other as much as possible to reassure each other
that we were doing this right. We helped each other.”
“We would get together in a group, we’d all read the same essay
and then we’d talk about it and we’d see how much we were
totally on the same page.”
Many assessors sought out informal training to help them.
“My PLA training is basically I trained myself”
“I wish we had training.” “Adjuncts never get training.”
“No training per se.” “Minimal if any and nobody went.”




“Who am I to evaluate anything?”
“I don’t know if I am ever right ever on any of
my assessments – all of them.”
“I fumbled my way through and did so many
of these. Maybe I did it right. I am not sure.”
“…you know there’s a little bit of playing God
in this, in how flexible you are in granting
students credit.”
“Blind leading the
blind”
“Ad hoc”
“I would like to
know someone else
is out there doing
what I am doing”
“Baptism by fire”
“Boldly going
where no one has
gone before”



External Outcome /Competency Based
Explicit Theory Based
Implicit
“I look at it as if somebody took a course on the topic, you know this is their final
exam, like this is an essay that they’re taking on this topic so they have
demonstrated they have learned and because it’s experiential learning it’s not all
that concrete like you have to address theory.”
“Our internal guidelines are basically anything that follows our syllabus; like
whatever they were supposed to be learning in the classroom environment, they
should be showing me in the portfolio.”
“instead of thinking of a 12 or 15 week syllabus with X number of topics on it and
seeing whether the student meets those requirements, if you can think about things
you wish a student would carry away from class and whether they wrote about it,
but it is more does the person know what you would hope someone would know
about this a year or so after taking the course, the kernel of knowledge that is
critical that remains afterwards.”
“I’m much more focused on learning outcomes like I think of it as taking a course.”
“There is no guess work here. We look for what should be known. Much of what is
taught in the classroom is theory and that is what I expect to see within the papers.
My assessment theory is based on them knowing the theory and the people
involved. If there are missing the point then they need to go back and get that
info.”
“I am looking for the degree of knowledge within the paper. The excellence in
writing is not necessarily linked to that knowledge so you have to look. The
information must be there or I will ask for more information.”
“it’s all based on Kolb’s experiential learning model and I look for evidence of the
learning and the circular form of Kolb’s model. They have to show me the learning
in the narration of the essay, but the experience is the real thing that they have to
have and learn from and then apply. If something is missing it is not what I give
credit for.”
In addition to Kolb’s Experiential Learning Cycle, Bloom’s Taxonomy was named
by some participants.


Many faculty have a difficult time expressing
their assessment activities and with producing
a working definition of college-level learning
Tacit knowledge is gained through experience
but faculty have limited recall of this
knowledge
“ I don’t know that I can explain it because it is sort of internally I
can tell with the portfolio essay, it’s just this is what I do, I can
tell.”
“I am comfortable with it. I’ve been doing it forever. I know what I
am doing and I have outlasted many faculty and directors. I can
read between the lines and can tell when learning is there and
when it is not. I consider myself knowledgeable about the field
itself and how PLA fits into college plans.”
“I can look at an essay and I can tell right away that they have
what they need in the essay. I look for key words that trigger
indications that learning has taken place. I don’t need everything
there, I need my key words and then I know.”



“When everything is said and done it only
needs to be a C level paper to get credit.”
“They just need passing level.”
“You just need a C to get credit and no more
and NO less either. I will pass you with
average work because you would pass my
class with average work.”





Difficult to define college level learning
The idea of portfolio development is
predicated on the idea of individual
institutional standards
Yet this boils down to individual standards
This research indicates that individuals are
working from own internal frameworks
While some have external institutional
frameworks, only a few have explicit
philosophical frameworks.




Training occurs in several formats, but mostly
informal and non-formal.
Individuals come out of this experience often
feeling inadequate to the task.
This is not so different from faculty in
traditional courses.
PLA has different meanings to different
individuals, but the task of assessment may not
be that different from others.