Transcript The Bologna Club: - DQP || Degree Qualifications Profile

The DQP line from proficiencies
to assignments to records
Cliff Adelman, Institute for Higher Education Policy
March 28, 2014
What are we going to talk about?
• Language, faculty, IR, and registrars
• How to write proficiency statements under which
student performance can be judged.
• Language signals: statement type, voice, diction
level.
• The line working backwards from assignments, and
the “Ah, hah!” moment for faculty.
• The challenge of recording proficiencies.
• The greater challenge of reporting student
proficiencies.
The form of proficiency statements,
what’s out:
• “Ability.” You don’t know a student has an “ability” to do
something until they do it. “Ability” and “capacity” are
white noise.
• “Awareness.” At best, it’s peripheral consciousness, and
you don’t award degrees based on consciousness.
• “Appreciation.” Tell us what a student does when they
“appreciate” something! Even music conservatories don’t
use the term.
• “Critical thinking,” the championship mush phrase of the
millennium. Describe the cognitive operations of “critical
thinking” and you have true assignment-oriented lead-in
verbs! You can drop the foggy short-hand now!
The form of proficiency statements: no lead
verbs, no assignments! So what’s in, Part I?
• Delineating: categorize, characterize, classify,
define, describe, determine, frame, identify,
prioritize, specify
• Explicating: articulate, clarify, explain, illustrate,
interpret, outline, translate
• Examining: analyze, compare, contrast,
differentiate, distinguish, extract, formulate, map
• Combining: assimilate, consolidate, connect,
integrate, link, synthesize, summarize
I think you begin to see what I mean, so. . .
Want some more cognitive action verbs for
proficiency statements? Try the following
categories:
• Inquiring: experiment, explore, hypothesize,
investigate, research
• Making: build, compose, construct, craft, create,
design, develop, generate, model, shape, simulate
• Operating: administer, control, coordinate, engage,
lead, maintain, navigate, optimize, plan, undertake
• Converging: collaborate, contribute, interact,
negotiate, participate
• Valuating: audit, appraise, assess, evaluate, judge
And so forth. Some of these are stronger prompts
than others; some could appear in two categories.
A lead into noun problems, too: Why is
language important in the proficiency world?
• It signals statement type (competence, discrete
outcome, metacompetence)
• It signals status of the proficiency (possession,
development, goal)
• Voice (declarative, imperative, subjunctive) signals
the student whether fulfilling or exceeding the
performance specified is assumed, required, or
simply desired. This is a matter of intentionality.
• Diction level (on a concrete/abstract continuum) is a
camera lens setting on the breadth, tractability, and
accessibility of the statement’s subject.
All this leads to our biggest noun problems:
“knowledge” and “understanding”
• The terms are too often used as synonyms. No deal!
• Bloom et al’s Taxonomy discards “understanding”
and replaces it with “comprehension,” a
combination of translation, extrapolation, and
interpretation, to which we might add description,
inference, visualization and testing.
• “Knowledge” is too often presented as an
assumption, i.e. that everyone knows what it is. We
feel too free to speak of “advanced knowledge” in
the face of the question, “What might ‘not so
advanced’ mean?”
The principal problem with “knowledge” is
that our language assumes it is something
possessed, a status and not an event. Really?
• It comes out as a tautology, as in “I have
possession of what I possess,” or
• in an example from a UK benchmarking
statement for accounting, “basic knowledge
and understanding is characterized by
knowledge of a topic” I am not kidding!
• “Knowledge,” though, leads to
specifications, and we understand this best
in the disciplines, and see it in Tuning.
Let’s mark, then, what is in “Tuning,” a
process now present on 6 continents
• “Tuning” is a ground-up faculty driven process to
create templates of reference points for student
learning outcomes in the disciplines.
• Then to go on and write student learning outcome
statements and their accompanying assessments
referenced to the template.
• And the disciplines are the locus for both outcome
statements and assignments, a natural by-product
of where faculty are located, where they are trained,
and how they self-identify.
• The result is “convergence,” not carbon copies.
Example of a “reference point” in
business administration: the Firm
• Not merely “the firm,” but the firm as a “valuechain” from procurement to customer service
• And with stops along the way in production (goods
or services), logistics, accounting, marketing, etc.
• In the European Tuning project, representatives
from universities in 15 countries speaking 11
languages agreed to this.
• Did that mean they all went home and taught “the
firm” the same manner with the same weightings?
Hell, no! But it meant they would use the “valuechain” concept in their own sweet ways.
“Tuning USA” first followed this path in
Indiana, Minnesota, and Utah
• You need a state system to pull it off---well, not
always.
• Unlike the Euros & others, we include the
associate’s degree, hence community college
faculty
• Also, unlike the Euros & others, we had students on
every disciplinary team
• Each state picked 2 or 3 disciplines; Indiana and
Utah independently selected history.
• Texas later joined with 4 engineering fields, and
Kentucky came in with 5 disciplines.
Back to the language of proficiency
statements and “knowledge”
While “knowing” is a verb, the position of
“knowledge” in Tuning is an irrevocable noun; the
reference points specify : “knowledge” of a
disciplinary what.
And that’s where the lists begin: legal regulations,
error analysis, transport phenomena, major wars,
poetic forms, auditing principles, and on and on.
Given the lists, the governing verb goes beyond
possession to presentation, e.g. demonstrate,
display, perform, etc. “Knowledge” becomes an
event, not a status.
When you put together all the
elements described above. . .
• You have the tools and forms for writing learning
outcome statements in your field.
• What you don’t have yet, are natural forms of
eliciting from students the behaviors that allow you
to validate whether they have demonstrated
proficiencies.
• But those natural forms are staring you in the face
because you, as faculty, create them every week.
They are called assignments. And they need no
more than tweaking to produce a clear connection
between proficiency objective and its validation.
You don’t have to look far for
generic cases, e.g.
• From the DQP’s bachelor’s-level for quantitative
fluency: “The student translates verbal problems
into mathematical algorithms so as to construct
valid arguments using the accepted symbolism of
mathematical reasoning and presents the resulting
calculations, estimates, risk analyses or
quantitative evaluations . . .”
• You have a hand-out that was a real-world
advertisement, a serendipitous find that we tweaked
to flow from the above proficiency statement. Let’s
talk about it, and see if we hit the “Ah, ha! moment”
And at the associate’s level
• For integrative knowledge, something the DQP does not
leave just for bachelor’s and master’s programs: “Describes
a key debate relevant to [a core field]. . . And shows how
concepts from core field can be used to address the .
…debate.”
• And then the assignment (published in DQP 2.0):
Prepare an exhibit of not more than five discrete 2-dimensional
pieces illustrating the range of chaos in color, drawing on at
least two of the major color theory sources, e.g. Goethe,
Kandinsky, Chevruel, in a 3-5 page catalogue of your exhibit. . .
Well, it’s not a perfect match, but close. There is a real debate
on color.
Similar problems, presented as assignments,
test questions, field projects, etc. can be
developed and offered in many fields
• Not all fields, to be sure, but you need enough of
them to assert that MKCC has a portfolio of
prompts to validate this type of quantitative
proficiency.
• And what you do for quantitative proficiency and
integrative knowledge you can do for all the others
in the DQP, or your version of a DQP.
• And of course an assignment cannot cover 3 or 4
proficiency territories.
Challenges of putting together a
set of touchstone assignments
• A review committee to determine validities
• Decision rules on distribution of qualifying
assignments across disciplines and levels
• Protection against concentrations of qualifying
assignments in two or three curricular areas
• Recognition that assignments change, and
establishing a process for continued change and
review
• Making sure that individual faculty will cover but a
limited number of proficiencies
Let’s be optimistic now, and
assume
• The ASU faculty senate has endorsed its own version of the
DQP; there are 36 proficiencies
• The Review Committee has marked 288 assignments as
DQP validations, with at least 5 for each proficiency, and
scattered across 50 departments/programs
• A separate student tracking analysis group has examined
the records of 1,000 students to make sure each of them
would have encountered all assignment-validating
proficiencies in the course of their undergraduate careers
(talk about a task!). And confirmed that such is the case.
WOW!
What’s missing? And What Does
One Do Next?
• Grades are missing. The DQP process does
not interfere with faculty judgment of student
performance. You can set alternative levels
of judgment, but not issue the judgments
themselves: that’s a faculty prerogative.
• Next step is the establishment of a record
system, its inputs, and its summative Degree
Audit conclusion. So. . .
Consider the following for
record-keeping
• Every student has an electronic template with the
36 proficiencies in boxes
• Faculty responsible for validated assignments enter
judgments (if binary, only checked if satisfied; if
multiple-leveled, then level) with a secure digital
entry
• You decide whether the student can receive
multiple checks in the same box. I prefer that if
faculty see the box previously checked, they don’t
have to duplicate the judgment.
• Credits continue, but in a separate account.
So when do you get a degree?
•
•
•
•
•
•
All 36 boxes checked
Credit requirements met
Residency and recency requirements met
No financial or disciplinary holds
“Opt-out” degree award policy (ask me!)
Registrar validates the whole; certifies the
Degree Audit.
Does this look like DQP “on the
ground”?
• It’s not on auto-pilot after established
• It’s missing provisions for transfers-in from
institutions that have nothing comparable (granted,
ASU transfer-in percentage is low).
• It’s missing key inclusion items for adjunct faculty.
• It opens potential conflicts from faculty whose
assignments are not included as validating.
• It does not answer the question of what ASU could
show to external parties other than the 36
proficiencies. Maybe retired assignments.
But of all the obligations we have in higher
education, our primary responsibility is to
students, and this DQP process is just that!
• The generation of knowledge and local community
economic stimulus can go on elsewhere, but, in our
ball game
• the core operating noun is “student”; the operating
verbs are those of specific actions by students; the
operating results are criteria for the award of
degrees to students. Get the idea?
• There still is dirt here; there are still rough edges.
You can clean it up, sand it, and make it work!