Transcript No Slide Title

Assaying Improvement
Lee Harvey
http://www.qualityresearchinternational.com
qualityresearchinternational.com
Introduction
Assaying is defined as:
‘the act or process of testing, especially
of analysing or examining metals and
ores, to determine the proportion of
pure metal’.
The presentation will explore some
issues about the nature of improvement
in higher education: the purity of the
enhanced silver.
qualityresearchinternational.com
Sequence
Interactive presentation
1. Development of higher education: 3
scenarios.
2. The impact of QA on the quality of
higher education qualifications.
3. Transformative learning.
4. Rankings.
5. Conclusion: inter alia rankings inhibit
transformative learning.
qualityresearchinternational.com
1
Positive scenario
• Period of rapid change due to external
forces.
• HE has adapted and delivered.
• Ever more research.
• Innovative teaching: student centred.
• Institutions have more responsibility.
• QA: additional work but has been positive
and resulted in a more transparent
education system.
qualityresearchinternational.com
1
Sceptical scenario
• Massification but not wider access.
• Deterioration in service offered to
students.
• Less one-to-one contact, fewer teaching
hours, less feedback, large alienating
classes.
• Students experience is fragmented.
• Student-centred, autonomous learning is
an excuse for not teaching.
• More research papers of less worth
(because of pressures to publish)
– a plethora of mediocre
journals.
qualityresearchinternational.com
1
Sceptical scenario
(continued)
• Instrumentalism: employment.
• Managerialism:
– ‘professional’ managers, disengaged from
academic practice.
– Human resource, accounting and marketing
departments run universities.
• Academic freedom is under attack.
• Quality assurance is a flimsy charade
pretending that all is well.
• Institutional managers: careerists
preoccupied with league tables rather than
with the academic development in their
qualityresearchinternational.com
institution.
1
Middle way scenario
• Access has improved.
• Class contact time has reduced but
lectures are not good sites of learning.
• Teaching and learning techniques have
improved considerably.
• Modern technology means information is
available in various forms and contact
does not need to be face-to-face.
• Expectations are transparent.
• Cross-fertilisation of ideas is (potentially)
greater.
qualityresearchinternational.com
1
Middle way scenario
(continued)
• Professional managers are necessary.
• The rumour of the demise of the power of
academics is premature.
• Institutions have more autonomy, albeit
tied to QA.
• Effectiveness of QA depends on purposes
and approaches and is a function of
engagement of all stakeholders.
qualityresearchinternational.com
2
Impact of last 20 yrs of QA
• Identifying impact:
– direct or permeable layers.
– ‘gold standard’ or ‘student learning
experience’.
• Gold standard view: despite QA, a degree
is not what it used to be:
– exclusive and different
– whether the exclusivity made it any better
(other than more marketable) is a moot point.
– Implication is that quality of a degree was
‘higher’, implying that students did more in
order to successfully achieve an award.
– This is highly contentious.
qualityresearchinternational.com
2
VC University of Surrey
“…the quality of the top 13%* of graduates
in the HE sector is at least as strong as it
was for the full cohort in the 1970s.
Many senior managers and government
leaders experienced a more elite education
system and may not fully appreciate the
balance between quality and diversity we
see today. The vast majority of the
additional 30% who gain a HE experience
today will benefit, raising the overall
educational attainment of the nation's
workforce significantly.” (Christopher Snowden, 2008)
* 13% of 43% of the cohortqualityresearchinternational.com
going into HE in England
2
Student experience
• Some evidence in support of the cynical
view about alienation of students and poor
face-to-face contact with staff.
• However, teaching and learning
techniques have improved (irrespective of
the role of quality assurance).
• A change in the role of the teacher and in
student-teacher communication (albeit not
always welcome by ‘traditional’ lecturers).
• Little research evidence of impact of QA.
qualityresearchinternational.com
2
Impact of QA: INQAAHE
• Delegates unanimous about +ve impact.
• Periodic reviews:change from one review
to the next
• Metrics show improvement:
•
•
•
•
•
•
Retention rates
Graduation rates
Level of graduate final award
Employment, employability attributes
Entry requirements
Numbers of graduates
• Institutional developments:
•
•
•
•
Quality on the agenda and taken seriously
Quality assurance units inside institutions
Quality-related mission statements
Policies and strategies, changed documentation/on the
agenda.
qualityresearchinternational.com
2
Impact of QA: Stensaker
• Power
– Supports new leaders, institutional image
debate, legitimates role for students etc.
• Professionalisation
– Formalisation/codification, academic/admin
co-operation, fear is gone.
• Permeability
– Demystifies HE, integrates dimensions of HE.
• PR:
– Promote importance of T&L, defence against
ranking lists.
I’ll come back to rankings
qualityresearchinternational.com
3
Transformative learning
• Basis of my work on quality.
• Transformative learning: student is a
participant in an educative process (not
products, customers, consumers, service users
or clients).
• Education is not a service for a
customer but an ongoing process of
transformation of the participant.
• Two elements of transformative quality
– enhancing the participant
– empowering the participant.
qualityresearchinternational.com
3
Enhancing
• A quality education is one that effects
changes in the participants and, thereby,
enhances them.
• A high-quality institution would be one
that greatly enhances its students (Astin,
1990).
• However, enhancement is not itself
transformative, any more than shining the
silver transforms the substance of the
silver object.
qualityresearchinternational.com
3
Empowering
• Giving power to participants to influence
their own transformation.
• Students taking ownership of the
learning process:
– self-empowerment, through increased
confidence and self-awareness.
• Developing students’ critical ability:
–
–
–
–
–
transcends taken-for-granteds
not ‘negative criticism’
questioning established orthodoxy
justifying their opinions
knowledge as a process in which they are
engaged, not some ‘thing’ they appropriate.
qualityresearchinternational.com
3
Developing critical ability
• Developing critical ability involves
encouraging students to challenge
preconceptions, their own, their peers
and their teachers.
• Critical transformative action involves
getting to the heart of an issue while
simultaneously setting it in its wider
context.
• Conceptually shuttling backwards and
forwards between what the learner
already knows and what the learner is
finding out, between the specific detail
and its broader significance, and
between practice
and reflection.
qualityresearchinternational.com
3
Deconstruction and reconstruction
• Transformative learning involves a
process of deconstruction and
reconstruction.
• A core concept identified as a pivot for
deconstructive activity.
• Deconstruction gets beneath surface
appearances.
• Core concept is used to ‘lever open’ the
area of investigation.
• Then a reconceptualised understanding
constructed.
qualityresearchinternational.com
3
Key element in quality
• Transformation learning is more than
adding
to is
a student’s
knowledge,
skills.
The jury
still out on
this.
• Evolution of the way students learn.
• Priority
I’m optimistic
to the transformative
that new approaches
notion of
quality.
A critical
transformative
that reinstate
trust
and focus onview
asks
how learning,
does quality
help
student
suchassurance
as in
transform
Scotland, the
will conceptual
impact on ability and
self-awareness
the student.
transformativeof
learning.
• So, for me, the criterion for evaluating
the
higher education
Butchanges
there is in
a monster
lurking inand
thethe
impact of quality assurance is: “has
wings!
transformative learning been enabled or
encumbered?”
qualityresearchinternational.com
4
Rankings
• provide easily interpretable information on
the standing of higher education
Link to quality is naïve. The construction of
institutions;
indices by which institutions or
•
stimulate competition
institutions;
departments
are ranked among
is arbitrary,
•
contribute and
to debates
about
the definition
inconsistent
based on
convenience
of ‘quality’ in higher education.
measures.
The operationalisations of the concept of
quality is cursory at best and nearly all
focus on just one notion of quality, that of
excellence.
qualityresearchinternational.com
4
Critiques of ranking
• Selection of indicators: no theoretical
In the THES ranking:
reflection, convenient data, bias.
Osaka, Japan went from position 69
• Weighting of indicators: no evidence of
in 2004 to 105 in 2005 and back to
theoretically underpinned weightings; no
70 in 2006.
consistency in weights between different
Ecole Polytechnique,
France, moved
ranking
systems.
from position 27 in 2004 to 10 in
• Reliability: newspaper publishers may like
2005 and to 37 in 2006.
the drama of significant changes in league
The University
Geneva
went
from
position
year onofyear
but this
hardly
not being ranked
2004,
to position
accurately
reflectsin
real
changes
in
88 in 2005 to
position
39 in 2006
institutions,
which
are rarely
so dramatic
(RFSU, 2008)
as to be noticeable within 12 months.
qualityresearchinternational.com
4
Critiques of ranking
• Statistical insignificance: rankings often
based on minute, statistically insignificant,
differences in scores.
• Institutional, political and cultural
environment: the context affects how
institutions operate and what they can do.
These contexts rarely borne in mind when
compiling international ranking lists.
• Focus of attention: rankings of institutions
treat them as homogeneous when there is
huge variability within institutions.
• Competition: ranking exacerbates
qualityresearchinternational.com
competition between
institutions
4
Impact of ranking
• Rankings have had an impact far beyond
that which their arbitrary design would
warrant.
• That is why rankings are important and, in
their current form, dangerous.
• A ranking position in a league table is a
statistic easily banded about by politicians
and university senior managers as well as
by teachers and union representatives
when it suits.
qualityresearchinternational.com
4
Example
• University of Malaya: dropped 80 places in
the THES rankings without any decline in
its real performance due to definitional
changes.
• This resulted in a replacement of the ViceChancellor and embarrassed the
university, which claimed in an
advertisement two months shy of the
2005 THES results, that it strived to be
among the 50 best universities by 2020.
• (Thakur, 2007, p. 89)
qualityresearchinternational.com
4
Meaningful?
• The Shanghai Academic Ranking of World
Universities
• Times Higher Education Supplement World
University Ranking.
• UNESCO initiated the International
Ranking Expert Group (IREG), which
adopted the Berlin Principles on Ranking
of Higher Education Institutions
• Participation: Asiaweek ended but
Maclean’s continues.
qualityresearchinternational.com
4
Stakeholder concerns
• Concerns despite the Berlin Principles
– Ministers meeting in Tokyo in 2008 concluded
that the ‘bias in the information base of
existing rankings towards research outcomes
could detract from efforts to improve
educational performance’ (OECD, 2008).
– Quality Agency (Stella and Woodhouse, 2006) argued
that ranking contravenes a fitness-for-purpose
approach and judges against a set of generic
criteria, which are harmful to institutional
diversity.
– The European Students’ Union (ESU, 2006) objects
to the elitism generated by rankings and claim
that they do not really inform students.
qualityresearchinternational.com
4
Stakeholder concerns
(2)
– Lecturers at the Higher Education and
Research Standing Committee in Oslo
considered ranking ‘irreconcilable with the
principle of equity’, ignore cultural context,
increases marketisation and most importantly:
“The pressures of the outcome of ranking
systems also deviates the attention of leaders
of HEIs from the students and the genuine
purpose and mission of HE… There is a real
risk that higher education institutions focus on
efforts to climb up the ranking ladder, ignoring
their mission in developing and disseminating
knowledge for the advancement of society.
Furthermore, ranking places too much
emphasis on institutions...”
(Education International,
qualityresearchinternational.com
2006, p.1)
5
Conclusion
• Rankings provide a threat to quality
processes.
• Rankings (despite poor construction) have
popular appeal and more credibility
amongst political decision makers than the
meticulous hard work of quality agencies.
• Rankings do not appear to ‘reward’
teaching.
• Rankings place a potential brake on the
development of critical transformed
learners. Developing a critical education is
not a way to move up league tables.
qualityresearchinternational.com
Thank you
[email protected]
www.qualityresearchinternational.com/glossary
qualityresearchinternational.com
Glossary
Analytic quality glossary
www.qualityresearchinternational.com/glossary
qualityresearchinternational.com
Impact
Quality
Assurance
Agency
action
Effect in
institution
qualityresearchinternational.com
Impact: permeable layers
QAA
action
QAA
action
QAA
action
QAA
action
National policies
External effects
Sector changes
Funding, expansion, teaching development
Effect in institution
qualityresearchinternational.com
4
Altbach (2006)
“Publication counts often stress established
refereed journals …published in English and
selected with the norms of the major academic
systems of the US and Britain in mind…. Using
international recognition such as Nobel Prizes as a
proxy for excellence downplays the social sciences
and humanities…and causes further disadvantages
for developing countries and smaller universities…
Using citation counts… emphasize material in
English and journals that are readily available in the
larger academic systems. It is well known, e.g.,
that American scientists mainly cite other
Americans and tend to ignore scholarship from
other countries. This may artificially boost the
ranking of US universities… (Altbach, 2006)”
qualityresearchinternational.com
• Teaching and research is about
inspiring creating motivating. Producing
cardboard boxes is about fine tuning
machines. Should we avoid the new
public managerial practices of lean
production, mbo, etc. or can we
improve through professional
management?
qualityresearchinternational.com
• How do you get academics to engage
with quality?
qualityresearchinternational.com
• How do you implement this in practice?
qualityresearchinternational.com
• Academics can relate easily to ideas
about critical thinking - what they don't
or won't do is link to QA/QE
qualityresearchinternational.com
• The QA documentation has improved
but individual academics still not highly
engaged with the process. Cynicism
increase with cost of audit!
qualityresearchinternational.com
• Academics will and do engage with
quality - they get turned off by form
filling and the administrative burden it
brings when there are a range of other
time demands.
qualityresearchinternational.com
• [High] fee-based h.e. inevitably
emphasizes the contractual nature of
the institution - student relationship.
Students and parents demand 'quality' how do we deal with this?
qualityresearchinternational.com
• Will it be possible to make progress
with transformative learning while
teaching is not sufficiently rewarded
and not viewed as important as
research?
qualityresearchinternational.com
• Management have forced our institution
to change our quality improvement
system, turning it into a useless tool.
We resisted but in the end we had no
choice! How do we tackle this and still
keep our jobs?
qualityresearchinternational.com
• We can't ignore them but MUST engage
strongly in the public debate - as we
have VCs silly enough to include
ranking in mission & strategic goals.
qualityresearchinternational.com
• How do you evidence improvement in
your definition of quality ie how to
measure students improved conceptual
ability + self-awareness?
qualityresearchinternational.com
• If you can't beat it, join the uni
management. Revolution from within!
qualityresearchinternational.com