Transcript No Slide Title

Preparing for QAA Academic Subject Review
Norman Jackson LTSN Generic Centre
•
•
•
•
National infrastructure for improving T&L
LTSN and its relationship to QAA
What is QAA trying to do?
Overview of QAA policies and how they relate
• Programme Specifications
• Subject Benchmarking
• Codes of Practice
• Academic Review
National infrastructure for improving T&L
Capacity
1992
2002
HEQC
QAA
ILT
HEFCE
ESRC
LTSN
research
LTSN is not an arm of QAA
In working with the QAA agenda LTSN sees its role as
* helping HE communities to understand the implications
of QAA policies and make effective use of them
* providing practical advice based on research and exemplification
* promoting an evidenced-based approach to evaluating policy
and its impact
* influencing QAA thinking on the way policies are interpreted
* encouraging HE to be self-critical in order
to learn and improve.
LTSN values
we believes that:
there is value in subject communities trying to understand
the nature of learning in their subject and describing this for
the benefit of the whole community.
there is value in teaching teams
* being explicit about the learning they are intending to promote
* explaining the methods they will use to enable students to learn
* using information provided by the subject community to reflect
self critically on their practice
What is QAA policy trying to do?
HEI purposes
and policies
external reference points
QAA policy framework
INSTITUTION
SUBJECT
NATIONAL
subject
benchmarks
programme
specifications
& progress files
qualifications
framework &
codes of practice
external reference points
Statutory/Prof. Body requirements
Academic Subject Review
INSTITUTION
SUBJECT
programme
specifications
& progress files
self-evaluation
subject
benchmarks
NATIONAL
qualifications
framework &
codes of practice
academic
subject review
external reference points
Statutory/Prof. Body requirements
QAA policies are promoting a consistent language
for the description and assessment of learning
An outcome is simply a result or consequence of an action
or process.
The outcome from a learning process is therefore
a learning outcome.
Learning outcomes are statements that predict
what learners will have gained as a result of learning
Example learning outcome statements
at the end of the degree programme students will
Knowledge & understanding
• have knowledge of the principal features of the English Legal System,
including general familiarity with its institutions and procedures
• show they understand the theory of electronic devices,circuits & systems
Cognitive skills
• be able to select and apply appropriate management techniques to
complex problems, analyse results, draw appropriate conclusions and
present results in an appropriate format
Subject practical skills
• have mastered fundamental synthetic techniques in organic and
inorganic chemistry with practice in related quantitative analytical techniques
General transferable skills
• have demonstrated that they can communicate succinctly orally
and in a variety of written formats
Explicit learning outcomes
curriculum design
& TLA methods
intended
learning
learning
process
assessment
criteria
learning
achieved
Explicit learning outcomes
generic
assessment
criteria?
programme
specification
intended
learning
learning
process
subject benchmark
learning outcomes
learning
achieved
subject benchmark
performance criteria
Programme Specification
Core information
Educational aims
Intended learning
* knowledge/understanding
* skills and other attributes
Teaching, learning
and assessment
methods to enable
student learning
Understanding how ouctcomes are enabled
curriculum building blocks
A
B
C
D
E
F
G
H
TPA
TPA P TPA
P
1
2
TPA
TPA
TPA
TPA
3
TPA
P
4
TPA P TPA
5
TPA
TPA
6
TPA
TPA PA TPA
PA
7
TPA
TPA
TPA P TPA
8
P
etc
T=taught
P=practised
A=assessed
HE Progress File
Transcript
Personal
Records
Personal
Development
Planning
PDP - A structured
and supported
process undertaken
by an individual to
reflect upon their
learning, performance
and / or achievement
and to plan for
their personal,
educational and
career development
goals
enhance the capability of students to learn through reflection
improve information about learning and achievement
PDP is an integrated and strategic process
containing a set of interconnected activities :
• recording (thoughts, ideas, experiences, evidence of learning)
• reviewing (making sense of it all)
• evaluating (judging self and determining what needs
to be done to develop/improve/move on)
• planning (how to achieve objectives or desired change)
• doing
(learning through the experience/learning more
or more effectively because of greater awareness)
• making effective use of knowledge & information about self
There is nothing new about PDP
The process of reviewing (revising) what has been learnt in
order to crystallise newly acquired knowledge and consolidate
understanding and then identify and rectify gaps and
deficiencies is central to effective student learning.
PDP simply captures the intellectual capacities, skills and
behaviours that underlie these processes and uses them to
help people review and evaluate themselves in a structured way.
Possible contexts for PDP
Academic curriculum
academic modules/units
purpose designed modules/units
research projects and dissertation work
academic tutorials
work placement or work experience
study overseas
strategies that promote a commitment to CPD
Work-based / practice-based experience
Extra-curricula activities
student training initiatives
working as a student rep. or SU officer
part-time paid or voluntary work
General characteristics of benchmarking
What is a benchmark?
* reference point
* criterion for measurement
* mark of distinction (excellence)
Benchmarking contains three ideas
* searching for, creating and
understanding the benchmarks
* evaluating own practice, process
results with benchmarks
* changing practice to emulate
or exceed benchmarks
QAA benchmarking


X
X


Subject benchmark statements are intended to embody
* defining principles or essence of a subject
* nature and extent of a subject – ‘subject territory and
programmes included in territory’
* attributes that a graduate in the subject might be expected
to display and demonstrate in terms of the subject
knowledge and understanding : subject skills and other skills
* the criteria that would be used to determine whether a
graduate satisfied the ‘threshold’ standard for the
award of an honours degree in the subject.
QAA expectations for benchmarking
‘Statements are not to be used to supplant wholesale the
learning outcomes of programmes and they are not to be used
as a checklist for reviewers.’
Handbook for academic review pg 7
Para 35. Institutions should be able to demonstrate how
subject benchmark information has been used to inform
decisions about the intended outcomes of programmes, and in
calibrating the overall demands of the assessment framework.
Benchmarking as a reflective tool
benchmark
learning
outcomes
programme
outcomes
benchmark
performance
criteria
generic
assessment
criteria
?
curriculum &
TLA methods
assessment criteria
used within the
curriculum
Possible uses of benchmark performance criteria
1. Checklist for assessment criteria in key final level
units that permit a range of outcomes to be demonstrated
2. Checklist for assessment criteria in compulsory
final level units
3. Checklist for assessment criteria for all final level modules
4. Checklist for assessment criteria in all curriculum units
Code of practice for the assurance of quality & standards
a reference point for HEIs and peer reviewers
Collaborative provision
Students with disabilities
 External Examining
 Academic appeals and student complaints
Assessment of students
 Programme approval, monitoring and review
Postgraduate Research Programmes
Guidelines for Distance Learning
Careers advice and guidance
Work placements
 most important for self-evaluation at programme level
HE minister Margaret Hodge June 15th THES
‘we want to ensure that we have high quality teaching
in all our universities. We need to develop something
that has the respect of all the players.’….
‘Students care a lot about teaching standards.’
‘the focus will be on the student, the removal
of barriers to university entry and raising standards.’
our ambition - 50% under 30s to have access to HE
Blunkett called for a 40% reduction in visits
The HEFCE/UUK position statement proposes
the following visit pattern
* external review of all departments who did
not achieve the threshold performance level
in Subject Quality Assessment.
* external review of a sample of departments/
schools achieving an excellent rating
(using the methodology before 1995) or who
achieved a rating of 3x4 and 3x3pts in the
six aspects using the 1995-2001 methodology.
All departments/schools will be expected to produce
* a self-evaluation describing the characteristics,
strengths and areas for development of the
programmes it offers within each subject area
This will not be published in order to encourage frankness.
* all departments/schools will be expected to publish
(ie make public) programme specifications for the
programmes they offer.
These will be an important source of public information.
Academic Subject Review differs from TQA
1. Underpinned by policy framework (PS, Subject
Benchmarking, Codes, Qualification Framework)
2. Form of peer review process is negotiated
3. sampling rather than universal visiting
4. Main focus on intended and achieved student
learning and how it is enabled
5. Generally, no observation of teaching
6. Judgements not overtly based on points
Like TQA underpinned by self-evaluation
Self-Evaluation Document provides the framework for review
Aims of subject provider
Learning outcomes
Curricula & assessment
Quality of learning opportunities
 Teaching and Learning
 Student progression and achievement
 Learning resources
• Maintenance /enhancement of standards and quality




Appendices
• Programme specifications
• Information about curriculum structures
/collaborative arrangements
A sea change in the balance of
accountability & development
Academic Subject Review will now be a developmental and
improvement-led process (self-evaluation in the light of external
reference points coupled to changes made in response to selfevaluation).
For many departments/schools professional and public
accountability will be within the institution’s own QA procedures
and through the provision of the self-evaluation to QAA reviewers.
For a proportion of departments/schools there will also be an
external accountability dimension to the process.
The goal should be to achieve the first position for all
departments/schools.