Transcript Document

External
Examiners’
Briefing Day
26 February 2015
Presenters
Mr Derek Milligan, Director of Academic Programme
Development
Dr Amanda Harvey, Associate Dean (Quality Assurance),
College of Health and Life Sciences
Dr Nicholas Holland, Senior Assistant Registrar
Professor Mariann Rand-Weaver, Pro Vice-Chancellor for
Quality Assurance and Enhancement
Brunel University London
Briefing outline
• Overview of External Examiner role
• Scrutiny and external moderation
• Panels and Boards of Examiners
• Programme and Award Regulations
• External Examiner annual reports
Brunel University London
Role overview
Appointment
• External Examiners are appointed to assist the University
with the quality and standards assurance of its
programmes
• Senate Regulation 4 governs the appointment of and role
of External Examiners
• At least 1 External Examiner per award bearing
Programme
• Appointment normally for 4 years but may be extended by
one further year
• External Examiners nominated by Heads of Department
and appointed by Senate
Brunel University London
Main Functions
 Scrutiny and external moderation of summative
assessments
 Involvement in Panels and Boards of Examiners which
confirm grades and awards
 Other meetings with staff/students as agreed (e.g. project
presentations)
 Annual Report to the VC on academic standards,
assessment processes, student performance,
comparability of overall standards with those of similar
programmes in other UK HEIs
Brunel University London
Brunel’s Regulations
All published at
www.brunel.ac.uk/about/administration/university-rules-andregulations/senate-regulations
 SR2 – Undergraduate programmes (Levels 1, 2, 3)
 SR3 – Postgraduate Taught programmes (Level 5)
Note that “levels” are not the same numbers as FHEQ.
 SR4 – Assessment Regulations
Brunel University London
Scrutiny and
external moderation
Brunel University London
Scrutiny of Summative Assessment Tasks (SR4.3-4)
 All summative assessments are subject to the approval of a group
designated by the appropriate Panel of Examiners.
 External Examiners' views should be sought on all examination papers
and other assessments (e.g. proposed coursework titles) at Level 2, 3
and Masters Level, before the papers are printed or other assessments
notified to students. Assessment criteria, marking schemes, and/or
indicative answers should be provided in order that he/she may consider
whether the questions:
a) are expressed clearly;
b) adequately assess the intended learning outcomes;
c) are in accordance with the overall assessment strategy for the
programme;
d) are of an appropriate standard.
 Changes proposed by the External Examiner under Regulation 4.3
should only be rejected with good reason after discussion with him/her.
Brunel University London
External Moderation of Summative Assessments
 Samples of assessed work made available together with full
set of provisional grades
 Not required to study each piece of work in detail
 Is the work of appropriate standard and comparable with
other UK HEIs?
 Do grades/marks assigned reflect the marking criteria and
published grade descriptors?
 Findings feed into Panel of Examiners consideration and
confirmation
Brunel University London
External Moderation Arrangements
• Samples of student work either sent prior to the Panel of Examiners
or available on the morning of the Panel meeting (by agreement)
• Student Work Samples and Information provided:
o 10% sample size
o Indication of where adjustments already made following internal
moderation
o Samples provided should represent full range of achievement
o 20% of Masters dissertations or all if fewer than 10 students
• All student work should be available on day of Panel of Examiners
Brunel University London
Recent strategic
developments at Brunel
New College/Department structures
• Brunel implemented a ‘transformational change’
programme (TXP) in 2014, re-structuring from 8 smaller
Schools into 3 large Colleges
• Each College has between 2 to 5 Departments
• Larger Departments have ‘Divisions’ for different subject
area – your appointment as an External Examiner is
likely to cover a large slice of a particular Division
Brunel University London
Colleges
• Business, Arts and Social Sciences (CBASS)
• Engineering, Design and Physical Sciences (CEDPS)
• Health and Life Sciences (CHLS)
Each is led by a Dean, supported by Vice-Deans Education
and Associate Deans Quality
Brunel University London
Other developments and activities
• REF results 2014
• Brunel is currently developing a new Education strategy,
with related Portfolio strategy and International strategy
• Brunel will also undergo QAA Review next year
 Our submission is due mid-Dec 2015. Your reports
from this summer will form a key tranche of evidence.
 Review visit will be mid-March 2016.
Brunel University London
Input from External Examiners
• Brunel greatly values your work and considers and acts
on your reports
• We also welcome any feedback about today’s event
Brunel University London
Panels and Boards
of Examiners
Panels and Boards of Examiners
 Panels and Boards are responsible directly to Senate
 Senate Regulation 4 governs Assessment, Panels of
Examiners, Boards of Examiners, Mitigating Circumstances
Panels and External Examiners
 Purpose is to ensure:
o Fairness to all students and each student
o Rigour of assessment
o Robustness of the results and decisions
o Maintenance of academic standards
Brunel University London
Panels of Examiners (PoE) – purpose
 To verify and confirm the marks/grades for each
module/block
 Not to look at individual students (unless error or additional
info presented)
 May make specific recommendations to Boards concerning
particular assessments
 Implement Senate Regulations 4.48-4.62
 Mitigating Circumstances Panels are different – see later
 Panels of Examiners are where collective academic
judgement is employed and EEs have a vital role to play
Brunel University London
Issues for Panels of Examiners
 Potential assessment design errors
 Academic judgment
 Moderation effectiveness and issues
 Extraordinary factors/events
 Re-marking/adjustment of results (see below)
 Justification of actions
 Recording of decisions at all stages!
Brunel University London
Panels of Examiners - Senate Regulation 4.60
“A Panel of Examiners shall not confirm grades/marks for an
assessment block until it is satisfied with the integrity and fairness of the
assessment(s) leading to the grades/marks. Where the Panel of
Examiners has insufficient confidence in the integrity and fairness of the
outcomes of an assessment, it shall take appropriate action in order to
achieve sufficient confidence. The Panel of Examiners may require the
reconsideration by assessors of the grades/marks for the complete
cohort of students taking an assessment. Only in exceptional
circumstances may the Panel directly adjust the grades/marks for a
particular assessment and must then must record the justification and
rationale for the adjustment. Grades/marks for an individual student may
not be adjusted, unless they have been wrongly recorded or additional
information is presented.”
Brunel University London
Role of External Examiners at Panels
 Full membership of the Panel of Examiners
 Contribute to the collective academic decision-making
 Have a distinct influence on the PoE (through their
independence and their overview)
 EEs are not empowered to change marks or grades –
all decisions are made collectively
 May ask to inspect any documents involved in the
assessment process
 See SR4.117 – 4.123
Brunel University London
Academic Judgement
 The vast majority of the collective academic
judgement is exercised in PANELS, not in Boards,
of Examiners
 Boards are more about judgement by academics,
rather than academic judgement – but there may be
extraordinary academic judgements needed
Brunel University London
Boards of Examiners (BoE) – purpose
 To receive confirmed marks/grades from Panels of
Examiners
 To consider individual student profile of achievement
 To take into account mitigating circumstances and determine
appropriate actions
 To decide on progression and re-assessment
 To recommend to Senate appropriate awards for each
completed/withdrawn student
 Implement Senate Regulations 4.48-4.54 and 4.63-4.76
 NB Boards now have few powers of discretion within the
regulations – but all decisions must still be justified
Brunel University London
Role of External Examiners at Boards
 Full membership of the BoE
 Contribute to the collective academic decision-making
 Have a distinct influence on the BoE (through their
independence and their overview)
 EEs are not empowered to change marks or
grades/progression/classification unilaterally – all
decisions are made collectively
 Assure themselves that due process has been followed
and appropriate consideration and decisions have been
made by the BoE
 See Senate Regulations SR4.117 – 4.123
Brunel University London
Vice-Chancellor’s Representative (VCR)
 Experienced member of the University Administrative staff –
normally from Registry
 Most BoEs have a Lead VCR and an Assistant VCR present
 BoE may not conduct business resulting in recommendation of
an award in the absence of the VCR
 VCR ensures accurate recording of BoE’s decisions and
transmission to students
 VCR also (i) ensures proceeding of a BoE are carried out in
accord with Senate Regulations and approved programme
specification(s), (ii) reports to the Academic Registrar any
irregularities, and (iii) reports to Academic Registrar where an
External Examiner has expressed dissatisfaction with the
proceedings or a decision of the BoE.
Brunel University London
Issues for Boards
 Mitigating circumstances
 Progression and re-assessment
 Awards
 Extraordinary decisions – discretion
 Justification of decisions
 Recording of decisions at all stages!
Brunel University London
Board Discretion
Limited scope:
 Action re Mitigating Circumstances
 Re-assessment timing
 Extraordinary situations
Also note: for treatment of AP(E)L, new processes
obviate need for BoE discretion
Brunel University London
Mitigating Circumstances Panels
 Report to Board of Examiners
 Consider MC submissions from students
 Have no access to academic results when considering
cases
 Consider the likely impact on studies/assessment
 Accept or reject MCs
 Make decisions directly re coursework submission
deadlines with MCs
 All other accepted MCs -> Board of Examiners
 MC Panel will identify serious cases to the BoE
Brunel University London
Mitigating Circumstances rules
 SR4 governs treatment of MCs + see Guidance on MCs
 A serious or significant unforeseen event, and its consequences,
which have significantly impaired the academic performance of a
student in one or more assessed activities.
 BoE actions taken in response to accepted MCs should ensure that
students are as little disadvantaged as possible as a result of the
impairment which occurred, while also preserving the integrity of
standards and awards.
 Potential BoE actions outlined in SR4.43-44
 Default actions are uncapped reassessment or, where confident and
appropriate, assignment of grades
 Extraordinary actions must be agreed with Chair of Quality Assurance
Committee
Brunel University London
Programme and
Award Regulations
Programme and Award Regulations
(SR2 and SR3)
Principles –
• Transparency of standards of awards
• Equity of treatment
• De-modularised Regulations (Assessment Blocks &
Study Blocks)
Note existence of ‘old’ Regulations – may still apply to
some trailing students
Brunel University London
Grades
• 17-point grade scale (A*,A+,A,A-,…F) with % mark
equivalence defined
• University Grade Descriptors are primary reference for
academic standards
• Grades are primary performance measure
Brunel University London
Brunel’s 17-point Grade Scale
Indicative Mark
Band
90 and above
80-89
73-79
70-72
68-69
63-67
60-62
58-59
53-57
50-52
48-49
43-47
40-42
38-39
33-37
30-32
29 and below
Non-submission
Brunel University London
Degree class
equivalent
1
1
1
1
2.1
2.1
2.1
2.2
2.2
2.2
3
3
3
Fail
Fail
Fail
Fail
Fail
Grade
Grade Point
A*
A+
A
AB+
B
BC+
C
CD+
D
DE+
E
EF
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
NS
0
Progression and Award rules
• Defined minimum grade profile requirements for progression and
threshold awards
• Classification determined by simple GPA requirement
• “Borderline” mechanism is strictly deterministic (no BoE discretion)
• No compensation/condonation mechanisms other than strict
borderline mechanism
• Credit is not “awarded” – credit is solely a volume/weighting metric
• “Core” assessments can be defined (must be passed at threshold
level)
• Standard reassessment rules across University – reassessment
volume right = limit
• No Trailing allowed (except for some cases of MCs and with individual
Senate approval)
Brunel University London
Core assessments or assessment blocks
 The designation ‘core’ indicates an element of assessment
or an assessment block in which a grade of at least D- (UG)
or C-(PGT) must be achieved in order that the student may
progress/be awarded
 Core status of any blocks/assessments must be defined in
the programme specification
 All Masters dissertations and all UG Final Year Projects
must be designated as core
Brunel University London
Grading and Marking at point of assessment
 Departments define whether they are using mark or grade
ENTRY at element level (by programme)
 All calculations and requirements for award are GRADEBASED
 SITS (student records database) is set up with details of
assessment elements and weightings
 Grade entry – SITS assigns a grade point for each element
and calculates weighted average of grade points.
 Mark entry – SITS finds weighted average of marks –
assigns corresponding grade point for assessment block
Brunel University London
UG Integration of Sandwich Placements
(SR2 2009-, 2.46-48)
• Special award name (defined in the Programme Specification)
available for sandwich students who have passed (D-) the
placement assessment.
• For this special award the placement module grade is included
in the Level 2 grade profile for award.
• Percentage contribution to Level 2 of the placement module is
defined in the Programme Specification.
• All students who have failed the placement module (or who
choose not to have it included in the award grade profile) are
eligible for the standard (non-sandwich) award.
• 2014/15 is third year of awards with integrated sandwich
placements
Brunel University London
Final Awards –
Classification mechanisms from June 2015 onwards
 Both UG and PGT degree classification mechanisms
simplified for all awards from June 2015
 Previous complex classification mechanism replaced by a
simple GPA requirement for each classification
 Previous complex (GPA – grade volume) mechanism
relegated (with a refinement) to be deterministic “borderline
mechanism”
 Sub-threshold grade constraints remain unchanged
 No student can be disadvantaged by the stepped borderline
mechanism (GPA thresholds have been refined and
additional steps defined) nor disadvantaged by the change
of classification mechanism
Brunel University London
UG Final Awards – Honours Bachelors
 Set out in SR2 Appendices
 Based on Level 2 and Level 3 Weighted Grade Profile (1:2 L2:L3
weighting of credit)
Class
Maximum
credit volume
Maximum of assessment
credit
blocks
volume of
containing
Grade F
core
assessments
below D-
1st
2.1
2.2
3rd
0
0
0
0
0
0
0
0
Maximum (non-weighted) credit
volume of non-core Grade Band E
(E+, E, E-)
Level 2
Level 3
Level 2 + 3
0
20
40
40
0
0
20
40
0
20
40
60
Minimum
weighted
GPA*
14.0
11.0
8.0
NA
 Requirements for Other UG Awards (Integrated Masters, Ordinary,
DipHE, CertHE) are also set out in SR2 Appendices
Brunel University London
PGT Final Awards – Masters
 Set out in SR3 Appendices
Class
Distinctio
n
Merit
Pass
Maximum
credit
volume of
Grades E+,
E, E-, F
Maximum credit
volume of
assessment
blocks
containing core
assessments
below C-
0
0
0
Maximum (nonweighted) credit
volume of noncore Grade Band
D (D+, D, D-)
Minimum
Taught Part
Weighted
GPA*
Minimum
Dissertation
Grade
0
0
14.0
A-
0
0
0
30
11.0
NA
BC-
 Requirements for PGDip and PGCert Awards are also set out in
SR3 Appendices
 If dissertation can be included in the profile for a PGDip award,
this must be set out in programme specification
Brunel University London
UG Final Awards New Borderline Mechanism
(PGT very similar)
•Set out in SR2 (and SR3 for PGT) Appendices
Class
Volume of grades (weighted by assessment block credit
value and by Level weighting) in Class or better
1st
2.1
2.2
Minimum Weighted GPA
At least 41% of grades (weighted) in Class or better
13.5
10.5
7.5
At least 45% of grades (weighted) in Class or better
13.0
10.0
7.0
At least 50% of grades (weighted) in Class or better
12.5
9.5
6.5
At least 54% of grades (weighted) in Class or better
12.0
9.0
6.0
At least 58% of grades (weighted) in Class or better
11.5
8.5
NA
At least 62% of grades (weighted) in Class or better
11.0
NA
NA
At least 66% of grades (weighted) in Class or better
10.5
NA
NA
Brunel University London
Entitlements to reassessment (UG)
 Defined limits of, and rights to, reassessment.
 Not at discretion of Boards of Examiners (unless MCs)
 Entitlement to reassessment in each Level up to a maximum
of 60 credits (Level 1) / 40 credits (Level 2, 3 and 5) where:
Student fails to achieve D- (Level 1, 2, 3) /C- (Level 5) in a
block or a core assessment, or fails to achieve a pass in
pass/fail assessments
 Grades for reassessments capped at D- (Levels 1-3) / C(Level 5)
Brunel University London
Entitlements to reassessment (PGT)
 Defined limits of, and rights to, reassessment.
 Not at discretion of Boards of Examiners (unless MCs)
 Entitlement to reassessment in taught part
assessments up to a maximum of 60 credits where:
Student fails to achieve C- in a block or a core
assessment, or fails to achieve a pass in pass/fail
assessments
 Grades for reassessments capped at C- (threshold
grade)
Brunel University London
18 July 2015
Resubmission of Dissertation
 Entitlement to revise and resubmit dissertation if
not achieved C- in first attempt
 Student must have achieved a grade of E- at the
first attempt – otherwise resubmission not
permitted
 Grades for reassessment capped at C- (threshold
grade).
Brunel University London
Presentation Title
45
External Examiner
Annual Reports
Report submission
•
All External Examiners submit a report annually to the
Vice-Chancellor – by 31st July each year
•
Request for reports and reminders sent out by the
Quality and Standards Office
•
Pro-forma/web form provided for the report
•
Fees paid once EE reports are received
•
You will be sent a formal response detailing how the
University is following up if you raise any concerns
•
External Examiners may also write in confidence to the
Vice-Chancellor at any time during their period of
appointment about any matter relating to their duties
Brunel University London
Response and publication
 College required to respond to the comments made
in the External Examiners Report both directly to the
External Examiner and in their annual monitoring
reports
 University will provide a central response to broader
issues, particularly regarding regulations
 External Examiner reports and University responses
published to current students and staff. Web page
also available to External Examiners.
 Fees paid once EE reports are received (reminder!)
Brunel University London
Your questions?
Thank you