Transcript Quality Assurance Changes AIVs Forum Jan 2013

Centre Quality Assurance
Briefing for AIVs January 2013
Agenda topics
1. The Regulatory Framework
2. Centre Monitoring – risk and sanctions
3. Direct Claim Status
4. Impact on AIVs
5. Pre-issue verification of assessment tasks
2
1. Regulatory Framework
QAA
Ofqual
• Regulates Access to HE
Diplomas
• Regulates other
qualifications
• Licenses Access Validating • Licenses Awarding
Agencies (AVAs)
Organisations (AOs)
• Requires AVAs to regulate • Requires AOs to
Centres
regulate Centres
• Regulations stable for last • Regulations changed
few years
in May 2011
3
1.1 OFQUAL’s role
• ensure assessments and qualifications are
reliable and consistent over time
• promote public confidence in regulated
qualifications and assessments
• secure efficiency and value for money
4
1.2 OFQUAL Regulations
• Introduced May 2011
• More stringent than previous regulations
• General Conditions of Recognition
– apply to recognised Awarding Organisations
• Criteria for Recognition
– apply to AOs seeking recognition
5
1.3 Impact of Conditions on
Centres
• Directly – none
• However Conditions affect:
– the way AOs regulate Centres
– policies and procedures required
– how qualification and assessment standards
are set and maintained
– quality assurance arrangements
6
2.1 Centre Monitoring: Risk-Based
Approach
• Each Centre allocated to a risk band based on:
– risk rating for the qualifications at the Centre
• as indicted by Ofqual
• as indicated from EV reports
– outcomes of annual monitoring visits
– Centre’s record of responding to action
• Risk band re-assessed at each visit
7
Banding
Score
Colour
code
General descriptor
Priority of actions
required
No action required, or
suggestions towards
improvement or best
practice
Low Risk
1
Green
Little or no risk to integrity of
assessment, qualifications, Centre
approval criteria, regulatory
Conditions and/or reputation of
OCNYHR or NOCN. Centre
performance is good.
Marginal
Risk
2
Yellow
Any risk to integrity is marginal.
Centre performance is satisfactory.
Suggested action
Amber
Concerns about a specific risk
indicator or indicators. Customized
or specific action required to ensure
integrity. Sanctions may be
imposed.
Action is required
Red
Major concerns about one or more
risk indicators which threaten
integrity. Urgent action required.
Sanctions may be imposed.
Immediate and urgent
action is required.
Moderate
Risk
High Risk
3
4
8
2.2 Risk Indicators
Table of Risk Indicators: Indicators 1, 2 and 3 are important for AIVs
1
Centre uses valid, reliable, fair and safe assessment methods and
tasks. Clear, comprehensive assessment guidance for assessors.
2
Assessment decisions made against unit learning outcomes and
assessment criteria at appropriate level, so evidence is valid, authentic,
sufficient and current. Clear and constructive feedback to learners
explains assessment decisions and helps them improve.
3
Robust internal verification, including pre-issue verification, confirming
validity and consistency of assessment tasks and decisions, constructive
feedback to assessors, appropriate standardisation, accurate records.
4
Centre provides appropriate and timely response to action points.
5
Centre compliant with qualification and Centre approval criteria and
requirements, quality assurance criteria and regulatory Conditions.
9
Exercise
In groups, use the information
provided to allocate the fictitious
Centres to a risk band. Be prepared
to share your group’s reasoning.
10
2.3 Sanctions
• Five Sanction Levels
• Applicable where Risk Band is 3 or 4
• Not always used, even where Risk is 3 or 4
• Intention is to apply Sanctions where they will
help to ensure the required action is implemented
11
Table of Sanctions that may be applied
Sanction
Rationale
0
None
 Little or no risk
1
Escalation to Centre
management, eg, QIP monitored by
 Limited confidence in specific risk
indicators
Level
Centre senior manager.
2
Additional development or quality
intervention visit, eg to check action
plan completion or EV following
learner re-assessment, etc
 Non-compliance
 Not responding to action points
3
(a) Suspension of registration
(b) Suspension of certification
Threat to learners
 Loss of integrity of assessment
 Danger of invalid certification claims
4
Withdrawal of Centre approval for
specific qualifications
Irretrievable breakdown in QA and
management of specific quals
5
Withdrawal of Centre approval for
all qualifications
Irretrievable breakdown in QA and
management of all quals
12
Exercise
In groups, decide whether any of the
fictitious Centres should be
sanctioned, and if so, how. Be
prepared to share your group’s
reasoning.
13
3. Direct Claims Status (DCS)
• DCS awarded to Centre for particular qual, if:
– Centre has an Accredited Internal Verifier (AIV);
– Centre has delivered the qual for at least a year;
– assessors & IVs satisfactory, Centre Risk Band = 1 or 2.
• Criteria reviewed during compliance visit
• Sample of assessed & verified work scrutinised
• Direct Claims Status removed if:
– AIV no longer verifies, or Centre stops delivering qual;
– Centre Risk = 3 or 4, or monitoring reveals concerns.
14
4.1 Impact on AIVs - role of the
AIV
AIVs act on behalf of OCNYHR:
• To conduct and/or oversee internal quality
assurance activity:
– Assessment tasks are high quality
– Assessment is high quality
– Internal verification is robust
• To ensure and assure that standards are met
• To enable RACs to be signed off
15
4.2 AIVs uphold standards
• AIV must check additional work where initial
samples identify an issue.
• If assessment decisions incorrect, AIV can:
• reverse assessment decisions;
• ask for work to be re-assessed;
• not sign off achievements.
• If previous decisions could be affected, AIV
to ask for review of assessment tasks
and/or all relevant work to be re-assessed.
16
4.3 AIV & Centre Monitoring
visits
• Visits check robustness of IV process
• QRs/EVs look for evidence that:
• Assessment tasks fit for purpose
• Learner work meets required standard
• Assessment is accurate
• IV robust, effective feedback to assessors
• If QR/EV needs to see more, integrity of
Centre’s IV is in question
17
Exercise
In groups:
• Discuss the implications of the Ofqual
regulations on your work as an AIV, considering:
– challenges that arise
– how you might address them
– how you might use the risk-based approach to help
– what OCNYHR could do to support you
18
5.1 Verification of assessment
tasks
• Every task must be verified
• before using the tasks with learners
• Includes tasks that are revised
• And extras devised for learners who need
to re-do an assessment
19
5.2 Assessment tasks must:
• be fit for purpose;
• permit reasonable adjustments while
minimising the need for them;
• allow learners to generate evidence that
can be authenticated;
• allow learners to reach the specified level
of attainment;
• allow assessors to differentiate accurately
20
Further information
• www.certa.org.uk
• www.ofqual.gov.uk
• www.qaa.ac.uk
• www.accesstohe.ac.uk
21