Transcript Academic Integrity

Competency-based
Assessment in TQA
Accredited Senior
Secondary Courses
Issues to be discussed in this forum…
Criterion- & competency-based assessment: how different are they?
Why use competency-based assessment in TQA courses?
Assessment: methods & issues
Quality Assurance: what, why, who and how?
Outcomes-referenced assessment:
“Occurs where students’ performances are measured
against explicit criteria…its focus [is] on achievement
against learning objectives or outcomes.”
(UTAS www.edu.au/tl/supporting/assessment/judgement.html)
Types of outcomes assessment:
• Criterion-based approach: Students’
achievement is measured against a range of
standards (eg from a ‘C’ to an ‘A’ rating) for
each learning outcome
• Competency-based approach (or ‘Mastery
learning approach’): Students’ achievement is
measured against a predetermined standard (eg
‘pass’/‘fail’) for each learning outcome.
The criterion- and competency-based
approaches share the same fundamental
assessment principle.
They are both concerned with assessing
learning outcomes against predetermined
standards.
In TQA accredited courses criterion-based
assessment has traditionally had three gradations
(ratings): ‘C’, ‘B’ and ‘A’ standards.
The recently introduced competency-based
assessment courses have a two gradations
(ratings): ‘competent’; and ‘not yet competent’.
In theory there is no reason to limit the number of
gradations in competency-based assessment –
both criterion- and competency-based
approaches to assessment are fundamentally the
same.
In practice the use of competency-based
assessment with two gradations is suited to some
kinds of TQA accredited courses.
What kinds of TQA courses use competency-based
assessment? (I)
Those with learning outcomes where the focus of
assessment is on:
• the learner’s ability to perform an activity (usually
in a defined context)
• the application of knowledge and skills in ‘real
world’ contexts.
What kinds of TQA courses use competency-based
assessment? (II)
Typically these are courses with learning outcomes
where it is more important to determine mastery of a
discrete learning outcome (ie. the learner ‘can do it’)
rather than measuring the degree to which a learner can
do something. Underpinning knowledge is implicitly
assessed within the context of completion of a task or
activity.
Eg. Learning outcome: Use a household fire
extinguisher safely
Assessment
Assessment instruments should make clear to the learner:
a) the nature of the specific assessment task (including the
size/format/due date etc)
b) which learning outcomes are being assessed
c) what the learner must demonstrate
d) how the assessment of the task relates to the final
assessment.
Assessment
Examples of common assessment methodologies
include:
direct observation
verbal/oral testing
practical testing
written testing
written tasks.
Assessment
The method chosen should ‘make sense’ within the
context of the course and the learning outcomes being
assessed.
• Direct observation of a practical competency/criterion
might be more valid than using a written task.
• Analysis of a product might be more valid when
the process is not as significant as the outcome or
cannot be easily observed.
Assessment
A marking guide should be prepared for each
summative assessment instrument. A marking guide
articulates ‘what must be demonstrated in the task’. For
example:
• a ‘correct answer’ list for a test and the number of
questions a learner must get right to ‘pass’
• a list of the specific features of a product or
process that must be demonstrated to ‘pass’.
Assessment
A marking guide should have a direct relationship to
the performance indicators described in the course.
The performance indicators may need to be elaborated
in order that they ‘make sense’ within a specific
assessment task.
Assessment
Final assessment decisions should be on-balance, holistic
ones based on the evidence of learning outcomes measured
against the standards stated in the course.
Considerations might include the:
relative weighting given to individual assessment tasks
consistency of demonstrated achievement
'currency' (evidence from the end of a course may have
greater relevance and reliability than that collected early
in the delivery).
Ratings (I)
Summative assessment of learning outcomes result in a
final rating for each competency.
The ‘competent’ rating is a final, overall rating and should
not be used as a grade in formative assessment/progress
reports.
Learners who are assessed as ‘not yet competent’ can be
provided with further opportunities to demonstrate
competency within the scope of the course’s design-time,
provider resources, and TQA reporting requirements.
Ratings (II)
Some providers use 'not yet assessed' in reports on
learner's progress (such as term reports).
While this is perfectly acceptable in this context, 'not yet
assessed' is NOT a rating that is reportable to the TQA.
Ratings (III)
Providers may use any form of grading they wish in
assessment so long as the final rating is reported as per
the course document (eg. ‘competent’, ‘not yet competent’,
‘highly competent’).
Examples:
% marks
marks out of 10
A, B or C
pass/fail
gold, silver, bronze
Awards
Each TQA accredited course contains a
description of award requirements. These are
algorithms (or rules) defining the relationship
between ratings and a final award.
Eg. To gain an award (‘Pass’) in the course the
learner must be assessed as ‘competent’ in all
five competencies.
Qualification
The qualification issued by the TQA on the
learner’s Qualifications Certificate comprises the
course name and the award gained.
Eg. Personal Pathway Planning: Pass
Quality Assurance & the TQA’s Role?
The TQA is responsible for the quality assurance
for qualifications it awards in senior secondary
courses it has accredited. This means that it has
to stand behind the statements made on
certificates as to the knowledge and skills of the
holder of the certificate.
Audit Model
• How are providers selected for an audit? Risk
based approach – ‘flags’ might include:
complaints; number of learners/size of provider;
major changes in staff; trends in data reported to
TQA; time since last audit.
• How do we know if we are/are not being audited
this year? TQA notification process.
Audit Model
• Who is audited? Provider not individual
teachers.
• Who does the auditing? TQA staff and
subject experts (when appropriate).
Audit Model
• Is an audit a ‘witch-hunt’? Opportunities for
confirmation of best practice, identification of
issues, continuous improvement.
• What happens if we ‘fail’ an audit? Audit
findings and opportunity to ‘clear’ them,
follow-up visits, report to Authority.
What will Auditors look for?
• Evidence that the course has been delivered
• Evidence that learners ‘have learnt’
• Evidence that assessment is based on the
course’s learning outcomes and standards.
Course Delivery
• Planning: did the course provider have an
articulated plan for the delivery of the
course and its assessment?
• Was the course actually delivered?
Planning (I)
A structured course delivery plan would articulate:
• the sequence of course delivery (what parts
of the course were delivered to learners and
in what order)
• when formative and summative assessment
occurs.
Planning (II)
A course delivery plan is sometimes called a
‘scope and sequence’.
Course delivery plans could be a ‘whole of
course’ plan or could be arranged as a sequence
of lesson, weekly, monthly or term plans
depending on individual teachers’/providers’
preferred methodology.
Was the Course Actually Delivered?
Evidences could include:
• records of attendance
• assessment tasks and student work (especially
examples of compulsory tasks such as keeping
a journal or log book)
• interviews with past and present students
• records of assessment.
Evidence that Assessment is Based on the
Course’s Learning Outcomes & Standards
Evidences could include:
• summative assessment tasks & marking guides
• some examples of student work showing the
application of marking guides
• assessment records and information about how they
were used to make final assessment judgements.
Assessment Records Should Contain:
• the name of each learner
• the competencies assessed and/or the assessment
instrument/s used to make the assessment
• the date of assessment
• the name of the assessor/s.
Assessment Records
The class assessment record could be a single document
for a whole class (such as a spreadsheet or paper-based
grid table) or comprise a set of records, one for each
learner.
The ‘Enhanced’ Audit Model
Used for courses with special risk factor (eg Essential
Skills courses with TCE ‘ticks’).
Same as other audit model but auditors will also wish to
view examples of the work of ‘borderline’ learners.
‘Borderline’ means the gaining or not the ‘Pass’ award.
Examples of the Work of ‘Borderline’ Learners (I)
• The nature of the work - major summative
assessment task/s that allows a judgement to be
made on all competencies/the award.
• How many? - use those that arise (do not ‘invent’
them). Number of examples will depend on the
number of borderline learners & the period that
course had been delivered.
Examples of the Work of ‘Borderline’ Learners (II)
• How kept? - archive copies (photocopies, CD-ROM or
other storage mechanisms).
• Where kept? - provider responsibility. System needs
to be robust enough to handle staff changes.