Transcript Document

Assessing Without Levels
30.04.2015
A tracking system for keys stage 3 needs to be manageable, proportionate and
quantifiable
To devise an approach to ‘Assessing Without Levels’ we need to:
1. Examine the requirements for tracking at key stages 3 and 4 and the evidence
available to support judgments on pupils’ progress across each key stage
2. Find a way to relate the measures we choose to descriptions of what a pupil has
learnt at any point in time.
3. Find a way to describe and quantify the progression that accompanies learning.
We invite colleagues in schools to comment on these proposals and describe
how these would relate to their own plans for Assessing Without Levels.
Please send your comments to [email protected] Thank you in advance.
www.4matrix.org/AWL
© New Media Learning Ltd.
The current position – tracking using the Flight Path tool
This approach has the advantage of making sure that pupils are on course to make expected progress.
In the example above: the school’s target is a B, the target should be an A to represent 4 levels of
progress, the Attainment 8 estimate is an A, and the pupil is tracking towards a C.
Assessing Without Levels
Several possible approaches
1. Record what has been taught and learnt; make a judgement about whether pupils are
in a high, middle or low category in regard to where one would expect them to be.
2. Record what has been learnt as Key Performance Indicators.
‘I can..’ Statements are an example of this approach.
Standardise the ‘I can’ statements against journeys pupils have taken towards expected
grades at KS4, i.e. ‘Bands’ in the case of the PiXL approach
3. Track progress towards the Attainment Target for each subject at key stage 3.
i.e. pupils are expected to know, apply and understand the matters, skills and processes specified in
the relevant programme of study, i.e. Subject Content list
4. Track back from GCSEs. Use Grade Criteria to decide on the most likely grade a pupil is
targeting.
5. There is a camp that says that you shouldn’t track pupils; you should only assess them
using standardised tests at the end of a key stage.
Assessing Without Levels
Which approach should we choose?
1. These are effectively RAG trackers which use different terms for the three
categories, e.g. Emerging, Developing, Exceeding - or - Ready, Expected, Mastered,
etc. Changing the terms used doesn’t solve the fundamental problem of ‘Levels’ which is their lack of definition. Also, many of these Mastery approaches don’t
quantify the value of each category. If there is no relationship between these terms
and the GCSE grades expected, then it will not be possible to quantify progress.
2. Key Performance Indicators (elemental ‘I can’ statements) are difficult to
standardise. i.e. to relate to grade criteria or levels. It is also difficult to process a table
of elemental ticks into an overall grade or level.
3. GCSE Grade Criteria is appropriate to use to evaluate progress at key stage 4.
However, they are distant from what will be taught at key stage 3 and would need
interpretation if used at key stage 3.
4. Attainment Targets will be used at key stage 3 to evaluate attainment.
These are defined as the knowledge, application and understanding of the matters,
skills and processes specified in the Subject Content. We would need to make these
attributes explicit and track their assimilation.
5. Only test at the end of a key stage. This would be fine if schools weren’t expected to
show evidence that they are managing pupil progress.
What data could be used for tracking?
1. Teaching Scheme information
2. Pupils’ response to what has been taught - marks, test scores
Possibly expressed on a +,
or - scale
3. Criteria to indicate acquisition of knowledge, skills and understanding (or ‘matters,
skills and processes’). e.g. ‘I can’ statements
4. KS3 - Extent to which Attainment Targets for each subject have been attained, i.e.
the acquisition of the knowledge, application and understanding of the matters, skills
and processes specified in the Subject Content
5. KS4 - Extent to which a pupil has developed GCSE grade criteria
The above list describes the range of evidence that could contribute to the tracking of progress.
All of it is valid, and much of it may be needed.
However, what key evidence should be used if the tracking system is going to be manageable,
proportionate and quantifiable?
Question
How can we illustrate a tracking point on a graph in terms of the
learning that it represents?
One Answer...
Click a dot to see the assessment that supports the teacher’s judgement.
Provide frameworks for schools to paste in :
1. their assessments, 2. ‘I can’ statements, or 3. teaching scheme tick lists.
etc.
Design feature
1. Clicking a point on the graph will display a table which shows the extent
to which a pupil has assimilated the subject content for the subject.
2. Click an arrow on this table to delve deeper and see the teaching
scheme, or pupil assessments which support this judgement. We could
provide a blank framework for schools to paste in their teaching schemes.
We could provide a way to convert module test information into tracking
points on a graph.
Question
How can the progress of pupils be presented visually so that we can track it?
Options:
1. Track an ascending scale reflecting the growth in learning
2. Track the path towards an expected grade.
There are issues associated with both approaches.
Basing tracking on expected outcomes
High
Middle
Upper Band
Low
Middle Band
Lower Band
7
8
9
10
11
Each pupil’s KS2 APS will describe a trajectory for them which will reflect the FFT chances
tables. We could assign QCA points values to their progress through each key stage .
Mock up of this tool
Mathematics
H
M
L
Upper
Band
We could track progress in QCA points through both key stages
towards target number grades, defined through FFT chances tables.
Additional instruments could provide point read-outs of a pupil’s
VA, P8 and A8, and show distance from target across subjects.
7
8
9
10
11
Translate KS2 APS to a band, i.e.
expected grade at GCSE
Basing tracking on concept of being ‘on track’ to attain a target level
Band 9
Band 8
Band 7
Band 6
Band 5
Band 4
Band 3
Low
Band 2
Band 1
7
8
9
10
11
Mock up of this tool
Assessment based on Subject Content
Assessment based on
Grade Criteria
Band 9
8A
Band 8
8C
Can be tracked using fine
grades within a band
Band 7
Band 6
Band 5
Band 4
We could track progress as being H, M or L
in a target GCSE number band. Additional
instruments could provide point read-outs
of a pupil’s VA, P8 and A8, and show
distance from target across subjects.
Band 3
Band 2
Band 1
7
8
9
10
11
Question
How is tracking at key stage 3 different to tracking at key stage 4?
Key Stage 3
At key stage 3 we could track the acquisition of the Attainment Target for
each subject, i.e. the extent to which the Subject Content has been learnt or
is assimilated, as shown by module tests etc.
We could also track against Grade Criteria as we do at key stage 4.
Key Stage 4
At key stage 4 we will be tracking towards the Grade Criteria for each subject
at GCSE. Standardised assessments will be used.
An aim will be to support pupil’s learning towards the higher GCSE grades
whilst recognising that the normalisation of grading will separate pupils’
attainments into a range of grades with different values.
Question
How are Core subjects different to Foundation subjects?
Many foundation subjects will be starting from the beginning, i.e. with no
prior attainment to consider.
Their starting point on any progress graph will be around zero.
A Flight Path for Foundation subjects may be different to one for Core
subjects.
Also, pupils might not study a foundation subject after year 9, so this key
stage represents an end in itself.
Core subjects will be part of a continuum of learning from key stage 2 and
will continue into key stage 4, so the case for applying GCSE grade criteria
throughout is stronger.
Question: Should we treat Core and Foundation subjects differently?
Foundation Subjects
New KS2 scale
Tracking based on Subject Content
Tracking based on Grade Criteria
Performance
4 Levels of Progress
Attainment 8 Estimate
7
8
9
10
11
At key stage 3 we are tracking the proportion of the Subject Content (AT) that is taught and learnt.
At key stage 4 we are tracking assessments using the grade criteria for each subject at GCSE.
Number
Grade
QCA
Core Subjects
New KS2 scale
Tracking based on Grade Criteria
Tracking based on Grade Criteria
Number
Above average PISA
performance
Student A
Baseline APS=5
Expected number grade=7
Tracking number grade=7
Progress=expected
Assimilation of Subject Content =?
Mastery of threshold concepts=?
Above average PISA performance
7
8
Expectation from KS2
Progress from KS2
9
10
11
At key stage 3 we are tracking the proportion of the Subject Content (AT) that is taught and learnt.
At key stage 4 we are tracking assessments using the grade criteria for each subject at GCSE.
Grade
QCA
Core Subjects
New KS2 scale
Tracking based on Grade Criteria
Tracking based on Grade Criteria
Student B
Baseline APS=3
Expected number grade=4
Tracking number grade=5
Progress=above expected
Assimilation of Subject Content =?
Mastery of threshold concepts=?
Above average PISA performance
Number
Above average PISA
performance
Expectation from KS2
Progress from KS2
7
8
9
10
11
At key stage 3 we are tracking the proportion of the Subject Content (AT) that is taught and learnt.
At key stage 4 we are tracking assessments using the grade criteria for each subject at GCSE.
Grade
QCA
What should the graph look like?
Our current flight path uses ascending lines which reflect the precision with which
we can currently plot lines of expectation from KS2 APS to KS4 APS.
Two things may lead us to approach progress and granularity differently:
1. With the introduction of number grades at GCSE there is a need to clarify the
QCA points values of number grades, and the size of the intervals between them.
This may have weakened our ability to use KS2 data to predict number grades.
2. The Mastery approach to managing progress implies tracking towards the higher
grade levels right through by concentrating on mastery of the ‘key concepts and big
ideas’. It doesn’t inherently quantify progression, but the concept of being ‘on
course’ to achieve the higher grades defines a particular tracking approach.
So another approach is to provide a grid which provides a background tracking
template, i.e. 3xL5s tend to lead to GCSE grades 7 and 8 etc.
Granularity in tracking is less likely to come from predictive data, and more likely to
come from teachers’ ability to assess whether pupils are tracking ‘strong’ ‘middle’
or ‘weak’ grades. i.e. fine grades.
KS2
Average
Level
6
CORE SUBJECTS
Expected academic journey based KS2 Levels
(Progress Bands)
KS4
Average
Grade
9
E
S
M
5
8
W
High
7
6
4
Mid
5
4
3
Low
3
2
2
1
Year
7
8
9
10
11
W
KS2
Average
Level
6
CORE SUBJECTS
Expected academic journey based KS2 Levels
(Progress Bands)
KS4
Average
Grade
9
E
S
M
5
8
W
High
7
Difference from
expected
6
4
5
Student C
Baseline APS= 5
Expected number grade=7
Tracking number grade=6
Progress=below expected
Above average PISA performance
3
4
Low
3
2
2
1
Year
7
8
9
Mid
10
11
W
What information is needed to exemplify the points on the graph?
Good planning will ensure that the Subject content has been turned into a teaching
scheme that shows how all of the content will be covered over years 7-9.
Supporting documentation may include:
-The Subject Content Map for each subject
-The Scheme of Work for the subject showing how all of the elements of the
subject content has been planned into the teaching units over key stage 3.
-The teaching scheme which identifies the learning attributes to be covered.
- Module test results which show the percentage of the taught programme that has
been learnt.
- Standardised tests and performance indicators which indicate which GCSE
number grade pupils are tracking towards
What assessment methods should be used to illustrate each point on the graph?
e.g. module tests, Standardised Tests and Performance Indicators to match the
teaching programmes
Multi-Layer Tracking Matrix
Mastery Layer
Learning Layer
Content Layer
Results Layer
Presentation Layer
Multi-Layer Tracking Matrix
Mastery Layer
Learning Layer
Content Layer
Results Layer
Presentation Layer
Multi-Layer Tracking Matrix
Mastery Layer
Learning Layer
Content Layer
Results Layer
Presentation Layer
Multi-Layer Tracking Matrix
Mastery Layer
Learning Layer
Content Layer
Results Layer
Presentation Layer
Multi-Layer Tracking Matrix
Age-related statements of achievement
Mastery Layer
Learning Layer
Content Layer
Results Layer
Presentation Layer
Multi-Layer Tracking Matrix
Mastery Layer
Learning Layer
Content Layer
Results Layer
Presentation Layer
Multi Level Tracking Matrix
What information is needed to support the teacher’s judgement that a pupil is at
any particular point on this graph?
We anticipate that schools will paste their assessment grids and planning
information into 4Matrix. What information is best made available through 4Matrix
and what information should stay on paper or in a mark book?
We can provide information that is already known, i.e. the Subject Content lists for
key stage 3, The Assessment Objectives for any GCSE subject at key stage 4.
Schools should add information that enable us to examine what has been taught,
what has been learnt, and how it relates to the Subject Content lists.
Also useful, will be age related statements for each subject.
Schools also need to be able to put in the results to module tests held at 8 week or
so intervals. These will become standardised over time.
Examples follow on the next slides.
Key stage 3
outcomes
Here we number of each element and note the verbs associated with the acquisition of the knowledge, application and
understanding of the matters, skills and processes specified in the Subject Content
Key stage 3 Scheme of Work for Computing
The scheme of work would describe the topic to be taught, the lesson content (not shown) the learning objectives
which reference the Subject Content and the Attainment Target reference.
Teaching Scheme for Computing
The Teaching Scheme would make reference to the Subject Content and Scheme of Work.
It would identify resources and describe the activities which would ensure that the full Subject Content was delivered
The advantages of this approach
An important issue with AWL is how to provide the evidence to exemplify what a dot
on a graph actually signifies.
The Multi-Layer Tracking Matrix will allow us examine the layers to show what
subject content was delivered, what learning attributes were taught, what
standardised scores were produced, and how pupil outcomes compared to agerelated statements in each subject.
Questions still to be answered
Question: Should we treat Core and Foundation subjects differently?
What information should be available to support the teachers’ judgements
about the progress of pupils?
Which of this would be useful to have in 4Matrix?
What assessment methods should be used to illustrate each point on the
graph e.g. module tests, Standardised Tests and Performance Indicators?