Transcript Preschool PowerPoint - Math August 3 Using Data

Using Data to
Improve Student
Achievement
Secondary Mathematics
Preschool Inservice
2006
Outcomes
1.
2.
3.
4.
Know why we need to look at data
Identify two types of tests
Understand three types of scores
Understand Summative & Formative
Assessments
5. Be able to interpret Summative
Assessment Reports
6. Know how to use data in
instructional planning for increased
student learning
Why Look at Data?
The purpose of data
is to give educators:
INSIGHT
DIRECTION
FEEDBACK
Types of Tests
Norm-Referenced
Test (NRT)
CriterionReferenced Test
(CRT)
What is a Norm-Referenced Test
(NRT)?
A standardized
assessment in which
all students perform
under the same
conditions.
Compares the performance of a
student to other students
nationally. Norm groups are
students in the same grade and
age.
Example: FCAT NRT
What is a Criterion-Referenced
Test (CRT)?
An assessment comparing one
student's performance to a
specific learning objective or
performance standard and not to
the performance of other
students.
For example: FCAT SSS (Sunshine
State Standards)
Summary NRT and CRT
TYPE
Normreferenced
Test (NRT)
Criterionreferenced
Test (CRT)
Shows how a Shows how a
student does student does
DEFINITION in relation to in relation to
a norm group a standard
EXAMPLES
FCAT NRT
PSAT
FCAT SSS
Class Tests
AP Exams
SAT/ACT
Types of
Scores
Raw Score (RS)
The number of items a student
answers correctly on a test.
John took a 20 item
mathematics test (where each
item was worth one point) and
correctly answered 17 items.
His raw score for this
assessment is 17.
Scale Score (SS)
Mathematically converted raw
scores based on level of
difficulty per question.
For FCAT-SSS, a computer
program is used to analyze
student responses and to
compute the scale score.
Scale Scores reflect a more
accurate picture of the
student’s achievement level.
Scale Score (SS)
Higher scale scores indicate higher
proficiency.
On a continuous, vertical scale
across grade levels you can track a
student's progress from lower to
upper grade levels on one scale.
Growth in scale score units
indicates growth in proficiency.
For FCAT-SSS, the Developmental
Scale Score is used to determine a
student’s annual progress from
grade to grade.
Gain Scores
Commonly referred to as
“Learning Gains”
The amount of progress
a student makes in one
school year.
Learning Gains: Who Qualifies?
Students from all subgroups (all
students, ESE, LEP, etc.) with
matched, consecutive year (i.e.
2005 & 2006) FCAT SSS results
(a pre- and post-test) in grades
4-10, who were enrolled in the
same school during the
October & February FTE Count.
Learning Gains: Which Scores?
Gains apply in reading and
math, not writing or science.
Pre-tests may be from same
school, same district, or
anywhere in the state.
Learning Gains:
What equals Adequate Yearly
Progress (AYP)?
A. Improve FCAT Achievement Levels
from 2005 to 2006
(e.g. 1-2, 2-3, 3-4, 4-5) or
B. Maintain “satisfactory” Achievement
Levels from 2005-2006
(e.g. 3-3, 4-4, 5-5) or
C. Demonstrate more than one year’s
growth within Level 1 or Level 2 determined by DSS Cut Points (not
applicable for retained students)
FCAT SSS Developmental Scale
MATH Graduation Passing Score
DSS 1889 or Higher
Grade
Level 1
Level 2
Level 3
Level 4
Level 5
3
375-1078
1079-1268
1269-1508
1509-1749
1750-2225
4
581-1276
1277-1443
1444-1657
1658-1862
1863-2330
5
569-1451
1452-1631
1632-1768
1769-1956
1957-2456
6
770-1553
1554-1691
1692-1859
1860-2018
2019-2492
7
958-1660
1661-1785
1786-1938
1938-2079
2080-2572
8
1025-1732
1733-1850
1851-1997
1998-2091
2092-2605
9
1238-1781
1782-1900
1901-2022
2023-2141
2142-2596
10
1068-1831
1832-1946
1947-2049
2050-2192
2193-2709
Learning Gains: Retainees
A retained student can only be
counted as making adequate
progress if he/she:
Moves up one level.
(e.g. 1-2, 2-3, 3-4, 4-5)
Maintains a level 3, 4, or 5.
Developmental Scale Score
Gains Table (DSS Cut Points)
Students achieving within Level 1 (or within Level 2) for two
consecutive years must gain at least one point more than those
listed in the table in order to satisfy the “making annual learning
gains” component of the school accountability system.
Grade Level
Change
Reading
Mathematics
3 to 4
230
162
4 to 5
166
119
5 to 6
133
95
6 to 7
110
78
7 to 8
92
64
8 to 9
77
54
9 to 10
77
48
Learning Gains: Activity
Using the data on the following
table, determine:
1. Which students made a
learning gain?
2.What percentage of the
teacher’s students made a
learning gain?
Data Display for FCAT Mathematics
Results
Student
04/05
Grade
Level
05/06
Grade
Level
Pre-test
Achievement
Level
A
7
8
Level 1
Level 2
Yes or No
Reason: A, B, or C
B
7
8
Level 4
Level 4
Yes or No
Reason: A, B, or C
C
7
8
Level 2
D
8
8
Level 1
Level 2
Yes or No
Reason: A, B, or C
E
8
8
Level 3
Level 3
Yes or No
Reason: A, B, or C
F
8
8
Level 1
G
7
8
Level 5
Pretest
DSS
1598
1486
Post-test
Achievement
Level
Level 2
Level 1
Level 4
Posttest
DSS
1743
1653
Learning Gain
Determination
Yes or No
Reason: A, B, or C
Yes or No
Reason: A, B, or C
Yes or No
Reason: A, B, or C
Teacher Learning Gains Based
on Data Display
Total Number of
Students with a
Pre and Post-test
who qualify for
learning gain
calculations:
Reason A
Increased 1 or
more
Achievement
Levels
7
2
Reason B
Maintains
“satisfactory”
levels
(3, 4, or 5)
Reason C
DSS Target
Gain
(More than a
year’s growth)
2
1
 5 out of 7 students made learning gains.
 71% of this teacher’s students made learning gains
and add points towards the school’s grade.
 No points are given to the school for Student F
because he was retained and stayed within level 1
– even though he made significant gains in DSS
points.
 No points are given to Student G because he
decreased a level.
Spring 2005
Florida Comprehensive Assessment Test (FCAT)
SSS Mathematics Student and Parent Report
Grade 10
Your 2005 Math Results
You have passed the grade 10
FCAT Mathematics test and
your score is on grade level.
You answered many of the
questions on FCAT correctly.
Aprobaste el examen de
Lectura del FCAT para el grado
10 y tu calificación está al
nivel. Respondiste a muchas de
las preguntas del FCAST
correctamente.
Ou pase eqzamen Leketi FCAT
yo bay Klas 10yèm nan epi ou fè
yon nòt ki nan nivo klas la. Ou
reponn anpil nan kesvon FCAT
yo san fot.
CONTENT SCORES
Number
Possible
Number
Correct
Number Sense & Operations
20
13
Patterns, Relationships, Algebra
10
6
Data, Statistics, & Probability
8
7
Geometry & Measurement
10
5
Objective
FCAT
Parent
Report
Types of Data
Results
(Summative)
Data used to make
decisions about
student
achievement at the
end of a period of
instruction.
Process
(Formative)
Data gathered at
regular intervals
during the
instructional period;
used to provide
feedback about
student progress
and to provide
direction for
instructional
interventions.
A Closer Look at Results Data
Examples:
What tools do we have?
FCAT Inquiry (Summative)
Teacher Tools for Data Collection
(Can be Summative or Formative)
Histogram
Pareto Chart
Run Chart
Scatter Diagram
Item Analysis
Histogram
Bar chart representing a
frequency distribution of student
scores
Heights of the
bars represent
number of
students scoring
at same level/score
Used to Monitor progress
Grade Distribution in 8th Grade Math
70
60
Frequency
50
40
30
20
10
0
0-10
11-20
21-30
31-40
41-50
51-60
Grade
61-70
71-80
81-90
91-100
Histogram: Pre-Algebra Midterm Test
Number of Students
16
14
12
10
8
6
4
2
0
40-49 50-59 60-69 70-79 80-89 90-100
Percentage Correct
Number of Students
Pre Test-8/12/06
16
14
12
10
8
6
4
2
0
F's
D's
C's
Grade
B's
A's
2nd Quarter
Number of Students
14
12
10
8
6
4
2
0
F's
D's
C's
Grades
B's
A's
Post Test-Final-5/23/07
Number of Students
12
10
8
6
4
2
0
F's
D's
C's
Grades
B's
A's
Histogram: Grade Distribution in 8th Grade Math
70
Frequency
60
50
40
30
20
10
0
0-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100
Grade
Activity
Lee County Grade 8 Mathematics Scores
Percentage of Students by Achievement Level
Year
Level 1 Level 2
Levels
1-2
Level 3 Level 4 Level 5
Levels
3-5
2001
23
22
45
33
14
9
56
2002
22
22
44
33
14
8
55
2003
20
23
43
35
14
8
57
2004
23
21
44
31
15
11
57
2005
21
20
41
34
15
10
59
2006
19
21
40
35
16
9
60
Activity Answer
Lee County FCAT Achievement Levels Grade 8
Percent of Students
70
60
50
40
30
20
10
0
2001
2002
2003
2004
Year
Levels 1-2
Levels 3-5
2005
2006
Pareto Chart
Use to:
Rank issues in order of occurrence
Decide which
problems need
to be addressed
first
Find the issues
that have the
greatest impact
Monitor impact of changes
100
100
90
90
80
80
70
70
60
60
50
50
40
40
30
30
20
20
10
10
0
Incorrect multiplication
Incorrect subtraction
No decimal
Mistake
Other
0
Cumulative percentage
Percent
Pareto Chart: Types of mistakes in Division Problems
Pareto Chart: Types of mistakes
in Division Problems
90
90
80
80
70
70
60
60
50
50
40
40
30
30
20
20
10
10
0
Cumulative percentage
100
Percent
100
Incorrect
multiplication
Incorrect
subtraction
No decimal
Mistake
Other
0
Pareto Chart Data
FCAT Errors by subtest for one student
Frequency of
Errors
Percentage of
Errors
Cumulative
Percentage
Geometry
7
7/24 = 29%
29%
Measurement
5
5/24 = 21%
50%
Algebra
5
5/24 = 21%
71%
Number Sense
4
4/24 = 17%
88%
Data Analysis
3
3/24 = 12%
100%
24
24/24 = 100%
Strand
Total
Megan Smith - Grade 9 - FCAT
Subtest Scores
100
90
80
Percent
70
60
50
40
30
20
10
0
Geometry
Measurement
Data Analysis
Algebraic Thinking
Subtest Categories
Number Sense
2005/2006 - 9th Grade - Lee County
FCAT Errors by Subtest
100
90
80
Percent
70
60
50
40
30
20
10
0
Geometry
Algebraic
Thinking
Number Sense
Measurement
Subtest Categories
Data Analysis
FCAT Scores for One
Teacher
100
291.5
NUMBER OF ERRORS
85
238.5
106
79.5
53
26.5
54
48
132.5
45
36
34
52 47
40 33
18
28 25 23
17 12
 Measurement
63
59
159
90
72
68
185.5
 Number Sense
81
77
212
99
27
18
8
6
0
9
0
Where are the errors (or
improvements)?
PERCENTAGE OF ERRORS
91 95
265
98
 Computation
 Geometry
 Statistics &
Probability
 Number Facts
 Whole Number
Concepts
 Patterns
 Problem Solving
 Fractions &
Decimals
 Computation in
Context
Run Chart
Use to:
Monitor progress over time
Display data in simplest form
Run Chart: Percent of Students Scoring at Least 80% on Weekly Math Quiz
100
90
Percent
80
70
60
50
40
30
20
10
0
1
2
3
4
5
6
Number of Students
7
8
9
Student Run Charts
The place
to start is
with
students
graphing
their own
progress.
Run Chart: Number of Math Vocabulary Words
Number of Words Correct
180
175
170
165
160
155
150
145
140
1
2
3
4
5
Week
6
7
8
9
Percent w/ avg. of at least 80%
Class Goal: By the end of 9 weeks, 100% of our class will have
an average of at least 80% on our weekly math quizzes.
100
Class Run Chart: Percent of Students Averaging at Least
80% on Weekly Math Quizzes
90
80
70
60
50
40
30
20
10
0
1
2
3
4
5
Week
6
7
8
9
Scatter Diagram
Scatter Diagram: Teacher Salary vs. FCAT Test Scores
FCAT Test Scores
5
4
3
2
1
0
0
20
40
60
Teacher Salary
80
100
Scatter Diagram: Number of Siblings
vs Grade Point Average
4.5
Grade Point Average
4.0
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
0
1
2
3
4
Number of Siblings
5
6
Scatter Diagram: Hours of Sleep
vs. Mistakes on Test
12
Mistakes
10
8
6
4
2
0
0
1
2
3
4
5
6
7
Hours of Sleep
8
9
10
Student
Score
Level
1
96
5
2
64
4
3
68
5
4
86
5
5
71
4
6
86
5
7
82
4
8
79
5
9
68
3
10
46
4
11
50
2
12
39
1
13
21
1
14
54
3
15
61
2
FCAT Level
Sopris-West Algebra Readiness
Score vs. FCAT Level
6
5
4
3
2
1
0
0
20
40
60
Readiness Scores
80
100
Radar Chart
Mathematics Scores Statewide Comparison for 2001 to 2006
Percent of Students Level 3-5
6th Grade
70
65
60
55
50
45
10th Grade
7th Grade
40
35
30
9th Grade
2001
8th Grade
2002
2003
2004
2005
2006
Example
8TH Grade Class of 2006
Total
Students
Tested
Percentage of Total Scoring at Level
1
2
3
4
5
3-5
7.1
28.3
29.8
27.3
7.5
64.6
22.1
25.5
28.9
18.4
5.1
52.4
5th
Grade
322
6th
Grade
353
7th
Grade
343
16.3
23
38.2
16.3
6.1
60.6
8th
Grade
439
13
21
40
18
8
66
1
8th Grade Class of 2006
Percentage of Students
At Each Level
3-5
70
60
50
40
2
30
20
5th Grade
10
6th Grade
0
7th Grade
8th Grade
5
3
4
Item Analysis
Use to:
Determine mastered
content
Determine most
common
mistakes
CLASSROOM TEST ANALYSIS
BENCHMARK
ASSESSED
ITEM
#
1
2
3
4
5
6
7
8
9
10
NUMBER
CORRECT
NUMBER
INCORRECT
NUMBER
PARTIAL
CREDIT
NUMBER
DISTRACTOR
A/1
NUMBER
DISTRACTOR
B/2
NUMBER
DISTRACTOR
C/3
NUMBER
DISTRACTOR
D/4
NUMBER
NO
ANSWER
ITEM ANALYSIS ACTIVITY
Student
q1
q2
q3
q4
q5
q6
q7
q8
q9
q10
1
1
1
4
4
2
3
3
1
1
2
2
1
2
4
4
2
3
4
2
2
2
3
1
2
4
4
2
3
1
3
2
2
4
1
2
4
4
2
3
3
1
1
2
5
1
3
4
4
2
3
1
1
3
4
6
1
2
4
4
2
3
3
4
3
1
7
3
2
4
4
2
3
2
4
4
2
8
1
2
4
1
2
3
3
1
3
2
9
1
2
3
4
2
3
3
1
4
2
10
1
2
4
4
4
3
1
1
2
1
11
4
2
4
4
2
3
1
2
3
3
12
1
2
4
4
4
3
3
1
3
3
13
1
2
4
4
2
3
4
2
3
2
Correct Answer
1
2
4
4
2
3
1
1
3
2
Using Formative Data
for Continuous Improvement
Random sampling of
end-of the year items
provides students a
constant review of
what has been taught
and a constant preview
of what is yet to be
taught.
The square root of
“n” is an ample
sample size for
accurate data, if
collected weekly
or bi-weekly.
Random Selection
Drawing from a “hat” or fishbowl
Dice
Computer spread sheets
Graphing Calculator
www.randomizer.org
Popsicle Sticks
Ping-pong balls
100- sided die from
Bingo
Gamescience (228) 392-4177
Transparency question
Data analysis provides:
Insight
and
Questions
Questions to Ponder…
What question are we trying to
answer?
What can we tell from the data?
What can we NOT tell from the data?
What else might we want to know?
What good news is here for us to
celebrate?
What opportunities for improvement
are suggested by the data?
Adapted from Getting Excited About Data, Edie Holcomb
www.corwinpress.com
Action
Provides
Answers!
Steps to
Improvement
Make
improvements.
ACT
STUDY
DO
PLAN
Analyze the
results.
Implement the plan.
What information have I
gained from my data?
What interventions can
I put in place?
Personal Action Plan
What data can I access?
What tools can I use to help me
monitor progress toward our
class goals?
What/who else do I need
to help me?
What is my start date?
S
A
D
P
How will I evaluate the results?