Transcript Preschool PowerPoint

Using Data to Improve
Student Achievement
Summer 2006 Preschool
CSDC
Outcomes






Know why we need to look at data
Identify two types of tests
Understand three types of scores
Understand Summative & Formative
Assessments
Be able to interpret Summative Assessment
Reports
Know how to use data in instructional planning
for increased student learning
Why Look at Data?
The purpose of data is to
give educators
INSIGHT!
Types of Tests
 Norm-Referenced Test
(NRT)
 Criterion-Referenced
Test
(CRT)
What is a Norm-Referenced Test (NRT)?


A standardized
assessment in which
all students
perform under
the same conditions.
It compares the performance of a student or
group of students to a national sample of
students at the same grade and age, called
the norm group.
What is a Criterion-Referenced Test
(CRT)?


An assessment comparing one student's performance
to a specific learning objective or performance
standard and not to the performance of other
students.
It tells us how well students are performing on
specific goals or content standards rather than how
their performance compares to a national or local
norming group.
Summary NRT and CRT
Norm-referenced
Test (NRT)
TYPE
Shows how a student
does in relation to a
DEFINITION
norm group.
EXAMPLES




FCAT NRT
SAT/10
TerraNova
PSAT/NMSQT
Criterion-referenced
Test (CRT)
Shows how a student does
in relation to a standard.




FCAT SSS
Classroom Tests
AP Exams
District Math
Assessments
Types of Scores
Raw Score (RS)

The number of items a student answers
correctly on a test.


John took a 20 item mathematics test
(where each item was worth one point) and
correctly answered 17 items.
His raw score for this assessment is 17.
Scale Score (SS)



Mathematically converted raw scores
based on level of difficulty per question.
For FCAT-SSS, a computer program is
used to analyze student responses and
to compute the scale score.
Scale Scores reflect a more accurate
picture of the student’s achievement
level.
Gain Scores

Commonly
referred to
as
“Learning
Gains”

The amount of
progress a student
makes in one school
year.
Learning Gains: Who Qualifies?

All students with a pre- and post-test,
including all subgroups (ESE, LEP, etc.).

All students with matched, consecutive year
(i.e. 2005 & 2006) FCAT SSS results, grades
4-10, who were enrolled in the same school
surveys 2 & 3 (FTE).
Learning Gains: Which Scores?

Gains apply in reading
and math, not writing
or science.

Pre-test may be from
same school, same
district, or anywhere in
the state.
Learning Gains:
What equals Adequate Yearly Progress (AYP)?
A. Improve FCAT Achievement Levels from 2005 to
2006 (e.g. 1-2, 2-3, 3-4, 4-5)
OR
B. Maintain “satisfactory” Achievement Levels from
2005-2006 (e.g. 3-3, 4-4, 5-5)
OR
C. Demonstrate more than one year’s growth within
Level 1 or Level 2 - determined by DSS Cut Points
(not applicable for retained students)
Developmental Scale Score
Gains Table (DSS Cut Points)
Students achieving within Level 1 (or within Level 2) for two
consecutive years must gain at least one point more than those
listed in the table in order to satisfy the “making annual learning
gains” component of the school accountability system.
Grade Level Change
Reading
Mathematics
3 to 4
230
162
4 to 5
166
119
5 to 6
133
95
6 to 7
110
78
7 to 8
92
64
8 to 9
77
54
9 to 10
77
48
Learning Gains: Retainees
A retained student can only be counted as
making adequate progress if he/she:
Moves up one level.
(e.g. 1-2, 2-3, 3-4, 4-5)
Maintains a level 3, 4, or 5.
Learning Gains: Activity
Using the data on the
following table, determine:


which students made a
learning gain
what percentage of the
teacher’s students made a
learning gain
Data Display for FCAT Reading Results
Student
04/05
Grade
Level
05/06
Grade
Level
Pre-test
Achievement
Level
Pretest
DSS
Post-test
Achievement
Level
Posttest DSS
Learning Gain
Determination
A
7
8
Level 1
Level 2
Yes or No
Reason: A, B, or C
B
7
8
Level 4
Level 4
Yes or No
Reason: A, B, or C
C
7
8
Level 2
D
8
8
Level 1
Level 2
Yes or No
Reason: A, B, or C
E
8
8
Level 3
Level 3
Yes or No
Reason: A, B, or C
F
8
8
Level 1
G
7
8
Level 5
1598
1486
Level 2
Level 1
Level 4
1743
1653
Yes or No
Reason: A, B, or C
Yes or No
Reason: A, B, or C
Yes or No
Reason: A, B, or C
Teacher Learning Gains Based on
Data Display
Total Number of
Students with a
Pre and Post-test
who qualify for
learning gain
calculations:
7




Reason A
Increased 1 or
more Achievement
Levels
2
Reason B
Maintains
“satisfactory”
levels (3, 4, or 5)
2
Reason C
DSS Target Gain
(More than a
year’s growth)
1
5 out of 7 students made learning gains.
71% of this teacher’s students made learning gains and add points
towards the school’s grade.
No points are given to the school for Student F because he was retained
and stayed within level 1 – even though he made significant gains in
DSS points.
No points are given to Student G because he decreased a level.
Class Record Sheet
for
Learning Gains
Types of Data
Results (Summative)

Data used to make
decisions about student
achievement at the end of
a period of instruction.
Process (Formative)

Data gathered at regular
intervals during the
instructional period; used
to provide feedback about
student progress and to
provide direction for
instructional interventions.
A Closer Look at Summative Data
Examples:
FCAT
Parent
Report
A Closer Look at Formative Data
Quizzes
Tests
Homework
Essays
Classwork
What tools do we have?


FCAT Inquiry (Summative)
Teacher Tools for Data Collection
(Can be Summative or Formative)
•Histogram
•Run Chart
•Scatter Diagram
•Item Analysis
•Pareto Chart
Histogram
Bar chart
representing a
frequency
distribution
of student scores
Grade Distribution in 8th Grade Social Studies
70
60
50
Frequency
Heights of the
bars represent
number of
students scoring
at the same
level/score
40
30
20
10
0
0-10
Used to monitor
progress
11-20
21-30
31-40
Grade
41-50
51-60
Histogram: Midterm Grade Distribution – 11th Grade American History
16
14
Number of Students
12
10
8
6
4
2
0
<50%
50-59%
60-69%
70-79%
Grades
80-89%
90-100%
Histogram: Final Exam Grade Distribution – 11th Grade American History
28
24
Number os Students
20
16
12
8
4
0
0-10
11-20
21-30
31-40
41-50
51-60
Grade
61-70
71-80
81-90
91-100
“L to J”
Grade Distribution in 8th Grade Social Studies
70
60
40
30
20
16
10
14
0
0-10
11-20
21-30
31-40
41-50
51-60
Grade
Number of Students
Frequency
50
12
10
8
6
Number os Students
28
24
4
20
2
16
0
50-59%
60-69%
70-79%
Grades
12
8
4
0
<50%
0-10
11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100
Grade
80-89%
90-100%
Run Chart
Use to:
Monitor progress over time
Display data in simplest form
Run Chart: Percent of Students Scoring at Least 80% on
Weekly Vocabulary Quiz
100
90
80
Percent
70
60
50
40
30
20
10
0
1
2
3
4
5
Week
6
7
8
9
Run Chart: Number of Words Defined Correctly on Weekly Quiz
180
175
Number of words
170
165
160
155
150
145
All Time Best
140
1
2
3
4
5
Week
6
7
8
9
Scatter Diagram
Use to:
Show relationships
Test for possible cause/effect
Scatter Diagram: Quiz Average vs. Test Average
100
Quiz Average
90
80
70
60
50
50
60
70
80
Test Average
90
100
Scatter Diagram: Hours of Sleep vs Mistakes on Test
12
10
Mistakes
8
6
4
2
0
0
1
2
3
4
5
6
Hours of Sleep
7
8
9
10
Use to:
Item Analysis
Determine mastered content
Determine most common mistakes
ITEM DATA RECORDING SHEET
FOR ANALYSIS OF CLASSROOM TESTS
Subject : _____________________
Record Answers (Letter) Chosen for Multiple Choice Items
Record Points Earned for Constructed Response Items
Student
#
1
Correct
Answer
#
2
#
3
#
4
#
5
#
6
#
7
#
8
#
9
#
10
#
11
#
12
#
13
#
14
#
15
#
16
#
17
#
18
#
19
#
20
CLASSROOM TEST ANALYSIS
BENCHMARK
ASSESSED
ITEM
#
1
2
3
4
5
6
7
8
9
10
NUMBER
CORRECT
NUMBER
INCORRECT
NUMBER
PARTIAL
CREDIT
NUMBER
DISTRACTOR
A/1
NUMBER
DISTRACTOR
B/2
NUMBER
DISTRACTOR
C/3
NUMBER
DISTRACTOR
D/4
NUMBER
NO
ANSWER
Pareto Chart
Use to:
Rank issues in order of occurrence
Decide which problems need to be addressed first
Find the issues that have the greatest impact
Pareto Chart: Types of mistakes in Division Problems
100
100
90
90
80
80
70
70
60
60
50
50
40
40
30
30
20
20
10
10
0
0
Incorrect multiplication
Incorrect subtraction
No decimal
Mistake
Other
Cumulative percentage
Percent
Monitor impact of changes
Pareto Chart: Types of mistakes in a Geography Quiz
100
90
90
80
80
70
70
60
60
50
50
40
40
30
30
20
20
10
10
0
0
Spelling
Location
Capitals
Mistake
Other
Cumulative percentage
Percent
100
Data analysis provides:
Insight
and
Questions
Questions to Ponder…
 What question are we trying to answer?
 What can we tell from the data?
 What can we NOT tell from the data? What
else might we want to know?
 What good news is here for us to celebrate?
 What opportunities for improvement are
suggested by the data?
Adapted from Getting Excited About Data, Edie Holcomb
www.corwinpress.com
Action
Provides
Answers!
Steps to
Improvement
ACT
STUDY
DO
PLAN
Make improvements.
Analyze the results.
Implement the plan.
What information have I gained from my
data? What interventions can I put
In place?
Personal Action Plan
What data can I access?
What tools can I use to help me monitor progress
toward our class goals?
What/who else do I need to help me?
What is my start date?
A
S
How will I evaluate the results?
D
P