Transcript Curriculum Mapping Session for Principals

Data Driven Decisions
Moving from 3D to D3
Malcolm Thomas
[email protected]
(850) 469-5386
www.escambia.k12.fl.us/adminoff/eval
How much water is impacted?
“Insanity is doing the same thing over
and over expecting a different result.”
Albert Einstein
Digging In the Data
One of the first steps:
Develop a common vocabulary that will
lead to common understanding.
NRT
Scale Score
FCAT
NPR
CRT
Stanine
DVSS
Percentile
Raw Score
NCE
NRT Score Types
• Scale Scores
– Use to compare year to year and to interpret trends in
performance
• National Percentile Rank (NPR)
– % in national sample scoring at or above student (ranks)
NOT equal interval
• Normal Curve Equivalent (NCE)
– Equal interval scale from 1 to 99 with average of 50
IS equal interval
• Stanine
– Scores divided into 9 equal groups
– Scores of 4 – 6 are considered average
Normal Distribution Curve
“A Band of Scores”
68%
95%
99%
-3
-2
-1
0
1
2
MEAN
 = Standard Deviation
3
FCAT CRT Scores (SSS)
Scale Scores 100 - 500
500
450
400
350
300
250
200
150
100
50
0
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9
Grade
10
FCAT SSS Scale Scores
• Scale scores have no meaning
in isolation
• Scale Scores are used to
determine achievement levels
• FCAT CRT (SSS) Scale score
range is 100 - 500
FCAT Developmental
Scale Scores (DVSS)
• Created to help understand students' year-to-year
progress.
• DVSS range from 86 to 3008.
• Students should receive higher scores as they move
from grade-to-grade according to their increased
achievement.
• Use DVSS to monitor academic progress each year.
DVSS Linear Progression
3000
2500
2000
1500
1000
500
0
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9
Grade
10
Let’s Review Scores By
Types by Test
•
•
•
•
•
NRT
Raw Score
Scale Score
NPR
NCE
Stanine
•
•
•
•
CRT (SSS)
Raw Score
Scale Score
DVSS
Achievement Level
Learning Gains
• Measure individual
student progress from
year to year
• Uses the Developmental
Scale Score and
Achievement Level
• Compares current scores
to scores of previous year
Making a Learning Gain…
1.
2.
3.
Improve FCAT at least one
achievement level (example move
from Level 1 to Level 2 or Level 3 to
Level 5)
Maintain satisfactory achievement
levels (Levels 3, 4, 5)
Demonstrate more than one year’s
growth within Level 1 or Level 2 (using
DVSS Gains Table)
Retained students MUST increase 1
achievement level or maintain
satisfactory achievement level (cannot
use DVSS method for retained
students)
Developmental Scale Gain
Grade Reading
Math
3 to 4
4 to 5
5 to 6
6 to 7
230
166
133
110
162
119
95
78
7 to 8
8 to 9
9 to 10
92
77
77
64
54
48
Developmental Scale Scores must be at least one point greater than the above cut-off
scores for a student to make a Learning Gain. This method of Learning Gain
calculation only applies to promoted students maintaining achievement 1 or 2.
Monitoring Student Progress …
Content Clusters
The way the FCAT SSS breaks down the
subject area tests.
The number of items varies across
disciplines and grade levels.
SSS Scores are used in
School Grade Calculations
School grade will consist of a total
of eight components in 2007
• Four (4) Proficiency components:
– Reading
– Math
– Writing
– Science **NEW**
• Four (4) Learning Gain Components:
– Reading Learning Gain
– Lowest 25% Reading Learning Gain
– Math Learning Gain
– Lowest 25% Math Learning Gain **NEW**
Revised School Grading Scale
2006
Points Required
(Points possible =
600)
Points Required
(Points possible =
800)
A
410+
525+
B
380-409
495-524
C
320-379
435-494
D
280-319
395-434
F
<280
<395
Grade
2007
Leadership is the key …
Source: Brandeis University
BARRIERS TO
DATA ANALYSIS
• Lack of training in • Outdated
technology
data use
• Unclear priorities
• No uniform data
collection
• Distrust of data
use
• Lack of leadership
at the school and
district level
Elements for Effective Use of Data
• Create collaborative / trusting
relationships
• Engage in data driven dialogue and
collaborative inquiry
• Learn what you can from standardized
tests
• Use multiple measures
• Examine data in variety of ways
Collaborative Inquiry
Questions usually lead to ….
. . . more questions
and more data collection.
When analyzing data
A Picture is Worth …
Collect Multiples Measures
• Example:
– Grade 3 Reading FCAT Scale Scores
– Grade 3 Reading National Percentile
Ranks
– Or Grade 3 Reading FCAT Achievement
Level Percentages for multiple years
Multiple Years of Data
Escambia Reading Level 1
Percentages Grade 3
Escambia Reading Level 1
Student Count Grade 3
100
1000
750
500
250
0
AL 1
2001 2002 2003 2004 2005 2006
AL 1 879 899 758 882 738 492
75
50
25
0
AL 1
2001 2002 2003 2004 2005 2006
AL 1 26
27
23
24
23
15
Comparing FCAT SSS Reading
to Reading NRT
Approximate 2005 FCAT Reading NRT National
Percentiles for Achievement Level Cut Points
100
AL5
AL4
NPR
75
DVSS 1926 =
NPR 60
AL3
50
AL2
25
0
AL1
Grade 3
Grade 4
Grade 5
Grade 6
Grade 7
Grade 8
Grade 9
Grade 10
AL1
21
22
30
28
35
41
49
52
AL2
32
33
48
48
53
73
75
79
AL3
62
65
74
74
78
94
93
91
AL4
91
90
92
94
94
98
96
95
AL5
100
100
100
100
100
100
100
100
Other Examples of Multiple Measures
Compare Student Reading or Math Scores with:
• Averages of State, District or School
• Attendance Data
• Achievement Scores And Discipline
Referrals
• GPA
• Content Areas By Grade Level
FCAT Myths to Avoid
• Performance for individual student
assessments should remain the same
or improve
• FCAT Proficiency equals grade level
• FCAT Proficiency levels are the same
for each grade level
• All data are accurate
Moving ...
Summative Assessments
• Summative assessments attempt to summarize
student learning at a specific point in time
– Most standardized tests are summative
– Excellent tool for shaping district / school or
course goals
• Major disadvantages
– time lag between assessment and results
– lack of specific information that can be used by
teachers for instruction
Formative Assessments
• Occurs when teachers provide information to
students in ways that enable the student to
learn better, or when students can engage in
a similar, self- reflective process
• Primary purpose support high quality learning
• Most effective for lower performing students;
although all students benefit
Formative Assessments
• Major Advantages
– Can occur in real time
– Maximize teachable moments in the
classroom
– Both teachers and students receive
information necessary for improvement
Profound Mistake
• Students NOT included as active
participants in the assessment process in
order for achievement to improve
Remedies
Create a balanced system of summative
assessments of learning and formative
assessments for learning
A Balanced Approach
Assessment of Learning
How much have students learned as of a
particular point in time?
Assessment for Learning
How can we use assessment to help
students learn more?
High Quality Classroom
Assessment
Teachers should be asking:
•
•
•
•
Why am I assessing?
What am I assessing?
What is the best assessment method?
How do I communicate the assessment
results?
Assessments Tools, strategies, and techniques to analyze
each student’s demonstration of accomplishments of specific
goals and criteria.
•
•
•
•
•
•
Paper-pencil tests
Exhibits
Interviews
Surveys
Observations
Other measures
Would you
rather be at
the beach?
Good assessment requires variety to ensure
opportunities for success for all students.
Traditional Assessment Methods
•
•
•
•
Selected response
Extended written response
Performance assessment
Personal oral communication
Selected Response
Students select an answer from a list or generate a
very brief answer.
Examples:
Multiple Choice
True / False
Matching
Fill in the blank
Label a diagram
Advantages of Multiple Choice
When there is only one correct answer with
several plausible alternatives to the correct
answer, MC is an effective tool to:
• Cover a variety of material effectively
• Be scored easily
• Provide formative information
Multiple Choice – D3
• Provide one, and only one, correct answer
• Include plausible options that demonstrate
a student’s level of understanding
• Maintain a homogeneous feel in style,
length, and visual display
• Colleague review
School Bus - Item
A teacher is planning a field trip and will
need school buses to transport students.
A school bus holds 36 students. If 1,128
students will be transported, how many
buses are needed?
A. 31
B. 31.33
C. 32
D. 36
School Bus - Item
A teacher is planning a field trip and will
need school buses to transport students.
A school bus holds 36 students. If 1,128
students will be transported, how many
buses are needed? Explain your
answer.
A. 31
B. 31.33
C. 32
D. 36
Begin with the end in mind . . .
• MC assessment could be designed so that
each stem presents a different level or skill
set
• Subsequent assignments could be
generated based on the students
response to the MC item
Formative Assessments
• Cautions
– Must be as rigorous as summative
assessment (FCAT)
– Data must be collected and analyzed
– Data must be used to create future
instruction
Which Type of assessment has the
greatest potential to alter classroom
instruction?
Which assessment will produce data
driven decisions in the classroom?
D3 = Effective Formative Assessment System
Moving to D3:
LESSONS LEARNED
•
•
•
•
•
It takes time.
It has to start at the top.
Progress has to be measurable.
Summative models are starting points.
Data–driven decision making can be a
powerful tool in changing student outcomes
and promoting continuous improvement.
70 Years ago ….
H. L. Menchen said:
“ For every complex problem
there is an answer
that is clear,
simple and …….
wrong.”
Tools for Data Analysis
• Evaluation Services Web Site
– Tools
– Documents
– Summative data and charts
– Student level disaggregation
- secure access (password required)
Malcolm Thomas
[email protected]
(850) 469-5386
www.escambia.k12.fl.us/adminoff/eval