Transcript Predicting MCAS - Data Team Consulting

Introduction To The
Middle School
Data Team
This Demo Uses Fictional Data For Purposes of Example
The Ultimate Goal is to Answer
Your Questions with Your Data
DataTeamConsulting.Com
1
Middle School Data Team
Purpose of the Data Team
The purpose of the Middle School data team is to use quantitative and
qualitative data, as much as possible, to improve teaching and learning.
Members of the Data Team
The purpose of this presentation is to invite you to become a member of
the data team. To provide a taste of the possibilities, we will present one
line of inquiry of interest to the current members. The current members:
Roberto da Costa
Samuel Guthrie
Xi’an Coy Manh
Danielle Moonstar
Sean Parker (consultant)
Rahne Sinclair
DataTeamConsulting.Com
2
How well do 5th grade Math MCAS scores predict
6th grade Math MCAS scores?
Scatterplot of 6th grade math scores vs. 5th grade math scores (n = 159).
We will use
MCAS scale
scores.
Advanced
220 = N.I.
240 = Prof.
260 = Adv.
Proficient
Needs
Improvement
Warning/Failing
DataTeamConsulting.Com
3
What does the scatterplot tell us?
Scatterplot of 6th grade math scores vs. 5th grade math scores (n = 159).
Initials: F.J.
Math MCAS
5th Grade: 222
6th Grade: 228
Initials: L.M.
Math MCAS
5th Grade: 280
6th Grade: 274
Initials: J.P.
Math MCAS
5th Grade: 244
6th Grade: 224
Initials: R.K.
Math MCAS
5th Grade: 246
6th Grade: 268
Initials: L.H.
Math MCAS
5th Grade: 238
6th Grade: 248
DataTeamConsulting.Com
4
Each data point is a small part of a much larger story.
Scatterplot of 6th grade math scores vs. 5th grade math scores (n = 159).
DataTeamConsulting.Com
5
We can make predictions based on the (limited) data.
Scatterplot of 6th grade math scores vs. 5th grade math scores (n = 159).
DataTeamConsulting.Com
6
The prediction line is “pulled” by every observation.
Scatterplot of 6th grade math scores vs. 5th grade math scores (n = 159).
DataTeamConsulting.Com
7
Once we have a prediction line,
we can use it to make predictions.
Scatterplot of 6th grade math scores vs. 5th grade math scores (n = 159).
Example:
For students
who score
240 on the
5th grade
Math MCAS,
we predict
that they will
score 242 on
the 6th grade
Math MCAS.
242
Of course, some students do much better (or worse) than our prediction.
DataTeamConsulting.Com
8
We can also use the prediction to identify students who
break the pattern.
Scatterplot of 6th grade math scores vs. 5th grade math scores (n = 159).
Example:
Some
students do
much better
on the 6th
grade Math
MCAS than
we would
predict based
on their 5th
grade scores.
Is there a pattern within the pattern?
DataTeamConsulting.Com
9
Is there a pattern within the pattern?
We have two valuable resources to answer this question.
• First and foremost, we have teachers with deep professional knowledge
of these students. For example, if given a list of names of students who do
better than predicted, teachers may be able to discern commonalities and
perhaps causes.
• Secondly, we have additional quantitative information that we can bring to
bear. For example, first semester grades provide information over and
above the previous year’s MCAS score. (Note that numbers alone cannot
get at causes unless we have random “treatment” and “control” groups.)
For grades, we will use an average of English, social studies, math and
science, because we know that all subjects are correlated with all MCAS
tests. (We will include more subjects in the average as we get the data.)
DataTeamConsulting.Com
10
Using grades to uncover the pattern within the pattern.
Students who earned an A in their first semester at Middle School: RED
DataTeamConsulting.Com
11
Using grades to uncover the pattern within the pattern.
Students
Studentswho
whoearned
earnedan
aB
A in
in their
their first
first semester
semester at
at Middle
Middle School:
School: Blue
RED
A
DataTeamConsulting.Com
12
Using grades to uncover the pattern within the pattern.
Students
Studentswho
whoearned
earnedaaCBinintheir
theirfirst
firstsemester
semesteratatMiddle
MiddleSchool:
School:Green
Blue
A
B
DataTeamConsulting.Com
13
Using grades to uncover the pattern within the pattern.
Students who earned a C in their first semester at Middle School: Green
A
B
C
DataTeamConsulting.Com
14
We can use the prediction lines to make predictions.
Predicted 6th grade math scores based on 5th grade math scores and first-semester grades.
Example:
For students
who score
240 on the
5th grade
Math MCAS,
we predict
differently
depending on
how they
perform in
the first
semester of
the 6th grade.
A
252
242
232
B
C
The previous year’s MCAS score is not destiny.
DataTeamConsulting.Com
15
We can use this information to flag students
who may need extra help.
Predicted 6th grade math scores based on 5th grade math scores and first-semester grades.
Example:
Almost all
students who
earn a C in
the first
semester of
the 6th grade
are at risk of
scoring less
than 240,
regardless of
how they
scored on the
5th grade
math MCAS.
A
B
C
16
Some students break the pattern,
and teachers can see who.
Predicted 6th grade math scores based on 5th grade math scores and first-semester grades.
Example:
Some
students do
much better
on the 6th
grade Math
MCAS than
we would
predict based
on 5th grade
scores and
first term
grades.
A
B
C
DataTeamConsulting.Com
17
We can construct plots for not only Math MCAS
but also for ELA MCAS.
Predicted 6th grade MCAS scores based on 5th grade scores and first-semester grades.
Math
ELA
280
Predicted 6th Grade ELA MCAS
Predicted 6th Grade Math MCAS
280
260
A
240
B
220
200
200
C
220
240
260
280
260
A
240
B
220
C
200
200
5th Grade Math MCAS
DataTeamConsulting.Com
220
240
260
280
5th Grade ELAMCAS
18
We can also construct plots to predict 7th grade scores.
Predicted 7th grade MCAS scores based on 6th grade scores and first-semester grades.
Math
ELA
280
Predicted 7th Grade ELA MCAS
Predicted 7th Grade Math MCAS
280
260
A
240
B
220
200
200
C
220
240
260
280
260
A
240
B
220
C
200
200
6th Grade Math MCAS
DataTeamConsulting.Com
220
240
260
280
6th Grade ELA MCAS
19
We can also construct plots to predict 8th grade scores.
Predicted 8th grade MCAS scores based on 7th grade scores and first-semester grades.
Math
ELA
280
Predicted 8th Grade ELA MCAS
Predicted 8th Grade Math MCAS
280
260
A
240
B
220
200
200
C
220
240
260
280
260
A
240
B
220
C
200
200
7th Grade Math MCAS
DataTeamConsulting.Com
220
240
260
280
7th Grade ELA MCAS
20
Future Lines of Inquiry
Future lines of inquiry for the data team:
• Does the MCAS/grades pattern hold up across the years? (The current
patterns are based on one cohort.)
• What other factors do we need to consider? E.g., special education
status, free lunch eligibility, classroom conduct as assessed by teachers.
• Based on professional experience and knowledge, what are the potential
causes? How can we dig deeper into the causes using qualitative and
quantitative data?
Future lines of inquiry for an action team:
What policies and interventions do the data suggest? (The data team and
the action teams should be separate. The purpose of the data team should
be informative.)
DataTeamConsulting.Com
21
Appendix I
GPA 4.0 Scale
(Continuous)
See Appendix II for
3-D Scatterplot
RVF Plot:
20
Assumption Checking
10
• Appears Normal
• Appears Linear
• No Apparent Outlier Problem
-20
-10
• Appears Homoskedastic
0
Residuals
• Independence Is Tricky
200
220
• No Stat Sig Interaction
DataTeamConsulting.Com
240
Fitted values
260
280
22
Appendix II
DataTeamConsulting.Com
23