Transcript faculty.ung.edu

Evaluating the Implementation
of Singapore Math:
A Large-Scale Study of 21
Elementary Schools
James Badger
Dianna Spence
North Georgia College & State University
NCTM Regional Conference 2009
Thursday, November 19
Agenda
 Singapore Math Overview & Examples

Dianna
 Overview of Research & Methodology

James
 Quantitative Findings
Student Performance (Test Scores)
 Dianna
 Conceptual Framework & Qualitative Findings
(Surveys, Observations, Interviews)

James
What Is Singapore Math?
 Curriculum based on elementary
mathematics teaching techniques used in
Singapore
 Initial curriculum: “Primary Mathematics”
 Created in 1981
 Developed by CDIS (Curriculum
Development Institute of Singapore)
 Revisions
 1992: stronger problem-solving
focus (2nd Ed.)
 1999: reduced content (3rd Ed.)
 2001 & forward:
adapted for U.S.
Why Singapore Math?
Trends in International Math/Science Study
 Singapore 4th
graders consistently
outperforming 4th
graders in other
countries
TIMSS: Mean Score, 4th Grade Math
COUNTRY
1995
2003
Singapore
590
594
Hong Kong
557
575
Japan
567
565
Netherlands 549
540
Latvia
499
533
England
484
531
Hungary
521
529
U.S.
518
518
Cyprus
475
510
Australia
495
499
New Zealand 469
496
Scotland
493
490
Slovenia
462
479
Norway
476
451
Source: http://nces.ed.gov/timss
Characteristics of Singapore Math
 Concrete  pictorial  abstract approach for
each concept
 Strong emphasis on place value
 Repetitive drill minimized: topics are
sequenced to reinforce/apply skills
 Problem solving based on conceptual
approach rather than memorization of rules,
“clue words”
Hallmark Strategies of
Singapore Math
9
2
 Number bonds

operations and part-whole relationships
 Mental math

7
6,325 + 400 = 6,725
leverages and reinforces place value
 Bar models

helps conceptualize arithmetic operations,
fractions, ratios, algebraic thinking
“12 of Jack’s
marbles are red,
which is 2/9 of
his collection…”
Example:
Place Value Disks
Thousands
Hundreds
100
100
100
537
+ 184
Tens
10
100 100
10
100
7
10
10
Ones .
10
1
1
1
1
10
10
10
10
2
10
100
10
10
1
1
1
1
1
1
1
1
Example:
Bar Modeling
“12 of Jake’s marbles are red, and these
make up 2/9 of his collection. How many
marbles in Jake’s collection are not red?”
12
6 x 7?= 42
6 6 6 6 6 6 6 6 6
Whole collection
Answer: 42 marbles in Jake’s collection are not red.
Classroom Best Practices
 Concrete  Pictorial  Abstract
3
+
4
3
4
 Emphasis on place value, mental math
 Conceptual approach, not rule-based
 Spiral approach to topics
Research Questions
1. Has the implementation of Singapore Math resulted
2.
3.
4.
5.
in higher student math scores?
Has the implementation of Singapore Math had a
positive impact on student interest and/or confidence
in mathematics?
Has the implementation of Singapore Math resulted
in measurable changes in the teachers’ attitudes
toward mathematics?
Is there fidelity in the implementation of the
Singapore Math curriculum?
How do elementary teachers implement the
Singapore Math curriculum?
Research Design
 County-wide implementation in a school district in
the Southeastern U.S.
 Research Setting


21 experimental elementary schools
 Every elementary school in the county
 All K-4 teachers used Singapore Math (first year)
3 control schools
 From another county with similar demographics
 State-approved curriculum (no Singapore Math)
 Participants
 One teacher in each grade (K-4) from each of the 24
schools volunteered to participate
Qualitative and Quantitative Data
i.
ii.
iii.
iv.
v.
vi.
Interviews with teachers & students
Participating teachers’ journals (4 times)
Classroom observations
Teacher and student survey – fall/spring
Video-taping of mathematics lesson (4 times) –
analysis using TPR (Teaching Performance
Record)
Standardized test scores
Quantitative Findings:
Standardized Test Scores
 What standardized test scores did we examine?


State criterion-reference test:
Criterion-Reference Competency Test (CRCT)
Nationally norm-referenced test:
Iowa Test of Basic Skills (ITBS)
 What patterns did we look for?



Degree of increase or decrease in mean score
Increase or decrease in students meeting
minimum requirements
Increase or decrease in percentile rankings
Student Test Performance:
Things to Keep in Mind
 Testing occurred during most teachers’ first year
using new curriculum
 Most students in higher grades (e.g.,
3rd and 4th) had not previously been taught
Singapore Math
 Data we are really interested in will not be
available for 3-4 more years.
Student Performance: CRCT
School Mean Math Score by Grade
Change in CRCT Math Mean Score
Grade 1
Decreased
20%
Change in CRCT Math Mean Score
Grade 2
Increased
80%
Decreased
15%
Increased
85%
Student Performance: CRCT
School Mean Math Score by Grade
Change in CRCT Math Mean Score
Grade 3
Change in CRCT Math Mean Score
Grade 4
Decreased
30%
Increased
70%
Decreased
15%
Increased
85%
Student Performance: CRCT
Percent Change in Mean Math Score
Percent Change in Math CRCT Mean Score
Grade 1 - Top 5 vs. Lowest 5
1.54%
1.35%
1.32%
1.04%
1.01%
0.03%
-0.13%
-0.15%
-0.16%
-0.62%
-5.00%
-4.00%
-3.00%
-2.00%
-1.00%
0.00%
1.00%
Mean Score Change
2.00%
3.00%
4.00%
5.00%
Student Performance: CRCT
Percent Change in Mean Math Score
Percent Change in Math CRCT Mean Score
Grade 2 - Top 5 vs. Lowest 5
2.34%
1.40%
1.22%
0.96%
0.83%
0.11%
0.02%
-0.01%
-0.13%
-0.28%
-5.00%
-4.00%
-3.00%
-2.00%
-1.00%
0.00%
1.00%
Mean Score Change
2.00%
3.00%
4.00%
5.00%
Student Performance: CRCT
Percent Change in Mean Math Score
Percent Change in Math CRCT Mean Score
Grade 3 - Top 5 vs. Lowest 5
4.32%
2.33%
1.92%
1.86%
1.77%
-0.24%
-0.49%
-0.51%
-0.93%
-2.64%
-5.00%
-4.00%
-3.00%
-2.00%
-1.00%
0.00%
1.00%
Mean Score Change
2.00%
3.00%
4.00%
5.00%
Student Performance: CRCT
Percent Change in Mean Math Score
Percent Change in Math CRCT Mean Score
Grade 4 - Top 5 vs. Lowest 5
3.72%
2.52%
2.10%
1.50%
1.37%
0.17%
0.01%
-0.30%
-0.83%
-0.86%
-5.00%
-4.00%
-3.00%
-2.00%
-1.00%
0.00%
1.00%
Mean Score Change
2.00%
3.00%
4.00%
5.00%
Student Performance: CRCT
Students Meeting Min. Requirements
Change in Percentage of Students
Meeting CRCT Math Minimum Requirement
Grade 1
Decreased
15%
Change in Percentage of Students
Meeting CRCT Math Minimum Requirement
Grade 2
Increased
85%
Decreased
5%
Increased
95%
Student Performance: CRCT
Students Meeting Min. Requirements
Change in Percentage of Students
Meeting CRCT Math Minimum Requirement
Grade 3
Change in Percentage of Students
Meeting CRCT Math Minimum Requirement
Grade 4
Decreased
25%
Increased
75%
Decreased
40%
Increased
60%
Students Meeting CRCT Math Req.’s
Change in Percentage Points
Change in Percentage of Students
Meeting Minimum Math CRCT Requirements
Grade 1: Top 5 and Lowest 5
15.4
13.9
12.3
9.4
8.2
2.3
0.1
-1.9
-2.5
-3.1
-40.0
-30.0
-20.0
-10.0
0.0
10.0
Change in Percentage
20.0
30.0
40.0
Students Meeting CRCT Math Req.’s
Change in Percentage Points
Change in Percentage of Students
Meeting Minimum Math CRCT Requirements
Grade 2: Top 5 and Lowest 5
24.5
9.8
9.6
9.1
8.9
0.3
0.2
0.2
0.1
-1.4
-40.0
-30.0
-20.0
-10.0
0.0
10.0
Change in Percentage
20.0
30.0
40.0
Students Meeting CRCT Math Req.’s
Change in Percentage Points
Change in Percentage of Students
Meeting Minimum Math CRCT Requirements
Grade 3: Top 5 and Lowest 5
34.0
28.4
13.9
13.7
12.8
-2.8
-3.5
-6.7
-8.0
-24.0
-40.0
-30.0
-20.0
-10.0
0.0
10.0
Change in Percentage
20.0
30.0
40.0
Students Meeting CRCT Math Req.’s
Change in Percentage Points
Change in Percentage of Students
Meeting Minimum Math CRCT Requirements
Grade 4: Top 5 and Lowest 5
27.8
27.7
19.2
11.5
7.7
-5.4
-5.6
-6.4
-7.0
-7.9
-40.0
-30.0
-20.0
-10.0
0.0
10.0
Change in Percentage
20.0
30.0
40.0
Student Performance: ITBS
Mean Percentile Ranking in Math
Change in ITBS Mean
Percentile Ranking in Math
Grade 4
Change in ITBS Mean
Percentile Ranking in Math
Grade 2
Decreased
30%
Change in ITBS Mean
Percentile Ranking in Math
Grade 3
Increased
70%
Decreased
0%
Increased
100%
Decreased
0%
Increased
100%
Student Performance: ITBS
Change in Mean Percentile Ranking
Change in Mean Percentile Ranking
on ITBS Math Scores
Grade 2: Top 5 and Lowest 5
11.39
10.32
9.11
8.41
8.20
-0.86
-1.79
-2.23
-6.00
-6.35
-30.00
-20.00
-10.00
0.00
10.00
Change in Mean Percentile Ranking
20.00
30.00
Student Performance: ITBS
Change in Mean Percentile Ranking
Change in Mean Percentile Ranking
on ITBS Math Scores
Grade 3: Top 5 and Lowest 5
17.29
15.83
12.44
12.11
11.91
5.70
4.65
2.43
1.08
0.98
-30.00
-20.00
-10.00
0.00
10.00
Change in Mean Percentile Ranking
20.00
30.00
Student Performance: ITBS
Change in Mean Percentile Ranking
Change in Mean Percentile Ranking
on ITBS Math Scores
Grade 4: Top 5 and Lowest 5
29.47
21.32
18.67
17.29
16.80
5.79
4.37
2.44
1.91
0.87
-30.00
-20.00
-10.00
0.00
10.00
Change in Mean Percentile Ranking
20.00
30.00
Theoretical Framework
Fidelity of Curriculum Implementation
(O’Donnell, 2008)
 Curriculum potential
 Teaching
 Curriculum-in-use
 Adaptation
Fidelity of Curriculum Implementation
(O’Donnell, 2008)
Guiding Questions
Curriculum profile
What are the critical
components of the
curriculum? What
ranges of variations
are acceptable?
What does it mean
to implement the
program with
fidelity (as defined
by school
administrators and
county supervisors)?
Teaching
How does one
distinguish good
teaching and
fidelity of
implementation to
good teaching
practices
prompted by the
curriculum
material?
Curriculum-in-use
How is the
curriculum and the
perceived
curriculum viewed
and implemented
by teachers? How
are curriculum
materials and
instruction
mutually
supportive and
reinforcing?
Adaptation
Does the curriculum
promote variation
and adaptation of
curriculum
implementation?
1. Survey Results
•
•
Teachers, Kindergarten in particular, indicated a
stronger affinity for and liked teaching
mathematics at the end of the 2008-2009 school
year than they had previously reported.
Content knowledge of mathematics is important
for effective teaching: teachers report some
degree of understanding and confidence in
teaching mathematics.
Survey Results, cont.
 A degree of satisfaction with the training and
resources for mathematics teaching in 2009 –
i.e. Singapore Math training and mentoring
initiatives were apparently noticed and
appreciated by many teachers.

Echoed in interview and journal data
2. Interview Results
 A fluid integration of the new curriculum: a
consequence of the training provided by the
county and ongoing support delivered by school
administration
 Teachers reported manipulatives frequently
integrated in the classroom
- value discs and number bonds cited as
fostering learning
Interview Results, cont.
 Teachers report students possessed a deeper
understanding of mathematical concepts with
the Singapore Math curriculum.
 Teachers claimed that they have higher
expectations of students in Singapore Math.
 Parents’ reactions to Singapore Math ranged
from enthusiasm to frustration.
3. Journal Results
 Teachers wrote that students liked using place
value disks

a helpful hands-on manipulative to assist students
grasp the concept of place value.
 Teachers wrote that students enjoyed
activities, games, and manipulatives.
-
the students showed enthusiasm in class.
Students described these parts of math class
as “fun”.
4. Classroom Observation
 Some teachers


tended to emphasize low-level cognitive
processes in their instruction
rarely asked students to draw associations to
real-world contexts


accountability pressure and time constraints?
preponderance of teacher instruction at the
expense of higher cognitive instruction, deeper
questioning, and more occasions for cooperative
student learning?
Second Year, 2009-2010
 No design changes in the second year of the
study – i.e. same data collection instruments,
teachers volunteer to participate, fifth grade
classes added, compare first and second year
data.
 Data collected in the second year will
- Determine student comprehension and
achievement, fidelity of curriculum
implementation.
Findings in Context
Sources of Data
 Survey Data
 Journal Data
 Interview Data
 Observation Data
 Standardized Test
Scores
Theoretical Framework:
Fidelity of Curriculum
Implementation
(O’Donnell, 2008)
 Curriculum potential
 Teaching
 Curriculum-in-use
 Adaptation
CHART
Preliminary Observations
 Teacher training and support
are essential
 Not a “drop-in” solution, especially at higher
grades (need phased approach)
 Parent “buy-in” is important
 Will take time to see full impact