Transcript easyCBM: Benchmarking and Progress Monitoring System

easyCBM:
Benchmarking and Progress Monitoring System
Jack B. Monpas-Huber, Ph.D.
Director of Assessment & Student Information
Shereen Henry
Math Instructional Specialist
Overview of the Train the Trainer Model
SEPTEMBER 15: FALL BENCHMARK WINDOW OPENS
September 14: Train the Trainers
We train you what you need to launch easyCBM at your school
We prefer two school training experiences:
1. ASAP: You train teachers how to log in, administer tests
2. Mid-October: You train teachers how to interpret reports, make sense of
data.
September 21, 4-5 pm: Follow-up Session (Shereen and Jack)
For people who need more support (during benchmark window) on purpose of
easyCBM, logging in, administering tests
OCTOBER 14: FALL BENCHMARK WINDOW CLOSES; PROGRESS
MONITORING BEGINS
Late October: Follow-up Session (Shereen and Jack)
Now that we have data…what do we do? Interpreting easyCBM reports and
effective progress monitoring
Organization of today’s training
I.
Overview of the Train the Trainer Model
II.
Why are we doing this? Rationale, purpose, and brief
overview of easyCBM
III.
Application: How easyCBM supports our PLC / RtI work
in Shoreline
IV.
How to use the easyCBM system
V.
Guided exploration using the demo site
VI.
Question & answers
WASL Math and Reading Achievement in
Shoreline, 1997-2009
% of s tude nts m e e ting or e xce eding s tandar d
100%
90%
80%
70%
Gr ade
Gr ade
Gr ade
Gr ade
Gr ade
Gr ade
60%
50%
40%
30%
20%
10%
0%
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
SOURCE: OSPI Report Card (http://reportcard.ospi.k12.wa.us)
4 Math
7 Math
10 Math
4 Re ading
7 Re ading
10 Re ading
Achievement Gap in Shoreline
Grade 4 WASL Math Achievement by Race, 1998-2009
% of s tude nts m e e ting or e xce eding s tandar d
100
80
As ian/Pacific Is lande r
Am er ican Indian
His panic
African Am e r ican
White
60
40
20
0
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
SOURCE: OSPI Report Card (http://reportcard.ospi.k12.wa.us)
Achievement Gap in Shoreline
Grade 10 WASL Math Achievement by Race, 1999-2009
% of s tude nts m e e ting or e xce eding s tandar d
100
80
As ian/Pacific Is lande r
Am er ican Indian
His panic
African Am e r ican
White
60
40
20
0
1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
SOURCE: OSPI Report Card (http://reportcard.ospi.k12.wa.us)
AYP Results: Distribution of “No” Cells
2008
GROUP
MATH READING TOTAL % OF 28
ALL
1
1
2
7.1%
AFRIC AN AMERIC AN 2
2
7.1%
HISPANIC /LATINO
2
2
7.1%
LOW INC OME
4
1
5
17.9%
SPED
7
8
15
53.6%
WHITE
1
1
2
7.1%
TOTAL
17
11
28
100.0%
% OF 28
60.7% 39.3% 100.0%
AYP Results: Distribution of “No” Cells
2009
Counts of "No" Cells
Content
Subgroup
Math Reading
Total
Limited English
1
1
Low income
6
6
Special education
6
1
7
Limited English
7.1%
7.1%
Low income
42.9%
42.9%
Special education
42.9%
7.1% 50.0%
13
1
14
92.9%
7.1% 100.0%
1As
of early August 2009. Includes results of WASL and WASL Basic assessments. Source is OSPI Report Card (http://reportcard.ospi.k12.wa.us).
SHORELINE PUBLIC SCHOOLS
SAT Mathematics
Achievement
SHORELINE PUBLIC
SCHOOLSby Race
SAT Mathematics
Class Achievement
of 2008 by Race
(Class of 2008)
Mexican or Mexican American (N=6)
452
Black or African American (N=15)
439
Ot her Hispanic, Lat ino, or Lat in American (N=6)
493
American Indian/Alaskan Nat ive (N=5)
540
Asian, Asian American, or P acific Islander (N=119)
582
T ot al (N=469)
561
White (N=277)
563
Ot her (N=22)
564
No Response (N=18)
563
200
300
400
500
600
Me an S AT S cal e S core
SOURCE: College Board. (2008). District Profile Report.
700
800
SHORELINE PUBLIC SCHOOLS
SAT Mathematics Achievement by Race
Class of 2009
Mexican or Mexican American
(N=5)
472
Black or Af rican American
(N=21)
422
Other Hispanic, Latino, or
Latin American (N=8)
544
American Indian/Alaskan
Native (N=5)
502
Asian, Asian American, or
Pacific Islander (N=103)
570
552
Total (N=434)
558
White (N=249)
Other (N=24)
538
No Response (N=19)
200
552
300
400
500
Mean SAT Scale Score
SOURCE: College Board. (2009). District Profile Report.
600
700
800
Need for Systemic Math Assessment in
Shoreline
Need more common formative assessment in
mathematics because:
•
Math is an area of concern (according to large-scale
evidence)
•
Achievement gap is acute in mathematics
•
Math is an area of intense scrutiny and community
engagement
•
Math is foundational; is critical to intervene early
•
New graduation requirements (Algebra II) warrant stronger
focus in math at lower grades
Basic Description of easyCBM
Online districtwide formative assessment system
Includes reading and mathematics measures for grades K - 8
Benchmark testing and progress monitoring (RtI framework)
Is fixed form, NOT computer adaptive (like DOMA was)
Measures NCTM Focal Points
Developed by education researchers at the University of
Oregon (who also develop DIBELS)
Formative assessment purposes1
Design
Purpose
Screening
Early identification of student’s strengths or
weaknesses for classification, placement, or
intervention
Diagnosis
Identify the causes of students’ learning problems
-- usually with intent to guide or modify instruction
or design differentiated instruction
Interim
benchmark
measurement
Provide a measure of students’ progress toward
achieving proficient performance on a standardsbased summative test or measure growth on a
measurement scale toward a final summative
assessment event
Progress monitoring
A special type of interim assessment that is
characterized by frequent, repeated assessment
1Stevens,
J. S. (2009). Washington State Diagnostic Assessment Guide. Office of the Superintendent of
Public Instruction.
Benchmark testing, 1/6
Purpose
To screen all students in order to identify those are below
grade level and need intervention
Three benchmark windows (determined by district)
Fall
Winter
Spring
Why three windows?
This first year is baseline year when we are collecting our own
norm data on how much Shoreline kids grow in math. It is
a linear growth model which requires three solid data
points.
Benchmark testing, 2/6
Three benchmark windows
Fall -- September 15 to October 14
Winter -- January 11 to 29
Spring -- May 31 to June 18
For a successful pilot, we need:
1. All K-8 students participate in all three benchmark
windows
2. All students take benchmark test at grade level
Benchmark testing, 3/6
What will the benchmark testing tell us?
Benchmark testing identifies students for intervention and
progress monitoring based on their percentile achievement
easyCBM is NOT criterion-referenced
No “cut score”
No “percent meeting standard”
easyCBM is norm-referenced
Results reported in percentiles
Student scores color-coded by percentile
Norm group is Oregon students (4 large districts last year)
Benchmark testing, 4/6
What percentiles will we use?
10th percentile and below - RED
20th percentile and below - YELLOW
50th percentile and below - WHITE
Hypothetical easyCBM Fall Benchmark Results
Oregon 2008-09 (Norm Group)
0.07
Proportion of students
0.06
0.05
0.04
0.03
0.02
0.01
0.00
0
4
8
12
16
20
24
Raw points
28
32
36
40
44
48
Benchmark testing, 5/6
How many Shoreline students will be identified for intervention?
Depends on how Shoreline students compare to Oregon norm group
Will vary by building and classroom
Hypothetical easyCBM Fall Benchmark Results
Oregon 2008-09 (Norm Group)
Shoreline 2009-10
0.07
Proportion of students
0.06
Assume 640 kids/grade level:
10% = 64 kids
20% = 128 kids
0.05
0.04
0.03
0.02
0.01
0.00
0
4
8
12
16
20
24
Raw points
28
32
36
40
44
48
Benchmark testing, 6/6
THINKING AHEAD TO NEXT YEAR:
What can I learn from my students’ prior year easyCBM results?
How does easyCBM sort our students relative to other math
measures?
Correlations with:
Measure of Student Progress (MSP)
Report card grades
What norms did we set for ourselves? How much do Shoreline
kids grow in math based on the three sets of benchmark
results?
What interventions worked most effectively?
To what extent can we link our percentile thresholds to MSP
level performance?
Progress monitoring
Purpose
To monitor progress, or growth, of lower achieving students toward grade level
goals, ideally in response to intervention/s
A series of short, parallel assessments to be administered semi-frequently (2-3
weeks apart) to document growth until student has reached 50th percentile
Can progress monitor at different grade levels by skill
Grade 7 Algebra - Shereen Henry
QuickTime™ and a
decompressor
are needed to see this picture.
Organization of today’s training
I.
Overview of the Train the Trainer Model
II.
Why are we doing this? Rationale, purpose, and brief
overview of easyCBM
III.
Application: How easyCBM supports our PLC / RtI work
in Shoreline
IV.
How to use the easyCBM system
V.
Guided exploration using the demo site
VI.
Question & answers