Transcript SPMS: Using Classroom Assessment and Reporting to Close

Use and Impact of the
Student Progress
Monitoring System
Mississippi Department of Education
Office of Instructional Programs and Services
Office of Student Assessment
MASS January 10, 2006
Assessment that is explicitly designed to
promote learning is the single most powerful
tool we have for raising standards and
empowering life long learning.
– Assessment Reform
Group (1999)
2
Overview



Student Progress Monitoring System (SPMS) is
designed to offer a web-based approach to help
districts track which students are mastering key
objectives and which are not.
The ultimate goal of SPMS is that teachers use
SPMS assessments to focus on reviewing, revising,
and improving daily instruction through rapid
assessment and feedback.
Through the daily/weekly review of student
progress, districts will have a better way to track
their expectations for meeting annual accountability
requirements and for preparing students for national
standardized tests.
3
District Use





Participation is a district and school decision.
Districts registration for SPMS may include all
schools or only targeted schools.
For the 2005-06 year, the state is funding the entire
per pupil cost, and the MDE intends to fully fund
SPMS for the 2006-07 school year.
Implementation depends heavily on local control;
therefore, districts can structure the use of the
system.
Some districts have chosen to use SPMS more for
“benchmark” testing, while others are using the
system for formative assessment data as well.
4
Application





The item bank currently contains math, reading, and
language arts items aligned to the “old” Mississippi
Curriculum Frameworks.
Coverage includes items for grades 1-12 in these
areas.
Coverage for science and social studies includes
items for grades 3-12.
SPMS also supports writing at grades 4, 7, and 10
(English II) with automated scoring.
Tests can be built by teacher, school, or district.
5
Application




Individual and summary reports enable teachers and
administrators to track, monitor, and disaggregate
data by various demographic categories at the
classroom, school, and district level.
Student portfolios enable teachers and students to
track progress in real time.
SPMS provides system-generated parent letters.
A new resource unit will be unveiled at the district
level at the end of January and will allow users to
use lesson plans, models, and activities already in
the system and to add personal resources for all
SPMS users.
6
Application

Reporting features:



State test results are automatically loaded
into the system.
Summary performance reports provide results
for curriculum competency and objective.
Item analysis and roster reports show
performance for each test and for each
student.
7
Status Comparison

2004-2005
 Districts: 63
 Schools: 244
 Students: 133,000+

2005-2006
 Districts: 116
 Schools: 647
 Students: 398,413
8
2005-2006 State Users


To date in 2005-2006, 247 schools across 78
districts are recording test results in the
Mississippi Student Progress Monitoring System.
Over 70,400 test sessions have been
administered to students in elementary through
high schools in the subject areas of reading,
math, writing, science, and social studies.
9
2005-2006 Users


SPMS testing began in August 2005, with
November and December being the heaviest
testing months in the school year thus far.
Testing continues as a brisk pace in 2006, with
over 5,000 tests already recorded in January.
10
2005-2006 Test Sessions by Month
SPMS Testing Activity Report
35000
30000
20000
2005/06
2004/05
15000
10000
5000
July
June
May
April
March
Feb
Jan
Dec
Nov
Oct
Sept
0
Aug
Volume
25000
Month
11
2005-2006 Test Sessions by Month
SPMS Testing Activity Report
35000
30000
20000
2005/06
2004/05
15000
10000
5000
July
June
May
April
March
Feb
Jan
Dec
Nov
Oct
Sept
0
Aug
Volume
25000
Month
12
Relationship Between SPMS Usage and
State Test Scores



Little is known about the nature of the tests
administered, the importance of the SPMS tests, and
how the data from SPMS were used to drive
instruction.
Investigations into the relationship between usage
of SPMS and performance on state assessments
show that the usage across the state in the 2004-05
year may not have been organized, integrated, or
strategic enough in order to show major trends
across the board.
Case studies provide the most insight into the
effectiveness of the SPMS.
13
Case Study: Forrest County
Grade
1
2
3
4
5
6
7
8
9
10
11
12
unknown
Total
Math
231
361
999
441
234
464
199
114
6
9
5
0
124
3187
Reading
61
232
715
257
269
369
470
300
4
17
5
2
26
2727
Writing
0
0
0
0
0
0
0
25
0
0
0
0
0
25
Social Science
0
0
10
0
0
62
248
974
0
15
72
2
0
1383
Science
0
17
15
19
18
47
80
58
6
19
6
4
0
289
14
Forrest County Grade 3 MCT Math

Gains from 2003-04 to
2004-05 for Grade 3 MCT
Math mean scale score


State gain:
 2.4 scale score
points
 499.8 to 502.2
Forrest County gain:
 10.9 scale score
points
 494.6 to 505.5

Gains for MCT Math from
2003-04 Grade 2 to 2004-05
Grade 3 % Proficient and
Above


State gain:
 0.2%
 91.7% to 91.9%
Forrest County gain:
 6.0%
 86.7% to 92.7%
15
16
Forrest County Grade 3 MCT Math
Black-White Gap from 2003-04 to 2004-05 for
Grade 3 MCT Math, % Proficient and Above

Forrest Co.
State
200304
200405
200304
200405
Black
71
82
87
88
White
96
96
96
96
Gap
25
14
9
8
17
Forrest County Grade 3 MCT Math

Economically Disadvantaged Gap from 2003-04 to 2004-05
for Grade 3 MCT Math, % Proficient and Above
Forrest Co.
State
200304
200405
200304
200405
ED
81
88
89
89
Non-ED
95
96
96
96
Gap
14
8
7
7
18
Case Study: Kemper County
Grade
1
2
3
4
5
6
7
8
9
10
11
12
unknown
Total
Math
0
10
0
97
0
0
264
423
0
0
0
0
0
794
Reading
0
7
0
12
3
113
771
1851
78
2
0
0
0
2837
Writing
0
0
0
0
0
20
0
0
0
0
0
0
0
20
Social Science
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Science
0
0
0
0
0
0
5
0
0
0
0
0
0
5
19
Kemper County Grade 8 MCT Reading

Gains from 2003-04 to
2004-05 for Grade 8
MCT Reading mean
scale score


State decrease:
 7.0 scale score
points
 563.6 to 556.6
Kemper County gain:
 9.4 scale score
points
 544.5 to 553.9

Gains from 2003-04 to
2004-05 for Grade 8
MCT Reading %
Proficient and Above


State decrease:
 -4.8%
 61.6% to 56.8%
Kemper County gain:
 14.2%
 41.1% to 55.3%
20
21
Kemper County Grade 8 MCT Reading

Economically Disadvantaged Gap from 2003-04 to 2004-05
for Grade 8 MCT Reading, Percent Proficient and Above
Kemper Co.
State
200304
200405
200304
200405
ED
37
54
47
44
Non-ED
52
60
77
72
Gap
15
6
30
28
22
Kemper County and SPMS 2004-05



Vicki Parker is a retired Mississippi educator and
certified Vantage trainer.
Vicki worked with Kemper County during the
2004-2005 school year.
Vicki will also be a trainer for the SPMS
Curriculum, Assessment, and Instruction (CAI)
modules.
23
MS In-State Vantage Contact



David Timmons, who taught in the BayWaveland School District, is a certified Vantage
Trainer.
He is now serving as the MS Contact Person for
SPMS.
His e-mail is [email protected].
24
New for 2006






3-in-1 bubble sheets with significant cost
reduction
Version 2.0 for district users
MCT practice tests
Science practice tests
MS-teacher reviewed items for social studies and
science
SPMS Curriculum, Assessment, and Instruction
(CAI) modules
25
Bubble Sheets for Off-line Testing
26
3-in-1 Bubble Sheet
27
Version 2.0



Version 2.0 will be ready for the 2006-2007
school year.
Vantage is making 2.0 available for users at the
district level only at the end of January for “trial
use”; 2.0 will not contain “live” files.
Version 2.0 will have a new resource site where
districts users will be able to enter lesson plans
or activities to the resource list already identified
within the system.
28
Version 2.0


Version 2.0 will have new functions to allow
users to nominate items to be included in the
SPMS database for all users.
Version 2.0 will have functionality to allow
external test data (such as DIBELS) to be
entered into the system.
29
MCT and Science Practice Tests




Release set for late January for first practice test
for grades 3 and 7, all three content areas
Second release set for February for first practice
test for all grades, all three content areas
Spring release for practice tests two and three
for all grades and content areas
Release in spring for practice science tests for
grade 5, grade 8, and Biology I
 (These are not the Riverside hard copy tests.)
30
MS Teacher Reviewed Items




New science and social studies items
All items reviewed by Mississippi teacher
committees
Items to be identified to distinguish them from
original SPMS items
Mississippi courses new to SPMS
 Science: Biology II, Physical Science,
Anatomy and Physiology
 Social Studies: MS Studies, US Government,
and Economics
31
New Math and Language Arts Items




Addition of math and language arts items to
SPMS was delayed so that new items will match
the revised frameworks.
Teachers will meet in late spring and/or early
summer to review items.
Items will be added to SPMS by January 2007.
New MCT2 will not be administered until spring
2008.
32
SPMS CAI Modules
33
SPMS CAI modules



Districts selected for Series
I or II were notified
January 6.
Ten teams were selected
for Series I and for Series
II based upon a rubric
evaluation of the district’s
application.
Letters regarding Series III
and IV will be mailed to
superintendents and SPMS
contacts next week.

Districts that applied but
were not selected for
Series I or II will be able to
submit a “Reapplication” to
be considered for the next
series.



Blank boxes
Non-users
Series V and VI sites have
not yet been selected, but
one will be in south
Mississippi, close to the
coast.
34
CAI Modules



Each series consists of three days of training,
each day separated by about two weeks.
Districts are asked to send three-member teams.
We suggest a central office person directly
involved the district’s plan for student
improvement, a school administrator, and a lead
teacher/SPMS-user teacher. The same team
should attend all sessions.
These series are not initial training sessions for
users. All team members who attend these
sessions should be active SPMS users.
35
CAI Curriculum Objectives
1) Gain a clear understanding of the alignment
between SPMS and the Mississippi Curriculum
Frameworks
2) Develop a scope and sequence guide to assist
with the integration of SPMS for instructional
purposes
3) Develop a plan to use SPMS to ensure student
growth in mastering Mississippi Curriculum
Framework objectives as well as local district
curriculum goals
36
CAI Assessment Objectives
1) Create appropriate SPMS assessments for
targeted objectives
2) Understand SPMS reports available
3) Utilize SPMS assessment data to inform
instruction
4) Establish district priority goals for increasing
student achievement
5) Design a district/school strategic plan and
program of action with measurable outcomes
and an evaluative component
37
CAI Instruction Objectives
1) Use SPMS assessment results to review/critique
instructional practices for targeted objectives
2) Explore SPMS reports to refocus instruction on
targeted curriculum, including pre-requisite skills
3) Connect SPMS reports to the use of researchbased instructional strategies
4) Examine SPMS reports to create learning
opportunities and revise instructional strategies
based on individual or group needs
5) Access resources through SPMS to assist in
providing quality instruction
38
Under Consideration



For the 2006-2007 school year, the MDE and
Vantage are working to have data available
when school begins.
The idea is to “promote” all SPMS students at
the end of the 2005-06 school year to the next
grade and/or next school.
This process would mean districts might have to
“clean up” the data, but the process would allow
districts access as soon as the 2006-07 school
year begins.
39
SPMS Advisory Committee




The SPMS Advisory Committee meets quarterly
during the school year.
The committee makes recommendations and
considers new functions.
Currently, there are 21 members on this
committee.
Michael Hood, assistant superintendent from
Montgomery County, is a “super” user of SPMS
and a member of the Advisory Committee.
40