Transcript Document

Michigan School Testing Conference
March 1, 2006
Harnessing the Wow Factor:
Software for Assessing Student Achievement
and Analyzing Assessment Results
Linda Glowaz, Lake Orion Community Schools
Ernie Bauer, Oakland Schools
Project Planning Process
• Fall 2003 – Meetings with focus groups
• Fall 2004 create RFP
• Oct 2004 – Meeting with Assessment,
Curriculum and Technology directors from
Oakland districts to discuss requirements
• Dec 2004 – RFP sent out to bid
• Jan 2005 – 10 responses received
• May 2005 – Committee selects products
• July 2005 – Oakland School BOE approval
What we are trying to do:
Provide Technology that Will Help
• Improve teaching and increase
learning for all
• Useful reports for teachers, principals and
district administration
• Common assessments tied to GLCEs
• Item banks tied to GLCEs
• Multiple district on-ramps
Oakland & LEA Members Only
(N = 15)
SAS-DAT Team -- Oakland only
Higher
St.
1.0
1.5
Disagree
Disa
2.0
gree 2.5
Not
3.0
Sure 3.5
Ag
4.0
ree
4.5
St. Agree
5.0
Usef ul reports f or teachers
Usef ul longitudinal reports f or administrators
Comprehensive data analysis tool
Usef ul reports f or administrators
Aligns to district curriculum
I
Sof tw are is user f riendly
m
Easy to create & manage tests
p
o
Usef ul item & test statistics f or administrators
r
t
Scanning & scoring is good
a
n Usef ul longitudinal student prof ile f or teachers
c
Allow s creation of multiple item types
e
Helps w ith collaboration on instruction
Provides item bank
SIS interf ace is good
Training model is good
Usef ul reports f or students & parents
Web based testing is good
Lower
Provides instructional resources
Items are arranged by “Importance” rating.)
Vendor A
Vendor B
Pearson
Oakland Schools Board of Education
agreed to spend up to $1,600,000
in 2005-06 to make
Pearson Benchmark “Lite” & Inform
available to all districts.
We have a CD available with nearly everything we did pretty well
documented, the RFP, rating forms, results of ratings, etc.
Major Parts of Each System
“Pearson Benchmark”
Student Assessment System
“Pearson Inform”
Data Analysis Tool
• Curriculum
Framework (GLCE’s) •
•
• Items
•
• Tests
•
• Administer tests
• Score & Report
Interface with SIS
Import external tests
Import Benchmark tests
Select & Analyze Groups
An assessment Portfolio
“for learning”
An electronic CA-60
“of learning”
– Graphs
– Drill Down
The Big Picture – The District
• Board Goal (7 years)
–
–
–
–
Student achievement
Align Curriculum
Communicate Curriculum
Assessment Strategies
•
•
•
•
MEAP
Norm-referenced tests
Common Assessments
Benchmark Report Cards
– Data-driven decision making
The Big Picture – High School
•
•
•
•
•
•
•
Global Vision
Data driven goals
Total secondary effort
Common assessments
True alignment to state benchmarks
Breakdown to 4 areas of mastery
Presentation of vision to Board at 2005 Winter
Workshop
What is Pearson Benchmark?
•
•
•
•
Comprehensive
Customizable
Web-based
District benchmark testing system and
reporting tool
What is Pearson Benchmark?
• Teachers can monitor student progress
– Against standards and benchmarks
– Address deficiencies
– Recheck mastery
• Result
– More targeted instruction
– Shorter assessment cycles
– More effective teaching and enhanced student
performance
Initial Challenges
• Teacher Fear
–
–
–
–
–
Is the data going to be used for teacher evaluations?
Is this going to take away “what” I’m teaching?
Is this going to change “how” I’m teaching?
How much of “my time” is this going to take?
How do I make this work for me?
(i.e., non-core areas)
• Benchmarks … what are those?
• Defining Common Assessments
Goal for High School
• By June 2006 every course in the high
school will administer common assessments
using Pearson Benchmark
– Scanned sheets
– Online
– rubrics
Timeline
•
•
•
•
Summer 2005:
September 27:
October 26/27:
Nov/Dec:
• January/May:
• June:
volunteer high school team
access to Benchmark partition
testing/scanning
try out different assessment
scenarios
additional course assessments;
middle school assessments
create and administer
benchmark assessments in all
high school courses and selected
middle school courses
We’re going to do WHAT?
….WHEN?
• Denial
– This too shall pass
• Panic
– We’re not ready
– Are you ready?
• Relief
– This is easier than we thought!
– I like what it does for me.
Making It Work
•
•
•
•
•
•
Be Organized
Ask for 2 Answer Keys
Be Available
Do It Yourself
Be a Cheerleader
Be a Duck
Participation
• 6 departments
–
–
–
–
–
–
•
•
•
•
Social Studies
Science
Mathematics
World Languages
Physical Education
Business
17 courses
46 teachers
88 sections
Over 5000 scan sheets (approximately 40% of students)
Before Assessment
•
•
•
•
•
Meet with department heads
Review exams with course teams
Create answer keys
Pre slug scan sheets
Verify data (pre assessment)
– Course benchmarks
– Class rosters
After Assessment
• Scan Sheets
• Verify data (post assessment)
– Assessed students
– results
• Distribute basic results to participating teachers
• Review detailed reports with participating teachers
(individually)
• All-staff professional development
– Share experiences
– Review reports
This program doesn’t give us answers …
it gives us questions!
Is this a TEST
Or
an ASSESSMENT?
The Results
Mastery By Benchmark
Individual Student Mastery
Test Item Analysis
Class Item Analysis Report
Impact of Participation
• Dialogue improved between teams
– Teachers discussed/modified student assessments
– Teachers discussed assessment question issues
– Teachers discussed assessment design
• Increased participation in benchmark testing
• Teacher’s comfort level of the common assessment process
was eased
• Teachers are beginning to “own it”
• Cultural and climate changes
“Technical Challenges”
• Assessment design
• Speed/Response time
– Benchmark structure
• Framework
• Creating items and tests
– Results
• Initial upload
• Deleting/correcting input errors
Where Do We Go From Here?
• Now through August
– Create and administer benchmark assessments
in all high school courses
– Develop and administer common assessments
in middle schools
– Design/modify instructional practices based on
data
– Create goals for next school year based on this
year’s data
Assess, Address, Progress
• Continuously measure student performance against state
and district benchmarks
• Create, deliver, score and report on benchmark-based
district and building level tests (online or offline)
• Apply a comprehensive suite of analytics with objective,
valid, and reliable evidence of student progress toward
meeting district and state benchmarks and standards
• Monitor student progress and inform instructional
decisions accordingly
Benchmark Test Results
Item Analysis
• Click on the question
number to see the
question itself.
• Click on the icon next
to the question number
to see a breakdown of
the item’s performance
by demographic
category.
Benchmark Test Results
Frequency Distribution
Pearson Benchmark
Benchmark Lite ends here
4th Grade (Reading* & Math)
School Year
2003-04
4
2004-05
2005-06
5th Grade (Science & Soc. Studies)
6
2003-04
4
2004-05
5
2005-06
7th Grade (Reading*)
2003-04
2005-06
9
5
7
6
8th Grade (Math, Science, Soc. Studies)
7
2004-05
5
2003-04
7
2004-05
8
2005-06
8
8
10
9
MEAP Proficiency Profiles & Levels
in Inform
• State Levels
– (apprentice, basic, meets, exceeds)
• % Correct
– (0-25%, 26-74%, 75-100%)
• PR 3 Bands
– (1-25%ile, 26-74%ile, 75-99%ile)
• PR 4 Bands
– (1-25%ile, 26-49; 50-74; 75-99)
• Strand Odds
– (Low, Moderate, High)
– (<30%; 30-70%; >70%)
Parent’s / Student’s Dashboard
OS Support
Oakland Schools’ Continuing
Role in Pearson Benchmark
We’re here for you…help is just a
phone call away!
Oakland Schools’ Continuing Role in
Inform
• Create structure for naming/filing queries
for
– Principals
– Teachers
• Create a consistent set of queries for each
• Teach all principals to run their own queries
• Get additional test data into Inform
Professional Development for LEA’s
• Using data to inform instruction
• Using Benchmark & Inform for grouping and
differentiation
• Using the Benchmark with Common Assessments
• Using the Benchmark for Classroom Assessments
• Administrator use of Inform
• SIP Planning using both products
Current Status
Pearson Benchmark & Inform
28
26
25
24
20
Number of Districts
20
18
16
12
8
4
0
Inform
Inform
Data Validation
(Completed, Scheduled,
or Planned)
Lite
Full
Benchmark