Transcript Document

A Look Inside the Smarter Balanced
Assessment System
Joe Willhoft, Executive Director
52nd Annual Northwest Mathematics
Conference
Hyatt Regency, Bellevue, Washington
October 11, 2013
Smarter Balanced
Assessment System Overview
“Not just another test”…Smarter Balanced is
being built by states for states
1 Preparing our students for a changing world
2
Connecting learning to life after high school –
career or college
teachers with a practical suite of
3 Supporting
resources
meaningful information to guide
4 Providing
student growth
5 Keeping educators in the driver’s seat
3
A State-led Assessment Consortium
•
•
•
•
26 member
states and
territories
representing
39% of K-12
students
23 Governing
States, 2
Advisory States,
1 Affiliate
Member
Washington
state is fiscal
agent
WestEd provides
project
management
services
4
A State-led Assessment Consortium:
Sustainability for 2014-15 and Beyond
•
Future
affiliation with
the Graduate
School of
Education
and
Information
Studies at
UCLA
5
Improving Teaching & Learning
Common
Core State
Standards
specify
K-12
expectations
for college
and career
readiness
Summative:
College and career
readiness
assessments for
accountability
Teachers and
schools have
information and
tools they need to
improve teaching
and learning
Formative
resources:
Supporting
classroom-based
assessments to
improve instruction
All students
leave
high school
college
and career
ready
Interim:
Flexible and open
assessments, used
for actionable
feedback
6
Individual Score Reports for
Grades 3-8 and 11
Overall Claim for Grades 3-8
ELA/Literacy
Mathematics
Overall ELA/Literacy Score
Overall Mathematics Score
Claim #1 - Reading
Claim #1 – Concepts &
Procedures
Claim #2 - Writing
Claim #2 –Problem Solving
Claim #4: Modeling & Data
Analysis
Claim #3 - Speaking and
Listening
Claim #3 – Communicating
Reasoning
Claim #4 - Research/Inquiry
7
Claims for the Mathematics Summative Assessment
Overall Claim for Grades 3-8
Overall Claim for Grade 11
“Students can demonstrate progress toward college and career
readiness in mathematics.”
“Students can demonstrate college and career readiness in
mathematics.”
Claim #1 - Concepts &
Procedures
“Students can explain and apply mathematical concepts and
interpret and carry out mathematical procedures with precision and
fluency.”
Claim #2 - Problem Solving
“Students can solve a range of complex well-posed problems in pure
and applied mathematics, making productive use of knowledge and
problem solving strategies.”
Claim #3 - Communicating
Reasoning
“Students can clearly and precisely construct viable arguments to
support their own reasoning and to critique the reasoning of others.”
Claim #4 - Modeling and Data
Analysis
“Students can analyze complex, real-world scenarios and can
construct and use mathematical models to interpret and solve
problems.”
Smarter Balanced
Educator Involvement and Progress
Educator Involvement Item Development
Activity
Total Educators
Item Author
147
Item Review - Accessibility
53
Item Review - Bias
50
Item Review - Content
138
Pilot Test Data Review
26
Pilot Test Range Finding
96
Stimulus Review - Accessibility
28
Stimulus Review - Bias
31
Stimulus Review - Content
32
Grand Total
601
10
Ongoing Involvement
•
•
•
Pilot Test in spring 2013 involved over 650,000
students from all Smarter Balanced states
Field Test in spring 2014 will involve over two
million students in all Smarter Balanced states
Standard setting in fall 2014 will invite all
teachers to participate in virtual “crowdsourcing”
11
Working with Higher Education
•
•
•
•
•
Ongoing state-level outreach to faculty and
administrators
– Five Regional Coordinators
– Higher education leads in each state
– Two higher education reps on Executive Committee
Higher education votes on major policies
College Content-readiness Policy
Career Readiness Task Force
Comprehensive validity research agenda
12
Major Milestones in Development of
Summative Assessments
✔
Cognitive
Labs
Apr – Aug
2012
✔
Small
Scale
Trials
Mar – Nov
2012
✔
Pilot
Testing
Feb –
May 2013
Early Q.C. of items & software; no student
results
Field Test
Mar –
June
2014
Deploy For
Operational
Use
Sep 2014
Full system
run-through;
Establish
performance
standards;
some results
13
Field Testing
•
•
•
•
March - June 2014
Sampling about 20% of students in Consortium
Purpose: Evaluate items and tasks for Smarter
Balanced pool…
 Statistical data analysis of 20,000+ items
 Divide items/tasks into secure (summative) pool
and open (interim) pool
 Conduct standard setting for different performance
levels (“cut scores”)
US Dept. of Ed flexibility to avoid double testing
14
Setting Performance Standards
•
•
•
•
•
Setting standards for four Performance Levels at each
grade
September 2014, using Field Test data
Grade-level panels of 30 per content area recommend
cut scores
– At least two representatives from each Governing State on each
grade level panel
Concurrent “crowd-sourced” recommendations from the
field
Recommendations presented to state chiefs for
approval in late September 2014
15
Smarter Balanced
Getting Ready for the Tests
Testing Window for Summative
Assessments in 2015
•
Each state sets its own schedule for the Smarter Balanced
summative assessments within the following windows:
– Grades 3-8 Testing can begin once sixty-six percent (66%) of a
school’s annual instructional days have been completed;
– Grade 11 Testing can begin once eighty percent (80%) of a school’s
annual instructional days have been completed; and
– All Grades Testing can continue up to and including the last day of
school.
•
Smarter Balanced is designed to have results returned
within 2 weeks of of testing being completed.
17
Technology Requirements:
Responding to School Needs
•
•
•
•
Smarter states have established standards for new and
existing hardware
Online “Readiness Tool”
– Schools and districts can evaluate technology readiness
Schools do NOT need one-to-one computers
– Illustrative example: A 600-student school can be supported by a
single 30-computer lab
– Smarter Balanced Readiness Calculator at:
http://www3.cde.ca.gov/sbactechcalc/
Pencil-and-paper option available for three-year
transition period
18
Accessibility & Accommodations
19
Accessibility & Accommodations
(from S.B. Guidelines)
20
Estimated Testing Times for 2014 Field
Test and 2015 Operational Assessment
Test Type
English
Language
Arts
Mathematics
COMBINED
Grades
CAT
Perf.
Task
Only
3-5
1:30
2:00
3:30
:30
4:00
6-8
1:30
2:00
3:30
:30
4:00
11
2:00
2:00
4:00
:30
4:30
3-5
1:30
1:00
2:30
:30
3:00
6-8
2:00
1:00
3:00
:30
3:30
11
2:00
1:30
3:30
:30
4:00
3-5
3:00
3:00
6:00
1:00
7:00
6-8
3:30
3:00
6:30
1:00
7:30
11
4:00
3:30
7:30
1:00
8:30
Total
Class
Activity
Total
Times are estimates of test length for most students. Smarter Balanced assessments are
designed as untimed tests; some students may need and should be afforded more time than
shown in this table.
21
Take a Peek Under the Hood…
The Smarter Balanced Practice Test
•
•
•
Shows item types and tools/features
Now available: Released May 29
Expanded full-feature version in spring 2014
http://www.smarterbalanced.org/practice-test
22
A Deeper Dive
Mathematics Content Specifications and Test
Blueprints
Test Specifications:
Click here….
Then, scroll to here…
Gr. 6 Math Blueprint: Claim 1 (DRAFT)
Gr. 6 Math Blueprint: Claims 2 & 4 (DRAFT)
Gr. 6 Math Blueprint: Claim 3 (DRAFT)
A Few Sample Items
Grade 3: Problem Solving
Steven is baking cookies. On one tray, he makes 5 rows
with 4 cookies in each row. He cannot fit any more cookies
on the tray. He puts some more cookies on another tray
the same size as the first tray.
Which number is a reasonable total for the number of
cookies on the two trays?
A. 9
B. 20
C. 34*
D. 60
30
Grade 3: Communicating Reasoning
31
Grade 4: Modeling and Data Analysis
32
Attending to Coherence
The Number Line Example
Grade 3
Grade 7
Grade 8
Learn More and Stay Engaged
www.smarterbalanced.org
•
•
•
Visit us for the latest
news and
developments
Sign up for our enewsletter
Follow us on
Twitter at
@SmarterBalanced