Transcript Preparing for the Common Core State Standards Mathematics Assessment October 30, 2012 Today’s Goals Participants will have the opportunity to: • explore the design of the SBAC.

Preparing for
the Common Core State Standards
Mathematics Assessment
October 30, 2012
Today’s Goals
Participants will have the
opportunity to:
• explore the design of
the SBAC assessment
system
• consider the design of
a balanced assessment
system for your
districts
Today’s Agenda
• Welcome, Context, and Introductions
• Opening Comments - Dr. Burkhardt
• Explore SBAC Content Specifications and Items
– SBAC Basics
– Summative Assessment Target Tables
– SBAC Tasks
– Computer Adapted Test Design
– AM Summary & Reflections - Dr. Burkhardt
Lunch & District Debrief
• A Balanced Classroom and District Assessment System
– Introduction to Formative Assessment – Dr. Burkhardt
– MAPS Tasks
• Closing Remarks - Dr. Burkhardt
• District Debrief & End of Day Reflections
CCSS States and the Smarter
Balanced Assessment Consortium
Balanced States
CCSSM States
SBAC - A Balanced System
Assessment Principles
1.
Assessments are grounded in a
thoughtful, standards-based
curriculum and are managed as
part of an integrated system.
2.
Assessments include evidence
of student performance.
3.
Teachers are integrally involved
in the development and scoring
of assessments.
4.
Assessments are structured to
continuously improve teaching
and learning.
5.
Assessment, reporting, and
accountability systems provide
useful information on multiple
measures.
Hugh Burkhardt, Ph.D.

Shell Center for
Mathematics
Education

Mathematics
Assessment
Resources
Project (MARS)

Mathematics
Assessment
Project (MAP)
7
Key Elements of the CCSSM
Connected to Issues of Assessment

CCSS Learning
Progressions


Within grades
Across Grades
 www.TurnonCCMath.net
 http://ime.math.arizona.edu/progressions/
8
Key Elements of the CCSSM
Connected to Issues of Assessment
Standards for Mathematical Practice
William McCallum
Reasoning and
explaining
Standards for
Mathematical
Practice
Tucson, April 2011
Modeling and
Using tools
Seeing structure
and generalizing
9
Smarter Balanced Assessment Consortium
Mathematics Content Specifications
Beginning with the basics!





Claims
DOK
Cluster Headings
Targets
Item Types
SBAC Basics - A Balanced System
SBAC Basics: Foundational Beliefs
Assessments should be structured to continuously
improve teaching and learning
■
Assess around learning progressions
■
Using Computer Adaptive Testing Technology
■
Creating opportunities for students and teachers to get
actionable feedback on student learning throughout the
year
■
Provide curriculum-embedded assessments that offer
models of good curriculum and assessment practices
■
Allowing close examination of student work and
moderated teacher scoring as professional development
SBAC Basics: Reporting Categories
“Each claim is a summary statement about the knowledge and skill
students will be expected to demonstrate on the assessment
related to a particular aspect of the CCSS for mathematics.”
Claim 1:
Concepts and
Procedures, ≈ 40%
“Students can explain and apply mathematical
concepts and interpret and carry out mathematical
procedures with precision and fluency.”
Claim 2:
Problem Solving
“Students can solve a range of complex well-posed
problems in pure and applied mathematics, making
productive use of knowledge and problem solving
strategies.”
Claim 3:
Communicating
Reasoning ≈ 20%
“Students can clearly and precisely construct viable
arguments to support their own reasoning and to
critique the reasoning of others.”
Claim 4:
Data Analysis and
Modeling ≈ 20%
“Students can analyze complex, real-world scenarios
and can construct and use mathematical models to
interpret and solve problems.”
≈ 20%
13
Same Target, Different Claims
For each task
consider:
Which claim or claims
do you see the task
supporting? Why?
14
SBAC Basics: Depth of Knowledge (DOK)
Measure of Cognitive Rigor
The level of task complexity.
Level 1: Recall and Reproduction

Requires eliciting information such as a fact, definition, term,
or a simple procedure, as well as performing a simple algorithm
or applying a formula.
Level 2: Basic Skills and Concepts

Requires the engagement of some mental processing beyond
a recall of information.
Level 3: Strategic Thinking and Reasoning

Requires reasoning, planning, using evidence, and explanations
of thinking.
Level 4: Extended Thinking

Requires complex reasoning, planning, developing, and
thinking most likely over an extended period of time.
DOK Level 1
Example - Grade 8
Select all of the expressions that have a value between 0 and 1.
87  8–12
74
7–3
1
2
3
(–5)6
(–5)10

1
3
9
DOK Level 2
Example - Grade 8
A cylindrical tank has a height of 10 feet and
a radius of 4 feet. Jane fills this tank with water
at a rate of 8 cubic feet per minute. How many
minutes will it take Jane to completely fill the
tank without overflowing at this rate?
Round your answer to the nearest minute.
DOK Level 3
Example - Grade 8
The total cost for an order of shirts from a company consists of the cost for
each shirt plus a one-time design fee. The cost for each shirt is the same
no matter how many shirts are ordered.
The company provides the following examples to customers to help them
estimate the total cost for an order of shirts.
• 50 shirts cost $349.50
• 500 shirts cost $2370
Part A: Using the examples provided, what is the cost for each shirt, not
including the one-time design fee? Explain how you found your answer.
Part B: What is the cost of the one-time design fee? Explain how you found
your answer.
DOK Level 4
Example - Grade 8
During the task, the student assumes the role of an
architect who is responsible for designing the best plan
for a park with area and financial restraints. The student
completes tasks in which he/she compares the costs of
different bids, determines what facilities should be given
priority in the park, and then develops a scale drawing of
the best design for the park and an explanation of the
choices made. This investigation is done in class using a
calculator, an applet to construct the scale drawing, and
a spreadsheet.
Cognitive
Rigor
Matrix
Structure of the CCSSM
DOMAIN
Number and Operations in Base Ten
4.NBT
Generalize place value understanding for multi-digit
whole numbers.
STANDARD
CLUSTER
1. Recognize that in a multi-digit whole number, a digit in
one place represents ten times what it represents in the
place to its right.
2. Read and write multi-digit whole numbers using base-ten
numerals, number names, and expanded form. Compare
two multi-digit numbers based on meanings of the digits
in each place, using >, =, and < symbols to record the
results of comparisons.
3. Use place value understanding to round multi-digit whole
numbers to any place.
SBAC Basics: Large Scale Assessment
Constraints
On the large scale summative assessment not
everything in the CCSSM can have equal priority
given time limitations. Cluster headings at each
grade level are categorized as Major (m), or as
Additional/Supporting (a/s).


About 75% - 80% of the items should come from
Major clusters for Claim 1
About 20% - 25% of the items should come from
Additional/Supporting clusters for Claim 1
SBAC Basics: Large Scale Assessment
Constraints

Identifying some standards
within “major” clusters and
others within
“additional/supporting” clusters
is not to say that anything in the
standards can be neglected. To
do so would leave gaps in
student preparation for later
mathematics. In other words,
all content is eligible for and
should be encompassed in the
assessment. (p.29)
SBAC Basics: Large Scale Assessment
Constraints

Working at the cluster level
helps to avoid obscuring
the big ideas and getting
lost in the details of specific
standards (which are
individually important, but
impossible to measure in
their entirety within the
bounds of reasonable
testing time). p.29
SBAC
Mathematics
Content
Specifications
Content Specifications for the Summative
Assessment of CCSSM

Details of the
Assessment
Specifications are
organized around
the four Claims that
will be used as
reporting categories
Claim 1:
Concepts and
Procedures, ≈ 40%
Claim 2:
Problem Solving
≈ 20%
Claim 3:
Communicating
Reasoning ≈ 20%
Claim 4:
Data Analysis and
Modeling ≈ 20%
Summative Assessment Target Tables
Currently under development by SBAC



Indicates Targets for the summative portion
of the Smarter Balanced assessment
Suggests what is taken as evidence of
student proficiency for a particular target
Articulates

Content (cluster heading and related standards)

Depth of Knowledge task assignments

Assessment method/Task types
Summative Assessment Target Tables
The cluster headings can
be viewed as the most
Claim 1:
effective means of
Concepts and
Procedures, ≈ 40%
communicating the focus
and coherence of the
standards. Therefore,
this content specifications document uses
the cluster headings as the targets of
assessment for generating evidence for
Claim #1. (p.29)
Structure of the CCSSM
DOMAIN
Number and Operations in Base Ten
4.NBT
Generalize place value understanding for multi-digit
whole numbers.
STANDARD
CLUSTER
1. Recognize that in a multi-digit whole number, a digit in
one place represents ten times what it represents in the
place to its right.
2. Read and write multi-digit whole numbers using base-ten
numerals, number names, and expanded form. Compare
two multi-digit numbers based on meanings of the digits
in each place, using >, =, and < symbols to record the
results of comparisons.
3. Use place value understanding to round multi-digit whole
numbers to any place.
Summative Assessment Target D
Claim 1 - Concepts and Procedures
Grade 4
Operations and Algebraic Thinking
Target D [m]: Generalize place value understanding for multi-digit whole
numbers. (DOK 1, 2)
Tasks for this target will require students to compare multi-digit numbers using >, =,
and <. Tasks should tap into students’ understanding of place value (e.g., by asking
students to give a possible digit for the empty box in 4357 < 43☐9 that would make
the inequality true). A smaller number of these tasks will incorporate student
understanding of rounding (e.g., explaining why rounding to a certain place would
change the symbol < or > to =).
In claims 2-4, students should see contextual problems associated with this target that
highlight issues with precision, including problems in Claim 3 that ask students to
explain how improper estimation can create unacceptable levels of precision and/or
lead to flawed reasoning. (pg. 34 - 35)
Cluster Headings RULE!

In the CCSSM the cluster
headings usually serve to
communicate the larger intent of a
group of standards. For example,
a cluster heading in Grade 4
reads: “Generalize understanding
of place value for multi-digit
numbers.”
Individual standards in this cluster pinpoint some
signs of success in the endeavor, but the important
endeavor itself is stated directly in the cluster
heading. In addition, the word generalize signals that
there is a multi-grade progression in grades K-3
leading up to this group of standards. (p.28)
Summative Assessment Targets
Claim 1 - Concepts and Procedures
Grade 4
Operations and Algebraic Thinking
A.
B.
C.
D.
E.
Use the four operations with whole numbers to solve problems.
Gain familiarity with factors and multiples.
Generate and analyze patterns.
Generalize place value understanding for multi-digit whole numbers.
Use place value understanding and properties of operations to perform
multi-digit arithmetic.
Number and Operations – Fractions
F. Extend understanding of fraction equivalence and ordering.
G. Build fractions from unit fractions by applying and extending previous
understandings of operations on whole numbers.
H. Understand decimal notation for fractions, and compare
decimal fractions.
Summative Assessment Targets
Claim 1 - Concepts and Procedures
Grade 4 continued
Measurement and Data
I. Solve problems involving measurement and conversion of
measurements from a larger unit to a smaller unit.
J. Represent and interpret data.
K. Geometric measurement: understand concepts of angle and
measure angles.
Geometry
L. Draw and identify lines and angles, and classify shapes by
properties of their lines and angles.
Claim 1- Concepts and Procedures
Students can explain and apply mathematical concepts
and interpret and carry out mathematical procedures with
precision and fluency.
Grade Level
Number of
Assessment Targets
3
11
4
12
5
11
6
10
7
9
8
10
11
16
Summative Assessment Target Tables
for Claims 2, 3, and 4 (≈ 60%)
Claim 2:
Problem Solving

Assessment Targets
for Claims 2, 3, and 4
are not divided into a
grade-by-grade
description.

A general set of
assessment targets
applicable across
grade levels.
≈ 20%
Claim 3:
Communicating
Reasoning ≈ 20%
Claim 4:
Data Analysis and
Modeling ≈ 20%
Pages 59 - 68
Summative Assessment Targets
Claim 2 – Problem Solving
Claim 2: Students can solve a range of complex well-posed
problems in pure and applied mathematics, making productive
use of knowledge and problem solving strategies.
A. Apply mathematics to solve well-posed problems
arising in everyday life, society, and the workplace
B. Select and use tools strategically
C. Interpret results in the context of the situation
D. Identify important quantities in a practical situation and
map their relationships.
Summative Assessment Targets
Claim 3 – Communicating Reason
Claim 3: Students can clearly and precisely construct viable
arguments to support their own reasoning and to critique
the reasoning of others.
A. Test propositions or conjectures with specific examples.
B. Construct, autonomously, chains of reasoning that justify or
refute propositions or conjectures.
C. State logical assumptions being used.
D. Use the technique of breaking an argument into cases.
E. Distinguish correct logic or reasoning from that which is flawed,
and—if there is a flaw in the argument—explain what it is.
F. Base arguments on concrete referents such as objects,
drawings, diagrams, and actions.
G. Determine conditions under which an argument does and
does not apply.
Summative Assessment Targets
Claim 4 – Modeling and Data Analysis
Claim 4: Students can analyze complex, real-world
scenarios and can construct and use mathematical models
to interpret and solve problems.
A.
B.
C.
D.
E.
F.
G.
Apply mathematics to solve problems arising in everyday life, society,
and the workplace.
Construct, autonomously, chains of reasoning to justify mathematical
models used, interpretations made, and solutions proposed for a
complex problem.
State logical assumptions being used.
Interpret results in the context of a situation.
Analyze the adequacy of and make improvement to an existing model
or develop a mathematical model of a real phenomenon.
Identify important quantities in a practical situation and map their
relationships.
Identify, analyze, and synthesize relevant external resources to pose
or solve problems.
Summative Assessment Targets Tables
As a table group select a grade level and skim
through the corresponding Targets for Claim 1.

Orient yourself to the grade level
Cluster headings with
standards
 DOK
 Related requirements in Claims
2–4

 Read one assessment target and explore related
targets in Claims 2 – 4.
 Share a feature that may suggest changes at the
system and/or classroom level.
39
SBAC Related Assessment Resources

Where can I go for
instructional and
assessment tasks
that reflect the types
of thinking students
will need to be able
to demonstrate on
the SBA?
SBAC Sample Math Tasks

Explore the SBAC collection www.tinyurl.com/SBsamples
 Be sure to push forward to explore
a variety of task types (Selectedresponse, Technology-enhanced,
Constructed-response,
Performance)
 Check out the two right tabs along
the top bar for a number of items.
41
SBAC Website Resources
http://www.smarterbalanced.org/smarter-balanced-assessments/
Click Here





Content Specifications (with some items at the end)
Item/Task Specifications (with more sample tasks)
Technology Enhanced Items (still more sample tasks)
Performance Tasks (you guessed it, more sample tasks!)
Guidelines
42
SBAC Website Resources
http://www.smarterbalanced.org/smarter-balanced-assessments/





Item and Task Types
Accessibility and Accommodations
For more information:
Visit the support for under-represented students webpage
Download the Accessibility and Accommodations Factsheet
Click Here
43
Oakland CCSS Initiative
Resources
The Common Core State Standards
Initiative (CCSSI) Oakland provide
support and direction for educators as
they move toward full implementation:
CCSS aligned curriculum and coherent
units of study to highlight needed shifts in
content related and pedagogical practices


Highlight Lessons Formative
Assessment
Resources (video, sample student work,
rubrics, instructional websites, etc.)
44
Atlas Curriculum Mapping
Units, lessons, formative assessments, and
other resources available in Atlas by Rubicon.
http://oaklandk12.rubiconatlas.org/public
45
Online CCSS
Curriculum Resources




Units of Study
Lesson resources
Assessment resources
Professional resources



Video
Sample student work
And more
46
Classroom and Systems Implications


As a group discuss implications
for your particular roles

instruction

assessments

teacher evaluation

student subgroups

classroom materials such as
textbooks, computers, etc.
Be prepared to share your
thinking with the whole group.
47
An Introduction to Computer
Adaptive Tests (CATs)
Not to be Confused with Computer
Administered Tests (CATs)! :-O
Computer Adaptive Tests…





Place student ability and test question
difficulty on the same scale
Modify the test based on student responses
Can assess student ability more efficiently
than “fixed-form” tests
Can be used to measure “growth”
Have a set of inherent challenges all of their
own.
Student 1
Ability /Difficulty
Less/
Easier
Test Scale Score
More/
Harder
Q1
Q2
Q3
Q4
Student 2
Test Scale Score
Less/
Easier
Ability /Difficulty
Q1
Q2*
Q3*
Q4*
More/
Harder
Things to Notice





Different students will get different items
Students aren’t presented with (many) items
that are far from their estimated ability
Since items are “targeted” to ability, the tests
can be shorter than fixed-form tests.
Potential for (real) Claim-level scores
The exact algorithm for selecting the next
item and when to stop is still under
development
CATs and Measurement Error

ALL Measurement involves error


As tests get longer, our estimates get better,
both in terms of



CAT is no different
Location (ability)
Error (precision)
Perhaps another graph will help
Morning Wrap-up & Questions

Dr. Burkhardt

What questions do
you have?
54
Lunch
55
Afternoon Wrap-up & Questions

Dr. Burkhardt

What questions do
you have?
56
End of Day Reflections
1. Pick an idea that came up today and that you
found particularly interesting. What is your
current thinking about this idea? What
questions do you still have?
2. What is your reaction to the work we did
today? What seems promising and/or
challenging at this point?
57
SB-CEU’s
SB-CEU sign-out is
available in the
lobby.
Don’t forget to
Sign-Out!
58
Contact us!








Mary Bouck
Hugh Burkhardt
Geraldine Devine
Jonathan Flukes
Dana Gosen
Jim Gullen
Valerie L. Mills
Carrie Zielinski
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]