Transcript Data: Digging Deeper in Steps 1 and 2

Going Deeper
Getting
withResults:
TBTs and the
Impacting
Ohio 5-Step
EachProcess
Student within All Students
Going Deeper with TBTs and the
Ohio 5-Step Process
2011-2012 SPDG Professional Development
Our Targets For Today
COMMON
Summative &
Formative
Questions
Formative
Classroom
Assessments
Embedded
Learning Targets
Aligning
Assessments
Categories of
Learning
Teacher Based Teams
Data:
Digging Deeper
in Steps 1 and 2
of Ohio 5-Step
Protocol
This document/product/software was supported in whole or in part by the U.S. Department of Education, Office of Special Education Programs,
(Award #Q27A090111A, CFDA 84.027A, awarded to the Ohio Department of Education). The opinions expressed herein do not necessarily reflect the
policy or position of the U.S. Department of Education, Office of Special Education Programs, and no official endorsement by the Department should be
inferred.
Step 1
Collect and
chart data
Step 5
Collect, chart and
analyze post data
The Ohio
5-Step
Process:
A Cycle of
Inquiry
Step 4
Implement changes
consistently across
all classrooms
Step 2
Analyze student
work specific to
the data
Step 3
Establish shared
expectations for
implementing specific
effective changes in the
classroom
Teacher Based Teams: Steps 1 & 2
Training Outcomes
• Analyze “common” assessment questions to
determine:
– Summative or Formative in Nature
– Distracters
• Deconstruct embedded learning targets and
identify category of learning for each
• Apply knowledge of how to determine cut
scores.
• Apply and analyze the use of classroom
formative assessments
Parking Lot
Materials
Working Agreements
Getting Started
Roles and Responsibilities
6
Facilities
Working Agreements
•
•
•
•
•
•
•
Stay Focused
Manage Electronic Devices
Practice Timely Attendance
Pass Notes instead of Sidebar Talking
Respectfully Challenge One Another
Listen Actively
Participate to the Fullest of Your Ability
Quick Check for Learning
•
•
•
Formative classroom
assessments for learning
•
Anticipation Guide
Statements that can be
reacted to without having
read the text.
Statements that challenge
beliefs.
General rather than
specific statements.
Statements that relate to
the upcoming content.
Inclusion Activity
Anticipation Guide
1. Answer each question
2. Talk at your table about your responses
Do you agree?
Think of literacy as a spine; it
holds everything together. The
branches of learning connect to
it, meaning that all core content
teachers have a responsibility
to teach literacy.
Ye
s
1. Yes
2. No
0%
No
0%
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Do you agree?
Offered a list of standards, we
should scrutinize each one but
also ask who came up with
them and for what purpose.
Ye
s
1. Yes
2. No
0%
No
0%
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Do you agree?
We test too much.
1. Yes
2. No
0%
Ye
s
No
0%
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
The Data Coach’s Guide: Love, Stiles,
Mundry & DiRanna, c. 2008
Annual
2-4 times a year
Quarterly or end of unit
1-4 times a
month
Daily - Weekly
Summative
district and
state
assessments
(aggregated,
disaggregated;
srand, item, and
student work)
Data about people,
practices, perceptions (e.g.,
demographic, enrollment,
survey, interview, observation
data, curriculum maps)
Benchmark common assessments (e.g.,
end-of-unit, common grade-level tests
reported at item level; aggregated,
disaggregated; strand, item, and student
work)
Formative common assessments (e.g., math
problem of the week, writing samples, science
journals, other student work)
Formative classroom assessments for learning (e.g., student
self-assessments, descriptive feedback, selected response,
written response, personal communications, performance
assessments)
Embedded
throughout our
Session
The Data Coach’s Guide: Love, Stiles,
Mundry & DiRanna, c. 2008
Annual
2-4 times a year
Quarterly or end of unit
1-4 times a
month
Daily - Weekly
Summative
district and
state
assessments
(aggregated,
disaggregated;
srand, item, and
student work)
Data about people,
practices, perceptions (e.g.,
demographic, enrollment,
survey, interview, observation
data, curriculum maps)
Benchmark common assessments (e.g.,
end-of-unit, common grade-level tests
reported at item level; aggregated,
disaggregated; strand, item, and student
work)
Formative common assessments (e.g., math
problem of the week, writing samples, science
journals, other student work)
Formative classroom assessments for learning (e.g., student
self-assessments, descriptive feedback, selected response,
written response, personal communications, performance
assessments)
More summative in the
types of questions
Questions are based on
specific deconstructed
learning targets
We must create/use
assessments that will
gather data that are
the right “grain size.”
That is, the information teachers need
from formative assessments must be
specific enough to be able to tell them
what to do next in their classrooms.
Kim Marshall (2008).The Teacher as Assessment Leader, Edited by Guskey, pg. 43
How We Assess
Learning Target
Students will be able to solve
problems requiring them to add
fractions with like and unlike
denominators.
What Is the Difference
Between These Two Items?
Summative
Solve:
½ + ¾ =
Formative
What is the least
common denominator
for the following
fractions?
(a) 4/6
(b) 1
(c) 1 1/4
(d) 4/8
1/2
(a) 24
(b) 12
(c) 6
(d) 9
2/3
3/4
Formative assessment provides
information about how to
differentiate classroom instruction
for the next lessons.
These (formative assessments) are not
improvised or spontaneous; rather, they are
administered at a particular time in the
teaching-learning process for the teacher to
get feedback about how to proceed.
Assessment that Makes Sense, Chris Jakicic, pg. 44 & 42
The Data Coach’s Guide: Love, Stiles,
Mundry & DiRanna, c. 2008
Summative
district and
state
assessments
(aggregated,
disaggregated;
srand, item, and
student work)
Data about people,
practices, perceptions (e.g.,
demographic, enrollment,
survey, interview, observation
data, curriculum maps)
Quarterly or end of unit
1-4 times a
month
Daily - Weekly
½ + ¾ =
(a) 4/6
(b) 1
(c) 1 1/4
(d) 4/8
Benchmark common assessments (e.g.,
end-of-unit, common grade-level tests
reported at item level; aggregated,
disaggregated; strand, item, and student
work)
Formative common assessments (e.g., math
problem of the week, writing samples, science
journals, other student work)
Formative classroom assessments for learning (e.g., student
self-assessments, descriptive feedback, selected response,
written response, personal communications, performance
assessments)
What is the least
common denominator for
the following fractions?
1/2
(a)
(b)
(c)
(d)
24
12
6
9
2/3
3/4
Can we mix summative and
formative type questions in
the same assessment?
I Don’t Like “Tricking” Students
Multiple choice items MUST be created with
“diagnostic” distracters that can pinpoint
what is understood and not understood. They
allow for teaching with precision.
Distracters are more difficult to
write than the correct answers!
Best Distracters
A plant is able to grow larger because
A. it gets its food
from the soil.
Misconception
A student who chooses this answer
does not understand that nutrients
are manufactured internally by the
plant.
B. it turns water and
air into sugar.
Oversimplification
The student understands that food is
manufactured internally but does not
understand that water & carbon
dioxide (from the air) are used to
make sugar and oxygen.
C. it has chlorophyll
to produce food.
Overgeneralization
The student does not understand that
some parasitic plants do not contain
chlorophyll.
D. it adds biomass
through photosynthesis.
Correct answer
Activity: Assessment Type
• In small groups, analyze the assessment to
determine whether each question is
summative or formative. How do you know?
• Choose 1 summative question and
rewrite it as formative.
• Choose 1 question and
determine what kind of
distracters are used.
So What May Be Missing
in Company-Generated Test Items?
• “We have trusted those we believe to be
more knowledgeable in test item
development and statistical validity and
reliability to develop our assessments.”
• “The problem isn’t that test makers have
failed in their task; the assessments that
textbook and testing companies have
designed have accomplished exactly what
they are capable of accomplishing.“
Popham 2001, 2008a
The Data Coach’s Guide: Love, Stiles,
Mundry & DiRanna, c. 2008
Annual
2-4 times a year
Quarterly or end of unit
1-4 times a
month
Daily - Weekly
Summative
district and
state
assessments
(aggregated,
disaggregated;
srand, item, and
student work)
HOW MUCH
IS TOO MUCH!?
Data about people,
practices, perceptions (e.g.,
demographic, enrollment,
survey, interview, observation
data, curriculum maps)
Benchmark common assessments (e.g.,
end-of-unit, common grade-level tests
reported at item level; aggregated,
disaggregated; strand, item, and student
work)
Formative common assessments (e.g., math
problem of the week, writing samples, science
journals, other student work)
Formative classroom assessments for learning (e.g., student
self-assessments, descriptive feedback, selected response,
written response, personal communications, performance
assessments)
Frequent Testing/Effects of Testing
.34
Hattie 2009
Feedback
.73
Hattie 2009
What Does Feedback Mean?
“The mistake I was making was seeing feedback as
something teachers provided to students – they typically
did not, although they made claims that they did at the
time, and most of the feedback they did provide was
social and behavioral. It was only then when I discovered
that feedback was most powerful when it is from the
student to the teacher that I understand it better. When
teachers seek, or at least open to, feedback from students
as to what students know, what they understand, where
they make errors, when they have misconceptions, when
they are not engaged – then teaching and learning can be
synchronized and powerful. Feedback to teachers helps
make learning visible. “ (Hattie 2009 pg. 173)
Formative Evaluation!
0.90
Hattie 2009
Formative Evaluation!
Pay attention to the formative effects of your
teaching, as it is these attributes of seeking
formative evaluation of the effects (intended
and unintended) of the programs that makes
for excellence in teaching.
0.90
Hattie pg. 181 2009
TRY USING THIS SCENERIO…
T1 We assess too much. I don’t have time for
teaching.
T2 Do you watch basketball?
T1 Yeah
T2 During a recent high school basketball playoff game,
the scoreboard completely malfunctioned midway
through the game. So the refs kept the score and time
on the sidelines. As it came close to the end of the
game, the visiting team was down by 2 points, but
they did not realize it nor how much time was left.
The clock ran out before they took their final shot.
T1 That’s not right!
T2 Of course not. If the scoreboard had been working,
the entire end of the game could have been different.
So you’d agree that a working scoreboard is critical for
sporting events, correct?
T1 Of course.
T2 At the end of the day, assessments where students
give teachers feedback is like fixing the scoreboard.
Relying on state tests is like covering up the scoreboard
at the beginning of the game and then uncovering it at
the end of the game to see if you won. At that point,
there’s nothing you can do to change the outcome! We
use benchmark common assessments to keep the
scoreboard uncovered so we can make the necessary
adjustments to be able to win the game.
Quick Check for Learning
3-2-1
Write 3 key terms related to
what you know or have
learned.
Write 2 questions you have.
Formative classroom
assessments for learning
Write 1 way you can apply
your learning.
So…
IF:
We understand … “the way assessment questions
are written determines the kind of
data we receive,”
THEN:
We need to ensure that all of the learning
targets embedded in our standards are
identified.
What Is “Deconstructing”?
aka
Unwrapping & Unpacking
“Taking a broad and/or unclear standard, goal,
or benchmark and breaking it into smaller,
more explicit learning targets that can be
incorporated into daily classroom teaching”
Stiggins et al. 2006
Deconstructing Standards into a
Progression of Learning Targets
A systematic process to identify embedded
learning targets in standard indicators so that
nothing essential is missed during instruction
Learning targets: What students should know,
understand and be able to do to master the
indicators
Sequence the targets into a developmentally
appropriate learning progression
Deconstructing the Golf Game
Will be able to perform
basic golf skills, model
acceptable social
behavior and apply
fundamental knowledge
of the rules of golf to
enable participation in a
regulation or modified
game of golf.
1. Perform Basic Golf Skills
• Proper grip
• Stance
• Alignment of feet to
target
• Distance of body from
ball
• Bend at the waist
• Swing fundamentals
• Wrist position at top of
swing
2. Acceptable Social Behavior
• All swings count towards
your score
• Clubs are not thrown
• Mark your ball when it is
on the putting green
• Keep the pace of play
• Do not talk or make
noise while others are
swinging
Tips for Deconstructing
Analyze the wording of the standard/objective to determine key
concepts and key skills
 Read through indicators
 Circle verbs to identify key skills
 Underline nouns and noun phrases to identify key concepts
Example:
Create grade-appropriate real-world problems
involving any of the four operations using multiple
strategies, explain the reasoning used, and justify
the procedures selected when presenting solutions.
Deconstructing Practice Activity:
Describe what happens when two forces act on
an object in the same or opposing directions.
1. I can give the definition of force.
2. I can tell what 3 changes might happen to an object when
a force is applied.
3. I can write in words and also create a diagram to explain
what happens when two forces act on an object in the
same direction.
4. I can write in words and create a diagram to explain what
happens when two forces act on an object in opposite
directions.
Writing Standard: Research to Build and Present Knowledge
12th Grade Research
Content:
• Short Research
Projects
• Sustained Research
Projects
• Self-Generated
Questions
• Multiple Sources of
Information
• Relevant Information
• Authoritative Print
Sources
• Digital Sources
• Advanced Searches
• Evaluation of Sources
• Flow of Ideas
• Audience & Purpose
• Plagiarism
• Citation Format
Writing Standard: Research to Build and Present Knowledge
7. Conduct short as well as more sustained research
projects to answer a question (including a self-generated
question) or solve a problem; narrow or
broaden the inquiry when appropriate; synthesize
multiple sources on the subject, demonstrating
understanding of the subject under investigation!
8. Gather relevant information from multiple authoritative
print and digital sources, using advanced searches
effectively; assess the strengths and limitations of
each source in terms of the specific task, purpose, and
audience; integrate information into the text selectively
to maintain the flow of ideas, avoiding plagiarism and
overreliance on any one source and following a
standard format for citation.
New Content Standards
 Choose a content area from the samples on your
table.
 In Pairs or Triads, “Deconstruct, Unwrap, Unpack”
one of them and record the Embedded Learning
Targets in the first column of the template .
 At your tables share your work.
Standard
Embedded Learning Targets
Type of Target:
Knowledge,
Reasoning, Skill,
or Product
Best
Assessment
Method
Quick Check for Learning
Formative classroom
assessments for learning
Linking
• 1st person makes a
statement about something
s/he has learned.
• Person to the right “links” to
the 1st person’s statement.
• Continue the “links” until
everyone at the table has
participated.
Four Categories of Learning
1. Knowledge: Fact-based learning easily assessed
2. Reasoning: Require application of knowledge to
make comparisons, predictions, or evaluative
judgments
3. Skills: Require an observable demonstration of
the proper application of knowledge.
4. Products: Require application of knowledge to
generate specific, predefined pieces of work
common to a particular domain of study
Stiggins et al. 2006
Knowledge Examples
• Identify metaphors and similes
• Read and write quadratic equations
• Describe the function of a cell
membrane
• Know the multiplication tables
• Explain the effects of an acid on a
base
Reasoning Examples
• Use statistical methods to describe,
analyze, evaluate, and make
decisions.
• Make a prediction based on evidence.
• Examine data/results and propose a
meaningful interpretation.
• Distinguish between historical fact
and opinion.
Performance/Skill Examples
• Measure mass in metric and SI units
• Use simple equipment and tools to
gather data
• Read aloud with fluency and expression
• Participates in civic discussions with the
aim of solving current problems
• Dribbles to keep the ball away from an
opponent
Product Examples
• Construct a bar graph
• Develop a personal health-related
fitness plan
• Construct a physical model of an
object
• Write a term paper to support a
thesis
FIRST GRADE
Standard/Benchmark:
Produce writing to communicate with different audiences for a variety of
purposes.
Learning Targets:
What are the knowledge, reasoning, skill, or product targets underpinning the standard?
Knowledge
Targets
Reasoning
Targets
Skill
Targets
Product
Targets
Hold a pencil correctly
Know what a
sentence is
Understand concept
of word choice
Distinguish the uses or
meanings of a
variety of words
(word choice)
Print letters correctly
according to DN methods
Space words
Use lines and margins
correctly
Stretch out sounds in words
to create a temporary
spelling of the word
Write sentences
with varied
beginnings.
Original Terms
New Terms
• Evaluation
• Creating
• Synthesis
• Evaluating
• Analysis
• Analyzing
• Application
• Applying
• Comprehension
• Understanding
• Knowledge
• Remembering
(Based on Pohl, 2000, Learning to Think, Thinking to Learn, p. 8)
Back to Your Deconstructing Template:
Complete the 2nd column (Category of
Learning) for each of the embedded
learning targets that you identified earlier.
Standard
Embedded Learning Targets
Type of Target:
Knowledge,
Reasoning, Skill,
or Product
Best
Assessment
Method
Quick Check for Learning
Formative classroom
assessments for learning
Whip Around
• Make a list of 3 items related
to your understanding and
application of types of targets.
• Everyone stands.
• One at a time each person
shares one item on his/her
list. Others cross off if their
own item is duplicated. Sit
when your 3 items have been
stated.
Aligning Learning Targets to
Appropriate Assessments
Type of
Target to
Be
Assessed
Selected
Response
Constructed
Response/ Essay
Performance
Assessment
Personal
Communication
Knowledge
Good Match
Good Match
NOT a Good
Match
Partial Match
Reasoning
Partial Match
Good Match
Good Match
Good Match
Performance
Skills
NOT a Good
Match
NOT a Good
Match
Good Match
Partial Match
Product
NOT a Good
Match
Partial Match
Good Match
NOT a Good
Match
Assessment Method
Stiggins 2006
Considerations in Creating
Assessments…
• Is it possible for me to create an
assessment that everyone in my
class can pass?
• Is it possible for me to create an
assessment that virtually no one in
my class could pass?
Important to Remember…
Include both formative level assessment
items as well as several advanced (or
summative) items
There is no magic number of items to
include at each level of proficiency;
simply ensure that multiple
opportunities exist for students to
demonstrate those varied levels.
Quick Check for Learning
Bump in the Road
Write down something
that you find confusing
or a skill or concept you
find difficult.
Formative classroom
assessments for learning
The Data Coach’s Guide: Love, Stiles,
Mundry & DiRanna, c. 2008
Annual
2-4 times a year
Quarterly or end of unit
1-4 times a
month
Daily - Weekly
Summative
district and
state
assessments
(aggregated,
disaggregated;
srand, item, and
student work)
Data about people,
practices, perceptions (e.g.,
demographic, enrollment,
survey, interview, observation
data, curriculum maps)
Benchmark common assessments (e.g.,
end-of-unit, common grade-level tests
reported at item level; aggregated,
disaggregated; strand, item, and student
work)
Formative common assessments (e.g., math
problem of the week, writing samples, science
journals, other student work)
Formative classroom assessments for learning (e.g., student
self-assessments, descriptive feedback, selected response,
written response, personal communications, performance
assessments)
Frequently check for
understanding:
Are the kids – all of the kids –
following what you are saying?
Asking “Does anyone have any
questions?” does not work.
(Ripley, 2010, p.5)
Formative Classroom
Assessments for Learning
•
•
•
•
•
•
•
•
•
•
Anticipation Guide
5+1, 10+1, 15+1
Pair/Share
3-2-1
Linking
Whip Around
Graphic Organizer
Bump in the Road
5 Seconds
Application Cards
What Do You Use?
Formative Classroom
Assessments for Learning
What Do You Use?
Samples
A. Student Reflection Chart
B. Checklist for Creating Common Assessments
C. Checklists for using Formative/Summative
Assessments
1.Divide samples at your table
2. Review (2 minutes) and present to the rest of
your group (1 minute per sample)
A. Benefits of Use
B. Ideas for Adaptation
Our Targets For Today
COMMON
Summative &
Formative
Questions
Formative
Classroom
Assessments
Embedded
Learning Targets
Aligning
Assessments
Categories of
Learning
QUESTIONS?????
References
Bambrick-Santoyo, Paul. (2010). Driven by data. Jossey-Bass
Blankstein, A. (2010). Failure is not an option: 6 principles for making student
success the only option. Facilitators Guide. Corwin
Fisher and Frey. (2007). Checking for understanding: formative assessment
techniques for your classroom. ASCD
Greenstein, L. (2010). What teachers really need to know about formative
assessment. ASCD
Gusky, et. al. (2009). The teacher as assessment leader. Solution Tree Press
Hattie, John. (2009). Visible learning: A synthesis of over 800 meta-analyses
relating to achievement. Routledge
Schmoker, Mike. (2011). Focus: Evaluating the essentials to radically improve
student learning. ASCD
This document/product/software was supported in
whole or in part by the U.S. Department of
Education, Office of Special Education Programs,
(Award #Q27A090111A, CFDA 84.027A, awarded to
the Ohio Department of Education). The opinions
expressed herein do not necessarily reflect the policy
or position of the U.S. Department of Education,
Office of Special Education Programs, and no official
endorsement by the Department should be inferred.