Transcript OAT Toolkit for Social Studies - ODE

OAT Toolkit for Social Studies
Using OAT Grade 5
Score Reports
Why analyze data?
“To improve student achievement
results, use data to focus on a few
simple, specific goals.”
Mike Schmoker
Data Analysis Cycle
Assessments:
Classroom, School,
District, State
Compile
Data
Standards
Standards
Classroom
Implementation
Determine
Aligned
Resources
Analysis,
Interpretation
and Dialogue
Curriculum Design
and Planning for
New Learning
and Re-Teaching
Enhanced Class Score Report
Page one is a welcome page. One item to
note on this page is the web address for
Ohio’s Instructional Management System
(IMS):
http://ims.ode.state.oh.us/ode/ims/
On this site you can access released test
items, model lessons, research on social
studies education and other resources.
SUCCESS Web Site
 See the bottom of page 1 for information on
the SUCCESS web site.
Ohio Achievement Test Results
 Compare your class average with
district and state averages.
Performance Levels
 View the percentage of students who
achieved each performance level.
Standard Categories
Standard Categories
Standard Categories
Standard Categories
Interpreting the Data
• On which standards were your students most
successful? Least successful?
• On which standards does your class exceed the
district or state average? Are those items
clustered around a specific skill or content?
• On which standards does your class fall below
the district or state average? Are those items
clustered around a specific skill or content?
Interpreting the Data
Consider the OAT data using three filters:
What does this mean for
•
•
•
Curriculum?
Instruction?
Assessment?
Interpreting the Data
Interpreting the Data
Interpreting the Data
Interpreting the Data
Question 25, Grade 5 OAT in Social Studies, 2007:
Which newspaper headline would give you information
about why a town is having an election next week?
A. Councilman Quits to Take Job in Private Industry
B. Council Votes to Increase Property Taxes
C. Mayor Takes Oath of Office
D. Crime Rate Rises in City
Interpreting the Data
Commentary:
This multiple-choice question asks students to use a source (a
newspaper headline) to identify a possible cause-and-effect
relationship to make an inference (a conclusion). Response A is
correct, because if a councilman quits or resigns from his position,
there is an empty position on the town council, which means that
the town will have an election. Response B is incorrect because a
town council voting for higher taxes does not involve an election.
Response C is incorrect because a mayor would not take the oath
of office before an election, but afterwards. Response D is
incorrect; because while current officials might not be re-elected if
crime goes up, an election would not occur solely because of a
rise in crime.
Discussion Questions
• Which benchmark is being assessed?
• How does this item test understanding of the
benchmark?
• What is the cognitive level of this item?
• What can we learn from student performance on
this item?
Constructed Response Items
Constructed response items include both short
answer and extended response. Though few in
number, they have a significant impact on the
total score.
• How did your students fare with these items?
• Do they frequently write in the classroom?
• Are they expected to answer questions
completely and to provide supporting details?
• Do they share examples of thorough
responses?
Interpreting the Data
Interpreting the Data
Question 11, Grade 5 OAT in Social Studies, 2007:
In your Answer Document, give one reason
English colonists came to Virginia and settled in
Jamestown.
Describe one way their experience was different
from what they had expected. (2 points)
Interpreting the Data
Scoring Guidelines for Item 11 (Short Answer):
Exemplar:
The English who settled in Jamestown were hoping to get rich quickly by
finding gold. In reality, there was no gold there, and the only way the
colony eventually prospered was by planting and selling tobacco.
Other correct responses may include but are not limited to:
Goals:
They wanted to live like gentlemen. They wanted to acquire land.
They wanted to make money for their joint-stock company.
Results:
Many fell ill from diseases.
Many starved.
The land was not suitable.
They had to work hard.
At one point they only survived due to food and water provided by local
American Indians.
Interpreting the Data
Scoring Guidelines for Item 11 (Short Answer):
2 point
1 point
0 point
Response includes one explanation why the English settled
in Jamestown AND a correct description of one way in which
reality was different from settlers’ expectations.
Response includes one correct explanation why English
colonists settled in Jamestown OR a correct description of
one way their experience was other than what settlers
expected. The noncredit part of the response may be
incorrect, overly general or vague, or missing.
The response does meet the criteria to earn one point. The
response indicates inadequate or no understanding of the
task and/or the idea or concept needed to answer the item.
It may only repeat information given in the test item. The
response may provide an incorrect solution/response and
the provided supportive information may be totally irrelevant
to the item, or possibly, no other information is shown. The
student may have written on a different topic or written, “I
don’t know.”
Questions to Consider
• Which benchmark is being assessed?
• How does this item test understanding of the
benchmark?
• What is the cognitive level of this item?
• What can we learn from student performance
on this item?
Grade-band Achievement Tests
MYTH: The grade five OAT is a test designed to
measure what students have learned in their fifth
grade year.
FACT: The grade five OAT test is a “Grade-band”
test designed to measure what students have
learned in their social studies classes in grades
three, four, and five based on Ohio’s Academic
Content Standards.
Interpreting the Data: Grade bands
Question 2, Grade 5 OAT in Social Studies, 2007:
“Where the press is free, and every man able to read, all is
safe.”
—Thomas Jefferson, 1816
Which American document guarantees the freedom described
above?
A.
B.
C.
D.
the Articles of Confederation
the United States Constitution
the Emancipation Proclamation
the Declaration of Independence
Interpreting the Data
Item 2 Commentary:
This multiple-choice question asks students to identify from a list of
documents the one that guarantees the freedom described in
Jefferson’s quote (…where the press is free). The United States
Constitution (Response B) is correct because the Constitution
includes the First Amendment which guarantees freedom of the
press, an important freedom in a democracy. The Articles of
Confederation (Response A) provided a framework for the
government of the original thirteen colonies with little mention of
individual rights. The Emancipation Proclamation (Response C) is the
document that freed slaves in Confederate states during the Civil
War. The Declaration of Independence (Response D) was for the
most part a list of grievances which the colonists had and urged
independence from England.
Questions to Consider
• How does this item test understanding of the
benchmark?
• What is the cognitive level of this item?
• What can we learn from student performance on
this item?
Next Steps
 Review alignment of lesson plans and assessments with the
knowledge and skills explained in the benchmark and grade-level
indicators.
 Work with third and fourth grade teachers to plan together. Start by
looking at how each grade contributes to understanding of the
benchmarks. How can we make connections between grade
levels?
 Review classroom resources and materials and identify gaps. Talk
to the Media Specialist about supplemental texts, Web resources,
or audio/visual materials.
 Review classroom assessments to ensure that they push students
to explain their answers and make connections between ideas.
“If we are not going to try to
improve what we do, there is
little sense in assessing it.”
William Glasser
Resources
Glasser, William. (1990) The Quality School: Managing
Students Without Coercion. New York: Harper Collins.
Mertler, Craig A. Interpreting Standardized Test Scores,
Strategies for Data-Driven Instructional Decision Making.
Sage Publications, Inc. Thousand Oaks, 2007.
Schmoker, M. First Things First: Demystifying Data Analysis.
Educational Leadership, vol.60 n5 p22-24 Feb 2003.