Transcript Assessing for College and Career Readiness:

Teaching and Assessing Higher Order Thinking Skills
for K-12 Teachers
DAY THREE: September 27, 2012




“Personalized,” Domain-specific HOTS
Comments to, and from others…
Topic(s) for Performance-based HOTSaligned Projects
Today’s emphasis: Instructional Strategies
 Critical Thinking
 Problem-Solving
 Analytic Reasoning
The mastery of content and lower order thinking
are particularly important prerequisites to
higher order thinking. Any lesser degree of
learning of prerequisites will result in
puzzlement, delay, inefficient trial and error at
best, and in failure, frustration, or termination
of effort at worst…”
- Gagne, Briggs and Wagner, 1988
“This division of basic facts from higher-order
thinking runs against common sense. How
middle schoolers may apprehend ‘historical
thinking’ without delving into the factual
details of another time and place far from
their own, is a mystery.”
- Mark Bauerlein, The Dumbest Generation, 2009
Lower Order Thinking Skills
Higher Order Thinking Skills
• cognitive strategies
• comprehension
• concept classification
• discriminations
• routine rule using
• simple analysis
• simple application
• complex analysis
Examples: Underlining main
ideas, outlining,
paraphrasing, mnemonic
devices to recall information
• creative thinking
• critical thinking
• decision making
• evaluation
• logical thinking
• meta-cognitive thinking
• problem solving
• reflective thinking
• scientific experimentation
• scientific inquiry
• synthesis
• systems analysis
“It is important that students acquire sufficient
declarative knowledge (e.g., concepts and facts)
and procedural knowledge (e.g., strategies and
algorithms) within a specific content domain
before they can be expected to engage in
complex reasoning processes.”
-- Marzano, 1992
HOTS are NOT simply an extensive collection
of Lower-Order Thinking Skills.
Consider an Example (Science):
Essential Question: “Why do some substances
burn, and other substances do not?”
Antoine-Laurent Lavoisier
(Adapted from The Ten Most Beautiful Experiments, by George Johnson, Alfred A. Knopf, 2008)
Why do some substances burn, while other substances do not? In the 1700’s,
Antoine Lavoisier and most other chemists accepted the notion that some kinds
of matter burned because of a mysterious substance called “phlogiston” (flow-jiston). The reason things burned was that they were rich in phlogiston, and as they
were consumed they released this “fire stuff” into the air. Set a piece of wood
aflame and it would stop burning only when its phlogiston was spent, leaving
behind a pile of ash. Wood, it logically followed, was made of phlogiston and ash.
Likewise, heating a metal under an intense flame, a process called “calcination,”
left a whitish brittle substance, or “calx.” Metal was thus composed of phlogiston
and calx. The process also worked the other way around. Calx, it was recognized,
resembled the crude ores mined from the ground, which were refined or reduced
by heating them next to a piece of charcoal. The charcoal emitted phlogiston,
which combined with the calx to recover the lustrous metal.
With phlogiston, scientists had a consistent explanation for combustion,
calcination, and reduction. Chemistry suddenly made sense. There was however
a problem: the calx left behind after calcination weighed more than the original
metal. How could removing phlogiston leave something heavier?
High School
Grades 9-12
Middle School
Grades 6-8
Upper Elementary
Grades 3-5
Lower Elementary
Grades K-2
(From our Previous Example – an “exit outcome”):
Students should be able to evaluate arguments,
identifying the strengths and weaknesses of
claims in light of the evidence and reasoning
used to support them.
Students should also be able to construct valid
arguments, and support them with evidence.
Young students can begin by constructing an argument for their own
interpretation of the phenomena they observe and of any data they collect.
They need support to go beyond simply making claims—that is, to include
reasons or references to evidence and to begin to distinguish evidence from
opinion. As they grow in their ability to construct arguments, students can
draw on a wider range of reasons or evidence, so that their arguments
become more sophisticated. In addition, they should be expected to discern
what aspects of the evidence are potentially significant for supporting or
refuting a particular argument. Students should begin learning to critique by
asking questions about their own findings and those of others. Later, they
should be expected to identify possible weaknesses in either the data or an
argument and explain why their criticism is justified. As they become more
adept at arguing and critiquing, they should be introduced to the language
needed to talk about argument, such as claim, reason, data, etc.
Framework for K-12 Science Education, National Academy of Sciences
Lower Elementary (Grades K-2): Students should…
 Know that people are more likely to believe your ideas
if you can give good reasons for them.
 Ask “How do you know?” when appropriate, and to
attempt reasonable answers when others ask them
the same question.
Adapted from Benchmarks for Science Literacy, AAAS
Upper Elementary (Grades 3-5): Students should…
 Support their statements with facts found in books,
articles and other sources.
 Seek better reasons for believing something than
“Everybody knows that…”, or “Just because…”.
 Recognize when comparisons or analogies used in an
argument might not be correct or fair.
▪ e.g. an electric wire is like a garden hose, Earth is like a peach
Adapted from Benchmarks for Science Literacy, AAAS
Middle School (Grades 6-8): Students should…
 Be aware that there may be more than one good way
to interpret a given set of facts.
 Be skeptical of arguments based on small samples of
data, biased data, or vague reasons.
 Criticize arguments where fact and opinion are
intermingled, or where conclusions don’t logically
follow from the evidence.
Adapted from Benchmarks for Science Literacy, AAAS
High School (Grades 9-12): Students should…
 Criticize arguments based on faulty, incomplete, or
misleading use of evidence.
 Know that convincing arguments need true supporting
statements and valid connections between them.
 Identify the hidden assumptions behind an argument to
assess its validity.
 Suggest alternative ways of interpreting the evidence in
an argument.
Adapted from Benchmarks for Science Literacy, AAAS
1. The 5-E Model of Inquiry applied to
“Critical Thinking”
2. Claim/Evidence/Reasoning applied to
“Problem-Solving”
3. Additional Strategies applied to
“Argumentation” and “Analytic
Reasoning”
“Inquiry focuses on the engagement of students to generate and
pursue the answers to questions through careful observation and
reflection. It is a multifaceted activity that involves making
observations; posing questions; examining other sources of
information to see what is already known in light of experimental
evidence: using tools to gather, analyze, and interpret data;
proposing answers, explanations, and predictions; and
communicating the results. Inquiry requires identification of
assumptions, use of critical and logical thinking, and
considerations of alternative explanations. It is far more flexible
than, for example, the rigid sequence of steps commonly
depicted in textbooks as the ‘scientific method.’ It is much more
than just doing experiments.”
– Llewellyn, Teaching Science Through Inquiry, NSTA Press 2005
Learning involves making sense of
1) prior experiences, and
2) new first-hand explorations.
Engage
Evaluate
Elaborate
Explore
Explain
Engage: A scientist has
made the claim, “For all
types of animals, an
average lifetime is about
one billion heartbeats.”
Suppose you wanted to
test this idea. Which
questions need to be
answered? What data do
you need?
Engage: “For all animals, a lifetime is about one billion heartbeats.”
Exploring the Data:
Animal
Cat
Dog
Hamster
Chicken
Monkey
Horse
Cow
Pig
Rabbit
Elephant
Giraffe
Whale
Weight (kg)
2
5
0.06
1.5
5
1,200
800
150
1
50,000
900
120,000
Avg. Heart Rate
(beats/minute)
150
90
450
275
190
44
65
70
205
30
65
20
Avg. Life Span
15
15
3
15
15
40
22
25
9
70
20
80
Animal
Human
Cat
Dog
Hamster
Chicken
Monkey
Horse
Cow
Pig
Rabbit
Elephant
Giraffe
Whale
Black Bear
Galapagos Tortoise
Hummingbird
Weight (kg)
Avg. Heart Rate
(beats/minute)
Avg. Life
Span
Avg. Heart Beats per
Lifetime (billions)
90
2
5
0.06
1.5
5
1,200
800
150
1
50,000
900
120,000
160
260
0.01
60
150
90
450
275
190
44
65
70
205
30
65
20
55
6
1260
70
15
15
3
15
15
40
22
25
9
70
20
80
25
177
5
2.21
1.18
0.71
0.71
2.17
1.50
0.93
0.75
0.92
0.97
1.10
0.68
0.84
0.72
0.56
3.31
Claim: An assertion or conclusion that addresses
the original question or problem.
Evidence: The facts, data, measurements or
calculations from an investigation that support
the claim.
 Appropriate: The evidence is relevant to the question
or problem; and
 Sufficient: There should be enough evidence to
convince others of the claim.
Reasoning: Link the evidence to the claim, explain
why the evidence supports the claim.
Example: A Disease Outbreak (Huang & Bayona, 2004)
Background:
An outbreak (epidemic) of gastroenteritis occurred in Greenport, a
suburban neighborhood, on the evening of April 28. A total of 89
people went to the emergency departments of the three local
hospitals during that evening. No more cases were reported
afterward. These patients complained of headache, fever, nausea,
vomiting and diarrhea. The disease was severe enough in 19
patients to require hospitalization for rehydration. Gastroenteritis
outbreaks like this are usually caused by the consumption of a
contaminated or poisoned meal. Meal contamination can often be
caused by pathogenic viruses or bacteria. However, acute outbreaks
are more often produced by toxins from bacteria such as
Staphylococcius spp., Clostridium perfringens, Salmonella spp. and
Vibrio cholerae. Food poisoning can also be caused by chemicals or
heavy metals, such as copper, cadmium or zinc, or by shellfish toxins.
Example: A Disease Outbreak (Huang & Bayona, 2004)
Identify the problem to be solved, the goal to be
reached, or the conclusion to be drawn:
1. Identify the source of contamination that
caused the outbreak of gastroenteritis (you will
present your “claim, evidence and reasoning”);
and
2. Suggest a plan to prevent future outbreaks.
Example: A Disease Outbreak (Huang & Bayona, 2004)
Inquire: What are some initial questions you have?
1.
2.
3.
Develop a plan: What are the steps you will take to find
answers to your questions?
1.
2.
3.
Example: A Disease Outbreak
Example: A Disease Outbreak
Example: The “Epidemic Curve”
Example: The epidemic team investigated the places where
affected persons, their relatives and neighbors ate that day.
Example: The epidemic team investigated the places where
affected persons, their relatives and neighbors ate that day.
Example: The epidemic team investigated the places where
affected persons, their relatives and neighbors ate that day.
Example: Once the implicated place was determined, the investigation centered on
the food. The following table includes the food items served in that place April 28:
Example: Once the implicated place was determined, the investigation centered on
the food. The following table includes the food items served in that place April 28:
None of the kitchen personnel were ill. The names of the kitchen personnel and their
participation in the food preparation are as follows:
 Manuel prepared the beef burritos and the potatoes,
 John prepared the salad and the fruit,
 Sally served all dishes except the ice cream, and
 Jane prepared the cheeseburgers and served the ice cream.

The ice cream was a commercial brand and was bought at a nearby supermarket.
Given that the epidemic team worked fast enough and the implicated meal(s)
was(were) identified before all food leftovers were discarded, food samples from
some meal leftovers were taken to the laboratory. In addition, stool samples were
taken from the kitchen personnel who prepared or handled each different food
item. The laboratory confirmed that Salmonella toxin was present in some of the
food samples and that one of the kitchen personnel had the same Salmonella
species. Furthermore, the Salmonella species found in the food and the kitchen
worker was the same species found in stool samples of the patients.
Claim: An assertion or conclusion that addresses
the original question or problem.
Evidence: The facts, data, measurements or
calculations from an investigation that support
the claim.
 Appropriate: The evidence is relevant to the question
or problem; and
 Sufficient: There should be enough evidence to
convince others of the claim.
Reasoning: Link the evidence to the claim, explain
why the evidence supports the claim.
Clear Instructions and Explanations;
2. Modeling of thinking skills;
3. Examples of Applied Thinking;
4. Feedback on student thinking processes;
5. Transitions (scaffolding) from initial support (e.g. teaching an
algorithm – specified set of steps for problem-solving) to
student-centered learning (e.g. heuristics – widely applicable
problem –solving strategies).
6. Combination of Direct Instruction with Guided Practice
7. Effective Questioning Strategies – pose paradoxes, dilemmas,
novel problems
1.
From King, Goodson & Rohani, 1998
http://www.youtube.com/watch?v=yTl9zYS3_dc
“An argument is a collective series of statements
to establish a definite proposition”
Argument: A set of sentences such that …


one of them is said to be true (the
conclusion); and
the others are being offered as reasons
(premises) for believing the truth of the
conclusion.
Words as clues: “if, then, since, therefore,…”
Arguments: Examples – Syllogisms (3-part)
Identify the premises and conclusions






All men are mortal.
Socrates is a man.
Therefore, Socrates is mortal.
Saginaw is north of Detroit.
Detroit is north of Toledo.
Therefore, Saginaw is north of Toledo.
Arguments: Examples – Syllogisms (3-part)
Identify the premises and conclusions






All men are mortal. (premise)
Socrates is a man. (premise)
Therefore, Socrates is mortal. (conclusion)
Saginaw is north of Detroit. (p)
Detroit is north of Toledo. (p)
Therefore, Saginaw is north of Toledo. (c)
Arguments: Examples – Syllogisms (3-part)
Identify the premises and conclusions






John is a bachelor.
John is single.
All bachelors are single.
All apples grow on trees.
All fruits grow on trees.
All apples are fruits.
Arguments: Examples – Syllogisms (3-part)
Identify the premises and conclusions






John is a bachelor. (P)
John is single. (C)
All bachelors are single. (P)
All apples grow on trees. (C)
All fruits grow on trees. (P)
All apples are fruits. (P)
Arguments: Examples (not stated as syllogisms)
Identify the premises and conclusions
The members of a Smith family are Aaron, Bob and
Char. Aaron is left-handed. Bob is left-handed.
Char is left-handed. Therefore, all members of the
Smith family are left-handed.
“I’m expected at the office at 8:00 am each weekday.
Since my drive takes half an hour, if I leave home
by 7:30 am, I should make it to work on time.”
Arguments:




Arguments are neither true nor false;
Premises, or assertions, are either true or false;
Arguments are either strong or weak
A strong argument:
 All the premises are true; and
 The conclusion follows from the premises.
“A good argument preserves truth.”
Types of Arguments:


Deductive: The truth of the premises
guarantees the truth of the conclusion.
Inductive: The truth of the premises makes
the truth of the conclusion more probable.
 Inductive arguments rely on the assumption that
the future will be like the past (the “uniformity of
nature”).
Types of Arguments: Which are Deductive, and
which are Inductive?
 “Susan is either a Democrat or a Republican.
Since she’s not a Democrat, she must be a
Republican.”
 “It has snowed in Michigan every December in
recorded history. Therefore, it will snow in
Michigan this December.”
 “Dom Perignon is a champagne, so it must be
made in France.”
Types of Arguments: Which are Deductive, and
which are Inductive?
 “Susan is either a Democrat or a Republican.
Since she’s not a Democrat, she must be a
Republican.”
(Deductive)
 “It has snowed in Michigan every December in
recorded history. Therefore, it will snow in
Michigan this December.”
(Inductive)
 “Dom Perignon is a champagne, so it must be
made in France.”
(Inductive)
Arguments: Validity vs. Truth
“When an argument is valid, then if its premises are
true, we can be certain that its conclusion is also
true.”

Valid arguments may contain false premises:
 All birds can fly.
 Penguins are birds.
 Therefore, penguins can fly.
All fish live in the ocean.
Whales are fish.
Whales live in the ocean.
Arguments: Using a Truth Table
“Is the Argument Strong?”
True Premises
Valid Argument
Invalid Argument
False Premises
STRONG
WEAK
WEAK
WEAK
Types of Inductive Arguments:
1.
2.
3.
4.
Arguments from Generalization
Causal Arguments
Arguments from Analogy
Arguments from Authority
All of these arguments CAN be valid and strong.
But we must consider some questions.
Arguments from Generalization:
“My Father-in-Law bought a car in Florida and got a
great deal. Florida must be a great place to buy
cars!”
“Helen is the Republican candidate, so she will be
opposed to gun control.”
Questions to pose:
1. How large was the sample size?
2. How representative was the sample?
3. Are there any counterexamples?
Causal Arguments:
“Married people live longer than single people. So
being married causes you to live longer.”
“I washed my car yesterday, so it’ll probably rain
today.”
Questions to pose:
1. How strong is the correlation?
2. Does the causal relationship make sense?
3. What causes what?
Arguments from Analogy:
“The universe is like a pocket-watch. Pocket
watches have designers. Therefore the universe
must have a designer.”
(William Paley, 18th century)
Questions to pose:
1. How similar are the two things being compared?
2. Are there any counter-examples?
Arguments from Authority:
“According to a research study, 93% of scientists favor
the use of live animals in scientific research.”
Questions to pose:
How were the data gathered? How were the questions
worded?
2. Who are the “experts?” Are they qualified in the
appropriate field?
3. Are these experts impartial? Are there any potential
sources of bias?
4. Are there any counter-examples (experts with contrary
opinions)?
1.
Logical Fallacies:
 Slippery Slope
 Black or White
 Ad hominem / Genetic
 Bandwagon
 Appeal to Nature
 Appeal to Emotion
 Cherry-picking Data
 Middle Ground
 Etc…
Issue: Hydraulic Fracturing
Tip-of-the-Mitt Watershed Council and
National Wildlife Federation
vs.
EnergyFromShale.org
Evaluating Arguments:
1. Conclusion: _______________
2. Premise #1 : _______________
3. Premise #2 : _______________
4. Hidden Premise (inference) #1 : ______________
5. Hidden Premise (inference) #2 : ______________
Evaluating Arguments: Questions
What general types of arguments are presented?
 Inductive or Deductive?
 Arguments from Generalization, Authority, Causation,
Analogy, other?
 What additional questions need to be asked (and answered)?
 Do you observe any potential “Logical Fallacies?”
 How strong is the argument?
▪ Are the premises accurate?
▪ Is there a valid connection between premises and conclusion?
Clear Instructions and Explanations;
2. Modeling of thinking skills;
3. Examples of Applied Thinking;
4. Feedback on student thinking processes;
5. Transitions (scaffolding) from initial support (e.g. teaching an
algorithm – specified set of steps for problem-solving) to
student-centered learning (e.g. heuristics – widely applicable
problem –solving strategies).
6. Combination of Direct Instruction with Guided Practice
7. Effective Questioning Strategies – pose paradoxes, dilemmas,
novel problems
1.
From King, Goodson & Rohani, 1998
Session 4: Online
 Develop HOTS-aligned Performance-based
Project(s)
 Identify the standard(s) to which the project is
aligned;
 Use some of the sample High-Impact Instructional
Strategies we’ve seen today.


Provide feedback/critique of other participants’
posted Project(s)
Begin thinking about how the student work from
your project(s) will be evaluated and scored.
Session 5: Face-to-face October 25th 4:30-7:30 pm

Assessing HOTS with Scoring Rubrics

Examining Student Work