What Will the Future Bring? A Narrow Window of Opportunity to Get Comprehension Instruction Right!!! P.
Download
Report
Transcript What Will the Future Bring? A Narrow Window of Opportunity to Get Comprehension Instruction Right!!! P.
What Will the Future Bring?
A Narrow Window of Opportunity to Get
Comprehension Instruction Right!!!
P. David Pearson
UC Berkeley
Slides posted at
www.scienceandliteracy.org
A Confluence of Opportunity
A model of comprehension processes
The Construction-Integration model of Walter Kintsch.
A framework that can be used to shape instruction in a productive
and flexible manner
The Four Resources Model of Peter Freebody and Allan Luke
A policy lever that can be used to frame the accountability system
to which we hitch our wagon
The Common Core Standards for English Language Arts
An assessment framework that could shape the terms of our
accountability system
The National Assessment of Educational Progress
A new rhetorical frame
Deeper Learning
Could the Stars be Aligned?
Let’s unpack each of these components a little
See how they fit together
See where they might take us in terms of curriculum and
pedagogy
Discuss the ways in which it could all unravel
THE CONFLUENCE…
Common Core
Kintsch
Four Resources
NAEP
The Common Core Standards…
Just to remind us
College and Career Readiness Standards
Common Core State Standards (grade by grade)
Assessments to measure their mastery
10 recurring standards for College and Career Readiness
Show up grade after grade
In more complex applications to more sophisticated texts
Across the disciplines of literature, science, and social studies
Affordances of the CCS
An uplifting vision based on our best research on the
nature of reading comprehension
Focus on results rather than means
Integrated model of literacy
Shared responsibility (text in subject matter learning)
An Uplifting Vision: ELA CCSS
Students who meet the Standards readily undertake the close,
attentive, reading that is at the heart of understanding and enjoying
complex works of literature.
They habitually perform the critical reading necessary to pick
carefully through the staggering amount of information available
today in print and digitally.
They actively seek the wide, deep, and thoughtful engagement with
high-quality literary and informational texts that builds knowledge,
enlarges experience, and broadens world views.
They reflexively demonstrate the cogent reasoning and use of
evidence essential to both private deliberation and responsible
citizenship in a democratic republic.
Focus on results rather than means*
Why?
Leave a place for each lower level to add its own signature
Some decisions about means really are local
Appropriate role for a larger body politic
Balance between our goals and our methods
*This is one issue on which there is reason to be concerned. Pick up later.
From the ELA Standards Document…
By emphasizing required achievements, the Standards
leave room for teachers, curriculum developers, and states
to determine how those goals should be reached and what
additional topics should be addressed.
Thus, the Standards do not mandate such things as a
particular writing process or the full range of
metacognitive strategies that students may need to monitor
and direct their thinking and learning.
Teachers are thus free to provide students with whatever
tools and knowledge their professional judgment and
experience identify as most helpful for meeting the goals
set out in the Standards.”
Integrated Model of Literacy
Two views of integration
Integrated Language Arts
Integration between ELA and disciplines
The CCSS are better on the interdisciplinary than on the
ELA integration
Corresponds to the actual uses to which reading and
writing are put.
Reading, writing, and language always serve specific
purposes
Reading and writing, not generically,
But about something in particular
The something in particular
What reading, writing and language look like in a domain
The information for a particular topic or unit or chapter
The information in a particular text
Language Arts
Social Studies
Science
Mathematics
Our current view of curriculum
A model I like: Tools by Disciplines
Academic Disciplines………..
Science
Social
Studies
Mathe- Literature
matics
Reading
Writing
Language
Early: Tools dominate
Academic Disciplines………..
Science
Social
Studies
Mathematics
Literature
Reading
Writing
Language
Later: Disciplines dominate
Academic Disciplines………..
Science Social
Studies
Mathe Literature
matics
Reading
Writing
Language
Weaving is even a better metaphor than a matrix
Writing
Language
Reading
math
literature
Social studies
Science
Shared Responsibility
English and Subject Matter
What we said before, reading and writing are always
situated in a topic and a purpose.
Knowledge fuels comprehension and writing.
Reading and writing, along with experience and
instruction, fuel knowledge.
Reading and writing and language work better when they
are “tools” for the acquisition of
Knowledge
Insight
Joy
Why sharing now?
The gap for college and workplace readiness
The increasing demands of an informational society
Finally addressing a problem that has always been there
Increasing awareness among disciplinary scholars
April 23, 2010 edition of Science.
The historical pathway to Kintsch’s
Construction Integration Model
Reader
Reading
Comprehension
Text
Context
Most models of reading have tried to explain
how reader factors, text factors and context
factors interact when readers make meaning.
Bottom up and New Criticism: Text-centric
Reader
Reading
Reading
Comprehension
Text
Context
The bottom up cognitive models of the 60s were
very text centric, as was the “new criticism” model
of literature from the 40s and 50s (I.A. Richards)
Pedagogy for Bottom up and New Criticism:
Text-centric
Since the meaning is in the text, we need to go dig it
out…
Leads to Questions that
Interrogate the facts of the text
Get to the “right” interpretation
Writerly readings or textual readings
Schema and Reader Response: Reader-centric
Reader
Reading
Comprehension
Text
Context
The schema based cognitive models of the 70s
and the reader response models (Rosenblatt) of
the 80s focused more on reader factors-knowledge or interpretation mattered most
Pedagogy for Reader-centric
• Since the meaning is largely in the reader,
we need to go dig it out…
• Spend a lot of time on
– Building background knowledge
– Inferences needed to build a coherent model
of meaning
– Readers’ impressions, expressions,
unbridled response
• Readerly readings
Critical literacy models: Context-centric
Reader
Reading
Reading
Comprehension
Text
Context
The sociocultural models of the 90s focused
on the central role of context (purpose,
situation, discourse community)
Pedagogy for Critical literacy models
• Since the meaning is largely in the
context, we need to go dig it out…
• Questions that get at the social, political
and economic underbelly of the text
– Whose interests are served by this text?
– What is the author trying to get us to
believe?
– What features of the text contribute to the
interpretation that money is evil?
CI: Balance Reader and Text: little c for context
Reader
Reading
Comprehension
Text
Context
In Kintsch’s model, Reader and Text factors
are balanced, and context plays a
“background” role--in purpose and motivation.
Pedagogical implications for CI
• Since the meaning is in this reader text
interface, we need to go dig it out…
• Query the accuracy of the text base.
– What is going on in this part here where it
says…
– What does it mean when it says…
– I was confused by this part…
• Ascertain the situation model.
– So what is going on here?
– What do you know that we didn’t know
before?
KintschModel
3
Knowledge Base
Text
1
Text Base
Experience
2
Situation Model
Inside the
Reader’shead
Out in the
world
NAEP
Locate and Recall
Interpret and Integrate
Critique and Evaluate
Common Core
Standards 1-3: Key ideas and details
Standards 4-6: Craft and structure
Standards 7-9: Integration of knowledge and ideas
CCSS
NAEP
Key ideas and details
Locate and Recall
Craft and structure
Interpret and Integrate
Integration of knowledge
Critique and Evaluate
and ideas
Range and level of text
complexity
Complexity is specified
but implicit not explicit
Consistent with Cognitive Views of Reading
Kintsch’s Construction-Integration Model
and Details What the text says
Locate
anda Recall
Decoder
Build
text base Key Ideas
Meaning
Makerand Ideas
of
Knowledge
Integrate
and Interpret
What the text means
Construct
aIntegration
“situation”
model
Craft
andtoStructure
What theit text
does
Critique
andknowledge
Evaluate User/Analyst/Critic
Put the
gained
work by applying
to novel
situations.
Kintsch
4 Resources
Text Base
Decoder
Situation Model
Meaning Maker
Put Knowledge to
Work
Text Analyst
NAEP
Locate and Recall
Says
Interpret and
Means
Integrate
Critique and
Does
Evaluate
CCSS
Key Ideas and
Details
Integration of
Knowledge and
Ideas
Craft and Structure
Kintchian Model
3
Knowledge Base
Text
1
Text Base
Reader as Decoder
2
Situation Model
Reader as Meaning Maker
Experience
Says
Means
Inside the head
Out in the
world
These consistencies provide…
Credibility
Stretch
Research “patina”
How could this all unravel?
We instantiate one or more of the components
inaccurately.
My candidates for conspiracies of good intentions
Close Reading: What the right hand giveth the left hand taketh away
Giving up on a tough assessment agenda
Close Reading
From literature classes
What do you think?
What makes you think so?
All about warranting claims about what the text means.
Thrilled when I saw the distribution of the first 9 CCSS
But…
www.corestandards.org/assets/Publishers_Criteri
a_for_3-12.pdf
The nature of the texts
C. Shorter, challenging texts that elicit close reading and
re-reading are provided regularly at each grade.
The study of short texts is particularly useful to enable students at a
wide range of reading levels to participate in the close analysis of
more demanding text.
Such reading focuses on what lies within the four corners of the text.
Questions and Tasks
A. A significant percentage of tasks and questions are text
dependent.
The standards strongly focus on students gathering evidence,
knowledge, and insight from what they read and therefore require
that a majority of the questions and tasks that students ask and
respond to be based on the text under consideration. Rigorous textdependent questions require students to demonstrate that they not
only can follow the details of what is explicitly stated but also are
able to make valid claims that square with all the evidence in the text.
Text-dependent questions do not require information or evidence
from outside the text or texts; they establish what follows and
what does not follow from the text itself
D. Questions and tasks require careful comprehension of
the text before asking for further evaluation or
interpretation.
The Common Core State Standards call for students to demonstrate a
careful understanding of what they read before engaging their
opinions, appraisals, or interpretations. Aligned materials should
thereforerequire students to demonstrate that they have followed the
details and logic of an author’s argument before they are asked to
evaluate the thesis or compare the thesis to others.
Materials make the text the focus of instruction by
avoiding features that distract from the text.
Teachers’ guides or students’ editions of curriculum materials should
highlight the reading selections. Everything included in the
surrounding materials should be thoughtfully considered and justified
before being included.
Given the focus of the Common Core State Standards, publishers
should be extremely sparing in offering activities that are not text
based.
5. The VAST unknown: CCSS and
Assessment
Assessments will make or break the CCSS movement
This is where we decide whether the movement is
Opportunity for reform
Or
Same old, same old
If assessments are not changed, these standards will not
make an iota of difference in teaching and learning
Short version of assessment…
With these standards, we’ll never get there with…
Multiple Choice or even short answer assessments as the primary
focus
These standards require us to engage kids in
Multiple day performance exams
Read within and across texts
Focus on project-based learning
Deeper learning
Have to return the the excitement of the mid 90s and get it right
this time.
I give PARCC and Smarter Balanced a 70% chance of getting it
right
108
The Players in the Assessment Game
Partnership for the Assessment of Readiness for College
and Careers: PARCC
Smarter Balanced Assessment Consortium: SBAC.
State Assessments
NAEP
Testing Industry
Constraints
Common Core Standards
Assessment consortia frameworks
Smarter Balanced Assessment Consortium (SBAC)
Partnership for the Assessment of Readiness for College and Careers (PARCC)
Different audiences and purposes (summative/formative/diagnostic)
Affordances
Learning progressions
Computer adaptive testing
Automated and distributed scoring
Improved psychometric tools
Through-Course, Interim/Benchmark
Assessment Visions
PARCC
Signal
& model good instruction
Rich & rigorous performance tasks
SBAC
Empirically
validate descriptions of learning
progressions
Through-Course Comprehension
Assessments & Learning Progressions
Should:
reflect
the interactive and multidimensional nature of
comprehension
assess readers’ abilities to understand, learn from, and use
text to accomplish specific purposes
provide transparent models of the demands of skilled
reading across a range of grades, disciplines, tasks
provide strong and informative predictors of success in
college, careers, and K-12
From Pearson, Valencia, and Wixson
An assumption/prediction?
Whatever model we develop, it is likely to be a hybrid
model.
Item format
Multiple Choice
Efficiency
Constructed Response Instructional Validity
Performance Tasks
Deeper Learning
Passage issues
Length and authenticity
Disciplines—Literature, Science and History
My focus
Given ourvast experience with MC and CR, I’ll focus on
performance tasks…
Except to say that well developed theories of mc items,
along with equally well-developed theories about classes
of distractors, are really important to decision validity and
the information value of test items.
We need to learn something from each and every response
a student makes, not just the right ones.
Performance Tasks: Why bother?
External validity
College ready
Career ready
Curricular validity
Powerful learning
Deeper learning
Consequential validity
What curricular activities will it lead teachers and students toward?
Sample Extended Constructed Response
Items (taken from CCSS)
9-10 ELA, Informational--Students analyze how Abraham
Lincoln in his “Second Inaugural Address” unfolds his
examination of the ideas that led to the Civil War, paying
particular attention to the order in which the points are
made, how Lincoln introduces and develops his points,
and the connections that are drawn between them.
11-12, ELA, Informational--Students delineate and
evaluate the argument that Thomas Paine makes in
Common Sense. They assess the reasoning present in his
analysis, including the premises and purposes of his
essay.
Sample Extended Constructed Response
Items (taken from CCSS)
11-12, ELA, Drama--Students compare two or more recorded or
live productions of Arthur Miller’s Death of a Salesman to the
written text, evaluating how each version interprets the source
text and debating which aspects of the enacted interpretations of
the play best capture a particular character, scene, or theme.
11-12, ELA Poetry--Students cite strong and thorough textual
evidence from John Keats’s “Ode on a Grecian Urn” to support
their analysis of what the poem says explicitly about the urn as
well as what can be inferred about the urn from evidence in the
poem. Based on their close reading, students draw inferences
from the text regarding what meanings the figures decorating the
urn convey as well as noting where the poem leaves matters
about the urn and its decoration uncertain.
Sample Extended Constructed Response
Items (taken from CCSS)
11-12, Informational Texts: Science--Students analyze the
concept of mass based on their close reading of Gordon Kane’s
“The Mysteries of Mass” and cite specific textual evidence from
the text to answer the question of why elementary particles have
mass at all. Students explain important distinctions the author
makes regarding the Higgs field and the Higgs boson and their
relationship to the concept of mass
Example of a New Standards Task from mid 1990s
Man and His Message
MLK
6-8 days, depending on class time
Culminating task: write an essay based upon choosing one
of several prompt options.
Pearson
Texts Encountered
A video about the Civil Rights Movement entitled, A Time for Justice.
An article about the Civil Rights Movement entitled, Confrontations.
An article about Ghandi from Scholastic's SEARCH magazine.
An oral rendition of King's Letter from a Birmingham Jail.
Printed versions of other King speeches.
An excerpt from a Time magazine account of the Rodney King riots in
East Central Los Angeles.
Two CNN video accounts of the riots: Rage of Despair and Roots of the
Problem.
Tasks Completed over the Period
collaboratively complete separate cognitive webs on key concepts
from the readings (Martin Luther King, Civil Rights Movement,
Non-Violent Resistance).
keep an ongoing log/chart of emerging learnings from all the
different texts (written, oral, or video).
answer straightforward "assignment-like" questions.
compare the similarities and differences between King and Ghandi
in a modified Venn diagram display.
the culminating essay
EVERYTHING can be scored
Affordances
Has the look and feel of powerful or deeper learning
Engages students in workplace like behaviors, including social
behaviors
Expands our conceptualization of what counts as a text
High capacity for engagement: interest and relevance
Maps onto many of the Common Core Standards for reading in
History
Could build professional community of teachers around
implementation and scoring
Constraints
Whose work is it anyway?
The inevitable dilemma of collaboration
Not just reading
(video and audio texts)
The usual suspects for performance tasks
Task generalizability
Scoring costs
Domain coverage
What counts for which standards
Example of a MEAP Inspired Pilot Task circa 2000 for a
Local Michigan District
School-wide Comprehension Assessment
Instructionally embedded (took a week out of the LA block)
Multiple text
Listening and reading
Reliance on multiple choice questions
Individual texts
Cross texts
Written Response to Reading
Position taken in response to the prompt question
Support from personal experience
Support from texts
Counts for both writing and reading comprehension depending on the
rubric used
Listening: Sister Anne’s Hands
Multiple Choice Question Stems
facts, relationships, inferences
This story is mostly about…
Sister Anne showed determination when she said…
What did Sister Anne mean when she said, “For me, I’d rather open
my door enough to let everyone in”?
The children learned much from Sister Anne. This selection tells us
that…
Kate Shelly and the Midnight Express
Multiple Choice Question Stems
facts, relationships, inferences
An important lesson of this story is…
How are Kate and her mother different?
In this selection, how do you know Kate showed determination and bravery
when crossing the Des Moines River Bridge?
Because Kate followed through, how would you predict she will face
problems in the future?
What dialogue does the author use to show you Kate has determination?
How do you know this story takes place in the past?
A Day’s Work
Multiple Choice Question Stems
facts, relationships, inferences
By showing determination, Francisco…
An important lesson from this selection is…
In this selection, why did Francisco and Grandpa leave the weeds?
This selection is not only about determination, it is also about…
Why did the author have Grandpa and Francisco speak in Spanish?
Cross Text Mult Choice Stems
facts, relationships, inferences
What important advice would both Grandpa and Kate give?
In both reading selections you read about main characters who…
How are Francisco and Kate different?
How were the characters rewarded for showing determination and
following through?
Applying Ideas to a Task
If you were trying to do something that was very hard, and you did
not think you could get it done, would you keep trying or quit? Use
examples from the two stories we read to support your decision.
Scoring
Answers questions by making connections bet ween
readings and u sing ideas from both readings to support
position taken
Answers questions and u ses ideas from at least
one story to support position taken.
Answers question and refers to
ideas in on e text
Answers question or
responds to theme
Writing in Response to Reading
Point Score 6
The student clearly and effectively chooses key or important ideas from each reading
selection to support a position on the question and to make a clear connection between
the reading selections. The point of view and connection are thoroughly developed
with appropriate examples and details. There are no misconceptions about the reading
selections. There are strong relationships among ideas. Mastery of language use and
writing conventions contributes to the effect of the response.
Affordances
In the direction of powerful and deeper learning, but…
Only one task for rubric-based scoring
Pretty good coverage of a range of cognitive targets vis a
vis question types.
Constraints
Does the reliance on MC format compromise its position
vis a vis powerful and deeper learning?
Limited to a single discipline—literature
Limited to a single genre—narrative
Limited to a single medium—text
Looking Ahead
Lots of dilemmas to manage
Back to the future and déjà vu all over again
Take advantage of new technologies, tools, and
understandings
Dilemmas to Manage
Social nature of embedded tasks
Domain coverage
Enabling skills or just the big outcomes
Dependence/independence across
Standards/cognitive targets/items
Issues of equity across populations, especially ELL and
LD populations
Déjà vu all over again
Build on what worked
Face the music on
Intertask generalizability
Scoring reliability and cost
Take advantage of new tools and technologies
Learning progressions (see SBAC)
But they are hard and different in reading
Discipline, topic, and text play a MAJOR role in shaping item
difficulty
We’ll just have to see how things scale in IRT models
Computerized scoring, but…
Easily corruptible
Will clever kids learn how to school the systems?
Computer adaptive testing
Garbage in-Garbage out
FINAL THOUGHT ABOUT ASSESSMENT
The promise of the ELA CCSS will not be realized
unless we create a new generation of reading
assessments that capitalize on the knowledge gained in
recent decades and the visions for the future.
Wending our way through yet another
perilous policy landscape…
A confluence of forces pointing us in the same direction
2. A wonderful opportunity
3. Some forces that can cause it all to unravel
1.
Old Chinese Proverb
May you live in interesting times…