Transcript Classroom Assessment - University of Central Florida

How do we know what
they know?
FASS Meeting
Orlando, FL
April 5, 2004
Arthur Eisenkraft ([email protected])
Are we listening?
• The optometrist
• The Duracell competition
• Two objects falling in a vacuum
• The cord of wood
No Child Left Behind
• Enormous concern about NCLB and other high
stakes assessments.
• NCLB – a potential nightmare –
– AYP - “need of improvement”
– public embarrassment
– best students leaving and the scores dropping more,
closing of schools.
• What can we do?
– This year alone 26,100 of the nation’s 91,400 have
been labeled “schools that need improvement.” (Sam
Dillon, 1 in 4 Schools Fall Short Under Bush Law, N.Y.
Times, January 27, 2004 at A21)
NCLB - Advice
• Assessment in Support of
Instruction and Learning:
Bridging the Gap Between
Large-Scale and Classroom
Assessment - Workshop
Report (2003)
Board on Testing and
Assessment, Mathematical
Sciences Education Board,
Center for Education
• www.nap.edu
The Deborah Meier Amendment
• A basic test should first be taken by the folks we
honor by electing to office.
• The people who legislate or mandate a test
should be required first to take it themselves to
ensure that it's measuring what they think it is.
It's a form of validity checking.
• They might even have their scores posted!
• Seeking Alternatives to Standardized Testing
• By Jay Mathews, Washington Post Staff Writer
Tuesday, February 17, 2004; 10:46 AM
Today’s Discussion
• Formative classroom assessment can
positively impact instruction and therefore
is our best approach to students
performing better on all tests.
• What can teachers do?
• What can we do to support teachers?
Brief History of Assessment
• When did it all begin?
– No Child Left Behind
– FCAS
– New York State Regents
How do we assess?
• Please discuss
–
–
–
–
–
–
–
–
–
–
–
–
Observation
Portfolio
Projects
Questioning
Paper and pencil
Interview
Presentation
Checklist
Skills
Self assessment
Quizzes
conferences
Classroom Assessment
The Grade Book
Tests
Quizzes
Homework
Class participation (?)
Lab reports
Attendance (X)
Projects
The Final Exam
Local
State – High stakes
These are often treated as summative though they do inform
as formative.
Other formative assessments include:
–Questions in class
–Practice tests
Get tests back immediately
• They can then be used for formative
assessment.
• How can anyone continue instruction
when you have a tool that informs you of
student understanding?
• Easily measured by supervisors and
students alike
Formative Assessment
• The value of formative assessment (Paul Black):
students often have limited opportunities to
understand or make sense of topics because
many curricula have emphasized memory rather
than understanding. Textbooks are filled with
facts that students are expected to memorize,
and most tests assess students’ abilities to
remember the facts.
What Goes Wrong?
Tests that do not correlate with understanding
•Force Concept Inventory (FCI)
•Regents exam question on moving galaxies
•Private Universe videotapes
What Goes Wrong?
Tests that do not correlate with understanding
•Force Concept Inventory (FCI)
•Regents exam question on moving galaxies
•Private Universe videotapes
We’re not testing what we teach
•Harris cartoon of mouse and maze
What Goes Wrong?
Tests that do not correlate with understanding
•Force Concept Inventory (FCI)
•Regents exam question on moving galaxies
We’re not testing what we teach
•Harris cartoon of mouse and maze
We’re not teaching what we test
•“Waldo” phenomenon
Improvements
Rubrics
•Clearly defined grading schema – matrix
•B++++ and A----•Have students help create rubric
•Ownership
•Motivation
•Have students self – evaluate with rubric
The grading rubric +/Student Grade
A
Teacher Grade
A
A
C
C
C
Improvements
Rubrics
•Clearly defined grading schema – matrix
•B++++ and A----•student and teacher comparisons: A,A or C,C or
A,C or C,A
•all require very different discussions
Saphier effective instruction
•testing for understanding
•how do you know what the students know?
Cognitive Empathy
With references from
The Skillful Teacher
-Jon Saphier
Checking for Understanding
Knowing when students don’t understand
suggests that teachers have means for
checking for understanding.
What means do we have for checking for
understanding?
Checking for understanding
• Presses on
– No test return for 3 weeks
– After math lesson, here’s your 25 problems
– Take a clean sheet, we’re going on
– No clue there are kids in the room
– Never asks students to explain
– Incorrect response, - can anyone else answer
Checking for understanding
• Presses on
• Reads cues
– Their looks
– Eye contact
– Nodding heads
– Asleep or awake
– Misbehavior
– I can see it in their eyes
Checking for understanding
• Presses on
• Reads cues
• “dipsticks”
–
–
–
–
–
–
–
White-boarding
Short quiz
Raise the hand
Choral answers
Cards with A or B
Raise fingers with 1(index) or 2
List answers on board – which answer is the best
Checking for understanding
•
•
•
•
Presses on
Reads cues
“dipsticks”
Uses recall questions
–
–
–
–
–
–
–
–
What do we already know
List examples
Who invented
Where’s waldo
Definitions
Name the parts of the microscope
The scientific method
Restating what is already known
Checking for understanding
•
•
•
•
•
Presses on
Reads cues
“dipsticks”
Uses recall questions
Uses comprehension questions
–
–
–
–
–
–
–
Explain
Justify
Compare
Apply
Calculate
Why
Summarize
Checking for understanding
•
•
•
•
•
•
Presses on
Reads cues
“dipsticks”
Uses recall questions
Uses comprehension questions
Anticipates confusion
– Photosynthesis, Kreb’s cycle
– Understanding that some kids are literal
• Underground railroad
– Misconceptions research
– Look at prior knowledge
– Teacher examining their own assumptions
A TEST for Checking for
Understanding
How do you know that a student
understands?
What evidence do you have?
How often should you be able to answer this
question?
The National Science Education Standards (NSES)
Less Emphasis On:
Assessing what is easily measured
More Emphasis On:
Assessing what is most highly valued
Less Emphasis On:
Assessing to learn what students do not know
More Emphasis On:
Assessing to learn what students understand
Instructional Models
• Karplus
– three-phrase learning cycle
• exploration, invention and discovery
• Lawson
– exploration, term introduction, and concept
application
• Bybee  5E
– Engage, explore, explain, elaborate, evaluate
• 7E clarification of 5E
5E
Engage
Explore
Explain
Elaborate
Evaluate
5E

--------------------------|
|
Engage
7E
Elicit
----------------|
|
----------------------------
Engage
5E

--------------------------|
|
Engage
Elicit
----------------|
|
----------------------------
Explore
7E
-------------------------------------------
Engage
Explore
5E

--------------------------|
|
Engage
7E
Elicit
----------------|
|
----------------------------
Engage
Explore
-------------------------------------------
Explore
Explain
----------------------------------------------
Explain
5E

--------------------------|
|
Engage
7E
Elicit
----------------|
|
----------------------------
Engage
Explore
-------------------------------------------
Explore
Explain
----------------------------------------------
Explain
-------------------------|
Elaborate
--------------|
-------------------------|
Evaluate
Elaborate
Evaluate
----------------|
---------------------------
Extend
4 Q Assessment Model
• What does it mean?
4 Q Assessment Model
• What does it mean?
• How do we know?
4 Q Assessment Model
• What does it mean?
• How do we know?
• Why should I believe?
4 Q Assessment Model
• What does it mean?
• How do we know?
• Why should I believe?
• Why should I care?
The other four questions
• What did you say?
• Should we take notes?
• When is class over?
• Will this be on the test?
Challenges
Identify who are we testing
oStudents
oTeachers
oSchools and districts
Challenges

Identify for what purpose
(from Classroom Assessment and the NSES, NRC)
oHelp students learn
oTo illustrate and articulate the standards for quality work
oTo inform teaching
oTo guide curriculum selection
oTo monitor programs
oTo provide a basis for reporting concrete accomplishments to interested
parties
oFor accountability
oCertification
Reporting individual achievement
Grading
Placement
Promotion
Accountability
oParents to taxpayers
(from High Stakes Assessments, NRC)
Challenges
Are we trying to use ONE instrument
for all (students, teachers, schools)?
for all purposes?
Understanding vs. belief
oMazur student taking FCI
Classroom Assessment and the
National Science Education
Standards (2001)
Center for Education
www.nap.edu
Knowing
What
Students
Know: The
Science and
Design of
Educational
Assessment
(2001)
Center for
Education
The Assessment Triangle
cognition, observation, and interpretation—
must be explicitly connected and designed
as a coordinated whole. If not, the
meaningfulness of inferences drawn from
the assessment will be compromised.
C
O
I
Assessment #1
• Question: What was the date of the battle
of the Spanish Armada?
– Answer: 1588 [correct].
• Question: What can you tell me about
what this meant?
– Answer: Not much. It was one of the dates I
memorized for the exam. Want to hear the
others?
Assessment #2
• Question: What was the date of the battle of the
Spanish Armada?
– Answer: It must have been around 1590.
• Question: Why do you say that?
– Answer: I know the English began to settle in Virginia
just after 1600, not sure of the exact date. They
wouldn’t have dared start overseas explorations if
Spain still had control of the seas. It would take a little
while to get expeditions organized, so England must
have gained naval supremacy somewhere in the late
1500s.
Comparison
• Most people would agree that the second
student showed a better understanding of the
Age of Colonization than the first, but too many
examinations would assign the first student a
better score.
• When assessing knowledge, one needs to
understand how the student connects pieces of
knowledge to one another. Once this is known,
the teacher may want to improve the
connections, showing the student how to expand
his or her knowledge.
Formative Assessment Research
• Black and Wiliam (1998) provide an extensive review of
more than 250 books and articles presenting research
evidence on the effects of classroom assessment.
• They conclude that ongoing assessment by teachers,
combined with appropriate feedback to students, can
have powerful and positive effects on achievement.
• They also report, however, that the characteristics of
high-quality formative assessment are not well
understood by teachers and that formative assessment
is weak in practice. High-quality classroom assessment
is a complex process
Return to bad practice
• We revert back to old, failed strategies
when under pressure
• Comfort food
• In dealing with our children
• You’ll poke your eye out
• What if everybody jumped off the Empire State
Building?
• Prison, drug addiction
Responding to the pressure
(NCLB, FCAS)
• If you go back to the worksheets and it doesn’t
work, you won’t be blamed in the same way
because this is what everybody does.
• If you continue with something that is closer to
the edge, you are more open to criticism
• This could be the reason for the blame
• Fail forward as Thomas Edison did
Support our teachers
• Help them to not succumb to the
pressures.
• Encourage them to improve their
instruction, not to revert to poor pedagogy.
• Give them permission to be better
teachers.
Questions that foster deep understanding rather than
questions that ask for repetition of memorized information
and conclusions. (Jim Minstrell and Emily van Zee)
• Ask or promote questions to open an inquiry and elicit students’
initial understanding and reasoning.
• Ask or promote questions to interpret and make sense of data in
order to generate new knowledge and understanding.
• Ask or promote questions to clarify or elaborate on observations and
inferences.
• Ask or promote questions to encourage learners to justify their
answers and conclusions or to explain their reasoning to go beyond
a mere stating of an answer.
• Ask or promote questions to extend or apply learned ides.
• Ask or promote questions that help learners monitor their own
learning.
Understanding by Design
• When do we generate test questions
• UbD (Grant Wiggins, Jay McTighe)
– Enduring understandings
– Evidence
– Instruction
Not in vain
• When studying about veins and arteries,
for example, students may be expected to
remember that
– arteries are
• thicker than veins,
• more elastic, and
• carry blood from the heart;
– veins carry blood back to the heart.
Sample test item
• Arteries
– a.
– b.
– c.
– d.
– e.
Are more elastic than veins
Carry blood that is pumped from the heart
Are less elastic than veins
Both a and b
Both b and c
The new science of learning
• does not deny that facts are important for
thinking and problem solving.
• Research on expertise in areas such as
chess, history, science,and mathematics
demonstrate that experts ’ abilities to think
and solve problems depend strongly on a
rich body of knowledge about subject
matter (e.g.,Chase and Simon,1973;Chi et
al.,1981;deGroot,1965).
Facts are not enough
• However,the research also shows clearly that
“usable knowledge ” is not the same as a mere
list of disconnected facts.
• Experts’ knowledge is
– connected and organized around important concepts
(e.g., Newton ’s second law of motion);
– “conditionalized” to specify the contexts in which it is
applicable;
– supports understanding and transfer (to other
contexts) rather than only the ability to remember.
Vein and artery experts
• Know the facts in the mc question
• also understand why veins and arteries have particular
properties.
– They know that blood pumped from the heart exits in spurts
– That the elasticity of the arteries helps accommodate pressure
changes.
– They know that blood from the heart needs to move upward (to
the brain) as well as downward and that the elasticity of an artery
permits it to function as a one-way valve that closes at the end of
each spurt and prevents the blood from flowing backward.
• They are better able to transfer
– Design an artificial artery strong enough to handle pressure with
or without elasticity (Bransford and Stein,1993).
Classroom environments
Formative assessments—ongoing assessments designed to make
students’ thinking visible to both teachers and students are
essential.
They permit the teacher to
• grasp the students’ preconceptions,
• understand where the students are in the “developmental corridor” from informal to formal thinking, and
• design instruction accordingly.
In the assessment-centered classroom environment, formative
assessments help both teachers and students monitor
progress.
Learner friendly assessments
• Provide students with an opportunity to revise
and improve their thinking (Vye et al.,1998b),
• help students see their own progress over the
course of weeks or months
• help teachers identify problems that need to be
remedied (problems that may not be visible
without the assessments).
• Problem based learning model
– Sport for the moon
Defining Science Content
• Facts, process, knowledge
– Order of the planets
• I went to a two day history seminar and
there was no content – I didn’t learn any
new dates.
• Where is the knowledge we have lost in
information. (T.S. Eliot)
Wait time study at King’s College
(Black & Wiliam, 2000)
• Questions opened up discussion that helped
expose and explore students’ assumptions and
reasoning.
• At the same time, wrong answers became useful
input, and the students realized that the teacher
was interested in knowing what they thought, not
in evaluating whether they were right or wrong.
• As a consequence, teachers asked fewer
questions, spending more time on each.
Wait time study at King’s College
(Black & Wiliam, 2000)
• In addition, teachers realized that their lesson
planning had to include careful thought about
the selection of informative questions.
• They discovered that they had to consider very
carefully the aspects of student thinking that any
given question might serve to explore.
• This discovery led them to work further on
developing criteria for the quality of their
questions.
Grades vs. comments
• Simply giving grades on written work can be
counterproductive for learning (Butler, 1988)
• In response, teachers began instead to concentrate on
providing comments without grades—feedback designed
to guide students’ further learning.
• Students also took part in self-assessment and peerassessment activities, which required that they
understand the goals for learning and the criteria for
quality that applied to their work.
• In these ways, assessment situations became
opportunities for learning, rather than activities divorced
from learning.
Sliding toward Socrates
• Follow-up research
– Grades vs. comments vs. grades & comments
• No distinction between instruction and
assessment
Your text and program
4
3
2
1
Beliefs and Practice
+
+
1
+
2
3
4
What you value
(e.g. inquiry, content)
Your text and program
4
3
2
1
Beliefs and Practice
+
1
2
3
4
What you value
(e.g. “telling and memorizing”)
Your text and program
4
3
2
1
Beliefs and Practice
+
+
+
+
1
2
3
What you value
4
Your text and program
4
3
2
1
Beliefs and Practice
1
o
o
o
o
2
3
4
What you value
(e.g. inquiry, content)
Your text and program
4
3
2
1
Beliefs and Practice
o
o
o
1
2
3
4
What we value
(e.g. telling and memorizing)
Our text and program
4
3
2
1
Beliefs and Practice
+
+
+
+
1
2
3
4
Content, pedagogy, assessment factors
Summary
• Return tests promptly
• Rubrics
• Understanding by Design
– Enduring undestandings, evidence, instruction
• Testing for undestanding
– Know what each student knows at the end of every class
• 7E learning model
• Assessment triangle: Cognition, Observation,
Interpretation
• Don’t revert to non-productive behavior
• Define science content to include inquiry and process
– Facts in meaningful context
Assessment in the movies
• Do we recognize this quality of
assessment?
In Conclusion
• The best way in which to deal with NCLB is to improve
instruction in the classroom.
• This can be done with improvements in formative
assessment
– Formative assessment has been shown to be effective at
improving student achievement.
– Help teachers learn “Testing for understanding” techniques.
• Provide teachers with the confidence and support to
continue to improve their teaching and resist the
temptation to revert to poor pedagogy and worksheets in
attempts to deal with NCLB.
• Give them permission to be good teachers.
Common Advice
• ….the wisdom to know what I can change
• My husband handles the important issues
“We must be the change we
want to see in the world.”
- Gandhi
Get inside a student’s head.
Cognitive empathy for the workings of
the learners’ minds -- an ability to put
themselves in the learners’ shoes
Unscrambling Confusion
Determining what students don’t understand
implies that teachers have ways of
unscrambling confusions that identify the
specific point(s) of misunderstanding and
deal with them.
What means do we have for unscrambling
confusions?
Unscrambling Confusions
• None
Unscrambling Confusions
• None
• Re-explains
Unscrambling Confusions
• None
• Re-explains
• Isolates point of confusion with pinpoint
questions
Unscrambling Confusions
• None
• Re-explains
• Isolates point of confusion with pinpoint
questions
• Perseveres and returns
Unscrambling Confusions
• None
• Re-explains
• Isolates point of confusion with pinpoint
questions
• Perseveres and returns
• Has student explain own current thinking
Three pillars of assessment
• a model of how students represent
knowledge and develop competence in the
subject domain
• tasks or situations that allow one to
observe students’ performance,
• an interpretation method for drawing
inferences from the performance evidence
thus obtained.
Improvements
Rubrics
•Clearly defined grading schema – matrix
•B++++ and A----•student and teacher comparisons: A,A or C,C or
A,C or C,A
•all require very different discussions
Eliciting prior understandings
•Fish is fish
Standards
(good instruction)
State exams
(good preparation)