Transcript Examining Student Work I PPT

Data for Student Success
Examining Student Work to
Inform Instruction
“It is about focusing on building a culture of quality data
through professional development and web based
dynamic inquiries for school improvement.”
Investigating Module: Examining Student
Work to Inform Instruction
• Professional Development
• Resources to support
• Inquiry tool to support
Module: Examining Student
Work to Inform Instruction
• www.data4ss.org
– Outcomes
– Agenda
– Work templates
– Protocol
– PowerPoint
• Resources for ISDs to adapt to their
audience for the module
• Please cite Data 4SS for work used
Beyond the Surface Level
You can’t “guarantee”
what you don’t monitor
Mike Schmoker
Moving to the student level…
Activity Objectives
• This activity shows us the power of
examining student work.
• This activity allows us to hear the
professional dialogue that occurs as
teachers come to agreement on criteria for
proficiency.
• This activity allows us to understand
teachers misconceptions and the
understanding of the GLCE assessed.
Protocol
• Middle School – Teachers analyzing
writing to a prompt
– Segment 2
– Segment 3
Comparative Item Analysis
• 6th grade
• Reading
• Is this GLCE assessed? If so, what has
been our school performance?
• How often is this GLCE assessed?
th
6
Grade Reading
Comprehension
Protocol
• Middle School – Teachers analyzing writing
to a prompt
– Segment 2
– Segment 3
• As you watch Segment 3, take notes on the
challenges faced by the teachers as they
begin the protocol.
Student Learning Problem
• How could the teachers have used this
information to begin the school year to
impact learning?
• What questions do you have about the
student learning problem, and what data
will answer those questions?
• Defining the learner-centered problem
shifts away from the blame game.
th
6
Grade Reading
Comprehension
Student Achievement
• Schmoker cites in Results Now
– Instruction itself has the largest influence on
achievement.
– The two things that matter most: What is
being taught and how well.
– Regardless of what a state policy or district
curriculum spells out, the classroom teacher
decides…what topics to cover.” (Manzo,
2003)
Student Work
• Examining student work is about teachers
looking at individual student demonstration of
learning – evaluating, determining instructional
needs, planning for instruction, teaching to the
objective determined as the next step in student
learning
– Student oral responses
– Student writing
– Student test results – answers to questions,
multiple choice, short answers
– Student performance
What Do Our Students Know And
What Are They Able To Do?
• What do our students know?
• What is the next instructional need?
– Based on student work, what is their next step in
learning? What is the learning challenge?
• Define the learner-centered problem – the
problem or challenge in a student’s
understanding or skills that interferes with the
student’s performance
• Focus must be on the learning
Moving Towards Knowing the Learner
• Analyzing student responses
– Understand
• a student’s response is the end product of his/her thinking
• there is a logic to the thinking process that the student
used
• Need to answer questions such as:
– Do students have any skills or knowledge to build
on?
– Do we need a total reteaching of a content?
– Are students lacking skills and/or content
knowledge?
– Is the design of the assessment itself an issue?
Examining Student Work
• Examining student work
– Neutral, observable data
– Challenges assumptions
– Helps build common understanding of
knowledge and skills students need
– Leads to discussions of work quality
• What are we considering proficient?
– Supports a culture of improvement
Examining Student Work
Protocol
• Resource book – Section 4
• Mathematics – Grade 6 MEAP Fall 2005
• What are we asking students to do? What
is the mathematics behind the task?
– Do the problem
– Make a list of the
skills/concepts/understandings
Examining Student Work
Protocol continued
• What would a proficient student need to do
to be successful?
– Make a list of the criteria for success
– Prioritize the list – What is critical for the
student to have in place to be proficient?
Examining Student Work
Protocol continued
• Examine the student work samples
against the prioritized list
• Sort into two piles – proficient, not
proficient
Examining Student Work
Protocol continued
• What are the strengths of the proficient
students?
• What are the challenges of the nonproficient students?
Examining Student Work
Protocol continued
Based on this information –
• What students need additional support?
• What are the next learning steps for these
students in the next 3-6 weeks?
Examining Student Work
Protocol continued
• How will you group them?
• What resource will you use? How will you
provide instruction?
• How will you know they have learned it?
What evidence will you collect? When?
Examining Student Work
Protocol continued
• For those students that are proficient, what
are the next learning steps in the next 3-6
weeks?
• How will you group them?
• What resource will you use? How will you
provide instruction?
Examining Student Work
Protocol continued
• How will you know they have learned it?
What evidence will you collect? When?
Examining Student Work
Protocol
• Knowing your LEAs, is examining student
work to this level common practice?
• How would using this protocol on a regular
basis impact student achievement?
• We have been looking from a teacher’s
point of view with the protocol, we need to
shift the work to explore what
administrators can do to support faculties
so that examination of student work can
become part of the school culture.
• What would need to be in place to develop
this culture, if we know that results happen
at the student level?
Building the Culture to Examine
Student Work
• What are the systemic pieces needed in a school to
measure student progress over time? What needs to
occur?
–
–
–
–
Decisions are made to collect data.
Assessments are developed and given.
Data is collected in some format and provided by teachers.
Data is analyzed and used to make instructional decisions by
teachers.
– Data and student samples are discussed by grade level teams
and administrators.
– Student work is used to determine proficiency and the next
learning steps.
A Principal’s Data Journey - An
Evolution
• Decision made three years ago to collect
data in comprehension – retelling
• Assessment given and data collected
• Discoveries
– Unable to determine what the data was
measuring.
– Conversations with teachers revealed
misconceptions about the assessment, what it
measured, and how it impacted student
learning.
Activity Objective
• This activity is designed to support
administrators to shift from data collection
to data analysis.
• This activity is designed to provide a
scaffold to begin data discussions.
• This activity will enable schools to shift the
emphasis from data collection to data
analysis.
Data Conference - Strategy
• What process needs to be in place to
begin the monitoring of student work?
• Scaffold for school leaders to begin to
monitor for student learning
• Resource book – Section 4
Scenario
• In 06/07 the District Curriculum
Coordinator mandated a requirement for a
comprehension assessment to be
completed 3 times a year K-8. (Data
Inventory)
• Student responses were to be scored and
data collected in a table format.
• This data was sent to the Curriculum
Coordinator and never seen again.
Scenario
• A 08/09 A new Curriculum Coordinator
has instructed principals to continue with
the assessment, but expects the principal
to analyze their building data as it relates
to the student achievement goals.
• Looking at the data collected, what do you
do with it?
• What process is used to begin to make
sense of the data?
Retelling Assessment
• What are we assessing? Why?
• A Retelling –
– Provides information on comprehension, sense of
story structure, and oral language complexity
– Provides insight on how students organize and
process text as well as interpreting and
comprehending text
– Allows the reader/listener to structure responses
according to personal and individual interpretations of
the text
Examine the Data
• Process for data analysis
– Determine proficiency
• On a 4 point rubric 3 is proficient
• On a 6 point rubric 4 is proficient
– Summarize the data
• What percent of the students are proficient?
– 1st data collection
– 2nd data collection
• Change in proficiency?
– Student work samples – available?
Examine the Data continued
• Using the Five Critical Questions of
Learning
– What is it we expect them to learn?
– How will we teach so that they can learn?
– How will we know when they have learned it?
– How will we respond when they do not learn?
– How will we respond when they already know
it?
Keep in mind…
• The quality of the learning information is
dependent upon the quality of the
assessment data gathered.
• The teacher needs to see the link between
the usefulness of the data being gathered
and the learning information about the
student it will provide.
Instruction for Learning
• It is what the teacher knows
and understands about how
the quality of the assessment
data determines the quality of
learning instruction that occurs
in the classroom
Planning the Data Conference
• Using PD resource – Planning A Data
Conference and the Five Critical Questions for
Learning – plan for the discussion
– As a team using the resources
– Plan individually how you would lead the
discussion
• Role play discussion
– Principal
– Teacher C – Mr. Dean
– Teacher A – Mrs. Jones
Data Conference Follow-Up
• As the principal, what are your next steps?
• As the teacher, what are your next steps?
• What will you see/hear in your classroom if
students are learning?
• How will you know? What evidence will
you collect?
What is evidence?
• Initial data conference will set the stage
• Set expectations
– Data to be collected
• What are we assessing?
– Data to be analyzed
• What is the data telling us?
– Evidence to bring
• Student work is the vehicle
Student Work – A Vehicle for
Learning
“Teachers have lately been required to
conduct exhaustive, student-by-student
reading assessments that can take days to
conduct. But few are told how to use their
results. We never encountered a single case
where teachers used these assessment
results to adjust or improve instruction; they
used them to group or regroup students.” –
Mike Schmoker
Protocol
• Elementary – Third grade teachers
analyzing student work – retelling
– Segment 2
– Segment 3
– Segment 4
• Middle School – Teachers analyzing
writing to a prompt
– Segment 2
– Segment 3
Summative versus Formative
Summative – Assessment of learning
Formative - Assessment for learning
How does this apply to a classroom?
Application to our LEAs
What do administrators and leadership
teams need to know and be able to do as a
result of this module?
Where are your LEAs in this first step of
data analysis?
Think about what you will need to put in
place to support them.
Starting With the End in Mind
• What is the non-negotiable by which we
are measured?
• What do we currently have that will
support our understandings of what is
being measured?
• Can teachers unpack the knowledge and
skills required by the state assessment?
Creating a Culture of Quality
Data
The most promising strategy for
sustained, substantive school
improvement is developing the
ability for school personnel to
function as professional
learning communities.
DuFour and Eaker, 1998
46
In Highly Effective PLCs…
Teams of teachers evaluate the effectiveness
of instruction and curriculum by establishing
student achievement goals. Teacher teams
discuss previous years’ trend data and
examine specific areas of program weakness.
Achievement goals can focus on reducing
failure rates, increasing access to rigorous
curricula, increasing the percent of students
attending college, and improving student
47
performance on district and state exams.
What is a PLC?
Professional
Learning
Community
48
Components of a PLC
•
•
•
•
Shared mission, vision and values
Collective inquiry
Collaborative teams
Action orientation and
experimentation
• Continuous improvement
• Results-oriented
49
The driving engine of a
PLC…
Is the collaborative team, on
which members work
interdependently to achieve a
common goal for which each
team member is mutually
accountable.
50
Why PLCs?
Because we must change…
• Every enterprise has to become a
learning institution and a teaching
institution. Organizations that
build in continuous learning in
jobs will dominate the twenty-first
century.
Drucker
51
Why PLCs?
Because we must change…
• You cannot have students as
continuous learners and effective
collaborators, without teachers
having the same characteristics.
-Fullan
52
Continuum of Community
Function
Toxic
Laissez-faire
Congenial
Collaborative
Accountable
Toxic – All about the teacher, adults not nice to one another or to the students
Laissez-faire – Teacher centered, autonomous, individual contractors
Congenial – Counterfeit, confuse niceness w/collaborative, focus not on kids
Collaborative – Have structures and skills in working together for improved
student achievement
Accountable – Able to acknowledge and deal with difficult data effectively;
Move beyond familiar solutions and approaches; Let go of instructional
practices that do not work; Call one another on unmet expectations or violated
norms.
Source: Skillful Leader II, Warnock presentation
53
CHANGE is difficult…
Those who undertake a 2nd Order
Change - such as transforming their
schools into Professional Learning
Communities must realize that change
is difficult but not impossible. Anxiety,
discomfort and conflict will accompany
2nd Order Change initiatives.
54
Change – The New Constant
• Leadership traits can positively
AND negatively effect
achievement (for students and
organizations)
• Second Order Change is the only
transformational change
55
1st and 2nd Order Change
First Order Change:
• Extension of past
• Within existing
paradigms
• Consistent w/ current
norms, values
• Incremental
• Linear
• Implemented w/ current
knowledge, skills
• Implemented by experts
Second Order Change:
• Break w/ past
• Outside existing
paradigms
• Conflicts w/ current
norms, values
• Complex
• Non-linear
• Requires new
knowledge, skills
• Implemented by
stakeholders
Balanced Leadership Research
• McREL’s work (69 studies/research)
has determined there are
–11 critical leadership responsibilities
when leading 2nd Order change
• As these responsibilities were
analyzed
–7 positively correlate
–4 negatively correlate
57
Positive Correlates to 2nd Order
Change
Change Agent
Comfortable planning w/ uncertain outcomes, consistently considers new,
better ways of doing things
Flexibility
Adapts leadership behavior to current needs, comfortable w/ and open to
dissent, can be directive or non-directive as situation warrants
Ideals & Beliefs
Strong professional beliefs (shared) about teaching and learning, behavior
consistent w/ beliefs
Intellectual Stimulation
Self, faculty, staff are aware of current theories and practices, discussion of
these is regular practice, continually reading an expectation
Knowledge of CIA
Knowledge of current best practices, provides guidance about effective
classroom practice, discussion of practices intentional and regular
Optimizer
Inspires self and staff to accomplish things thought to be beyond grasp,
positive attitude about ability of staff to accomplish substantial
goals, driving force behind major initiatives
Monitor / Evaluate
Regular monitoring and evaluation of curriculum, instruction
and assessment is expected, discussed, planned for
58
Negative Correlates to 2nd Order
Change
Communication
Being accessible, develop effective means to
communicate regularly with staff, maintain open and
effective types of communication
Culture
Foster shared beliefs and a sense of community and
cooperation, promote sense of well-being and
understanding of purpose (shared vision of what is the
important work)
Input
Involve staff in important decisions and implementation,
provide opportunity for input and policy development
Order
Provide and enforce clear structures, rules and
procedures for staff and students, establish
routines for running organization
59
Complex Change…
• Requires leadership greater than one
person can provide. Distributed
leadership maximizes the collective
strengths of individuals in a coherent,
adaptive, and sustainable
organization. It requires individuals
to assume responsibility, take action,
learn and grow for the benefit of the
whole.
60
Managing Complex Change
Source: Ambrose, Adapted from Villa and Thousand (1995)
Required Elements
Result
Vision
Skills
Incentives
Resources
Action Plan
Collegiality
Positive
Change
***
Skills
Incentives
Resources
Action Plan
Collegiality
Confusion
Vision
***
Incentives
Resources
Action Plan
Collegiality
Anxiety
Vision
Skills
***
Resources
Action Plan
Collegiality
Resistance
Vision
Skills
Incentives
***
Action Plan
Collegiality
Frustration
Vision
Skills
Incentives
Resources
***
Collegiality
Treadmill
Vision
Skills
Incentives
Resources
Action Plan
***
Isolation
61
8 Common Mistakes
• Allowing too much complacency
• Establish a sense of urgency
• Failing to create powerful guiding coalition
• Create guiding coalition
• ID critical number of champions re: change process
• Underestimating the power of vision
• Vision = direction, alignment, inspiration
• Without direction, individuals “do their own thing”
• Lack of direction leads to “permission” or “debate”
8 Common Mistakes
• Under communicating the vision by a power
of 10
• 2nd Order Change requires credible and frequent
communication
• Important to communicate vision often
• Unite leadership focused on vision
• Action aligned to vision
• Permitting structural, cultural obstacles to
block change process
• Confront obstacles: PD, structures, staff, policy
63
8 Common Mistakes
• Failing to create short-term wins
• Build on incremental gains; avoid “long march”
• ID goals, performance criteria, celebrate
• Declaring victory too soon
• Know difference between “win” and “victory”
• Successful change must be anchored in culture
• Celebration impetus to tackle bigger problems
• Neglecting to anchor changes firmly in
culture
• “Change” is part of the way we do things
• Behaviors must be rooted in social norms, shared values
64
Bruce Tuckman's 'Forming Storming'
Team Development Stages Model (1965)
FORMING
PERFORMING
STORMING
NORMING
65