Transcript Simcoe Country District School Board

Using Data to Improve Student
Learning
MISA Regional Workshop
March 24, 2006
Damian Cooper
Plan~Teach~Assess
Tel: (905) 823-6298
Email: [email protected]
www.damiancooperassessment.com
Outcomes
Understand the importance of evidence-based
decision making
 Know how to use data effectively in a
decision-making inquiry process
 Understand the importance of using multiple
and appropriate sources of data in an
accountability framework
 Understand basic principles and concepts
pertaining to the use of data
Know how to act upon data to improve student
learning
Terminology
Data?
Information?
Evidence?
“We live in a society that is data
rich but information poor.”
Robert H. Waterman
“What gets measured
gets done.”
Tom Peters
•Teach to the test?
•What gets tested gets
taught.
•“Test” what we teach.
“Why do we avoid data? The reason is fear – of
data’s capacity to reveal strengths and
weaknesses, failure and successes. Education
seems to maintain a tacit bargain among
constituents at every level not to gather or use
information that reveal where we need to do
better, where we need to make changes. Data
almost always point to action – they are the
enemy of comfortable routines. By ignoring
data, we promote inaction and inefficiency.”
Schmoker
The effective use of data
depends on simplicity and
economy.
Schmoker, 2003
Why Do We Use Data in our
Schools?
Individually, jot down 1-3 reasons
Share with a partner
As a group, what are your top 3 reasons?
Guide to using Data in School Improvement Efforts,
Learning Point Associates, 2004
Connecting Mission Statements and
Evidence
Mission Statement
Evidence We Have
Evidence We Need
Love of learning in a
safe caring
environment.
-discipline referrals
-vandalism
…
-student survey data
…
Our Speer-it gives us
strength to grow, to
learn, and to succeed.
-EQAO data
-report card data
…
-parent attitudinal data
…
Your turn…
Evidence We Have or Evidence We Need?
Demographic Data
•Enrollment by grade level
•Racial/ethnic composition
•Student mobility rate
•Percentage of limited English proficient students
•Percentage of students receiving special education services
Student Data
•Student participation rates in nonacademic services and programs
•Student participation in co-curricular activities and the degree to which it is representative of the overall composition of the
student body
•Percentage of students who exceed graduation requirements
Data on Teaching and Learning
•Usage data from library, information, and media services
•Number of students moving among ability groups, especially into groups with more challenging course work.
•Participation in professional development and evidence of impact on improved teaching
•Evidence of how analysis of data at the school level has resulted in specific improvements
•Evidence of how technology has contributed to increased use of data for decision making
Adapted from Getting Excited About Data • Edie L. Holcomb • 1999 Corwin Press, Inc.
Evidence We Have or Evidence We Need?
Data on School, Family, and Community Partnerships
•Data on family involvement in school activities and the degree to which the families involved are representative of the
overall student body
Indicators of Success
•Results of standardized tests for each of the last 5 years, disaggregated according to the largest and most significant
subgroups in the school
•Results of non-standardized, or alternative, assessments developed at the school level.
•Percentage of students in various educational and employment categories a year following graduation
•Daily student attendance
•Student dropout rate
•Daily teacher attendance
•Teacher turnover rate
•Results of climate surveys
•Percentages of students involved in various types of safety, discipline, and drug issues
Adapted from Getting Excited About Data • Edie L. Holcomb • 1999 Corwin Press, Inc.
Student Database
Who are these students?
(Demographics)
What are they
perceiving
about the
learning
environment?
(Perceptions)
What are they
experiencing?
(School
Processes)
What do they know?
(Student Learning
Results)
Multiple Measures / Data Elements
Data: Inputs and Outputs
 Demographics
 Affect
 Readiness to
learn
 Growth
 Opportunities
to learn
 Achievement
 Relationships
 Instruction
 Progress
 Interventions
Triangulation of Data
Use of different data sources to maximize
validity and reliability
Data from different participants
Data from different stages in a process
Triangulation of Data:
Student Achievement
Standardized Test
Data (normreferenced)
Valid &
Reliable
Picture of
Student
Achievement
Classroom
Assessment
Data
Standards-based
Test Data
(criterionreferenced)
Reliability and Validity in Assessments
Reliability
Validity
Working with Data
3) Gather
Data
4) Examine
Data
2) Check
Assumptions
1) Identify
Problem &
Formulate
Question
6) Develop
Improvement
Plan
5) Translate
Data into
Information
Checking Our Assumptions
Before examining a set of data, we often
suggest an hypothesis, based on certain
assumptions, to explain a problem or
current state.
Checking Our Assumptions
Examine these Gr. 3 EQAO Mathematics
data.
What trends do you notice over time?
Suggest some hypotheses to account for
these trends.
4 C’s of Examining Data
Concealing
Complete
Consistent
Comparisons
From Assessment Training Consortium/Ottawa-Carleton District School Board
4 C’s of Examining Data
Complete?
From Assessment Training Consortium/Ottawa-Carleton District School Board
4 C’s of Examining Data
Consistent?
What observations can you make
about this school’s Gr. 3 EQAO
Mathematics data in terms of
“consistency”?
From Assessment Training Consortium/Ottawa-Carleton District School Board
4 C’s of Examining Data
Comparisons?
What observations can you make
about this school’s Gr. 3 EQAO
Mathematics data in terms of board
and provincial comparisons?
From Assessment Training Consortium/Ottawa-Carleton District School Board
4 C’s of Examining Data
Comparisons?
What observations can you make
when you compare 2004-05 Gr. 3
EQAO Mathematics data with
report card data?
From Assessment Training Consortium/Ottawa-Carleton District School Board
Gr. 3 Mathematics Overall
Report Card Achievement
%-age of
students at
each
report card
grade.
45
40
35
30
R
D
C
B
A
25
20
15
10
5
0
1st Rpt.
2nd Rpt.
3rd Rpt.
Checking Our Assumptions
Examine the same Gr. 3 Report Card
Mathematics data, disaggregated by strand.
Suggest some hypotheses to explain the
data.
What further classroom level data do you
need to gather?
Gr. 3 Mathematics Report Card
Achievement by Strand
%-age of
students at
each report
card grade
level.
60
50
40
R
D
C
B
A
30
20
10
0
Number Measmt. Geometry Pattern.
Data
Mgmt.
How Do We Improve Student
Learning…and Achievement?
Planning
Instruction
Assessment
Classroom Management
Characteristics of Effective Schools
 A Safe and Orderly Environment
 A Clear, Focused Vision for Learning
 A Climate of High Expectations for Success
 A Focus on High Levels of Student Achievement
 Instructional Leadership
 Frequent Monitoring of Student Progress
 Strong Home-School Relations
School Improvement Planning: A Handbook for Principals, Teachers, and School Councils • Education Improvement Commission •
http://eic.edu.gov.on.ca
How Do We Improve Student
Learning…and Achievement?
“Strategic planning presumes that the
most vital, high-leverage thinking is
done primarily by “planners” before the
school year begins, rather than by
teaching practitioners throughout the
school year.”
Schmoker, 2004
The primary purpose of
analyzing data: improving
instruction to achieve greater
student success.
Schmoker, 2003
How Do We Improve Student
Learning…and Achievement?
1. What kinds of data am I able to gather about
each of these classroom variables?
2. How can I gather such data?




Planning
Instruction
Assessment
Classroom Management
Differentiating
Instruction & Assessment
Goal: success for all.
Instruction
 Students bring different
knowledge &
experience to school
 Students learn at
different rates
 Students learn in
different ways
Assessment
 Not all students are
able to demonstrate
their learning in the
same way
 Not all students
respond the same way
to test pressure
 Some students need
more scaffolding than
others
How Do We Differentiate
Instruction and Assessment?
Instruction
 Vary pace of instruction
 Vary depth & complexity
of materials
 Vary teaching strategy:
direct/indirect, guided,
whole-class, small group,
individual
 Vary amount of
intervention/independence
Assessment
 Provide choice of
assessment task
 Provide choice of
assessment environment
 Vary complexity of
assessment task
 Adjust scaffolding of
assessment task
A Model Lesson
1. Identify Learning Goal
2. Activate Prior Knowledge/Experience – hook, engage, personal
experience
3. Diagnostic Assessment of Prior Knowledge
4. Present New Learning – teacher, small groups, etc
5. Check for Understanding – orally
Differentiated
6. Practice New Learning/Scaffold New Learning
7. Self/Peer Assess Practice Work
8. Review and Consolidate Learning – tie back to learning goal
9. Apply New Learning to New Context
10. Assess Learning
(Differentiated)
Research on Effective Assessment
•The provision of effective feedback to students
•The active involvement of students in their own
learning
•Adjusting teaching to take account of the results of
assessment
•Recognition of the profound influence assessment
has on motivation and self-esteem
•The need for students to be able to assess
themselves and understand how to improve
Crooks, 1988; Black & Wiliam, 1998
Assessment for Learning
“Assessment for learning is any assessment
for which the first priority in its design and
practice is to serve the purpose of promoting
students’ learning. It thus differs from
assessment designed primarily to serve the
purposes of accountability, or of ranking, or
of certifying competence.”
Black, Wiliam et al. 2004
Assessment of Learning
“Assessment of learning includes those tasks
that are designed to determine how much
learning has occurred after a significant
period of instruction. The data from such
assessments is often used to determine report
card grades.
Assessment for Learning:
am I already doing it?
 Do teachers routinely share learning goals with
students so they know where they are heading?
 Do teachers routinely communicate to students
the standards they are aiming for before they
begin work on a task?
 Do teachers routinely have students self and peer
assess their work in ways that improve their
learning?
Assessment for Learning:
am I already doing it?
 Do teachers’ questioning techniques include all
students and promote increased understanding?
 Do teachers routinely provide individual
feedback to students that informs them how to
improve?
 Do teachers routinely provide opportunities for
students to make use of this feedback to improve
specific pieces of work?
Rapid Results: the Breakthrough Strategy
“The breakthrough strategy focuses on small
but immediate improvements and
operationalizes and accelerates the effectiveness
of teamwork, goals, and data … The key is to
pay attention to already existing approaches that
work, and work fast.”
Schmoker
Celebrating Success
“ Celebration, recognition and reward only
require a varied repertoire of methods and a
sincere heart. Teachers must believe that the
methods are more than a bald attempt to
manipulate or control behaviour. They must see
praise and recognition as an extension of a
leader’s character.”
Schmoker
Planning for Improvement
Working with Data
3) Gather
Data
4) Examine
Data
2) Check
Assumptions
1) Identify
Problem &
Formulate
Question
6) Develop
Improvement
Plan
5) Translate
Data into
Information
School and System Improvement
Professional learning communities use evidence
and intuition in order to work and talk together
to review their practices and to increase their
success…In a professional learning community,
the culture changes – everyone sees the big
picture and works for the good of the whole
community. Professional learning communities
bring together culture and contract. They value
both excellence and enjoyment.
Hargreaves, 2004
The Big Ideas of Professional
Learning Communities DuFour, 2004
 Ensuring that students learn
 A Culture of Collaboration
 A Focus on Results
The Big Ideas of Professional
Learning Communities DuFour, 2004
Ensuring that students learn:
-Focus on learning, (as well as) on teaching
-What do we want each student to learn?
-How will we know when each student has
learned it?
-How will we respond when a student
experiences difficulty in learning?
The Big Ideas of Professional
Learning Communities DuFour, 2004
 A Culture of Collaboration
-move beyond “Yes but…” and “If only…”
-partnerships
-sharing knowledge and learning
-team work
-community
-conversations
The Big Ideas of Professional
Learning Communities DuFour, 2004
 A Focus on Results
-establish baseline data
-set improvement goals
-work together to achieve goals
-gather frequent evidence of progress
-identify strategies that lead to the greatest gains
-share these with colleagues
Goals
Goals give teamwork meaning.
Goals allow educators to analyze and adjust
practice.
Goals and the commitment that they generate
are the glue that holds the team together.
A few specific, clear and measurable goals are
the most vital ingredient to purpose.
Connecting Our Goals to the Desired State
Goal
Student Achievement
Desired State
Results
Actions
Student Achievement
Current State
1.
2.
3.
4.
5.
6.
7.
What specifically needs to be done?
How does it need to be done?
What is the timeline for doing it?
Who is responsible for getting it done?
What resources are needed?
How will we monitor progress?
How often and by what means will we
communicate progress?
8. How will we celebrate success?
Committing Our Plan to Paper
School Action Plan
Improvement Goal:
Current State,
including
Evidence
Indicators
of Desired
State
(may
include
targets)
Specific
Measures
(tools used
to gather
data)
Strategies
(specific actions that will be taken)
Resources
(improvement
teams,
materials)
Responsibility
Timelines
Communicating
Progress
Celebration
of Success
Effective Goals
Poor Goal Statement:
Support high expectations for student
achievement with a specific focus on literacy
and numeracy.
Revised Goal Statement :
Improved student achievement with a specific
focus on literacy and numeracy.
S.M.A.R.T. Goals
Specific
Measurable
Attainable
Results-oriented
Time-bound
Administrator’s Role
 Leader
Do I have the teams in place?
___
Are the right structures in place to facilitate team work?
___
Am I creating a passion for system improvement?
___
Do I model evidence-based decision making?
___
Do I regularly assess the progress/implementation of our
improvement plans?
___
Do I ensure that our budget aligns with our goals?
___
Adapted from School Improvement Planning: A Handbook for Principals, Teachers, and School Councils • Education Improvement
Commission
Administrator’s Role
 Professional Developer
Do I provide time for staff to ask questions and dialogue about
evidence?
___
Do I know what is available from the Board to help us with
gathering data?
___
Do our PD activities focus on achieving school improvement
goals?
___
Do I know the strengths of my staff that will assist with our
system improvement plan process?
___
Do I provide appropriate professional development opportunities
for staff?
___
Adapted from School Improvement Planning: A Handbook for Principals, Teachers, and School Councils • Education Improvement
Commission
Administrator’s Role
 Communicator
Do I clearly explain the process to staff, school councils,
parents and other community members?
___
Can my staff articulate the sources of evidence we are
using ?
___
Am I using evidence when I am communicating about
the system?
___
Adapted from School Improvement Planning: A Handbook for Principals, Teachers, and School Councils • Education Improvement
Commission
Resources
1. Assessment Training Consortium, Linking Achievement Data to School
Improvement, 1996, Toronto, Ontario.
2. Educational Leadership, February, 2003
3. Holcomb, Edie L. Getting Excited About Data: How to Combine People, Passion,
and Proof, Corwin, 1999. ISBN. 0-8039-6739-X
4. Learning Point Associates, Guide to Using Data in School Improvement Efforts,
2004 (available on-line)
5. Northwest Regional Educational Laboratory, Evaluating Whole-School reform
Efforts: A Guide for District and School Staff, 2000 (available on-line)
6. Schmoker, Mike. Results: the Key to Continuous Improvement, ASCD, 1996.
ISBN. 0-87120-260-3
7. Schmoker, Mike. The Results Fieldbook, ASCD, 2001. ISBN. 0-87120-521-1
8. Tomlinson, Carol Ann. How to Differentiate Instruction in Mixed Ability
Classrooms, 2nd. Edition, ASCD, 2001. ISBN. 0-87120-512-2