Transcript Slide 1

Integrating Student Standardized
and Interim Assessments
with National Board's
Teacher Development Process
The Eleventh MARCES/MSDE Event
October 20, 2011
Session Goals
• Overview of how National Board assesses teacher practice and
provides standardized feedback based on the Five Core
Propositions
• A case study of integrating both student standardized and interim
assessments with teacher professional development
• Bringing together the results of teacher and student assessments
to enhance teacher practice and student learning
• Considerations for taking these lessons to scale
A Way Forward
• The U.S. educational accountability movement has not
yet delivered desired results
– U.S. performance on the OECD’s PISA*
• 14th in reading
• 17th in science
• 25th in math
• We need to support teaching practice – the tools and
methods of our learning communities
(Organization for Economic Cooperation and Development’s Programme for International Student Assessment)
• Over the last 20 years, the nation’s focus on measuring
educational results has grown and intensified
• Historically, two separate conversations
• Student outcomes
• Teacher evaluation
• NBPTS has always believed that the success of teachers in
promoting student learning should be a defining measure of
an accomplished teachers
• Teacher demonstration of student learning is integral to
becoming an NBCT
Balanced Focus
Policy - Outcome
• Student Achievement
• Teacher Evaluation
Practice - Input
• Student Learning
• Teacher Growth
The Challenge
How do we leverage accomplished teacher
practice to address the societal need of
graduating well-educated students?
NBPTS’ Approach
Provide states and school districts whole
school transformation tools and processes
that are:
– Successful
– Systematic
– Scalable
– Sustainable
Take One! Whole School Transformation
• Validate a process to improve teaching effectiveness and student
learning in low-performing schools
• Assess NB’s Take One! professional development feeder pattern
approach in strengthening teacher effectiveness and student
outcomes
• Determine feasibility of taking to scale
• Compare two feeder school systems, using one as a control
• Through December 2013
• Initial year, 2010-11, funded by the Gates Foundation
To better understand
appropriate measures of
teacher practice linked to
student performance,
NBPTS convened a task
force of educational
experts in assessment,
reform and measuring
teacher quality.
Task Force Members
Lloyd Bond, University of North Carolina, Greensboro
Peggy Carr, National Center for Education Statistics
Linda Darling-Hammond, Stanford University
Doug Harris, University of Wisconsin, Madison
Frederick Hess, American Enterprise Institute
Robert Linn (chair), University of Colorado, Boulder
Lee Shulman, Carnegie Foundation
Clarity in Language
Greater clarity will net better outcomes for college and
workforce preparedness
• Student achievement is the status of subject-matter
knowledge, understandings and skills at one point in
time and is commonly measured by standardized
tests.
• Student learning is the growth in subject-matter
knowledge, understandings and skills over time.
Learning is gauged by comparing student mastery at
successive points in time.
Recommendations for Teacher Evaluation
• Be grounded in student learning, not student achievement
– “The only defensible way to determine teacher effectiveness is to focus on
the gains that students have realized over the period during which the
teacher provided instruction.”
• Employ measures of student learning explicitly aligned with the
elements of curriculum for which the teachers are responsible
– “Tests may need to be differentiated to address the needs of the groups of
students being taught, including students with disabilities or languageacquisition needs.”
Criteria for Large-Scale Standardized Tests
As a measure of teacher effectiveness, they must meet the
following minimum conditions
• Curriculum-related scale with equivalent unit of measure along a
considerable continuum of achievement
• Information on validity of tests for assessing special populations
• A data system that tracks students and links to teachers
• Alignment between tests and state curricula
From Learning to Measuring
What is Tested Does Count,
But Much of What Counts Is Not Measured
What is formally tested in core subjects only
Knowledge and learning that can be measured
All classroom learning
Why Jefferson County School District??
National: NBPTS and
Gates
Regional: Southern
Regional Educational
Laboratories (SREB)
State: AL Dept. of Education
and Samford University Staff
Local:
Jefferson County Schools
(Superintendent
Hammonds)
Factors That Make The Difference
• A Structured Approach
• Buy-in that Creates a Shift in School Culture
• Clarity of Roles
• Feeder Systems
A Structured Approach: Alignment of Teacher Tasks
Take One
Question
Alabama Quality
Teaching Standards,
Teacher Evaluation,
Professional
Learning Plan
2. What are the
relevant
characteristics of
this class that
influenced your
instructional
strategies for this
learning sequence:
1.5 Provides
instructional
accommodations ,
modifications, and
adaptations to meet
the needs of each
individual learner.
ethnic, cultural, and
linguistic diversity;
the range of abilities
of the students; the
personality of the
class?
4.7 Understands the
characteristics of
exceptionality in
learning.
Response to
Instruction
(RTI)
Continuous
Improvement
Plan
(CIP)
Tier I
Part I of the CIP
Progress
monitoring all
students
Identification of
needs
NBPTS
Standards
For
Entry 2
I. Knowledge of Learners
II. Knowledge of the Field
of Literacy: Reading–
Language Arts
III. Equity, Fairness, and
Diversity
IV. Learning Environment
V. Instructional Resources
VI. Instructional Decision
Making
VII. Assessment
VIII. Integration
IX. Reading
XIV. Teacher as Learner
Buy-in to Change School Culture
• Critical Mass of Teachers
– Works collaboratively with peers around a common purpose
– With colleagues, addresses issues across the feeder system
– Focused dialogue and peer support
• Teacher Leader
– Augments principals with instructional leadership
– Facilitate collaborations, structured discussions, feedback and support
• Principals
– Demonstrated leadership through changed interactions with teachers
– Structure of environment that supports focused collaboration
– Serves as key instructional leader
• School District
– Superintendent involvement & presence
– Deployment of human capital
– Alignment of resources supportive of implementation
Project Approach
• Train-the-trainer model
– Project Director and Coordinator provide leadership and support to principals
who will meet once monthly to receive training on guiding their faculty.
• All-participant group meetings
– Host four meetings to build professional learning communities among the
feeder school system meetings will focus on, for example, reflective thinking.
• Small group meetings
– Twice monthly, principal and facilitator will meet with faculty to guide the
process, lead peer reviews, and provide teacher support.
Feeder System
• Permits a systems approach, creating more feedback loops
during the entire K-12 experience
• Facilitates a more long-term connection to students, enabling
teachers to preview & follow-up about students
• Strengthens collaboration among teachers at different
schools to build connections, strengthen practice & resolve
issues
• Enables early identification of student issues, while successful
intervention is meaningful
• Enables principal cohort & learning community focused on
instruction & classroom improvements
Feeder System Characteristics
• Eight low-performing schools
• 63% of schools receive Title I funding
• 58% of feeder pattern’s students receiving
free/reduced lunch
• High school has failed to meet AYP for the last 5
years
• 210 teachers participating – 70% of all teachers
Measures
Freq.
Broader
Learning
Community
Assessment
National Board Take One!
Respondent Group
Yearly
Engagement Survey, Pre and Post Yearly

Principal
Certified
Staff
Classified
Staff






Parent
Student


Standardized Tests
• Alabama Reading and Math
Test/SAT-10 (Grades 3-8)
Yearly

• EXPLORE and PLAN (Grades 8-11)
Add’l Student Growth Measures
(e.g., attendance rates,
promotion/retention rates, graduation
/dropout rates)
Yearly

Take One!
One Entry of NBPTS’ Certification Process
Portfolio
Student
Work
Assessment Center
Exercise 1
Whole Group
Video
Entry 3
Exercise 5
Exercise 3
Exercise 4
Small Groups
Video
Entry 2
Documented
Accomplishments
Exercise 2
Exercise 6
NBPTS’ Certification Assessment
2-Factor Confirmatory Factor Analysis
Port. #1
Teaching
Skill
ε
Port. #2
ε
Port. #3
ε
ε
Port. #4
AC #1
ε
ε
AC #2
ε
AC #3
Content
Knowledge
AC #4
AC #5
AC #6
ε
ε
ε
Summary of Standardized Feedback – Year 1
Feedback/Comment area
Overall (N=165)
A. More evidence of knowledge of child development, knowledge of
students, and insight into learning
B. Evidence of knowledge of the subject as you appropriately design and
deliver your classroom lessons
C. Evidence that demonstrates your ability to employ effective instructional
strategies to facilitate individual participation
D. Evidence that you develop a relevant learning sequence
38.8%
E. Evidence of your ability to foster a purposeful learning environment in
which all students are encouraged to participate
F. Evidence of your ability to set worthwhile goals for learning.
50.9%
G. Evidence that you use a variety of appropriate resources to enhance
student learning.
H. Evidence that you employ appropriate assessments and provide
constructive feedback.
I. Evidence that you are able to describe your own practice and reflect on
instructional decisions
64.2%
47.3%
63.0%
38.8%
53.9%
66.7%
6.1%
MIDDLE
(N=29)
HS
(N=41)
58.3% 41.7% 30.8%
41.4%
46.3% 38.8%
B. Evidence of knowledge of the subject as
you appropriately design and deliver your
52.6% 42.9% 83.3% 41.7% 58.3% 53.8%
classroom lessons
51.7%
31.7% 47.3%
C. Evidence that demonstrates your ability
to employ effective instructional strategies
to facilitate individual participation
31.6% 66.7% 50.0% 50.0% 41.7% 84.6%
72.4%
80.5% 63.0%
D. Evidence that you develop a relevant
learning sequence
15.8% 33.3% 33.3% 50.0% 29.2% 38.5%
58.6%
41.5% 38.8%
E. Evidence of your ability to foster a
purposeful learning environment in which
all students are encouraged to participate
26.3% 57.1% 33.3% 75.0% 45.8% 30.8%
51.7%
63.4% 50.9%
F. Evidence of your ability to set worthwhile
52.6% 66.7% 33.3% 66.7% 54.2% 46.2%
goals for learning.
48.3%
53.7% 53.9%
G. evidence that you use a variety of
appropriate resources to enhance student
learning.
84.2% 57.1% 33.3% 50.0% 54.2% 69.2%
69.0%
68.3% 64.2%
H. evidence that you employ appropriate
assessments and provide constructive
feedback.
68.4% 52.4% 50.0% 58.3% 70.8% 69.2%
86.2%
61.0% 66.7%
A. More evidence of knowledge of child
development, knowledge of students, and
insight into learning
I. evidence that you are able to describe
your own practice and reflect on
instructional decisions
ELEM
(N=19)
ELEM
(N=21)
26.3% 33.3%
0%
14.3%
ELEM
(N=6)
0%
0%
ELEM
(N=12)
25.0%
ELEM
(N=24)
Overall
(N=165)
ELEM
(N=13)
Feedback/Comment area
8.3%
0%
0%
4.9%
6.1%
Learning Community Engagement and Leadership Survey
Formative and Summative Administration
Engagement
Perceptions About the
Learning Community
–
–
–
–
–
–
Principal
Students
Parents
Certified Staff
Classified Staff
Community Members
Leadership 360
Perceptions About the
Principal’s Behaviors
–
–
–
–
–
Principal
Supervisor
Peers
Certified Staff
Classified Staff
Leadership Behaviors
Cluster
Behaviors
Thinking
Gathering Data
Analyzing Information and Ideas
Thinking Flexibly
Reflective Practice
Developing
Valuing Others and Their Perspectives
Collaboration
Developing Human Capital
Inspiring
Cultivating Ownership and Support
Building Confidence
Communication
Achieving
Planning, Implementing and
Empowering
Goal Setting and Monitoring
Stakeholder Centered
Mapped to the Standards in Behavioral Language
Cross-Rater Items
Engagement
Survey Dimension
1. Level of Academic Challenge
2. Student-Staff Interaction
3. Active and Collaborative Learning
4. Active and Collaborative Teaching
5. Enhancing Education Experiences
6. Student Commitment Levels
7. The School Environment
8. Parent-Learning Community
Interaction
9. Work and Community Engagement
10. Organizational Citizenship
11. Work Withdrawal
12. Turnover Intentions
13. Broader Learning Community
Engagement
Respondent Group
Broader
Learning
Community

Principal
Certified
Staff
Classified
Staff
Parent
Student
x2

















































Questions to Respondent Group
“Teachers present materials at a level our students can
understand.”
“I present materials at a level our students can
understand.”
“Teachers present materials at a level my child can
understand.”
“My teachers present materials at a level I can
understand.”
“My teachers give lessons at a level I can
understand.”
Pre-post Reporting
Benefits of Student Measures
• Longitudinal results
• Guide student development
• Positions college as a viable opportunity
Alabama Reading and Math Test
The 4th & 5th grade teachers are new
Grade
4
Grade
4
Grade
5
Grade
5
Grade
5
Grade
5
Math 2011 (post)
614
619
633
638
660
630
Math 2010 (baseline)
587
624
589
622
627
613
+27
-5
+44
+16
+33
+17
Reading 2011 (post)
611
613
624
629
653
591
Reading 2010 (baseline)
593
596
619
600
641
570
+18
+17
+5
+29
+12
+21
Math Gain
Reading Gain
How Are We Doing?
Some Interim Assessments
What is formally test in core subjects only
Knowledge and learning that can be measured
All classroom learning
Interim Integrated Classroom Evaluation
data
Engagement/
efficacy
Knowing
Your
Students
background
attendance
Know Students and Subject Area
The Architecture of Accomplished Teaching
Areas to know about your students
• Where they are intellectually,
socially & emotionally
• Whether they have special
learning needs
• How they feel most comfortable
learning
• What they know & do not know
• What they can & cannot do
• What they are interested in
• How any issues or event taking
place in their lives may affect
learning
Resources
• Personal observation of students
• Interest inventories & surveys
• Student work products
• Report cards
• Records from previous schools
• Standardized tests scores
• IEPs
• Student extracurricular activities
• Results from diagnostics testing
• Previous teachers
• Guidance counselors
• Family members
• Students themselves
Step 1: Know Students and Subject Area
Interim Integrated Classroom Evaluation
Pre
Interim
Post
Student
Engagement
Classroom
Practice
Peer Video
Review
Deep
Questioning
Teacher
Monitoring
Evaluate Student Learning
The Architecture of Accomplished Teaching
Step 4: Evaluate
Student Learning
Following instruction,
evaluate student
learning to see if goals
were met.
Analyzing Your Video
• Extent of classroom involvement
• Extent of student engagement in lesson
• Evidence of student learning
• Student success in meeting learning
goals
• Types of questions you asked
• Opportunities for students to ask
questions
• Adjustment in lesson so learning goals
achieved by every student
Interim Integrated Classroom Evaluation
• Evaluate practice
and learning
against rubric
Review
Reflect
• Identify teacher
and student
• Strengths
• Weaknesses
• Content
• Pedagogy
• Culture
Modify
Reflect on Teaching Practice
The Architecture of Accomplished Teaching
Reflection
• Link reflection to knowledge of your
students
• Link reflection back to goals of lesson
• Cite specific strengths and weaknesses
• Describe appropriateness of your
instructional choices
• Provide clear, consistent & convincing
evidence & examples of student
learning
• Discuss how you continue with your
instruction
• Discuss any modifications to advance
student learning
Step 5: Reflect on Teaching
Practice
• What would I do
differently?
• What are my next steps?
Take One! Submission Rate Comparison
General
Population
Take One! Whole
School Transformation
50%
80%
Participation Increase
2010-11
2011-12
N
Total N
%
N
Total N
%
NBC
N/A
N/A
N/A
42
277
15%
Take One!
215
310
69%
219
277
79%
NBCP
N/A
N/A
N/A
2
8
25%
Principal Take One!
N/A
N/A
N/A
4
5
80%
TOTAL
215
310
69%
267
290
92%
Status
Preliminary reactions are positive
• Teachers - talking, planning, and communicating with each other
• Teachers - excited to have the opportunity to work with others in their
field from across the feeder pattern, an atypical experience
• Principals - connecting with teachers in new ways through Take One!
• Principals - communicating about student support
• Superintendent – encouraging his central office staff, principals,
teachers and other staff to build on the success
• Everyone -- Identifying strengths and weaknesses across the schools in
the feeder pattern
• Everyone - seeing the students as “their students,” a collective
responsibility
Where Are We Going?
• Refined year one and building year two
• Developing a structured process with a supporting suite
of tools that:
– Defines the elements for success
– Road-tested and readily available
– Is affordable to implement & sustain
– Facilitates continued school growth – multi-year
– Is research-based
• Getting the word out
– Department of Education
– Alabama publications
Evaluation and Modification of Program
Year One
• Teacher selects favorite
lesson to teach
Year Two
• Teacher evaluates data and
selects lesson to support
area of student need
• External experts support
teachers after school
• Identify and train schoolbased teacher leaders to
build capacity
• After school cohort
meetings
• In-school cohort meetings
Measurement is only part of the story
Moving from measuring teacher effectiveness
to…
Building teacher capacity
Move from 30% teachers based on student
achievement to 100% teachers based on
student growth
Provides the ‘how to’ for developing a
sustainable a teacher evaluation and
support system that delivers learning for
all students
For more information
on NBPTS, visit
nbpts.org