Transcript National Evaluation of the Eisenhower Professional

What Can Districts and Schools Do
to Make Professional Development Work?
Andy Porter
Vanderbilt University
June, 2004
Study Design
• District and IHE Interviews
– 363 district coordinators (88%); 92 IHE coordinators (87%)
• Teacher Activity Survey
– teachers’ professional development experiences
– 1025 teachers (72%), representing 657 activities
in 358 districts and IHEs
• Case Studies
– 10 in-depth in 5 states; 6 exploratory
• Longitudinal Study of Teacher Change
– teacher classroom practice
– 30 schools, 10 districts, 5 states
2
Study Timeline
1997
Winter
Spring
1998
Summer
Fall
District and
IHE Interviews
Case Studies
Winter
Spring
Reports
Summer
Fall
Winter
Spring
2000
Summer
Fall
Winter
Interviews
Exploratory Cases
In-depth Cases
Teacher Activity
Survey
Longitudinal
Study of
Teacher Change
1999
In-depth Cases
Survey 1
Wave 1
1st Report
Exploratory Cases
Wave 2
Survey 2
Wave3
2nd Report
Mostly National
Description
3rd Report
Mostly
Impact on
Teachers
3
This Presentation
• Characteristics of Effective Professional Development
(PD)
• Effects of Professional Development on Changes in
Teaching Practice
• District Management and Implementation Strategies
Related to Effective Professional Development
4
Characteristics of Effective PD
• Identify key features of professional development
• examine the relationship between features of professional
development and teacher outcomes
5
Teachers’ PD Experiences
• Enhanced knowledge and skills, and change in practice are the
expected outcomes of teachers’ PD experiences
• What features might be related to these outcomes?
– Sponsorship
• District vs. IHE
– Structural Features
• Type: traditional vs. reform
• Duration: amount (hours) and time span
• Collective participation
– Core Features
• Focus on content knowledge
• Active learning: discussing classroom implementation, being
observed, sharing knowledge, evaluating student work
• Coherence: continuity of learning, alignment, building professional
community
6
Teachers are More Likely to Participate in Traditional than
Reform Types of PD
50
45
45
40
30
30
District
24
25
20
Reform
Traditional
35
IHE/NPO
20
15
15
11
9 8
10
3
3
5
0
0
In-district
workshop
or institute
College
course
Out-of-district
workshop
or institute
Out-of-district
conference
6 7
1
Study
group
5
2 2
Teacher
network
Mentoring
2
Committee
or task force
1
3
Internship
1 1
Resource
center
Other
7
Compared to Districts, IHE Activities are Longer
100
90
Hours
80
70
60
62
59
50
40
37
30
20
23
10
0
Traditional
Reform
Districts
Traditional
Reform
IHE/NPO
8
Emphasize Content Knowledge More,
Percent of teachers reporting major
emphasis on content knowledge
100
68
65
Reform
50
49
50
Traditional
0
IHE/NPO
District
Activity Type
9
Offer More Opportunities for Active Learning
Percent of Teachers
100
54
50
53
55
45
43
41
33
35
31
25
16
9
15
10 13
29
District
IHE/NPO
11
5
0
Types of Opportunities for Active Learning
10
and Are More Coherent with Teachers’ Goals
and Other Activities
100
88
90
81
80
69
Percent of Teachers
70
60
54
District
50
IHE/NPO
40
36
33
30
20
10
0
Consistent with goals
Builds upon earlier activities
Followed-up with additional activities
Continuity of learning
11
But District Activities Are More Likely
To Be School-Based
50
Percent of Teachers
40
30
District
IHE/NPO
20
18
13
11
11
10
0
All teachers in department or grade level
All teachers in school
School-based participation
12
PD and Teacher Outcomes
Sponsorship
Structure
Quality
Outcomes
Time Span
.27
Sponsor
(IHE vs District)
Focus on
Content
Knowledge
.08
.10
.27
.33
.08
.30
.30
.21
.06
Active
Learning
Contact Hours
-.11
Enhanced
Knowledge &
Skill
.14
.05
.10
.44
.09
Type
(Traditional vs
Reform)
Controls
Schools % Poverty
.-13
School % Minority
Teacher’s Gender
Subject (Math & Science)
Grade Level (El, Middle, High)
In-field Certification
Teaching Experience
.21
.08
.42
.25
.13
.10
Collective
Participation
.08
Coherence
13
Change
in Teaching
Practice
Summary from National Data: 6 Features
of Quality Make PD More Effective
•
Six Features of the Quality of Professional Development
Predict Self-reported Change in Teachers Knowledge, Skills,
and Teaching Practice
Structural Features
Core Features
reform type (vs. tradition)
focus on content knowledge
duration
active learning
collective participation
coherence
14
Structural Features
Teacher Change
Core Features
•
Teacher knowledge and skills are enhanced by
professional development that emphasizes content,
provides opportunities for active learning, and is coherent
with teaching goals, standards and assessments, and
professional community.
•
Longer, reform type, and school-based professional
development activities are associated with content
emphasis, active learning, and coherence.
15
Longitudinal Study of
Teacher Change
Do teachers’ experiences in professional
development activities contribute to changes in
teaching practice?
16
Longitudinal Study of
Teacher Change

30 schools, 10 districts, in 5 states

1 elementary, 1 middle, and 1 high school in each district

Schools were selected based on:


participation in Eisenhower-assisted activities over the 1997-98 year

diverse approaches to professional development
All mathematics and science teachers surveyed in each
school
17
Sample for the Longitudinal Study of
Teacher Change
 17 of the sample schools, or 57%, are high-poverty,
compared to the national average of 25%
 287 teachers responded to all three waves of the survey
 Among responding teachers:
• 74% are female and 18% are minority
• 12% of mathematics and 18% of science teachers
are novices
18
Overview of Longitudinal Teacher
Survey
The longitudinal survey provides...
 three years of data on teaching practices (1996-97, 1997-98, and 1998-99)
• content covered
• specific teaching strategies for higher-order learning
 two years of data on participation in professional development activities
(1997-98 and 1998-99)
• specific teaching strategies for higher-order learning
• six quality features
19
Effects of Participation in PD on
Teaching Practices: Approach
 We measured:
baseline teaching practice in 1996-97
characteristics of a professional development activity
teachers participated in during 1997-98
teaching practice again in 1998-99
 We looked at effects on:
specific teaching practices
20
Effects of Participation in PD Focused on
Specific Teaching Strategies for Higherorder Learning
 We asked teachers about:
 Frequency of use of specific teaching strategies in
mathematics and science instruction
 Whether the professional development activity focused on
using the strategies
 We examined the effects of professional development in
three areas related to higher-order learning:
Use of technology
Instructional methods
Student assessment
21
Technology Use for
Higher-order Learning
 Calculators or computers to develop models or
simulations
 Computers to write reports
 Calculators or computers for data collection and
analysis
 Computers to access the Internet
22
Instructional Methods for
Higher-order Learning
 Work on independent, long-term (at least one week) projects
 Work on problems for which there is no immediately
obvious method or solution
 Develop technical or mathematical writing skills, including
using equations, graphs, tables, and text together
 Work on interdisciplinary lessons (e.g., writing journals in
class)
 Debate ideas or otherwise explain their reasoning
23
Student Assessment
for Higher-order Learning
 Essay tests
 Performance tasks or events
 Systematic observation of students
 Math/science reports
 Math/science projects
 Portfolios
24
Independent Variables
 For each of the teaching strategies, we modeled the effects
of three variables describing the professional development
teachers received:
Whether the professional development focused on the
strategy (relative focus);
Whether the professional development activity also
focused on closely related topics (mean focus); and
The quality features of the professional development
experience.
25
Effects on the Use of Calculators and Computers to Develop
Models of Professional Development Focused on Specific
Technology-Use Strategies, and Active Learning
1
Teacher's use of calculators or computers to develop models
0 = almost never
1 = some lessons
2 = most lessons
3 = every lesson
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
No focus on strategy, no active learning
Focus on set of technology-use strategies
no active learning
Focus on set of technology-use
strategies, high active learning
Characteristics of professional development activity
26
Effects on the Use of Calculators and Computers to Develop
Models of PD Focused on Specific Technology-Use Strategies
and Collective Participation
Teacher's use of calculators or computers to develop models
0 = almost never
1 = some lessons
2 = most lessons
3 = every lesson
1.5
1.4
1.3
1.2
1.1
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
No focus on strategy, no collective
participation
Focus on set of technology-use
strategies, no collective participation
Focus on set of technology-use
strategies, collective participation
Characteristics of professional development activity
27
Effects of Professional Development
on Teaching Practice
Well Designed Professional Development is Effective
 Professional development activities that focus on a specific,
higher-order teaching strategy increase teachers’ use of this
strategy in the classroom.
 Features of high quality—reform type, collective
participation, active learning, and coherence—increase the
impact of professional development activities that focus on
specific higher-order teaching strategies.
28
District Management and
Implementation of PD
• identify key features of district management and
implementation of professional development
• examine the relationship between district management and
implementation and the quality of professional
development
29
What Shapes District-sponsored
Professional Development?
• Building a vision for professional development:
–
Alignment with standards and assessments
–
Coordination with other programs: working together
and co-funding
• Implementation:
–
Continuous improvement: indicators, needs
assessments, evaluation, and guidance to schools and
providers
–
Teacher participation in planning
30
Effects of Management and Implementation on PD Quality
Building a Vision
Implementation
Portfolio Features
.15
.12
Alignment
.40
Participation in
“Reform”
Types Activities
Average Duration
.16
Continuous
Improvement
.14
.16
Coordination
(Co-funding)
Opportunities for
Collective
Participation
.20
.15
.20
Teacher
Participation
in Planning
Opportunities for
Active Learning
.17
Controls
District Poverty
District Size
Cluster Status
Consortium Status
.16
.19
Targeting
31
Summary from National District Data:
Building a Vision and Implementing It
• Alignment with standards and assessments and co-funding
with other programs are associated with reform types of
professional development, portfolios of higher quality
activities, and greater targeting.
• Continuous improvement is important for translating a
vision into practice, but districts may have insufficient
capacity.
• Teacher planning is linked to more opportunities for active
learning and more targeting of teachers of special needs
students.
32
Implications from our National Data
for Improving PD
• Encourage content focus, duration, opportunities for active
learning, coherence, and school-based professional
development.
• Continue to emphasize alignment with standards and
assessments, and co-funding with activities funded by
other programs.
• Build district capacity to foster continuous improvement.
33
Summary
 Large-scale change in teaching practice would require
districts and schools to specify the areas of teaching in
which change is desired.
 Then, districts and schools would need to plan and provide
professional development activities that focus on these
areas, that are aligned with other reforms, and that have the
6 characteristics of effective professional development:
longer, reform type, school-based activities that focus on
content, provide opportunities for active learning and are
coherent with teachers’ goals, standards and assessments
and professional community.
34
35
Conclusions
 We know what makes professional development work, and
it is expensive, so it can’t be used for all teachers.
 But, superintendents and principals want to reach all of
their teachers, and this creates a dilemma.
 There are two options:
(1) provide more money for professional development or
(2) develop a targeting strategy.
36
Citations
Porter, A.C., Garet, M.S., Desimone, L.M., & Birman, B.F. (2003, Spring). Providing effective professional development:
Lessons from the Eisenhower Program. Science Educator 12(1), 23-40.
Garet, M.S., Porter, A.C., Desimone, L., Birman, B.F., & Yoon, K.S. (2001, Winter). What makes professional development
effective? Results from a national sample of teachers. American Educational Research Journal 38(4), 915-945.
Desimone, L., Porter, A.C., Garet, M., Suk Yoon, K., & Birman, B. (2002, Summer). Effects of professional development
on teachers’ instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis
24(2), 81-112.
Desimone, L., Porter, A.C., Birman, B.F., Garet, M.S., & Suk Yoon, K. (2002, October). How do district management and
implementation strategies relate to the quality of the professional development that districts provide to teachers?
Teachers College Record 104(7), 1265-1312.
Desimone, L., Garet, M., Birman, B., Porter, A., & Suk Yoon, K. (in press). Improving teachers’ in-service professional
development in mathematics and science: The role of postsecondary education. Education Policy.
Birman, B., Desimone, L., Porter, A.C., & Garet, M. (2002). Designing professional development that works. Educational
Leadership 57(8), 28-33.
Birman, B.F., & Porter, A.C. (2002). Evaluating the effectiveness of education funding streams. Peabody Journal of
Education 77(4), 59-85.
37