Transcript Longitudinal Study to Measure Effects of MSP Professional

Symposium - Evaluating the Quality of Professional Development:
Implications for Districts and States
Measuring Differences in
the Quality of Professional
Development
Michael S. Garet
Kwang Suk Yoon
American Institutes for Research
Andrew C. Porter
Vanderbilt University
Focus

How to conceptualize and measure
the full set of professional
development experiences districts
offer and teachers participate in
Levels of analysis
District portfolio of Professional
Development (PD)
 Specific PD activities
 Specific time-periods during which
PD takes place (months, semesters)
 Teacher collection of PD experiences

Levels of analysis (continued)
District Portfolio
A1
tp1
tp2
Other PD
A2
tp1
Teacher 1 PD
tp2
A3
A4
tp1
tp1 tp2 tp3
Teacher 2 PD
Features of PD activities

Structural features
– Type
– Duration (contact hours and span)
– Collective participation

Core features
– Content focus
– Active learning
– Coherence
Design of MSP evaluation

4 MSP projects

Within each project, 2 groups of
teachers
– Treatment group: teachers targeted for MSP
participation
– Comparison group: matched teachers or
schools
Design of MSP evaluation (cont)

Data on professional development
collected through Professional
Development Activity Log (PDAL)
– teachers provide 1 log on each activity
each month
– data collected for 15 months (July 2003
- October 2004)
Professional Development Activity Log
(PDAL)



The PDAL is a web-based, self-administered,
longitudinal data collection tool for teachers
to record their professional development
experiences in detail with the assistance of a
series of structured prompts
Teachers log on to their password-protected
web account and fill out their PDAL at regular
intervals
Visit www.PDAL.net for more information
PDAL data structure
(teacher by activity by month)
Teacher
Mr. Anderson
Ms. Lopez
Mrs. Kelly
Mr. Lee
Mrs. Smith
No. of logs
Activity
A
B
C
A
D
E
E
F
G
Jul-03 Aug-03 Sep-03 Oct-03 Nov-03 Dec-03 Jan-04 Feb-04 No. of Logs
1
1
1
3
1
1
1
1
4
1
1
1
1
4
1
1
1
1
1
1
4
1
1
1
1
2
1
1
2
3
1
2
4
2
3
2
1
20
Contact Hours
(teacher by activity by month)
Teacher Activity
Mr. Lee E
F
total
Jul-03 Aug-03 Sep-03
10
5
5
0
0
5
10
5
10
Oct-03 Nov-03 Dec-03 Jan-04 Feb-04
0
0
0
nd
nd
5
0
0
nd
nd
5
0
0
nd
nd
# of
contact
hours
20
10
30
% of months with PD (math)
Percent of months
100%
80%
60%
Comparison
Treatment
40%
20%
0%
Site 1
Site 2
Site 3
Site 4
ct
Se
p
Ju
l
A
ug
N
ov
D
ec
Ja
n
Fe
b
M
ar
A
pr
M
ay
Ju
n
O
Se
p
Ju
l
A
ug
Hours per activity per month
Contact hours by calendar month
40
30
20
10
0
Contact hours per month (math)
Hours per month
25
20
15
Comparison
Treatment
10
5
0
Site 1
Site 2
Site 3
Site 4
Topic intensity (science)
Hours per topic per month
25
20
15
Comparison
Treatment
10
5
0
Site 1
Site 2
Site 3
Site 4
Next steps
1. Exploring teachers’ PD experiences
– Is there a relationship between PD
participation and change in instructional
practice and achievement? Which teachers
receive high-quality PD?
Next steps (cont.)
2. Using the PDAL to examine specific
PD activities and portfolios.
– Do MSP activities have the intended
features? Are some activities of particularly
high quality?
3. Examining the reliability and validity of
the PDAL
Contact Information
Michael S. Garet
[email protected]
Kwang Suk Yoon
[email protected]
Andrew C. Porter
[email protected]
Visit us
www.PDAL.net