Studying Teacher Professional Development: Challenges and Possible Solutions

Nicole Kersting Research Scientist

LessonLab Research Institute, Santa Monica, CA

1

Overview of Presentation

• PD Program under study • Challenges to implementation & possible solutions • Challenges to evaluation & possible solutions 2

Study Background and Goals

• Builds on key findings of TIMSS 1999 Video Study.

• Aims at helping 6 th -grade mathematics teachers to maintain the cognitive level of rich mathematics problems.

• Key topic areas: fractions, ratio/proportion, and variables/expressions 3

PD Program

Can we get teachers to effectively engage students in thinking about math concepts?

• Three video-based modules, each focusing on: • Mathematics content knowledge • Analysis of videotaped lessons (strategies for maintaining problems’ richness and student lens) • Teaching of videotaped lesson • Sharing meetings • Total of 45 face-to-face hours – 7 pull-out days (42 hours) distributed during the year – 3 sharing meetings (3 hours) 4

Research Questions

• Does the PD program impact

teacher knowledge

?

• Does the PD program impact

teaching practices

?

• Does the PD program impact

student learning

?

5

Sample

• Predominantly low-income district in Los Angeles • 5 Title-1 middle schools • All 6 • All 6 th th 2004) -grade teachers (=71); PD program made mandatory by the district -grade students (approx 4400 in 137 classes) • Student population: 73.6% Latino and 25.9% African American; 40% are English learners • 6.2% of students are mathematics proficient (as measured by state tests in 6

Design

Year 1 Random assignment within schools and tracks: Treatment and no treatment group Year 2 All teachers participated in PD program (1 year vs. 2 years of PD) 7

Key Measures - Teachers

• Content and pedagogical content knowledge – Multiple-choice survey: Items from Study of Instructional Improvement (Ball & Hill, University of Michigan); California Professional Development Institutes (MPDI - UC office of the president); and developed in-house – Administered at 3 points in time; 2 equivalent forms • Video Analysis Assessment Administered prior and at the end of Year 2 • Compliance Ratings 8

Key Measures – Teaching Practices

• Videotapes of lessons including a rich problem • Fieldnotes from classroom observations 9

Key Measures - Students

• State standardized tests (most distal) • Subset of district quarterly assessment items targeting key content areas • In-house student performance assessment items (most proximal) 10

Key Features of PD Program

• District mandated teacher participation • Multi-site randomized field trial • Title 1 schools • Multi-track schools and calendars 11

Challenges to Faithful Program Implementation Possible Solutions

• Full participation of all teachers • Opposing instructional goals  Involve every level at the school and district • Complications of multiple track calendars  Constant contact with the person at the school “Try to get on their calendar and try to stay on it”.

12

Challenges to Faithful Program Implementation Possible Solutions

• Role of math coaches  Don’t make assumptions about district and school personnel position responsibilities.

• Changes in school and district personnel  Don’t presume personnel stability.

13

Challenges to Experimental Design Possible Solutions

• Group contamination: students are moved across classrooms  obtain class rosters to assess percent of students moved.

• Teacher attrition (especially for programs with more than one year implementation).

 be conservative to ensure sufficiently large sample size for inferential statistics 14

Challenges to Experimental Design Possible Solutions

• Compliance: 6/28 teachers in Year 1 and 5/38 teachers in Year 2 were rated as fully compliant  Create compliance scale to be able to evaluate differential program effects for compliant and non-compliant teachers.

15

Challenges to Experimental Design Possible Solutions

• Threats to the validity of measures – Low motivation (PCK survey is time consuming) – Resistance to being assessed (exposes teachers’ lack of knowledge)  embedded measures of teacher knowledge • Lack of control over collecting student measures  Use own resources to collect data 16

Conclusions

• Conducting experimental research in schools (particularly in disadvantaged schools) is time consuming and difficult • Partnerships between researchers and schools are extremely valuable • Disadvantaged schools deserve our attention 17