Transcript Improving Teacher Quality Grants, Cycle 1: External

Improving Teacher Quality
Grants, Cycle 5:
External Evaluation Report
December 8th, 2008
University of Missouri
Evaluation Team
Evaluation Team
Principal Investigators
Sandra Abell
Fran Arbaugh
Mark Ehlert
John Lannin
Rose Marra
Graduate Research
Assistants
Kristen Hutchins
Ya-Wen Cheng
Michele Lee
Dominike Merle
S. Rená Smith
Context of the Evaluation
• Improving Teacher Quality Grant program,
Cycle 5, 2007-2008
– Focus on high-need schools (Title II)
– 6 funded professional development projects
– Science and mathematics, grades 4-8
– Formative and summative evaluation
Funded Projects
Project Title
Higher Education Institution;
Principal Investigator
Grade Level;
Content Focus
Project
Year
Making Science Accessible to 4-8 Grade Students
though Inquiry and Literacy
Lincoln University; Dr. Saha
Grades 4-9; Science
1 of 1
Stimulating Inquiry in Math and Science:
Developing a Math/Science Community
Missouri Southern; Dr. Messick
Grades 4-8;
Mathematics and
Science
3 of 3
Science and Mathematics Achievement from Rural
Teachers
Missouri State; Dr. Plymate
Grades 4-8;
Mathematics and
Science
2 of 3
Physics for Elementary and Middle School
Teachers: Constructing an Understanding of
Physics
Rockhurst; Dr. Hegarty
Grades 4-8; Science
and Mathematics
2 of 3
Scaffolding Authentic Learning by Inquiry: A FieldBased Project for Middle School
University of Central Missouri; Dr.
Sarkar
Grades 6-8; Science
3 of 3
Science Education and Quantitative Literacy: An
Integrated, Inquiry-Based Approach
University of Missouri-Rolla; Dr.
Samaranayake
Grades 7-8;
Mathematics and
Science
3 of 3
Participant Summary
• 170 participants: 155 teachers, 6 pre-service teachers, 1
paraprofessional, and 8 administrators;
• More taught mathematics and/or science at the end of
Cycle 5 than at the beginning;
• Only 17.9% were new to the ITQG program;
• Taught in 64 different Missouri school districts, and 4
private schools;
• Directly impacted 13,282 students in the 2007-2008
school year.
Percentage of Participants from
High-Need Districts
39.0%
61.0%
% participants from high-need districts
% participants from non high-need districts
Model of PD Evaluation
ITQG Objectives
• Improve teacher pedagogical knowledge
and practices in inquiry-based instruction
• Enhance teacher use of assessment
• Increase teacher content knowledge
• Improve student achievement
• Impact pre-service teacher education in
higher education institutions
Teacher Knowledge and Practice
of Inquiry
From the Cycle 5 RFP:
Improve teachers’ pedagogical knowledge and
practices that utilize scientifically-based
research findings and best practices in inquirybased instruction.
Teacher Knowledge and Practice
of Inquiry
Teacher Knowledge and Practice
of Inquiry
Teacher Knowledge and Practice
of Inquiry
Models for Inquiry
• Learning Cycle (Lincoln)
• Curriculum Specific (Rockhurst)
• 3 Principles of Learning (MSSU)
Teacher Knowledge and Practice
of Inquiry
• What aspects of instructional practice were
emphasized in your project?
•What other aspects of instructional practice
should be considered?
Teacher Assessment Knowledge
From the Cycle 5 RFP:
Enhance teachers’ use of assessment to
monitor the effectiveness of their instruction
Teacher Assessment Knowledge
Teacher Assessment Knowledge
• What aspects assessment practices
were emphasized in your project?
•What other aspects of assessment should
be emphasized?
Teacher Content Knowledge
Content Knowledge Test Results
Survey Items
Level of Knowledge in Targeted Topics
Relevance of Growth in Content
Knowledge
Confidence in Content Knowledge
Results of Teacher Knowledge Tests
80
70
60
%Correct
50
40
71
30
20
45
54
50
45
74
73
38
36
31
10
0
Lincoln-Science
MSU-Science
Pre-test
Rockhurst-Science
Post-Test
Post-Test #2
MSU-Math
Teachers’ Ratings of Growth in Content Knowledge
9
7.8
8
7.7
7.5
7.2
Average Rating (0 to 10)
7
6.0
6
5.2
5
4
3
2
1
0
Remembered
End of Summer
Science Knowledge
Math Knowledge
End of Project
Ratings of Relevance in Gains in Content Knowledge
4.5
4.2
3.9
4
3.6
Avg Relevance Rating (0 to 5)
3.5
3.5
3
2.5
2
1.5
1
0.5
0
End of Summer
End of Project
Science
Math
Teacher Ratings of Confidence in Content Knowledge
3
2.7
2.6
2.5
Avg Confidence Rating (0 to 3)
2.3
2.1
2
1.5
1
0.5
0
End of Summer
End of Project
Science
Math
Student Achievement
 Teacher ratings of PD influence on student
learning
 Teacher developed and administered pre- and
post-tests
 MAP exam results
 Science Levels
 Math Levels
 Math Gains
<graphic>
<graphic>
<graphic>
<graphic>
<graphic>
Perceptions of How Change in Practice from PD Improved Student Learning
Using inquiry-based/problemcentered teaching
Managing inquiry-based/problemcentered classrooms
Participating in classroom activities
as your students would
Implementing activities in your
classroom
Developing materials for use with
your students
Increasing student motivation
Collaborating with other teachers
Improving my content knowledge
Creating lessons aligned with GLE’s
Assessing student learning
Using technology effectively to
enhance your teaching
Analyzing student performance data
0.0
0.5
1.0
1.5
Average Rating
2.0
2.5
3.0
Average Student Performance on Tested Science and Math Content
80
70
% Correct
60
50
40
76
30
70
68
65
52
49
20
39
70
64
47
38
29
26
10
28
0
MSU
Lincoln Science
MSSUScience
Rockhurst Science
Pretest
UMRScience
Posttest
MSSU-Math UMR-Math
Percent Proficient on MAP Science Exams
60.0
|-------- Grade 5 --------|
|-------- Grade 8 --------|
|-------- Grade 11 --------|
50.0
10.0
46.2
49.8
38.1
40.1
44.7
47.5
33.2
47.2
28.8
20.0
33.5
48.6
30.0
23.7
Percent Top 2
40.0
0.0
High-Need
Not High-Need
High-Need
PD Schools
Not High-Need
High-Need
Non-PD Schools
Not High-Need
Percent Proficient on MAP Math Exams
42.5
46.7
36.0
45.6
46.3
22.8
34.0
33.6
36.4
40.6
37.4
37.5
46.1
49.0
37.2
32.4
37.2
31.3
30.0
35.5
31.6
Percent Top 2
40.0
45.2
47.7
48.8
46.5
50.0
51.5
52.2
50.7
53.3
60.0
20.0
10.0
0.0
3
4
HN - PD Partic.
5
HN - Not PD
6
Grade
7
Not HN - PD Partic.
8
10
Not HN - Not PD
Changes in MAP Math Proficiency for Same Student Cohorts
10.0
8.4
8.0
8.0
6.0
4.2
4.0
3.1
%Top 2
2.0
1.1 1.2
2.5
2.3
1.8
1.3
0.0
-2.0
-4.0
-0.1
-0.7
-1.5
-2.1
-3.0 -2.8
-3.6
-3.1
-6.0
-6.9
-8.0
-9.0
-10.0
3 to 4
HN - PD Partic.
4 to 5
5 to 6
Grades Compared
HN - Not PD
6 to 7
Not HN - PD Partic.
7 to 8
Not HN - Not PD
Impact on Higher Education
• Inclusion of pre-service teachers in PD (e.g.,
Rockhurst)
• Development of new courses for pre-service
teachers (e.g., MSU)
• Changes to current curriculum and instruction
(e.g., MSSU)
• Plans to develop a university Science Education
Center (e.g., Lincoln)
• Renewed partnerships with K-12 (e.g.,
Rockhurst NSF grant proposal)
Cycle 7 Evaluation—Next Steps?
• What is missing in the evaluation?
• What else should we ask?
• What other suggestions for the
evaluation do you have?
Questions
Copies of the Cycle 5 Report and Executive
Summary available at:
www.pdeval.missouri.edu