Transcript Slide 1
USING RESEARCH TO IMPROVE
TEACHING AND LEARNING
Israel Ministry of Education, Culture and Sports
Department of the Centers for
Professional Development of Teachers
November 28, 2005
USING RESEARCH TO IMPROVE
TEACHING AND LEARNING
Edward Crowe
Teachers for a New Era
Carnegie Corporation of New York
Academy for Educational Development
National Commission on Teaching and America’s Future
Hunter Foundation of Scotland
Scottish National Executive
High Quality Teacher Preparation
Why does it matter?
Essential precondition for these outcomes:
Student readiness for college success
College admissions
Student retention in colleges and universities
Graduation rate outcomes
Compelling evidence that teachers matter…
How teachers make a difference
Value-added assessment
Explaining differences in student learning
Differences across districts—5%
Across schools within districts—30%
Teachers within schools within districts—65%
www.sas.com/govedu/edu/research.html
How have we responded?
Revised accreditation standards and processes
Licensure and program approval policies
Teacher testing
“Highly qualified” teachers
Alternate routes into the classroom
Evidence that what we’re doing is working?
Beginning Teacher Attrition in the United States:
A Serious Problem
50%
40%
30%
20%
10%
14%
24%
33%
40%
46%
1st Year
2nd Year
3rd Year
4th Year
5th Year
0%
International Study:
Why are teachers a focus?
Research shows that teaching quality is a key influence
on student learning
Teachers are the largest item in school budgets:
-- The teaching workforce is large (2.5% of total labour force)
-- Teachers’ compensation: 63% of spending on schools
Teachers’ roles are changing
Efforts to improve schools will fail if there are serious
shortfalls in teacher supply and quality
8
Teaching Quality
Why does it matter?
Efforts to improve schools will fail if there are serious shortfalls
in teacher supply and quality (OECD)
Demonstrable links between teacher experience and student
learning (Sanders, New Teacher Center)
Impossible to achieve key student learning outcomes without
effective teachers:
Student readiness for college success
Adequate and diverse supply of skilled talent in key
employment fields (National Research Council, National
Science Foundation, Thomas Friedman)
Evidence that what we’re doing is working?
Teacher Turnover in the United States
A Revolving Door
2,376,677
69%
Not In Transition
539,778
16%
Leaving
534,861
15%
Incoming
Total Teaching Force, 1999-2000: 3,451,316
DEVELOPING A RESEARCH AGENDA
Professionally
Rewarding Career
Paths
Hiring Incentives
Professional Development
Performance Pay
Board Certification
Schools
Professional
Teacher
Preparation
Professional
Teaching
Conditions
Professional
Teaching
Conditions
Professionally
Rewarding
Career Paths
Schools
Small Schools
Leadership
and
Mentoring
Professional
Teacher Preparation
Professionally
Rewarding
Career Paths
Schools
Professional
Teaching
Conditions
Careful Selection, Academic Preparation,
Clinical Practice & Induction Residencies,
Technology, and Assessment of Teachers & Programs
Accreditation, Licensing & Certification
Professional
Teacher Preparation
Professionally
Rewarding
Career
Advancement
Conditions
for
Professional
Teaching
Practice
High-Performing
Schools
Professional
Teacher
Preparation
Support from
Replace Solo
Induction to
Teaching with
Accomplished
Learning
Teaching, with
Communities
Professional
st
Compensation 21 Century
Learning
Preparation, Induction,
Assessment
NO DREAM DENIED:
A Pledge to America’s Children
Organize schools for success
Invest in quality teacher preparation
Create professionally rewarding careers for teachers
www.nctaf.org
Some Key Research Questions
Teacher Preparation and Professional Development
Students Preparing to be Teachers
What students bring to the program (student characteristics
predictive of success)
o Demographics
o Attitudes and values
o Previous education and work experience
Preparing New Teachers
(also applies to professional development programs)
The “treatment” (the program) and its effects on
o Candidate knowledge
o Teaching skills and teaching performance
Program Outcomes
Research on important outcomes
o Teaching performance
o Persistence in the profession
o P-12 student achievement
Studying Teacher Education:
Report of the AERA Panel
on Research and Teacher Education
www.aera.net/publications/?id=793
Improving the quality of education research
Applying Multiple Social Science Research Methods
to Educational Problems
American Psychological Association
National Research Council
American Educational Research Association
National Science Foundation
www7.nationalacademies.org/cfe/Multiple_Methods_Workshop_Agenda.html
Learning from Attempts to Improve Schooling:
The Contribution of Methodological Diversity
Stephen W. Raudenbush
University of Chicago
Focus on evaluating the impact of interventions designed to improve
teaching and learning
Randomized experiments are the gold standard for assessing these
effects
Learning from Attempts to Improve Schooling:
The Contribution of Methodological Diversity
Success depends on a well-integrated, methodologically diverse
research effort:
Precisely define viable instructional aims and outcome variables
Identify, refine, and test promising new interventions
Clarify the instructional needs of children in various settings
Learn whether interventions work, how they work and for whom
CONCLUSION: Diverse methods can be combined, and a healthy
scientific community should collaborate to achieve these aims
Learning from Attempts to Improve Schooling:
The Contribution of Methodological Diversity
Recommendations
“Research using a variety of methods is essential:
o To define the student outcomes we seek to change and to build and
validate assessments of those outcomes;
o To support novel thinking about how best to intervene, to support
preliminary studies of those interventions; and
o To enable educators to test the feasibility of implementing those
interventions in ordinary school settings”
--Raudenbush, 2004
National and International
Research Efforts
On Teaching and Learning
Teachers for a New Era
Carnegie Corporation of New York
Annenberg and Ford Foundations
Academy for Educational Development
RAND
www.teachersforanewera.org
Teachers for a New Era
3 design principles
Decisions based on evidence
Engagement with the arts and sciences
Teaching as academically taught clinical profession
Teachers for a New Era
Key design principle: Decisions based on evidence
Assessments linking program preparation to teaching
performance
Assessments of teacher impact on student learning
Teachers for a New Era
11 colleges and universities
4 pupil assessment pilot studies in 2004-05
7 pupil assessment pilot studies in 2005-06
Continuation of first 4 studies in 2005-06
Scottish Teachers for a New Era
www.abdn.ac.uk/cass/bulletin/school.shtml#b
Improving Teacher Supply and Effectiveness
Phillip McKenzie and Paulo Santiago
Directorate for Education
Meeting of OECD Education Ministers
Raisin
34
www.www.oecd.org/edu/teacherpolicy g the Quality of
Learning for All
Dublin, 18-19 March 2004
Campbell Collaboration
• The Campbell Collaboration is an international effort whose
mission is to prepare, maintain, and make accessible
systematic reviews of studies on the effects of interventions
• Funders: Smith Richardson, Robert Wood Johnson,
Rockefeller and Knight Foundations; governments of
Denmark, Norway, Finland
www.campbellcollaboration.org
Campbell Collaboration
Objectives
• Use standards for quality of evidence that are transparent and
can be critiqued
• Focus on randomized field trials first and on high quality
nonrandomized field trials second
Campbell Collaboration
Systematic reviews of research evidence
Randomized trials completed and reviewed:
Teacher professional development programs
Evaluation of Reading First Program
www.campbellcollaboration.org/Bellagio/papers
Ohio Teacher Quality Partnership
Project Focus
• Teacher candidates, impact of program design, program quality outcomes,
P-12 student learning
• Funding support from Ohio state education and higher education agencies,
Knowledge Works Foundation, Carnegie Corporation, Gund Foundation
(Ohio), Proctor & Gamble, Joyce Foundation, U.S. Congress
www.teacherqualitypartnership.org
Teacher Preparation Accountability System
Value-Added Teacher Preparation Program Assessment Model
Louisiana Board of Regents
Board of Elementary and Secondary Education
http://asa.regents.state.la.us/TE/value_added_model
Teacher Preparation:
Does the Pathway Make a Difference?
Project Focus
Which teachers are most effective in improving student
outcomes?
What characterizes their preparation, pathways into teaching
and qualifications to teach?
What is the cost effectiveness of various pathways into
teaching?
www.teacherpolicyresearch.org
Assessments of Teaching Performance
CLASS--Classroom Assessment Scoring System [Robert
Pianta, University of Virginia]
http://convention.allacademic.com/aera2004/
view_presenter.html?part_id1=905783
SOLO--Structure of Observed Learning Outcomes
www.learningandteaching.info/learning/solo.htm
Reliability and Validity of Instruments
Connecting research and practice
Developing the missing links
Research on Teaching and Learning
Value-added studies
Some current efforts:
Boston Public Schools
Dallas Independent School District
Denver Public Schools
Milwaukee Public Schools
RAND study funded by Carnegie Corporation
New York State:
All 4th grade teachers, 1994-2002
Project Focus:
Impact of assessment and accountability
Boyd, Lankford, Loeb, and Wyckoff (2005)
www.teacherpolicyresearch.org
Impact of assessment-based accountability
on teacher turnover
Since NY State use of fourth grade testing, the turnover rate of fourth grade
teachers has decreased relative to teachers in other grades.
Decrease is consistent across urban, suburban and rural areas and across
schools with differing levels of student performance.
Little evidence that high-ability teachers are more likely to leave fourth
grade
Drop in turnover less for more experienced teachers than for recent hires,
especially in suburban districts.
New 4th grade teachers in lowest performing schools more likely to have
attended highly competitive undergraduate institutions.
Public Education Fund-Chattanooga
Highly effective teaching
High school reform
Teacher development--reading and urban education
Instructional leadership development
www.pefchattanooga.org/www/docs/3/new_society/
www.pefchattanooga.org/www/docs/5/effective_teaching/
Evidence Based Education-UK
Create evidence (teaching and learning)
Disseminate evidence (to teachers, schools, policy makers)
Promote a culture of evidence
Work against ‘non-evidence-based’ policy and practice
University of Durham
Curriculum, Evaluation and Management Centre
www.cemcentre.org/ebeuk/default.asp
Battelle for Kids
Improve teaching and learning
Project SOAR
Analyze, share and use data
78 Ohio school districts
718 schools
200,000 students (20% of all grades 3-8 in Ohio)
SAS Institute
www.battelleforkids.com/b4k/rt/about/our_work
SAS Institute
Schooling effectiveness
Schooling effectiveness “technology”
Education value-added assessment system (EVAAS)
Bill Sanders, June Rivers
Papers, resources, other links
www.sas.com/govedu/edu/research.html
The biggest challenges in research
on teacher education and professional development?
Keeping the focus on outcomes
“In the end the final test of a medical school
is its outcome in the matter of clinicians.”
Abraham Flexner (1910)
Designing and using metrics for success
Set goals
Establish indicators
Measure progress
Students
Teachers
Preparation programs
Professional development initiatives
Schools
Policy
After the Test:
Closing the Achievement Gaps With Data
Bay Area School Reform Collaborative
Compared improving and failing schools on achievement gap
School culture and leaders supportive of intensive focus on assessment
Definition of equity includes student achievement
Teachers learn about and use assessment tools regularly
Successful schools function as learning communities
www.ncrel.org/gap/studies/basrc.htm
Value-added assessment studies
Teacher Effects
Estimate effects of teachers on student learning gains or
achievement growth
Use of complex models to account for nonrandom pupil
assignments, variability in non-school inputs, background
variables, student attributes
www.rand.org/pubs/monographs/2004/RAND_MG158.sum.pdf
Overview of value-added assessment
Student learning gains the “gold standard” for evaluating
teaching quality and preparation program effectiveness
But…. many challenges as research projects move forward with
value-added assessment
Some value-added challenges
Access to relevant data
Complexity of methods
Research capacity of teacher education programs
Commitment to assessment and using evidence
P-12 test quality
Opening the VAM “black box”
Interpreting the results
The Flexner Report
“Society reaps at this moment but a small fraction of the
advantage which current knowledge has the power to confer.”
Abraham Flexner (1910)
[Bulletin No. 4, Carnegie Foundation for the Advancement of Teaching]
www.carnegiefoundation.org
Key Flexner Recommendations
Medical education in the U.S. and Canada
Raise entry standards for students
Close 80% of medical schools
Strengthen state oversight
Faculty should engage in research
Practitioners should be scientists
Full-time clinical faculty
Medical school-hospital clinical ties
The future of university teacher preparation?
USING RESEARCH TO IMPROVE
TEACHING AND LEARNING
Edward Crowe
Teachers for a New Era
Carnegie Corporation of New York
Academy for Educational Development
National Commission on Teaching and America’s Future
Hunter Foundation of Scotland
Scottish National Executive