Transcript Uncovering indicators of effective school management
Uncovering indicators of effective school management in South Africa using the National School Effectiveness Study
Stephen Taylor Department of Economics, Stellenbosch University PSPPD Project – April 2011 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
2
Motivation (the problem) • • • •
Low quality education a poverty trap to many children in historically disadvantaged schools Question: Poverty itself or the characteristics of schools in poor communities?
• SACMEQ II and III: Poor South African children performing worse than equally poor children in other African countries This despite substantial resource shifts to correct for apartheid inequalities • Historically disadvantaged schools have been largely unresponsive to additional resources • Consequence: Perpetuation of a “2 systems” system How does the literature explain this?
Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Motivation (the literature)
3 • • • • Resources do not necessarily make a difference: • the ability of schools to convert resources into outcomes is the crucial factor (Van der Berg, 2008) Socio-economic status (SES) has a dominant impact on the distribution of achievement Studies based on large sample surveys have typically struggled to identify specific aspects of effective management and teaching practice that explain performance.
• Crouch and Mabogoane (1998): 50% of variance explained by “management efficiency” • Van der Berg and Burger (2002): 2/3 variance explained by SES, racial composition & school resources; remainder probably due to unobserved “management efficiency”.
Largely due to data limitations most large surveys Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
4
Data • • • • •
National School Effectiveness Study (NSES) • JET Education Services & RNE Literacy and numeracy testing: • • • Grade 3 (2007) Grade 4 (2008) Grade 5 (2009) same individuals Principal questionnaires (2007, 2008, 2009) Teacher instruments (2008, 2009) • • Teacher comprehension and maths test Extensive review of learner workbooks Greater potential to uncover indicators of effective management an teaching Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results – overall scores
2007 (grade 3) 2008 (grade 4) 2009 (grade 5) Gain 2007 - 2008 Gain 2008 - 2009 2-year gain Literacy
20.15 29.59 37.73 9.43 8.14 17.57
Numeracy
29.38 35.50 47.04 6.12 11.54 17.66 5 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results: Numeracy scores by province
6 Numeracy 2007 Numeracy 2009 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results: Literacy achievement by SES
7 0 20 40 60 Literacy score 2009 (grade 5) Quintile 1 Quintile 3 Quintile 5 Quintile 2 Quintile 4 80 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union 100
Results: Literacy achievement by SES
8 0 1 2 SES (min = 0, std dev = 1) Literacy 2007 (grade 3) Literacy 2008 (grade 4) Literacy 2009 (grade 5) 3 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union 4
Results: Numeracy achievement by ex-department
9 0 20 40 60 Numeracy score (%) Numeracy grade 3 (DET) Numeracy grade 4 (DET) Numeracy grade 5 (DET) 80 100 Numeracy grade 3 (HOA) Numeracy grade 4 (HOA) Numeracy grade 5 (HOA) Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results: Numeracy achievement of African language students by ex-department
0 20 40 60 Numeracy score 2008 Ex-DET/Homelands schools 80 Historically white schools 100 10 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results: Indicators of effective management and teaching
Percentage of students in schools where more than 25 maths topics were covered
Ex-department
DET (B) HOR (C) HOD (I) HOA (W) Total
Percentage > 25 topics
26% 25% 38% 75% 29%
Number of students
6306 849 86 591 7832 11 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results: Indicators of effective management and teaching
Mean number of literacy exercises found in the “best” learner’s book
ex-department
DET (B) HOR (C) HOD (I) HOA (W) Total
Mean number of exercises
33.43 62.40 72.44 75.21 39.58
Number of students
6478 837 102 580 7997 12 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results: Extended writing
100 90 50 40 30 20 80 70 60 85 70 88 19 23 13 10 0 No exercises with paragraphs 1 or 2 exercises with paragraphs 3 to 9 exercises with paragraphs More than 10 exercises with paragraphs Unspecified Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results: Maths teacher knowledge
10 days 75 hours can be written as .... days .... hours
Teacher score
0 1 2 3 4 5 Total
Number of students
210 2130 2774 2168 1408 1209 9899
%
2.12 21.52 28.02 21.9 14.22 12.21 100
Cumulative %
2.12 23.64 51.66 73.56 87.79 100 100
Mean Numeracy 2008
37.27 33.04 33.50 34.14 34.77 46.92 35.44 14 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results: Multivariate analysis • • • • •
Are teachers with better subject knowledge located in more affluent schools?
And is it this affluence driving the association of student achievement with teacher knowledge?
The need for multivariate analysis to disentangle this.
After accounting for the influence of SES, what school and teacher characteristics are associated with student achievement?
What distinguishes better and worse-performing schools within poor communities?
15 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results: Multivariate analysis •
4 multivariate regression models estimated in the education production function tradition: • • • OLS regression predicting Literacy achievement in grade 4 OLS regression predicting Numeracy achievement in grade 4 2 more sophisticated techniques to model gain scores 16 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
17
Results: Multivariate analysis
Explanatory variables Student characteristics
Student SES Male Young Old Household size: large Read 1 to 3 times a week Read more than 3 times Books at home: 1 to 10 Books at home > 10 Home language English Speak English 1-3 times Speak English 4+ English on TV 1-3 times English on TV 4+
School characteristics
Mean School SES Mean School SES squared Pupil-teacher ratio Teacher absenteeism zero LTSM Inventory good Problems with students index Curriculum planned using year schedule
Teacher characteristics
Full year learning programme Constant R-squared statistic N 0.39* -2.48*** -0.40 -2.84*** -1.89*** 1.37** 2.39*** 0.60 1.17* 8.42*** 1.75*** 1.86** 0.85* 3.35*** -9.13*** 3.35*** -0.18** 1.93* 1.66* -0.96* 1.46~ 1.55~ 29.69*** 0.4591 10 860 (0.18) (0.26) (0.46) (0.33) (0.37) (0.44) (0.62) (0.39) (0.48) (1.52) (0.38) (0.68) (0.39) (0.44) (1.77) (0.45) (0.07) (0.81) (0.80) (0.43) (0.81) (0.87) (3.45) Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results: Multivariate analysis • • Literacy
grade 4 (2008) Estimated effects of change in characteristics on the literacy national average (Original sample mean = 26.57%) Teacher absenteeism zero LTSM Inventory good Curriculum planned using year schedule Full year learning programme Combined effect of improved characteristics
Predicted new mean Gain
27.84 27.36 27.18 27.18 29.85 1.27 0.79 0.61 0.61 3.29 18 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results: Multivariate analysis • • Numeracy
grade 4 (2008) Estimated effects of change in characteristics on the numeracy national average (Original sample mean = 34.21%) Assessment record keeping No timetable available Teacher absenteeism zero Maths teacher test score: 100% Maths topics covered: 25 plus Combined effect of improved characteristics
Predicted new mean Gain
35.08 34.45 36.01 36.38 37.20 42.29 0.87 0.24 1.80 2.17 3.00 8.08 19 Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Results: Multivariate analysis:
Modelling the literacy gain scores ( Historically black schools only ) 20
Explanatory variables
Mean School SES Facilities index (2008) Monitoring through class visits No timetable available (2008) Principal absent Teacher punctuality good More than 2 English mark records Paragraph writing: none Literacy exercises: more than 27 Years teaching: 4 to 9 Years teaching: 10 to 19 Years teaching: 20 plus Time dummy (1 st year) Constant R-squared N ~ p<0.10, * p<0.05, ** p<0.01, *** p<0.001 Standard errors in parentheses
[A] Pooled gains step 2
0.39 0.14~ ( 0.35
) ( 0.08
) -1.67** 0.94~ 1.44* -1.72** 1.34* 0.40 -5.33
*** 0.1214 390 ( 0.65
) ( 0.53
) ( 0.64
) ( 0.57
) ( 0.55
) ( 0.51
)
[B] 2-year literacy gains
1.37* 0.27~ 2.16* -2.72 -4.03*** 3.03*** 3.76*** -4.12*** 2.35* 1.03 2.64 3.83* 6.10
** 0.3976 195 ( ( ( 0.63
) 0.15
) 0.90
) ( 1.93
) ( 1.13
) ( 0.91
) ( 1.13
) ( 1.01
) ( 0.96
) ( 1.87
) ( 1.61
) ( 1.67
) Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Conclusions and Policy Implications
21
• • • •
Resource variables were not amongst the most important factors predicting achievement Several indicators of effective school management and teacher practice that are associated with student achievement have been identified • • even within the large historically disadvantaged section of the school system. This is an advance on earlier analyses An organised learning environment: • curriculum planning for the full year, a functional timetable, good quality inventories for LTSM, low teacher absenteeism and up-to date assessment records Extensive coverage of curriculum and exercises Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union
Conclusions and Policy Implications
22
• • •
Policies should empower teachers to cover curriculum and administer exercises: • • At the top: clearly communicated curriculum requirements Also, textbooks and workbooks that make worked examples easier for both teachers and students to implement.
Command and control measures to enforce adherence to best practices?
• Probably not… Explore ways to attract, train and support better principals, and to replace those at the head of dysfunctional schools.
Programme to Support Pro-Poor Policy Development A partnership between the Presidency, Republic of South Africa and the European Union