Transcript Slide 1
Place-based Education & Academic Achievement Prepared by: Michael Duffin, PEER Associates, Inc. Prepared for: the Place-based Education Evaluation Collaborative (PEEC) November 16, 2005 Suggested citation: Duffin, M., Chawla, L., Sobel, D., & PEER Associates (2005). Place-based education and academic achievement. Retrieved November 14, 2005 from http://www.peecworks.org/PEEC/PEEC_Research/S0032637E Place-based or environment-based education uses the environment as an integrating context (EIC) across disciplines. It is characterized by interdisciplinary learning, team teaching, hands-on learning experiences that often center on problem-solving projects, learner-centered education that adapts to students’ individual skills and abilities, and the exploration of the local community and natural surroundings. The following studies show that this approach not only increases students’ environmental literacy, but delivers other important benefits as well. A Review of Research and Evaluation on Impacts of Place-based Education on Student Academic Achievement Part 1: Ten studies from across the United States – Collectively cover 16 states, 265 schools – Recent (1998-2005) – Various combinations of standardized test scores, interviews, observations, demographics, documents Part 2: Results from the Place-based Education Evaluation Collaborative (PEEC) and CO-SEED – Basic theory of change – Dose-response measurement strategy – Direct measures of student achievement Ten Student Achievement Studies from across the United States Environmental Education: Improving Student Achievement (Bartosh, 2004, Washington State) Design: Statistically compared 77 pairs of demographically equivalent schools (fully implementing EE for at least 3 years v. schools w/o or just starting EE program) Standardized tests (Washington Assessment of Student Learning, and Iowa Test of Basic Skills) Electronic survey to evaluate the teaching & learning environment Gilford Elementary School, Gilford, NH Environmental Education: Improving Student Achievement (Bartosh, 2004, Washington State) Findings: Schools with systematic EE programs showed higher, statistically significant, test scores on standardized tests in math, reading, writing, and listening Pattern of EE school students having higher scores persisted for all five years of data investigated (1997-2002) Gilford Elementary School, Gilford, NH EE schools used natural areas more, had more EE prof. devel., and more support from parents, community, and administration Closing the Achievement Gap: Using the Environment as an Integrating Context for Learning (Lieberman & Hoody, 1998, national scope) Design: EIC programs in 40 schools in 12 states (California, Colorado, Florida, Iowa, Kentucky, Maryland, Minnesota, New Jersey, Ohio, Pennsylvania, Texas, Washington) Interviews with over 250 teachers and principals and over 400 students Four surveys about site characteristics 14 comparisons between EIC and traditional programs Gilford Elementary School, Gilford, NH Closing the Achievement Gap: Using the Environment as an Integrating Context for Learning (Lieberman & Hoody, 1998, national scope) Higher scores on standardized measures of academic achievement (reading, writing, math, science, social studies, GPA) Findings: In the 14 schools that compared EIC v. traditional programs, 36 out of 39 measures showed better performance by EIC students Reduced discipline, classroom management problems; Increased engagement and enthusiasm for learning; Greater pride, ownership in their accomplishments. Gilford Elementary School, Gilford, NH California Student Assessment Project: The Effects of Environment-based Education on Student Achievement (SEER, 2000, California) Design: 8 paired sets of students: one class exposed to EIC programs and the other without (twice from same school, six times from different, neighboring schools matched by demographics/SES) Evidence came from standardized test scores, site visits, and teacher surveys and interviews Haley School, Roslindale, MA California Student Assessment Project: The Effects of Environment-based Education on Student Achievement (SEER, 2000, California) Findings: Substantiates and builds upon findings from the Closing the Achievement Gap study . Haley School, Roslindale, MA The Effects of Environment-based Education on Students’ Achievement Motivation & The Effect of Environment-based Education on Student’ Critical Thinking Skills and Disposition Toward Critical Thinking (Athman (Ernst) & Monroe, 2004, Florida) Design: Norm-referenced tests (Achievement Motivation Inventory, Cornell Critical Thinking Test, California Measure of Mental Motivation) and interviews with selected students Studies published in two separate peer-reviewed articles (Journal of Interpretation Research, and Environmental Education Research) Haley School, Roslindale, MA 400 students, grades 9 & 12 in 11 Florida high schools The Effects of Environment-based Education on Students’ Achievement Motivation & The Effect of Environment-based Education on Student’ Critical Thinking Skills and Disposition Toward Critical Thinking (Athman (Ernst) & Monroe, 2004, Florida) Controlling for GPA, gender and ethnicity, EIC programs significantly raised scores on all three tests Critical thinking attributed to environmental themes, open-ended research projects, student voice and empowerment, connection to community Motivation attributed to learning experiences tailored to students’ interests/strengths, and applied to real-life issues/problems Haley School, Roslindale, MA Findings: Environment-based Education: Creating High Performance Schools and Students (NEETF, 2000, national scope) Design: Case studies of schools with environment-based programs (five individual schools, a model school program involving five schools, and a statewide program) Compared scores on statewide standardized tests between students from environment-based programs and various comparison groups such as district or state Hinsdale School, Hinsdale, NH Augmented with qualitative data, presumably from interviews, observations and/or documents Environment-based Education: Creating High Performance Schools and Students (NEETF, 2000, national scope) Findings: Hinsdale School, Hinsdale, NH All 3rd graders at Hawley Environmental Elementary School in Milwaukee, WI passed the Wisconsin Reading Comprehension Test (as compared with only 25% of the total Milwaukee public school population). At Isaac Dickson Elementary School in Asheville, NC, 4th grade students achieved a 31 percentage point increase in math achievement in just one year. Scores on college admission ACT test were higher for students from the School for Environmental Studies in Apple Valley, MN, than their peers in the district, the state, and the nation. At Condit Elementary School in Bellaire, TX, 3rd grade students who took part in the research-based environment program demonstrated higher-level thinking skills according to locally developed instruments. Environment-based Education: Creating High Performance Schools and Students (NEETF, 2000, national scope) Findings: First graders in the EIC classroom at Kruse Elementary in Pasadena, TX, performed higher on the Iowa Test of Basic Skills in all categories. Since incorporating environmental issues into the curriculum, Tompkinsville Elementary (and other Kentucky schools) have increased their achievement in science, reading, and social studies on statewide tests. All five schools in Florida’s Model Schools in EE program showed steady increases over five years on Florida Writes and FCAT tests. Hinsdale School, Hinsdale, NH Effects of Outdoor Education Programs for Children in California (American Institutes of Research, 2005, California) At-risk 6th grade students, 4 schools, 3 hands-on, ecologyoriented outdoor programs Design: Delayed treatment design (119 students in treatment group, control group of 106 students participated later in the year) Student, teacher, and parent surveys (pre-, post-, 6-10 weeks post), plus site visits and interviews Measured social/personal skills, stewardship of the environment, knowledge/understanding of science, and benefits for English Language Learners Beebe School, Malden, MA Effects of Outdoor Education Programs for Children in California (American Institutes of Research, 2005, California) Findings: Science scores on post-tests higher for treatment group Teachers reported increases in self-esteem, conflict resolution, relationship with peers, problem solving, motivation to learn, and behavior in class No significant change in environmental stewardship scores Beebe School, Malden, MA An Evaluation of the National Wildlife Federation’s® Schoolyard Habitat Program® in the Houston Independent School District (Danforth, 2005, Texas) Design: Compared 3 pairs of Houston schools, matched based on demographics Treatment group included 306 4th graders in schools implementing SYH, control group included 182 4th graders not doing SYH Dearborn Middle School, Roxbury, MA Measures include tests scores (Texas Assessment of Skills and Knowledge- Reading, Math, not Writing), attendance, and demographics 3rd grade as pre-, 4th grade as post- An Evaluation of the National Wildlife Federation’s® Schoolyard Habitat Program® in the Houston Independent School District (Danforth, 2005, Texas) Findings: SYH participation correlated positively and fairly strongly with increased math scores (.30) SYH participation correlated negatively and more weakly with improved reading scores (-.15) Author argued that the SYH curriculum did not address reading as directly as math Dearborn Middle School, Roxbury, MA Overall, non-whites and/or African Americans showed more improvement; Attendance results were mixed; Free lunch status correlations were not significant They Remember What They Touch: The Impact of Place-based Learning in East Feliciana Parish (Emekauwa, 2004, Louisiana) In 1999-2000, began Project Connect, a district wide place-based math and science initiative Design: 5 elementary/middle schools, over 2000 students, 80% African American, 85% free lunch 52 teachers participated in 1 or more of 3 consecutive summer trainings Investigated 4th grade ELA, Math, Science, Social Studies scores on Louisiana Educational Assessment Program (LEAP 21), 1998-2002 Compared district to state for % of students at “unsatisfactory” level Haley School, Roslindale, MA They Remember What They Touch: The Impact of Place-based Learning in East Feliciana Parish (Emekauwa, 2004, Louisiana) Performance gap between district and state decreased for all areas Findings: Greatest individual school success occurred at Slaughter Elementary where three of the district’s placebased leadership team teach Haley School, Roslindale, MA A Study to Keep Your Eye On (Falco, 2004, South Carolina EIC School Network) 10 middle schools participated in EIC implementation in South Carolina In the first year all of the schools showed some degree of improved attendance, behavior, and academic achievement Watch to see if future reports from this group continue to build on their growing body of evidence and also provide more details about the method and data for their investigations Woodsville Elementary School, Woodsville, NH Student Achievement Investigations from PEEC and CO-SEED Basic Theory of Change for Student Academic Achievement Changes in educator practice Placebased Education Program in a school (e.g. increased collaboration, interdisciplinary integration, use of local resources, professional growth & engagement etc.) Increased student engagement & enthusiasm Improved student academic achievement Changes in school culture Marzano claims that that number is actually closer to 20%, with 13% deriving from teacher-level factors, and 7% attributable to school-level factors Higher Lower Coleman report claimed that schooling accounted for only 10% of the variance in student achievement (or rR2 = .10) (Measures of Intended Outcome) If participants with less dose have lower outcomes, and those with more dose have higher outcomes, then the program is likely to be an active ingredient Program “Response” PEEC’s Dose-Response Measurement Strategy Less More Program “Dose” (Exposure + Implementation) Weight status predicts 17-19% of cost for treating cardiovascular disease Link A in the Logic Chain Changes in educator practice Placebased Education Program in a school (e.g. increased collaboration, interdisciplinary integration, use of local resources, professional growth & engagement etc.) Very diverse sample (4 programs in 55 schools; Whole school change & Prof. development models; Urban, rural, suburban; Grades K-12) Averages from an aggregate of 12 survey items show PEEC dose accounts for 19% of variance in Overall Educator Practice Figure S4. Teacher Practice (overall module) Scale: Strongly Disagree = 1, to Strongly Agree = 4 Scale: Twice per year or less = 1, to Once a week or more = 4 338 educator surveys Average scores for items about teacher collaboration, meeting of curriculum goals, confidence, energy, growth as a teacher, and use of local people and places for teaching(teapracmea) PEEC Cross-Program Survey Results 2003-04 Changes in Educator Practice (Link A) From PEEC educators surveys, 20 03-04 4.0 3.0 2.0 1.0 0.0 The best fit multiple regression line above shows that 19% of the variability in survey response is predicted by dose of a PEEC program. The result is statistically significant. rR2= .19, p = .000, n = 342. 1.0 2.0 3.0 4.0 Dosage composite Scale: No exposure to a PEEC program = 0, Very high PEEC program exposure and implementation = 4 Link B in the Logic Chain Changes in educator practice (e.g. increased collaboration, interdisciplinary integration, use of local resources, professional growth & engagement etc.) Increased student engagement & enthusiasm PEEC and CO-SEED Survey Results 2003-04 Increase in Student Engagement (Link B) (rR2 Same effect size = .16) for PEEC cross-program From CO-SEED educator surveys, 2003-04 Scale: Strongly Disagree = 1, to Strongly Agree = 4 Averages from an aggregate of 3 survey items show CO-SEED dose accounts for 16% of variance in (adult reports of) Student Engagement in Learning Figure S3. Student Engagement in Learning (index) Average scores for index consisting of three items about the enthusiasm, preferences for traditional style learnin g activities, and selfdirecte dness in their CO-SEE D projects (xsel) 338 educator surveys for PEEC, 245 educator surveys for CO-SEED 4.0 3.0 2.0 The best fit multiple regression line above shows that 16% of the variability in survey response is predicted by dose of CO-SEED. The result is statistically significant. rR2= .16, p = .000, n = 245. 1.0 0.0 1.0 2.0 3.0 4.0 Dosage composite Similar results for (adult reports of) Student Civic Engagement Scale: No exposure to CO-SEED = 0, Very high CO-SEED exposure and implementation = 4 Link C in the Logic Chain Increased student engagement & enthusiasm Improved student academic achievement PEEC and CO-SEED Lit Review 2003-04 Improved Student Academic Achievement (Link C) Citing over 40 research studies, Marzano states (2003, p. 144) … “The link between student motivation and achievement is straightforward. If students are motivated to learn the content in a given subject, their achievement in that subject will most likely be good.” PEEC and CO-SEED Survey Results 2003-04 Improved Student Academic Achievement (Link C) Similar effect sizes for student reports of Enthusiasm for Learning, both for CO-SEED and PEEC Effect sizes get smaller (for students) as one gets further from the intervention (directed first at educators)…the closer you are to the fire, the more heat you feel From CO-SEED student surveys, 2003-04 Scale: Strongly Disagree = 1, to Strongly Agree = 4 Raw scores from a single survey item shows CO-SEED dose accounts for 5% of variance in (student reports of) “CO-SEED helps me get better grades” Figure S10. Helps with Grades and Tests (item) Raw scores for item "CO-S EED helps me do better on tests and get better grades" (x10) 721 student surveys for PEEC, 680 student surveys for CO-SEED 4 3 2 1 The best fit multiple regression line abov e shows that 5% of the variability in survey respo nse is predicted by dose of CO-SEED. The result is statistically significant.r R2= .05, p = .000, n = 680. 0.0 1.0 2.0 3.0 4.0 Dosage composite Scale: No exposure to CO-SEED = 0, Very high CO-SEED exposure and implementation = 4 Efforts to directly measure student academic achievement at CO-SEED sites Interesting but Inconclusive Investigation of Writing Scores on Statewide Standardized Tests (Beebe and Gilford Schools, 2003) Previous evaluations had generated (unsolicited) assertions from interviewees at all sites that COSEED improved student writing At the Beebe school in Malden, MA, MCAS writing scores had increased in the three years since CO-SEED arrived, but roughly similar trends were found across the district Young Achievers School, Jamaica Plain, MA In Gilford, NH, 3rd grade NHEIAP writing scores showed some but not a very strong correlation with level of CO-SEED involvement, though all Gilford students outperformed the state avg. Extremely Strong Pattern in NHEIAP Test Scores but Inconclusively Linked to CO-SEED (Gorham Schools, 2005) Interviewee claimed that COSEED helped Gorham students become top performers in the state on NHEIAP tests, so we looked at scores from 1997-2004 100 90 80 Cohort 1 70 Percentile rank For the last several years, Gorham 3rd graders score/rank very low, but by 6th grade the scores/ranks are very high Changes in Gorham State Rank: Math Cohort 2 60 Cohort 3 50 Cohort 4 40 Cohort 5 30 Cohort 6 Cohort 7 20 Scores/ranks drop off a bit by 10th grade, but still high and there was no where to go but down 10 0 Grade 3 Grade 6 Grade 10 Grade level Gorham Middle School, Gorham, NH Similar patterns for Math, Language Arts, Science, but pattern may pre-date CO-SEED First Grade Academic Achievement as a Function of CO-SEED/ Community-Based Units (Young Achievers School, 2005) Design: Principal says “One thing we know is that kids’ writing is much more interesting, complex, and detailed if they’ve had rich experience…The current first grade has about a third of the kids who didn’t have Kindergarten here and in general it is breathtaking the difference in the academic achievement. Our Kindergarten has the strongest place-based education in the school, especially with language development.” First grade is also strong. 3 measures (Direct Reading Assessment, TERC Math, YA Writing Assessment) tracked in YA’s assessment database Young Achievers School, Jamaica Plain, MA Compared 1st graders with one v. two years of exposure to strong PBE teachers First Grade Academic Achievement as a Function of CO-SEED/ Community-Based Units (Young Achievers School, 2005) Findings: 1st graders w/ more place-based education outperformed peers on all measures Writing TERC Math: Assessment: 1 year vs. 1 year 2 vs. year vs. tenure 2 year tenure at Y.A. Y.A. DRA Assessment: 1 year 2 year tenure at at Y.A. 100.00% 2.5 36 90.00% 32 2.25 80.00% Mean Score Mean Mean Score score 28 70.00% 2 24 60.00% Grades K-1 68.60% 20 50.00% 1.75 1.7 1.68 Grade 1.65 1 only 60.71% 1.63 16 40.00% 1.61 1.64 1.61 1.55 1.5 12 30.00% Grade only Grade 1 only Grade 11only (N=14) (N=14) (N=14) 1.54 1.44 Grades K-1 13.2 8 20.00% 1.25 Grade 1 only 9.57 4 10.00% 0.00% 0 1 Writing-Content Writing-Organization Writing-Language TERC DRAMath Assessm Assessm ent ent Assessm ent Young Achievers School, Jamaica Plain, MA Writing-Mechanics Writing-Overall Grades K-1 Grades K-1 Grades K-1 (N=25) (N=25) (N=25) Effects of CO-SEED on Standardized Test Scores (MCAS) at the Beebe Health & Environmental Magnet School (Beebe School, 2005, Massachusetts) CO-SEED worked with Beebe 19992003, helped secure CSR funding to continue work 2002-2005 Design: Several lines of evidence suggest that the environmental theme has become embedded in the school culture Before analyzing MCAS scores, we predicted that Beebe would deviate from the typical pattern and increase performance relative to district and/or state in the following content areas: – – – – Math (mostly near 3rd and 4th grade) English Language Arts – Writing Life Science Earth Science Beebe School, Malden, MA Effects of CO-SEED on Standardized Test Scores (MCAS) at the Beebe Health & Environmental Magnet School (Beebe School, 2005, Massachusetts) Findings: Typical pattern: State performs highest, then Beebe, then district % correct MCAS - Grade 5 Earth Science 80 75 70 65 60 55 50 45 40 35 30 Beebe District State 2000 2001 2002 2003 2004 2005 year Beebe School, Malden, MA Effects of CO-SEED on Standardized Test Scores (MCAS) at the Beebe Health & Environmental Magnet School (Beebe School, 2005, Massachusetts) Findings: Only a few deviations from the typical pattern (6th & 8th grade Math, 8th grade Life & Earth science) correct %% correct Gr. 8 Life Science MCAS -MCAS Gr. 8- Math, Open Response 80 75 70 70 65 65 60 60 55 50 55 45 50 40 45 35 40 30 35 30 Beebe Beebe District District State State 2000 2001 2002 2003 2004 2005 year2003 2004 2005 2000 2001 2002 Beebe School, Malden, MA Effects of CO-SEED on Standardized Test Scores (MCAS) at the Beebe Health & Environmental Magnet School (Beebe School, 2005, Massachusetts) Analysis mildly supported the prediction for two areas (Math & Earth Science) Findings: Analysis strongly supported the prediction in one area (Life Science) Analysis did not support the prediction in one area (Writing, the typical pattern persisted in both grades 4 and 7) Future prediction: Strongest results will continue to show up in the upper grades (i.e. where students have the highest cumulative dose of the environmental/ place-based theme integration) Beebe School, Malden, MA References American Institutes of Research. (2005). Effects of Outdoor Education Programs for Children in California. Sacramento. Retrieved March 15, 2005 from http://www.sierraclub.org/insidetheoutdoors/downloads/outdoorschool_finalreport.pdf Athman, Julie & Monroe, Martha. (2004). The effects of environment-based education on students’ achievement motivation. Journal of Interpretation Research, 9(1): 9-25. Bartosh, O. (2004). Environmental education: Improving student achievement. Unpublished master's thesis, The Evergreen State College, Olympia, Washington. Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfield, F. D., & York, R. L. (1966). Equality of educational opportunity. U.S. Government Printing Office, Washington, DC. Danforth, P. (2005). An evaluation of the National Wildlife Federation’s Schoolyard Habitat Program in the Houston Independent School District. Unpublished master’s thesis, Texas State University, San Marcos, TX. Duffin, Michael, Powers, Amy, Tremblay, George, & PEER Associates. (2004). Place-based Education Evaluation Collaborative: Report on cross-program research and other program evaluation activities, 2003-2004. Retrieved October 6, 2004 from http://www.peecworks.org/PEEC/PEEC_Reports/S0019440A. Emekauwa, E. (2004). They remember what they touch: The impact of place-based learning in East Feliciana parish. Rural School and Community Trust. Retrieved November 16, 2005 from http://www.peecworks.org/PEEC/PEEC_Research/0009D4FB-007EA7AB.0/Emekauwa%20East%20Feliciana%202004.pdf Ernst, Julie Athman & Monroe, Martha. (2004). The effect of environment-based education on student’ critical thinking skills and disposition toward critical thinking. Environmental Education Research, 10(4): 507-522. Liebermann, J., & Hoody, L. (1998). Closing the achievement gap: Using the environment as an integrating context for learning. State Education and Environment Roundtable, San Diego, CA. Marzano, R. J. (2003). What works in schools: Translating research into action. Association for Supervision and Curriculum Development, Alexandria, VA. National Environmental Education Training Foundation (NEETF). (2000). Environment-based education: Creating high performance schools and students. Retrieved October 1, 2004 from http://neetf.org/pubs/NEETF8400.pdf State Education and Environment Roundtable (SEER). (2000). California Student Assessment Project: The Effects of Environment-based Education on Student Achievement. Retrieved July 14, 2005 from http://www.seer.org/pages/csap.pdf Wang, G., Zheng, Z., Heath, G., Macera, C., Pratt, M., & Buchner, D. (2002). Economic burden of cardiovascular disease associated with excess body weight in U.S. adults. American Journal of Preventive Medicine, 23 (1), 1-6. Place-based Education & Academic Achievement Prepared by: Michael Duffin, PEER Associates, Inc. Prepared for: the Place-based Education Evaluation Collaborative (PEEC) November 16, 2005 Suggested citation: Duffin, M., Chawla, L., Sobel, D., & PEER Associates (2005). Place-based education and academic achievement. Retrieved November 14, 2005 from http://www.peecworks.org/PEEC/PEEC_Research/S0032637E