Transcript Presentation title

Uncovering the Sources of
Student Learning Challenges:
Analyzing and Interpreting Data
December 4, 2014
Roanoke, VA
Welcome and Introduction
Aimee Evan
Virginia Middle School Research Alliance Lead
REL Appalachia
Regional Educational Laboratory (REL) Program
• U.S. Department of Education, Institute of Education Sciences (IES).
• RELs provide regional support for:
– Applied research and evaluation.
– Technical support and information sharing to build capacity to use data for
improved education outcomes.
3
REL Appalachia’s Mission
• Meet the applied research and technical support needs of Kentucky,
Tennessee, Virginia, and West Virginia.
• Bring evidence-based information to policymakers and practitioners:
– Provide support for a more evidence-reliant education system.
– Inform policy and practice for states, divisions, schools, and other
stakeholders.
– Focus on high-priority, discrete issues and build a body of knowledge over
time.
4
What Does REL Appalachia Do?
• Assess regional research needs by monitoring emerging education issues
and challenges.
• Maintain and refine research alliances through ongoing dialogue between
educators in each region and researchers.
• Provide analytic technical support to increase use of data and analysis to
understand policies and programs, make decisions, and support effective
practice.
• Conduct research and evaluation studies of rigor and method appropriate
to the questions the studies attempt to answer.
• Distribute results of REL research across the region.
• Coordinate and partner with other RELs and federal, state, and local
education research and technical assistance organizations.
5
Research Alliances
• What is a research alliance?
– A partnership between education stakeholders and REL Appalachia.
• What is the purpose of a research alliance?
– As partners, REL Appalachia and alliance members develop and carry out a
research and analytic technical assistance agenda on priority topics.
• Who are the education stakeholders in an alliance?
– May include representatives from one or more schools, divisions, state
education agencies, and other organizations (e.g., colleges and universities).
6
How Do We Do the Work: REL Appalachia Staff
• Justin Baer, Director, REL Appalachia
• Lydotta Taylor, Alliance Lead, REL Appalachia
• Kellie Kim, Analytic Technical Support Lead, REL Appalachia
• Becky Smerdon, Early Warning Systems Content Lead
• Aimee Evan, VMSRA Lead
• Angela Estacion, VMSRA Project Lead
7
What Is the Virginia Middle School Research Alliance?
• Collaborative working group of practitioners and researchers.
– Superintendents, assistant superintendents, directors of curriculum &
instruction, directors of assessment & testing, principals, teachers from the
following divisions:
•
•
•
•
•
•
Campbell County
Greene County
Harrisonburg
County
Louisa County
Nelson County
Norton City
•
•
•
•
•
•
Pulaski County
Rockbridge County
Russell County
Salem City
Smyth County
Staunton City
– Executive directors of SURN (School-University Research Network) and VSUP
(Virginia School-University Partnership).
– Virginia Department of Education (VDOE).
– REL Appalachia researchers in rural education, early warning systems, data use.
8
Goals of the Virginia Middle School Research Alliance
• Assist middle school practitioners in using data to inform instructional
decisionmaking and improve student outcomes by:
– Identifying struggling students who need additional support.
– Selecting, implementing, and monitoring interventions to support students.
• Focus on expanding the state’s early warning system (EWS) to middle
schools.
9
Virginia Middle School Research Alliance Projects
• Catalog EWS data, assessments, and interventions currently being
collected/used in schools.
• Document how data are currently being used in schools, and what
supports and barriers are in place to help or hinder use.
• Determine the most powerful data to use to identify students in need of
further assistance.
• Provide workshops on using data efficiently and effectively to:
– Identify struggling students and determine how to target resources to meet
their needs.
– Monitor each student’s progress.
10
Virginia Middle School Research Alliance Future Work
• Workshops* will continue to build the capacity of educators to interpret
meaning from data in order to best meet students’ needs by:
– Improving classroom instructional strategies (Spring 2015).
– Improving division and schoolwide strategies (Summer 2015).
– Workshop on determining the most powerful data in your own division to
identify struggling students (Winter 2015).
*These activities are currently under review by IES.
11
Workshop Overview and Goals
Aimee Evan
12
Introductions
• Name
• Role
• Division
13
Goals for Participants
• Understand a process to use data to identify sources and causes of
students’ learning challenges.
• Recognize what data sources can be used to accurately identify sources of
students’ learning challenges and to monitor their progress toward
improvement.
• Learn what building leadership can do to facilitate and improve teachers’
use of data.
• Understand your own and your school’s status in developing data-use
capacity.
14
Logistics
• Binder Navigation
• Housekeeping:
– Agenda
– Bathrooms
– Tabs
– Breaks
– Survey
– Lunch
15
Introduction to the
Early Warning System Process
and Foundational Elements
Aimee Evan
What Is an Early Warning System (EWS)?
• Provides a systematic way to identify (“flag”) students early who are at
risk of failure.
• Grounded in research.
• Relies on readily available (and familiar) data:
– ABCs: Attendance, Behavior, and Course grades/assessment results.
• Provides information that is actionable by educators in schools and
divisions.
• Requires educators to diagnose further student needs, and to use
professional judgment to support at-risk students.
• Targets resources to support at-risk students while they are still in
school, before they go too far down the road of academic failure and
drop out.
• Examines patterns and identifies school climate issues.
Sources: Allensworth & Easton, 2005, 2007; Balfanz,
2009; Balfanz & Herzog, 2005
17
The EWS Continuous Improvement Cycle
Step 1: Establish
roles and
responsibilities
Step 7: Evaluate and
refine the EWIMS
process
Step 6: Monitor
students and
interventions
Step 5: Assign and
provide
interventions
Step 2: Use the EWS
Middle Grades Tool
Step 3: Review the
EWS data
Step 4: Interpret the
EWS data
Source: Therriault et al., 2013
18
The EWS Continuous Improvement Cycle
Infrastructure and Support Are Just as
Important as the Process.
19
EWS – A Systems Perspective
A. Data and Computer
Data Systems
Step 1: Establish roles
and responsibilities
Step 7: Evaluate and
refine the EWIMS
process
Step 6: Monitor
students and
interventions
B. Educator Knowledge &
Skills for Data Use
Step 5: Assign and
provide interventions
Step 2: Use the EWS
Middle Grades Tool
Step 3: Review the
EWS data
Step 4: Interpret the
EWS data
C. School Organization
for Data Use
20
References
Allensworth, E. M., & Easton, J. Q. (2005, June). The on-track indicator as a predictor of high school graduation.
Chicago: University of Chicago, Consortium on Chicago School Research. Retrieved from
http://ccsr.uchicago.edu/sites/default/files/publications/p78.pdf
Allensworth, E. M., & Easton, J. Q. (2007, July). What matters for staying on-track and graduating in Chicago
public high schools: A close look at course grades, failures, and attendance in the freshman year. Chicago:
University of Chicago, Consortium on Chicago School Research. Retrieved from
http://ccsr.uchicago.edu/sites/default/files/publications/07%20What%20Matters%20Final.pdf
Balfanz, R. (2009). Putting middle grades students on the graduation path: A policy and practice brief.
Baltimore: Johns Hopkins University, Everyone Graduates Center. Retrieved from
http://new.every1graduates.org/putting-middle-grades-students-on-the-graduation-path-a-policy-andpractice-brief/
Balfanz, R., & Herzog, L. (2005). Keeping middle grades students on-track to graduation: Initial analysis and
implications. Presentation at the second Regional Middle Grades Symposium, Philadelphia, PA.
Therriault, S. B., O’Cummings, M., Heppen, J., Yerhot, L., Scala, J., & Perry, M. (2013). Middle grades early
warning intervention monitoring system implementation guide. Washington, DC: National High School
Center at American Institutes for Research. Retrieved from http://betterhighschools.org/EWS_middle.asp
21
Local Example of Foundational Elements:
How Northside Middle School Uses Data
Lori Wimbush, Principal
April Griffin, English Teacher
Christina Hall, Math Teacher
Linda Shiflett, Special Education Teacher
Laurie Spickard, Data Specialist
“Schools to Watch”: Northside Middle School
23
Year of Change
All Students
Disadvantaged
SWD
Black
Hispanic
White
LEP
60
66
65
81
69
2010-11
94 90 88 94 90 94 79
90
84
78
85
83
92
71
White
65
Hispanic
79
Black
89 80 70 75 77 93 69
SWD
2009-10
All Students
LEP
Reading Reading Reading Reading Reading Reading Reading Mathematics Mathematics Mathematics Mathematics Mathematics Mathematics Mathematics
Disadvantaged
Northside
Middle
School
24
New Math SOL Results
All Students
Disadvantage
d
SWD
Black
Hispanic
White
LEP
Northside Middle School Mathematics
2010-11 Results
2011-12 Results
90
88
84
80
78
66
85
68
83
83
92
81
71
79
2011-12 State
Results
2012-13 Results
68
85
54
79
40
61
52
73
61
77
75
87
59
73
2012-13 State
Results
2013-14 Results
71
85
57
78
41
61
55
76
64
79
77
86
59
86
2013-14 State
Results
74
61
43
60
67
80
62
25
New Reading SOL Results
All Students
Disadvantage
d
SWD
Black
Hispanic
White
LEP
Northside Middle School Reading
2010-11 Results
2011-12 Results
94
93
90
87
88
75
94
83
90
93
94
94
79
72
2011-12 State
Results
2012-13 Results
89
79
81
68
66
42
80
62
84
61
93
83
80
44
2012-13 State
Results
2013-14 Results
75
79
59
68
43
42
59
62
65
61
82
83
54
44
2013-14 State
Results
75
59
43
59
65
82
54
26
2013–14 Annual Measurable Objectives (AMOs)
Proficiency Gap Dashboard for Federal Accountability
Reading
Reading
AMO
Target
AMO
Result
All Students
69
75
YES
Gap Group 1: Students with Disabilities, English Language
Leaners, Economically Disadvantaged Students
(unduplicated)
59
60
Gap Group 2: Black Students
57
Gap Group 3: Hispanic Students
60
Key:
YES = Met objectives based on the current year result
TS = Too small; objective not evaluated due to too few students
NO = Did not meet objective
- = No data for group
N/A = Not applicable
Reading
Mathe- Mathe- Mathematics matics matics
Met AMO AMO
Target
Target
AMO
Result
Met
AMO
Target
66
85
YES
YES
57
76
YES
66
YES
56
76
YES
50
3YR
60
79
YES
3YR = Met objective based on the 3 year average result
R10 = Met objective by reducing failure rate by at least 10 percent
< = A group below state definition for personally identifiable
results
* = Data not yet available
27
Changes as a School
• Promote atmosphere where ALL students can improve!
• 90 minutes Math and English each day.
• All teachers remediate during class or before/after school.
They turn in weekly SOL evaluation sheets.
28
Weekly SOL Evaluation
29
More Changes as a School
• Use Student-Based Performance By Question/By Teacher to
drive instruction.
• Continue to use regular common assessments analysis to
drive instruction.
30
Student Performance By Question/Teacher
31
Analysis by Question
32
Analysis by Student
33
Changes as a School, continued
• Monthly and bi-weekly data meetings by subject area:
– Common planning.
– Benchmark analyses.
34
Benchmark Analysis
35
Benchmark Analysis
36
Benchmark Analysis
37
Math
• Common planning and Math Team meetings:
– Analyze common assessments and share successful
teaching strategies.
– Communicate across grade levels to help use common
language when teaching concepts.
– Continuous cumulative reviews.
– Error analysis after assessments.
– Various math programs.
– Math Tutors help support teachers to fill in gaps in
learning.
38
English
• Students reading 20 minutes per day:
– Teachers model reading.
– Conference with students about various topics.
• Students choose what to read and work on needed skill:
– Allows students to make choices all year on their reading
interests as well as assignments.
– Students make choices and they take ownership of their
learning.
• Ultimate goal is for students to become lifelong readers.
• All teachers in our building can count on students having a
book to read during any down time.
39
Special Education
• Use data to determine self-contained Math and English
classes.
• Re-evaluate student learning plans.
• Use data to drive IEPs.
• Look at the individual student and his/her specific needs.
• Focus on improvement for ALL students.
40
Social Studies
• Remediation occurs the month prior to SOLs, during lunch.
• Creation of computer lab with laptops to allow for increased
computer use.
• Continuation of benchmark analysis meetings.
41
Connect with Us!
www.relappalachia.org
@REL_Appalachia
Aimee Evan
[email protected]
703-655-3695
42