Transcript Slide 1

Southwest Ohio
2014 Data Workshops
Using the HSTW Assessment and
MMGW Survey Data
to Take Action
High Schools & Middle Schools
Mike Ross
HSTW/MMGW Ohio Data Consultant
Southern
Regional
Education
Board
Susanne Davis
HSTW/MMGW Ohio Assessment Coordinator
1
Workshop Objectives





Southern
Regional
Education
Board
Understand components of the HSTW
and MMGW Assessment Reports
Identify gaps in student achievement
and instructional practices
Identify new challenges
Link key Assessment data to the Site
Review Process (SRP)
Explore how Assessment data may
provide critical baseline or
“readiness” levels for implementing
common core state standards
2
HSTW Key Practices




High Expectations
Program of Study
Academic Studies
Career/Technical
Studies
 Work-Based
Learning
Southern
Regional
Education
Board
 Teachers Working
Together
 Students Actively
Engaged
 Guidance
 Extra Help
 Culture of
Continuous
Improvement
3
MMGW Key Practices
 Aligned academic
core
 All students matter
 High expectations
and extra help
 Engaging students
 Teachers working
together
 Support from
parents
 Qualified teachers
 Use of data
 Use of technology
for learning
 Strong leadership
Southern
Regional
Education
Board
4
Why Are Data Important?
 Replace hunches with facts
 Enable us to tackle root causes instead of
symptoms
 Enable us to determine if we are
accurately tracking effects
 Assist in understanding how school and
classroom practices impact achievement
Southern
Regional
Education
Board
5
Data Can’t Help When…
 The data are not valid or reliable
 The appropriate questions are not asked
after reviewing the data
 Data analysis is not used for making wise
decisions
 There is no commitment by teachers and
administrators to change school and
classroom practices when data indicate the
need for change
Southern
Regional
Education
Board
Make conscious efforts to develop a data-rich
culture at your school!
6
Multiple Measures
Perceptions
School
Processes
Effort
Student
Achievement
Demographics
Southern
Regional
Education
Board
Based on Victoria Bernhardt’s Multiple Measures Model
Categories of Data
“Measures of student learning help us understand
how students are performing and what students
know as a result of instruction.
Student
Achievement
Southern
Regional
Education
Board
Based on Victoria Bernhardt’s Multiple Measures Model
Categories of Data
School
Processes
Southern
Regional
Education
Board
“…programs, practices, and instructional
strategies…that produce school and
classroom results.”
Based on Victoria Bernhardt’s Multiple Measures Model
Categories of Data
Perceptions
Southern
Regional
Education
Board
“A particular view, judgment, or appraisal formed in the
mind about a particular matter...a belief stronger than
impression and less strong than positive knowledge.”
Based on Victoria Bernhardt’s Multiple Measures Model
Categories of Data
“Statistical characteristics of human
populations…builds the context of the school
…for which change is planned and takes place.”
Demographics
Southern
Regional
Education
Board
Based on Victoria Bernhardt’s Multiple Measures Model
A Complete Picture
SAT/ACT
Scores
HSTW
Assessment
Industry
Certification
Exams
State Test
Scores/AYP
AP/IB
Enrollment
Southern
Regional
Education
Board
Course
Failures
Discipline
Referrals
Comparisons
of Assessments to OAA/OGT
Southern
Regional
Education
Board
HSTW Assessment
OGT/OAA
 Based on NAEP
Standards
 3 areas
 Administered to
12th graders
 Content through
grade 12
 Multiple choice
 Based on Ohio
standards
 2 to 5 areas
 Administered (initially)
to 3rd through 10th
graders
 OGT – through gr. 10
 Multiple choice &
constructed response
13
Comparisons
of Assessments to OAA/OGT
Southern
Regional
Education
Board
HSTW Assessment
OGT/OAA
 Reports Readiness
Standards Generalize to high
school and college
success
 Scale scores
 Cut scores/
Proficiency Levels
 Random Sample
 Others?
 Not predictive of
college success
 OGT tied to
graduation
 Scale scores
 Cut scores/
Proficiency Levels
 All students
 Others?
14
2014 Assessment & Surveys
Ohio Participation
 2014 HSTW Assessment
 125 Ohio sites (41 SWO)
 6,314 Ohio 12th grade students (2,286 SWO)
 4,580 Ohio teachers (1,754 SWO)
 2014 MMGW Surveys
Southern
Regional
Education
Board
 95 Ohio sites (31 SWO)
 12,118 Ohio 8th grade students (5,518 SWO)
 1,789 Ohio teachers (653 SWO)
15
Approaching the Report
 Before analyzing data, ask yourself
“who participated in this assessment?”
Southern
Regional
Education
Board
 Random sample or all students?
 Career/technical or all students?
 Who completed the assessment?
• Pages 62, 71, 86, 97 of High School
assessment report
 What is your H.S. category? – A, B, C, D
Refer to top left of each page and
descriptions on page iv.
16
High School Standard Setting –
Readiness Standards
Basic
Southern
Regional
Education
Board
Reading
Mathematics
Science
250-271
257-291
258-285
Students performing at this level or above
are able to enter postsecondary studies
without needing additional preparation
and/or are able to pass most employer
exams for entry-level jobs.
17
2014 HSTW Assessment Results All Ohio Sites
Percentage of Students Meeting Readiness Goals
100%
85%
85%
85%
80%
65%
60%
60%
57%
63%63%
59%
60%63%
HSTW
Goal =
85%
55%
40%
20%
0%
Southern
Regional
Education
Board
Reading
Ohio 2014
Mathematics
Ohio 2012
All Sites 2014
Science
HSTW Goal
Source: 2014 HSTW Assessment – Ohio Composite, pg.1
18
2014 HSTW Assessment Results All Ohio Sites
Percentage of Students at Each Performance Level
60%
Reading
40%
25%
27%
8%
63%
Math
37%
39%
21%
4%
19%
4%
60%
Science
Southern
Regional
Education
Board
40%
Below Basic
38%
Basic
Proficient
Advanced
Source: 2014 HSTW Assessment – Ohio Composite, pg.1
19
2014 Ohio Achievement Assessment
8th Grade Preliminary Results
Percentage of Students at Proficient or Above
100%
87% 86%
75%
80%
80%
77%
75%
75%
68% 69%
60%
40%
20%
0%
Reading
Southern
Regional
Education
Board
Mathematics
2014
Source: ODE website
State Std
Science
State
Standard
= 75%
2014 Ohio Achievement Assessment
8th Grade Preliminary Results
Percentage of Students at Each Proficiency Level
Reading 3% 9%
Southern
Regional
Education
Board
Math
6%
Science
5%
32%
32%
13%
Limited
25%
41%
27%
33%
Basic
Source: Ohio Department of Education
24%
Proficient
16%
Accelerated
14%
20%
Advanced
Main Summary Reports
 Now that we know who participated
and how Ohio students performed,
how did your school do overall?
 H.S. - Executive Summary, pp. 1-2
 M.S. – refer to your OAA data
Southern
Regional
Education
Board
22
2014 All Ohio HSTW Sites Composite, p. 1
2014 All Ohio Career Centers Composite, p. 1
Key Indicators of Student Achievement
or Power Indicators (report page 2)
• 12 Student and 6 Teacher Indicators
• Associated with very high mean
scores
• ETS has found these indicators to be
statistically predictive of student
achievement
Southern
Regional
Education
Board
25
2014 All Ohio HSTW Sites Composite, p. 2
Activity Time
High School - Utilizing
the Executive Summary
& Analyzing Differences
 Report pp. 1-12
 HS Handout #1
Middle School – Reviewing
State Achievement Data
 State Report/Data
 MS Handout #1
Southern
Regional
Education
Board
28
Activity Processing
 What value do these data
provide?
 How can you use this executive
summary and other features in
the reports in planning at your
school?
Southern
Regional
Education
Board
29
Utilizing the
High School Overview
and
Middle School Indices
Southern
Regional
Education
Board
30
More Information
 Now that we’ve gotten a picture of
overall performance, where can I get
some more details?
 H.S. - Executive Summary – pp. 1-2
 H.S. - Overview – pp. 3-24
 M.S. - Indices – pp. 6-8
Southern
Regional
Education
Board
31
Semantics
 Indicator
 Index or indices (pl.)
 Common meanings and
distinctions
Southern
Regional
Education
Board
32
“Our Definition”
 Indicator –
Southern
Regional
Education
Board
a student experience(s)
and/or improvement
strategy(ies) that are
associated with higher
levels of student
achievement
33
Definition
 Examples of Indicators
 Used a graphing calculator in
math assignments at least weekly.
Prepared a written report of
science lab results at least
monthly.
Southern
Regional
Education
Board
Used computer skills to complete
an assignment or project in C/T
classes at least weekly.
34
Definition
 Index –
A scale that combines multiple
items into a single value.
Southern
Regional
Education
Board
Each index is built around a
common theme which HSTW or
MMGW believes is a key element
of student achievement.
35
…be mindful
 “Although we cannot prove
a causal connection between
the indices and student test
performance, it can be said
that faithful adherence to the
Key Practices is associated
with better student
achievement.”
Southern
Regional
Education
Board
36
Understanding the Indices
 Based on the HSTW Key Practices
 Example – Literacy Across the Curriculum
 A series of indicators (and frequency)
are identified for each practice
 Example – Students often revised their
essays or other written work several times
to improve their quality
 The number of indicators experienced
determines the level of emphasis
Southern
Regional
Education
Board
 Example – Intensive = 8-10 indicators
Source: 2014 HSTW Assessment – Ohio Composite, pg.13
High School Indicator Data
(source: 2014 All Ohio HSTW Sites Composite, p. 26)
The MMGW Indices
2014 MMGW Student Survey Ohio Composite, p. 7
2014 MMGW Student Survey Ohio Composite, p.17
2014 MMGW Student Survey Ohio Composite, p.18
2014 MMGW Student Survey Ohio Composite, p.19
Activity Time
A More Detailed Picture of Your
School’s Performance
High School –
Utilizing the Overview
& the HSTW Indices
 Report pp. 1 & 13-23
 HS Handout #2
Middle School –
The MMGW Indices
 Report pp. 6-7
 MS Handout #2
Southern
Regional
Education
Board
44
Share
 On what three categories of
indicators did most of your students
report an intensive emphasis?
 What actions did you take that may
account for these results?
 What challenges did you identify by
looking at these data?
Southern
Regional
Education
Board
45
The Importance of
Leadership and School
Improvement
Southern
Regional
Education
Board
46
Key Practice:
Continuous Improvement
Southern
Regional
Education
Board
Use student achievement and
program evaluation data to
continuously improve school
culture, organization,
management, curriculum and
instruction to advance student
learning.
47
Improvement Cycle
Southern
Regional
Education
Board
Teachers’ Perception on
Continuous School Improvement
Percentage Who Strongly Agree
56%
54%
40% 43%
Southern
Regional
Education
Board
42% 43%
Goals and Priorities
Clear
2014
Demanding yet
supportive
2012
55%
45% 47%
Teachers continually
learning
High-Scoring Sites
Source: 2014 HSTW Assessment – Ohio Composite, p.57
49
Teachers’ Perception on
Continuous School Improvement
Percentage Who Strongly Agree
53%
40% 39%
Southern
Regional
Education
Board
47%
40% 39%
Teachers and admin
work as team
2014
Use data
continuously
2012
43%
32% 33%
Intense Emphasis
High-Scoring Sites
Source: 2014 HSTW Assessment – Ohio Composite, p.57
50
2014 MMGW Teacher Survey Ohio Composite, p. 4
Activity Time
High School –
Leadership & Continuous
Improvement
 Report pp. 187, 198,
201, 210-213
 Handout #3
Middle School –
Leadership & Continuous
Improvement
 Teacher Survey
Report pp. 4 &16-19
 Handout #3
Southern
Regional
Education
Board
52
Engaging Instruction
…a key to ensuring rigorous and
challenging learning experiences
Southern
Regional
Education
Board
Key Practice
 Engage students in academic and
career/technical classrooms in rigorous
and challenging assignments using
research-based instructional strategies
and technology.
Southern
Regional
Education
Board
Four Types of Engagement
MMGW Student Report, pp. 33-40
 Intellectual - students work with new
concepts, explain their reasoning, defend
their conclusions and explore alternative
strategies. Students have confidence in their
ability to succeed and academic success is
important for future goals.
 Emotional – students have opportunities to
Southern
Regional
Education
Board
choose projects or areas of further study
related to their interests and goals. Students
who are engaged emotionally are able to
relate what they are learning to their own life.
Four Types of Engagement
MMGW Student Report, pp. 33-40
 Social – Student work in teams in class,
participate in extracurricular activities,
have friends at school, feel a sense of
loyalty and belonging to the school, and
believe in the legitimacy of school. Social
engagement motivates students to stay in
school.
 Behavioral - Students go to class
Southern
Regional
Education
Board
prepared and actively participate, seek
assistance when needed, and take
challenging classes.
SREB Literacy Goals
 Students will read the equivalent of 25
books per year across the curriculum.
 Students will write weekly in all classes.
 Students will use reading and writing
strategies to help them understand and
use the content of all classes.
 Students will write investigative research
papers in all classes.
 Students will be taught as if they were in
honors language arts classes.
Southern
Regional
Education
Board
Actions for Engaging
Students in Research-Based
Instructional Strategies






Southern
Regional
Education
Board
Project- or Problem-based learning
Cooperative learning
Student-designed research
Integrated, interdisciplinary studies
Integrating technology
Effective direct instruction
Literacy Across the Curriculum or Engaging Literacy
–Refer to p. 24 M.S and p.14 H.S. for complete lists
EXAMPLES OF INDICATORS
 They often revised their essays or other written work
several times to improve their quality. (Similar)
 They completed short-writing assignments of one to three
pages for which they received a grade in their English
classes at least monthly. (Similar)
 They read an assigned book and demonstrated
understanding of the significance of the main ideas at least
monthly. (Similar)
 They stood before the class and made an oral presentation
on a project or assignment to meet specific quality
requirements at least once a semester. (Similar)
Southern
Regional
Education
Board
Literacy Across the
Curriculum – H.S.
Percentage of Students Experiencing
an Intensive Emphasis on Literacy
Across the Curriculum
36%
24%
Ohio 2014
Southern
Regional
Education
Board
25%
Ohio 2012
High-Scoring Sites 2014
Source: 2014 HSTW Assessment – Ohio Composite, p.1
Engaging Literacy Across the
Curriculum – M.S.
Percentage of Students Experiencing
an Intensive Emphasis on Literacy
Across the Curriculum
43%
36%
Ohio 2014
Southern
Regional
Education
Board
34%
Ohio 2012
All Sites 2014
Source: 2014 MMGW Student Survey – Ohio Composite
Literacy Across the
Curriculum – H.S.
Percentage of Students Meeting Readiness
Goals by Level of Emphasis on Literacy
Across the Curriculum
64%
71%
51%
67% 70%
56%
65% 68%
52%
Low
Moderate
Intensive
Southern
Regional
Education
Board
Reading
Mathematics
Science
Source: 2014 HSTW Assessment – Ohio Composite, p.14
Literacy in All Classes
 Choose texts for students that are
demanding enough to help develop strong
reading skills
 Work with students directly in mastering
texts - this is how students learn how to
learn in each subject
 Students need more experiences in
reading complex materials, analyzing
texts, discussing what they read, and
writing reports based on their reading.
Southern
Regional
Education
Board
Activity Time - Literacy
High School – Engaging
Instruction
 Report pp. 47-48, 7784, 104 & 132
 HS Handout #4
Southern
Regional
Education
Board
Career Centers –
Quality CT; Integrating
Academics into CT ;
Work-Based Learning
 Report pp. see
workbook handout for
report pages
 CC Handout #5
Middle School –
Engaging Classroom
Practices
 Report pp. 22-24,
25-27, 28-30
 MS Handout #4
64
Engaging Numeracy Across the Curriculum
–Refer to p. 27-29 M.S and p.15 H.S. for complete lists
Southern
Regional
Education
Board
EXAMPLES OF INDICATORS
 They used math in classes other than mathematics at
least monthly. (Similar)
 Their mathematics teachers sometimes or often
showed how mathematics concepts are used to solve
problems in real-life situations. (Similar)
 They often developed and analyzed tables, charts and
graphs in their school work. (Similar)
 They solved mathematics problems other than those
found in the textbook at least monthly. (Similar)
 They used a graphing calculator to complete
mathematics assignments at least weekly. (Similar)
Numeracy Across the
Curriculum – H.S.
Percentage of Students Experiencing an
Intensive Emphasis on Numeracy Across
the Curriculum
53%
38%
Southern
Regional
Education
Board
Ohio 2014
40%
Ohio 2012
High-Scoring
Source: 2014 HSTW Assessment – Ohio Composite, p.1
Engaging Numeracy Across
the Curriculum – M.S.
Percentage of Students Experiencing an
Intensive Emphasis on Numeracy Across
the Curriculum
43%
38%
31%
Southern
Regional
Education
Board
Ohio 2014
Ohio 2012
All Sites
Source: 2014 MMGW Student Survey – Ohio Composite
Numeracy Across the
Curriculum – H.S.
Percentage of Students Meeting Readiness
Goals by Level of Emphasis on Numeracy
Across the Curriculum
73%
70%
62%
60%
39%
70%
59%
43%
38%
Low
Moderate
Intensive
Southern
Regional
Education
Board
Reading
Mathematics
Science
Source: 2014 HSTW Assessment – Ohio Composite, p.15
9 Actions to Build Students’
Math Abilities
Southern
Regional
Education
Board
 Develop students’ fluency in the language
of mathematics.
 Help students find a purpose for
mathematics.
 Teach students to organize and classify
mathematics information.
 Help students gain a thorough
understanding of fractions, decimals and
percentages.
 Use multiple representations of information.
9 Actions to Build Students’
Math Abilities (contd.)
 Develop students’ understanding of
appropriate applications of technology in
mathematics.
 Emphasize the role of mathematics in the
workplace and real world.
 Help students find a problem-solving
strategy.
 Foster students’ confidence in and
appreciation for mathematics.
Southern
Regional
Education
Board
Activity Time - Numeracy
High School – Engaging
Instruction
Southern
Regional
Education
Board
 Report pp. 48-49, 9295, 131
 HS Handout #4
Career Centers –
Quality CT; Integrating
Academics into CT ;
Work-Based Learning
 Report pp. see
workbook handout for
report pages
 CC Handout #5
Middle School –
Engaging Classroom
Practices
 Report pp. 22-24,
25-27, 28-30
 MS Handout #4
71
Engaging Science Curriculum and Instruction
–Refer to p. 30-32 M.S and p.16 H.S. for complete lists
Southern
Regional
Education
Board
EXAMPLES OF INDICATORS
 Their science teachers often showed how scientific
concepts are used to solve problems in real-life
situations. (Similar)
 They used science equipment to do science activities in
a classroom or laboratory at least weekly. (Similar)
 They used computers or technology to do science
activities at least monthly. (Similar)
 They prepared a written report of their lab results at
least monthly. (Similar)
 They participated in a classroom discussion relating
science to everyday life at least monthly. (Similar)
 They worked with other students in their class on a
challenging science assignment or project at least
monthly. (Similar)
Engaging Science – H.S.
Percentage of Students Experiencing
an Intensive Emphasis on Engaging
Science
21%
15%
Ohio 2014
Southern
Regional
Education
Board
16%
Ohio 2012
High-Scoring Sites
Source: 2014 HSTW Assessment – Ohio Composite, p.1
Engaging Science – M.S.
Percentage of Students Experiencing
an Intensive Emphasis on Engaging
Science
33%
31%
22%
Ohio 2014
Southern
Regional
Education
Board
Ohio 2012
All Sites
Source: 2014 MMGW Student Survey – Ohio Composite
Engaging Science – H.S.
Percentage of Students Meeting
Readiness Goals by Level of
Emphasis on Engaging Science
65% 68%
68% 68%
52%
49%
Reading
Southern
Regional
Education
Board
65% 67%
48%
Mathematics
Low
Moderate
Science
Intensive
Source: 2014 HSTW Assessment – Ohio Composite, p.16
How can you improve the
quality of instruction?
 Focus on improving professional
practice
 Common planning time
• Integrating
• Common unit planning, syllabi, exams, etc.
Southern
Regional
Education
Board





Peer observations
Demonstration classrooms
Professional development and follow-up
Team teaching
Teacher evaluation; annual plans for
improvement
Activity Time - Science
High School – Engaging
Instruction
Middle School –
Engaging Classroom
Practices
 Report pp. 49-50, 104-108
 HS Handout #4
 Report pp. 22-24,
Southern
Regional
Education
Board
Career Centers –
Quality CT; Integrating
Academics into CT ;
Work-Based Learning
 Report pp. see workbook
handout for report pages
 CC Handout #5
25-27, 28-30
 MS Handout #4
77
Share
 What does this type of data tell us
about quality instruction in our
schools?
 How can these data be used to
impact student achievement?
 What actions can be taken to
increase the quality of instruction?
Southern
Regional
Education
Board
Monitoring and Updating
School Improvement Plans
Southern
Regional
Education
Board
79
Data Tools
Compare your site 2012 & 2014 vs.
All Ohio Sites vs. All Network Sites
hstwohioregions.org
“Assessments” Tab
2014 Data Tools – Ohio High School Sites
or
2014 Data Tools – Ohio Middle Grade Sites
Southern
Regional
Education
Board
80
Improvement Plans
and Processes
Site Action Plans (SAP)
CCIP
OIP
Site Review Process (SRP)
Work from one plan!
Southern
Regional
Education
Board
81
Ohio Improvement Plan
Southern
Regional
Education
Board
SRP Seven Phase Process
Phase 1
Introduction to
SRP process
Phase 2
Phase 7
Prepare for the
SRP
Implement and
update SIP
Review Data
Site
Review
Process
Phase 6
Present
completed SRP
report/SIP plan to
staff
Complete DOP
Worksheets
Phase 5
Southern
Regional
Education
Board
Phase 3
Plan On-site
Review
Complete the
final SRP report
including SIP
Phase 4
Complete the Onsite review and
share exit
summary
83
Next Steps:
How to Use the Assessment Report
 Show the “positives”
 Identify the challenges
Southern
Regional
Education
Board
 Use state assessment & HSTW/MGA
report findings that explain why
 What actions can you take to change
that?
 Walk through the report
 Identify strengths and weaknesses
 Identify improvement actions
 Develop plan to share findings with the
faculty and other stakeholders
84
Next Steps:
Identify and Prioritize Actions
 List areas in need of improvement
 Rank items based on impact on student
achievement and completion rates.
 Identify short-term goals
 Implementation in the first year
 Have one item in each of the four areas of
structural, instructional, support and leadership
change
 Identify long-term goals
Southern
Regional
Education
Board
 Implementation in years two and three
 Have one item in each of the four areas of
structural, instructional, support and leadership
change
85
Essentials for Moving Forward
Southern
Regional
Education
Board
 Keep the improvement plan “alive”
 Keep staff informed and involved
 Establish necessary structures and
processes (committees or focus
teams, regular meeting schedules,
agendas, and meeting notes)
 Use resources from SREB, state, and
other sites
 Collect evidence to determine the
effectiveness of improvement efforts
 Make “mid-point” corrections and
revise improvement plan at least once
a year
 Develop teacher leaders to make this
happen!
86
Website information and workshop documents:
www.hstwohioregions.org
click “Assessments”
www.sreb.org
click “SREB Programs”
HSTW or MMGW
Southern
Regional
Education
Board
87
Contact Information
 Susanne Davis, HSTW SW Ohio
[email protected]
(513) 677-5641
 Mike Ross, HSTW SW Ohio
[email protected]
(937) 996-4694
 Missy Kramer, HSTW SW Ohio
[email protected]
(513) 619-2330
 Fran Cowart, SREB - HSTW
[email protected]
(404) 879-5611
Southern
Regional
Education
Board
 Jordan Hertl, SREB - MMGW
[email protected]
(404) 879-5575
88
Wrapping it all up-Remember…
All schools want to improve but few
want to change. The fact remains
that to improve, one MUST change.
Remember that if nothing changes,
nothing changes!
Southern
Regional
Education
Board
89