Transcript Slide 1

Looking Beyond
Traditional Sources
for School Data
The Ohio School Improvement Institute
November 15, 2007
Presented by: Mike Ross and Jim Posta
A Data Continuum –
Where on this continuum is your school?
Don’t Know
“Sure we use data…
Data Rich
Data Smart
Don’t Care
I think.”
Information Poor
Information Rich
An “Overload” of Data???
OGT score rosters…frequency distributions…extra-help options
and sign-in sheets…quarterly failure reports… IPDP… ICP…
IRN…disaggregated OGT percent passing…subscale analysis…test
item analysis…Student Assessment…Teacher Survey…Annual
Progress Report…Ohio School report card… student grades…
scheduling demand… similar district comparison… course
enrollment…curriculum mapping …attendance by
date…attendance by day of week… tardies… discipline…free and
reduced lunch…disability profile…categorical report…IEP
achievement...parent income level…drop-out rate…graduation
rate…completer status…national certification …health
records…mobility…families on public assistance…books
read…credit deficiencies…teacher licensure data…CCIP…program
evaluation reports… student survey … parent/community
survey… open house attendance…parent conference
attendance…student participation in extra-curricular
activities…safe school survey…equity survey… harassment
complaints… ACT…SAT… Board of Regents remediation
reports… suspensions… expulsions…gifted and
talented…advanced placement…post-graduate follow-up
survey…percent students entering post-secondary
training…alternative program…reasons for drop-out…level of
technology
How Schools Measure the Depth
of HSTW and MMGW Implementation
The HSTW and Middle Grades Assessment:
• NAEP or content standards – referenced
assessment in Reading, Mathematics and
Science
• Student survey of school and classroom
practices
• Student transcript analysis (HSTW)
• Faculty Survey
Annual Report
Technical Assistance Visit
Assessing Practice
Getting Started Questions
Why do we need to more
effectively use data at this school?
•
What are some examples of how
we are currently using data?
•
What additional data may help us
in our school improvement efforts?
•
Getting Started Questions
How may be cultivate more data
inquiry and dialogue at our school?
•
What are the obstacles to more
effectively using data?
•
What conditions can support our
efforts?
•
•
What benefits can we realize?
Familiar Sources of School Data
•Ohio
School Report Card
•Ohio Achievement Tests (OAT)
•Ohio Graduation Test (OGT)
•Certification Examinations – Career Tech
•Completer Status
•Graduation Rate
•Attendance
•Adequate Yearly Progress
•HSTW Assessment
•MGA – Middle Grades Assessment
Quantitative versus Qualitative
•
Is one form of data better than the
other?
•
What is the purpose for using the
data?
•
Both forms can be effectively utilized!
Beyond the Traditional Data Sources
•
•
•
•
•
ACT
College Remediation
Similar District Performance
Walk-Through’s
Questionnaires - Surveys
ACT
“ACT has identified the minimum score
needed on each ACT test to indicate a
50% change of obtaining a B or higher or
about a 75% chance of obtaining a C or
higher in the corresponding first-year
college course. To increase your college
readiness, consider taking additional
rigorous coursework before you enter
college.”
- ACT Website
From the 2007 ACT Rigor Report

“Today we have empirical evidence that
college readiness also means workplace
readiness. While not every student plans to
attend college after high school, many of the
jobs now being created in a highly
technology-based economy require a level
of knowledge and skills comparable to that
expected of a the first-year college student
(ACT,2006c). College and work readiness
should therefore be an expectation not only
for traditional college-bound high school
students, but for all students at the high
school level.”
Measuring the Rigor
of Your School’s Curriculum – Is It Rigorous?
ACT Benchmark
English
Math
Reading
Science
Readiness Scores
18
22
21
24
Note: These scores were empirically derived based on the
actual performance of students in college.
Focus: The number and percentage of students meeting or
exceeding ACT’s College Readiness Benchmark Scores (more
meaningful than focusing on composite scores only).
Measuring the Rigor
of Your School’s Curriculum – Is It Rigorous?
Limitations of Mean Scores
Core versus Non-Core
ACT Core
•English – At Least 4 Years
•Math – At Least 3 Years
•Social Studies – At Least 3 Years
•Natural Sciences – At Least 3 Years
Key Questions:
•Are your “core students” performing significantly better
than your “non-core” students?
•How does the performance of your “core students”
compare to the national average?
•How will you increase the numbers of students who take
the “core” and beyond “core” requirements?
ACT – High School Student Opinion Survey
Explores high school students’ perceptions of their
school’s programs, services, and environment.
Indicate your level of satisfaction with each of the
following characteristics of this high school. (very
satisfied to very dissatisfied)
•Quality
of Classroom Instruction
•Assistance provided in course selection and class
scheduling
•Laboratory facilities
•Grading policies and procedures
•Assistance provided in selecting and planning a
career
•Library or learning resources center
ACT – High School Student Opinion Survey
•Teachers
are willing to help students with their
studies outside of class.
•This school should offer more courses in which a
student can get some practical work experience.
•Teachers at this school try to provide a good
education for students.
•This school provides enough individual help for
students having trouble in courses.
•Students are encouraged to participate in
classroom activities.
•(Opportunity for schools to add additional
questions to the survey.)
College Freshmen Remediation
What percentage of your
graduates require remediation
(English or mathematics) at an
Ohio-supported college or technical
school?
•
•2005
Graduate data now available
•From
High Education Information (HEI)
through the Ohio Board of Regents
•Historical
data available back to 1998
College Freshmen Remediation
School
Took
ACT
Test
Rpt. ACT
Core Data
ACT Core
Not
Compl.
Central
City
H.S.
134
94%
23%
Ttl. Enroll.
Remedial Math
Remedial Eng
R. Math or Eng
R. Math & Eng
Ttl.
Enroll
Main
Campus
159
21%
14%
26%
9%
119
18%
9%
22%
6%
Source for Your School’s Data:
http://qry.regents.state.oh.lus/cgi-pub/site_map?name=rem_rep&jsflag=yes
Similar District Comparison
Similar district groupings allow
schools to compare their OAT/OGT
performance with schools most like
them based on:
•
•Number
of students (ADM)
•Poverty as a percentage of ADM
•Types of jobs and education level of population
•Agricultural property, population density, etc.
Similar District Comparison Report
Rank
IRN
District
County
ADM
0
45112
Wilmington City
Clinton
3,199
1
44602
Oregon City
Lucas
3,732
2
44784
Sidney City
Shelby
3,821
3
45773
Elida Local
Allen
2,398
4
44651
Port Clinton City
Ottawa
1,755
Seneca
2,888
(continues)
20
44891
Tiffin City
Source for Districts Similar to Your School District:
https//:webapp2.ode.state.oh.us/similar_districts/Similar_Districts.asp
Etc
Similar District Comparison
Use similar district comparisons:
Benchmark for higher levels of performance
on OAT and OGT (percentages at all levels of
proficiency).
Benchmark for higher attendance and
graduation rates.
Identify schools to contact and/or visit for
learning about new strategies and resources.
Replicate “best practices.”
Similar District Comparison
How to…
•Use
Excel or similar application
•Download
data from Ohio Dept. of Education
•Use
similar district list to flag the 19 districts most
like your district/parse data as necessary
•Insert
preferred calculation columns, etc.
•Sort/filter/report
•Distribute
as needed
compiled data to appropriate groups
•Analyze
and identify potential schools for follow-up
contacts and/or visits
•Source:
http://www.ilrc.ode.state.oh.us/Downloads.asp
Process Observation
Classroom Walk-Through
Goals of Classroom Walk-Through
Principal as “lead learner” (DuFour)
Frequent classroom visits by principal or
designated observer
Real-time data to make decisions for
school improvement planning
professional development planning
School-wide reflective practice
Increased student achievement
Process Observation
Classroom Walk-Through
Typical Classroom Walk-Through Model
•
Snapshot of Teaching and Learning – 3 to 5 min.
Identification of instructional strategies. level of
engagement, survey of learning environment,
curriculum alignment, focus, etc.
•
•
Capture observed data
•
Analysis of Information Collected
Reflection with Teacher and/or Staff (if not
personally identifiable)
•
Guides School Improvement!
Process Observation
Classroom Walk-Through
“Mining the Data”
“In a faculty meeting, I shared that the data
indicates that on average, the teachers are
using teacher-centered instructional strategies
almost 60% of the time, which is positively
correlated with the percent of time the students
…actively on-task around 40% of the time.”
-Alan
-Blog
Knoblauch
at leadertalk.org
Questionnaires – Untapped Source
Measure Effectiveness
Identify Potential Problems
Invite Feedback
Extra Help Program
Advisory Program
Literacy Requirements
Level of Engagement
Senior Project
Level of Student Expectations
Staff Awareness/Buy-In
Others?
…collecting data
to guide your school’s
improvement efforts
Questionnaires – All Kinds
Multiple Choice
Constructed
Response
Ranking
Rating
Yes/No
Nominal
Questionnaires – Power of Disaggregation
Separating results by groups that make up the
surveyed population.
•Gender
•Ethnicity
•Grade Level
•Previous School Attended
•Attendance
•Concentration
•Teaching Assignment/Content/Department
•Others?
“Disaggregation is not a
problem-solving strategy. It is
a problem-finding strategy.”
-Victoria Bernhardt
Questionnaires – Do’s and Don’ts
Do
Have
a clear purpose and state it
Use clear and concise questions
Arrange questions logically
Use items that allow for easy tabulation
Don’t
Overwhelm
the respondent
“Lead” the respondent
Use jargon or confusing terminology
Unrealistic timeline for returning
Sample Questionnaire
These questions will help us to improve our school. Answer
them honestly. There is not a right or wrong answer.
Just think about this quarter when you answer the
questions.
1. How often did you complete short writing assignments of
one to three pages that were graded in your English
class?
A. Weekly or several times a week
B. Once or twice each month
C. At least once during the quarter
D. Didn’t do this
Note: Item could be repeated for social studies, math,
science, etc.
Sample Questionnaire
2. How often were you asked to revise your writing by a
teacher at this school?
A. Weekly or several times a week
B. Once or twice each month
C. At least once during the quarter
D. Didn’t do this
3. How often did you voluntarily use the after-school
help center?
A. Weekly or several times a week
B. One to three times each month
C. At least once during the quarter
D. Didn’t do this
Sample Questionnaire
4. How often were you required by a teacher to attend
the after-school help center?
A. Weekly or several times a week
B. At least once each month
C. At least once during the quarter
D. Didn’t do this
5. How often did you have to answer a short answer or
extended response question on a quiz or test?
A. Weekly or several times a week
B. Once or twice a month
C. At least once during the quarter
D. Didn’t do this
Resourceful Books
Data Analysis for Comprehensive Schoolwide
Improvement by Victoria L. Bernhardt, Eye on
Education, 1998.

Getting Excited about Data: Combining
People, Passion, and Proof to Maximize Student
Achievement by Edie L. Holcomb, Corwin Press.

The Three-Minute Classroom Walk-Through:
Changing School Supervisory Practice One
Teacher at a Time by Carolyn J. Downey, SAGE
Publishing, 2004.

…contact information
Jim Posta
HSTW/MMGW
NW Ohio
(419) 530-7220
[email protected]
Mike Ross
HSTW/MMGW
SW Ohio
(937) 996-4694
[email protected]
Use Data!