Transcript October 9 Data Workshop

Using Multiple Sources of Data
to Measure Success
December 15, 2009
School Improvement Webinar Series www.acteonline.org/multimedia.aspx
Your Moderator, Host and Presenter
Diana
Rogers


Regional
Coordinator
HSTW NE
Ohio Region
Catherine
Imperatore


Electronic
Media Manager
ACTE
Mike
Ross


HSTW/MMGW
School
Improvement
Coach
HSTW SW Ohio
Region
Sound Check


Can everyone hear me?
If not, please check the volume on your
computer
Questions

To ask about the content
type a question in the Q&A
panel and send to All
Panelists.
Questions will be
addressed at the end of
the presentation

For technical problems or
any other questions, type
in the Chat panel and
send to the Host.
Replay or Register for Webinars
School Improvement Webinar Series
Archived
Assessing Academic Rigor
Archived
Developing Effective School Improvement Teams
Archived
Motivating Students to Participate in Assessments
Jan 19, 2010
Developing a School-wide Literacy Plan
Feb 16, 2010
Establishing an Effective Advisor/Advisee Program
Mar 16, 2010
Developing a School-wide Numeracy Plan
Apr 13, 2010
Using the Technical Assistance Visit Report




Replay/register www.acteonline.org/multimedia.aspx
Invite your colleagues to register
Complete webinar survey
Graduate credit available
Why do we need multiple sources?



Single sources of data don’t provide us with
the complete picture!
Reliance on a single data source is
incomplete!
We need multiple sources of data to more
accurately identify causes of the problem and
to find more appropriate solutions.
Why do we sometimes have “tunnel
vision” when looking at data?



Emphasis on paper and pencil testing for
accountability
“Reaching the standard,” “making the cut-off
score,” etc. may blur our focus
Absence of a data culture…and resources
and time for creating one!
Data are critical
for all parts of the
improvement
process – from
initial needs
assessments to
monitoring progress
and evaluating
outcomes/results
A Data Continuum – Where on this
continuum is your school?
Don’t Know
“Sure we use data…
Data Rich
Data Smart
Don’t Care
I think.”
Information Poor
Information
Rich
Delving into Data

School culture is everything!
–
–
–
–
Data are accessible and usable.
Data are viewed as an invaluable resource for
improvement.
Data serve as a basis for inquiry, reflective
dialogue, problem-solving, and decisionmaking.
Discrepant data provide the “teachable
moment” in the school improvement process.
Bernhardt’s Suggested Use of Data





Replace hunches with facts concerning what
changes are needed
Identify the root causes of problems so we
can then solve the problems
Assess needs to target our services on
important issues
Know if goals are being accomplished
Determine if we are “walking our talk”
From the work of Victoria L. Bernhardt,
Data Analysis for Comprehensive School
Improvement, 1998, Eye on Education.
Quantitative versus Qualitative
•
Is one form of data better than the other?
•
What is the purpose for using the data?
•
Both forms can be effectively utilized!
Categories of Data
“Measures of student
learning help us
understand
how students are
performing and what
students
know as a result of
instruction.”
Student
Learning
From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.
Categories of Data
School
Processes
“…programs, practices, and
instructional
strategies…that produce
school and classroom
results.”
From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.
Categories of Data
Perceptions
“A particular view,
judgment, or appraisal
formed in the
Mind about a particular
matter...a belief stronger
than impression
And less strong than
positive knowledge.”
From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.
Categories of Data
“Statistical characteristics of
human populations…builds the
context of the school…for
which change is planned and
takes place.”
Demographics
From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Educatio
Categorizing Data




D – Demographic
P – Perception
SP – School Process
SL – Student Learning
From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.
What category?

An additional hour of extra help is available after
school this year.

Only 78% of boys scored at the proficient level in
reading as compared to 92% of girls.

For two years, “making inferences using non-fiction
passages” has been our lowest performing area.
Poll Activity
How would you categorize the following data:
D – Demographic, P – Perception, SP – School Process, SL – Student
Learning
1. Our math scores have declined during the past three years.
2. Students classified as low socioeconomic status score less
well on constructed response test items.
3. Last week, problem-based learning was observed in about
35% of the classrooms.
4. None of the students in the focus group discussion
mentioned project work as a “quality” learning experience.
5. Over 90% of parents said they were satisfied with the new
grading scale.
How may more than one category
interact for a better data analysis?



How do students who regularly use the writing lab
perform on their senior projects compared to those
who don’t?
Does the reading performance of males increase
with the number of hours intervention they have
experienced?
Do the parents of underperforming students have
confidence in the school’s enhanced intervention
programs?
Categories of Data - Interactions
School
Processes
Perceptions
Demographics
Student
Achievement
From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.
How Data Can Be Used





To guide improvement efforts!
Provide students with feedback on their performance
Gain common understanding of what quality
performance is and how close we are to achieving it
Measure program effectiveness
To understand if what we are doing is making a
difference
More Ways Data Can Be Used







Make sure students don’t “fall through the cracks”
Know which programs are getting the results we
want
To get to the real causes of problems
To guide curriculum development
Promote accountability
Meet state and federal requirements
(all from V. Bernhardt)
From the work of Victoria L. Bernhardt, Data Analysis for Comprehensive School Improvement, 1998, Eye on Education.
An “Overload” of Data???
•
OGT score rosters…frequency distributions…extra-help options and sign-in
sheets…quarterly failure reports… IPDP… ICP… IRN…disaggregated OGT
percent passing…subscale analysis…test item analysis…Student
Assessment…Teacher Survey…Annual Progress Report…Ohio School
report card… student grades… scheduling demand… similar district
comparison… course enrollment…curriculum mapping …attendance by
date…attendance by day of week… tardies… discipline…free and reduced
lunch…disability profile…categorical report…IEP achievement...parent
income level…drop-out rate…graduation rate…completer status…national
certification …health records…mobility…families on public
assistance…books read…credit deficiencies…teacher licensure
data…CCIP…program evaluation reports… student survey …
parent/community survey… open house attendance…parent conference
attendance…student participation in extra-curricular activities…safe school
survey…equity survey… harassment complaints… ACT…SAT… Board of
Regents remediation reports… suspensions… expulsions…gifted and
talented…advanced placement…post-graduate follow-up survey…percent
students entering post-secondary training…alternative program…reasons for
drop-out…level of technology
Sources of Data for Measuring
Performance and Practices









State Assessments
Teacher Assessments
Course Failure (ninth-grade)
ACT/SAT Results
Attendance Rates
Graduation Rates
Certification Exam Results
Post-Secondary Readiness
Assessing Readiness Practice
Sources of Data for Measuring
Performance and Practices









Instructional Review
Staff Experience Chart
Remedial Studies Reports
Follow-up studies
Drop-out exit reports
Master Schedule
Focus Group Interviews
Graduate Feedback
Assessing Practice
Compiling Data Sets




Collect related data appropriate to a
particular goal, objective or strategy
Organize data for review
Develop thoughtful questions for making
meaning
Draw conclusions
Example of Data Sets

Overriding Goal: To close achievement
gaps, meet state and federal
accountability requirements…to maintain
high expectations and extra help.
–
What data sets are appropriate for
addressing this goal?
Elements of Data Sets

Organize data for review
–
How should we best organize the data?
Elements of Data Sets

Overriding Goal: To close achievement
gaps, meet state and federal
accountability requirements…to maintain
high expectations and extra help.
–
What questions will help us to make the
data meaningful?
Elements of Data Sets

Overriding Goal: To close achievement
gaps, meet state and federal
accountability requirements…to maintain
high expectations and extra help.
–
What conclusions may be draw from
analyzing our data?
…on being the best professional
“Over everything else, all the tools they have in their toolboxes,
the difference between ineffective and effective teachers is
that effective teachers reflect on their teaching in meaningful
ways.”
“Ineffective teachers may work hard, but they don’t move along
the continuum of self-reflection from unaware, to
consciousness, to action, to refinement.”
- Pete Hall
ASCD 2005
• Outstanding Young Educator Awardee
Data Helps Us to Reflect



Effective teachers consciously use data as a source
of guidance and reflection.
The use of multiple sources of data provides a more
accurate and complete picture of performance and
effectiveness.
Make time for using data and reflecting on your
practice.
Recommended Resources
Book:
Victoria L. Bernhardt, Data Analysis for
Comprehensive School Improvement, 1998,
Eye on Education.
Website:
Common Core of Data, National Center for
Educational Statistics http://nces.ed.gov/ccd/
Questions
 To ask about the content
type a question in the Q&A
panel and send to All
Panelists.
Questions will be
addressed at this time
 Or an email response will be
sent to you after the
webinar.
Question

Do you have examples of data sets used
by schools to measure student success?
Question

What professional development is available
to assist school teams in learning more
about using multiple sources of data?
More Q & A

Questions and responses
Contact Information
If you have questions or would like to learn more
about using multiple sources of data to measure
success, please contact:
Mike Ross, School Improvement Consultant
 [email protected]
Next Webinar in the Series
Developing a School-wide Literacy Plan
Paulette Dewey, HSTW/MMGW Coach
January 19, 2010
from 11:30 – 12:30 ET
Thank you for participating!
Reminders…
 Register for future webinars or to view archived webinars at
www.acteonline.org/multimedia.aspx or
www.hstwohioregions.org
After leaving today’s webinar…
 Please complete the webinar survey.
 If you are interested in graduate credit, remember to print a copy
of the survey.
Please click the X to exit the webinar. Have a great day!