Transcript Webinar Title - ISFIS Iowa School Finance Information Services

September 18, 2014 Special Topics
Webinars: School Improvement
Margaret Buckton, Partner
Susie Olesen, School Improvement Enthusiast
Assessment and Data
“Data. Data. Data. I can’t make bricks
without clay!” Sir Arthur Conan Doyle
© Iowa School Finance Information Services, 2014
1
Webinar Reminders
• Update us with your email address [email protected]
• PowerPoint on ISFIS web site at
http://sites.google.com/site/iowaschoolfinance/Home/webinar-recordings
• Power Point on Skills Iowa web site at http://www.skillsiowa.org/?q=PL
• Use question pane to pose questions
• Ask questions. We will find the answer if we don’t know it today. If
we don’t answer during the Webinar, we’ll get back to you.
• This series of 9 webinars pairs with ISFIS conference for one credit
hour (15 hours of content) for administrator license renewal
• This series of 8 webinars pairs with 1 day at the ISFIS office on
September 10 for one hour of admin. license renewal
Dates, Topics and Links to Register
•
•
•
•
•
Thursday, July 17, 2014, 9 AM – What’s happening in school with students?
Thursday July 31, 2014, 9 AM –What’s happening with teachers?
Thursday, August 14, 2014, 9AM – What’s happening with school leaders?
Thursday, August 28, 2014 9 AM – Professional Development
Thursday, September 4, 2014 9 AM – Collaboration
• Thursday, September 18, 2014, 9 AM –
Assessment and Data
• Thursday, October 2, 2014, 9 AM – TLC Model
• Thursday, October 16, 2014, 9 AM – TLC Model
• Thursday, October 30, 2014, 9 AM – What’s next in my school?
3
Using Webinar Information Later
• PPT, Recording and related tools posted on the
Webinar Page and also the Skills Iowa professional
leaning page: http://www.skillsiowa.org/?q=PL
• Itemized list of contents is searchable. Find what you
need when you need it via Google search box
• Use PPT or information with leadership teams or with
PLCs or data teams to get the conversation going
• Content for school improvement meetings
• Or shoot us an email and we’ll send you what you
need.
4
Agenda
• Why assess?
• Uses of Data in the School Improvement Process
–
–
–
–
–
–
How are students performing?
What is going on in classrooms?
Set goals for student performance
Select content for instructional improvement
Monitor ongoing student learning (formative)
Monitor implementation of instructional improvement
(implementation)
– Program Evaluation (Summative Data)
• Other Uses of Data Related to Improvement
5
Quotes on data brought to you by:
Web Analytics Action Hero, 31 Essential Quotes
on Analytics and Data
http://www.analyticshero.com/2012/10/25/31-essentialquotes-on-analytics-and-data/
The Data Science of Digital Marketing, Spinnakr Blog
http://spinnakr.com/blog/data-2/2013/03/44-more-ofthe-best-data-quotes/
• “Facts do not cease to exist because
they are ignored.” Aldous Huxley
• “Anything that is measured and watched
improves.” – Bob Parsons
• “We are drowning in information and starving
for knowledge.” – Rutherford D. Rogers
• “True genius resides in the capacity for
evaluation of uncertain, hazardous, and
conflicting information.” – Winston Churchill
Why do schools spend time and
money testing students? Put some of
your thoughts in the chat pane. . . . .
Why Assess?
The Nation and the State:
• Confirm that all kids have access to a great education
(insure equity)
• To prove to taxpayers and constituents that schools are
doing the job . . .or
• To prove to taxpayers and constituents that schools aren’t
doing the job
• To justify expenditures and direct resources
• To change behavior and beliefs (prove examples of what’s
possible)
• To comply with mandated data collection requirements so
state can draw down federal $$
“In God we trust, all others must bring data.” W. Edwards
Deming
Why Assess?
Districts and Schools
• To comply with state and federal law
• To get information to change behavior
– Change instruction based on what kids know and can do
– Determine what teachers need to learn more about
– Avoid wasting time– focus teaching on unmastered skills
and challenge students
– Confirm that all kids have access to a great education
(insure equity)
– Insure student success
“The price of light is less than the cost of darkness.”
Arthur C Nielsen
Required Assessments
• Iowa Tests as part of Iowa’s NCLB plan (3rd-11th grade; math,
reading, science) proficiency at 41st Percentile.
– Current plan does not report student growth
– Iowa is unlikely to obtain federal waiver since student achievement (test
scores) are not part of Iowa’s teacher evaluation system.
– Final recommendation of Council on Educator Development are due
November 2016
• District must have a second assessment as part of the NCLB
compliance plan (included in CSIP)
• NAEP – some districts are chosen to participate in some grades
• All districts must have some assessment for an Early Warning
System for early elementary (could be FAST)
• Teaching Strategies GOLD Early Childhood Assessment to PK
students and a valid and reliable universal screening instrument,
as prescribed by the DE, to every kindergarten student by Oct. 1
“A point of view can be a dangerous luxury when substituted for
insight and understanding.” Marshall McLuhan, Canadian
Communications Professor
Quotable
• “…(A)nother important way to understand the different uses
and impacts of assessment is to see the assessment system as
a structure which both provides information and influences
what people do.”
The Teaching and Learning Research Programme
• “….(F)ormative and summative are not labels for different
types or forms of assessment but rather describe how
assessments are used.”
The Teaching and Learning Research Programme
http://www.tlrp.org/
12
More…
• “We need to get the word out to the nation’s teachers
that formative assessment is capable of triggering big
boosts in students’ achievement—the educational
equivalent of a cure for the common cold.”
James Popham, Professor Emeritus, UCLA
• "We have to think about accountability in a very different way.
We have done a splendid job of holding nine-year-olds
accountable. Let me suggest as a moral principle that we dare
not hold kids any more accountable than we expect to hold
ourselves."
Doug Reeves, Founder, Leadership and Learning Center
13
Where are we?
Student Performance
• How are our students doing in various, specific areas?
–
–
–
–
–
–
–
State Tests
Interim Assessments
Common Formative Assessments
Attendance
Graduation rates
Surveys
Engagement
“Not everything that can be counted counts, and not
everything that counts can be counted.” Albert Einstein,
Physicist
What kind of classroom environment are our
students experiencing?
•
What are students doing?
–
–
–
–
–
–
•
What are teachers doing?
–
–
–
–
–
–
–
•
Reading?
Writing?
Interacting?
Working in groups?
Working alone?
On computers doing what?
Interacting with students around content?
Asking questions? What kind?
Monitoring student work?
Modeling expected learning/behavior?
Email ?
Lecturing?
Working at their desks?
How are our classrooms organized; what do we see?
– Books and other text for reading?
– Student work displayed?
– Places to meet together? Places to get individual work done?
•
What are we teaching?
– How long for various subjects?
– Interdisciplinary work?
– Higher order thinking and more concrete factual content?
•
How are we assessing?
Set goals for student performance:
• What is our biggest area of need? Be as specific as
possible (reading comprehension, math problem
solving, informational writing, etc.)
• Look at data that helps everyone understand what’s
possible.
• Set long term and short term goals related to student
performance.
• Focus. Don’t try to do it all at once.
• Determine how you’ll measure the goals. Remember,
the state test alone isn’t enough. Need multiple
measures.
“Errors using inadequate data are much
less than those using no data at all.”
Charles Babbage
Select content for improving instruction.
• Be specific and focused about where you want to see student
improvement (writing informational text, reading comprehension,
math problem solving, science heuristics, etc.)
• Scan the literature about what kind of instruction can change
student performance in the area you wish to improve. (Ask 3-6
experts in the area you wish to improve to recommend 3 or 4 good
studies related to your area of improvement. See if there are
common recommendations.)
• Seek technical assistance. Schools can rarely do this alone. (AEAs,
consultants you know and trust, etc.)
• Determine what new instruction you want everyone to learn. Be
sure to be clear about:
– The moves of the new instruction
– The frequency and fidelity of implementation required for student
learning results.
• Embed learning in on-going PD system that ensures the support
needed for implementation with fidelity
Monitor Ongoing Student Learning
• Determine how you will regularly monitor
student learning and respond instructionally
– CFAs
• Student Writing
– Daily work
• PWIM words
– Interim assessments
• Skills Iowa principal
“Never confuse motion with action.”
Benjamin Franklin
Monitor Instructional Initiative
• Observations by other teachers, administrators, coaches –
expert and peer
–
–
–
–
–
Video
Rehearsals in collaborative teams
With students
Other teachers
Administrators
• Self reports like logs that help guide the work
• Walk throughs
• Listening – discussions of how it went, problems
encountered, how students responded
“Without big data analytics, companies are blind and deaf,
wandering out onto the Web like deer on a freeway.”
Geoffrey Moore, Author of Crossing the Chasm & Inside the
Tornado.
Program Evaluation
• Summative (pre-program benchmark and postprogram measure)
• Need multiple measures
• Determination isn’t always an affirmative yes or
no – could be “Those who implemented with
fidelity saw significant gains. Need to determine
how to elevate fidelity and frequency for all staff.”
“If we have data, let’s look at data. If all we
have are opinions, let’s go with mine.” – Jim
Barksdale, former Netscape CEO
• Where do we go next?
Do any of these processes align to a
focus area in your school? Which
processes do you want to be sure to
include in your future work?
Principal Use of Teacher Effectiveness Measures
for Talent Management Decisions
Gates Study at Vanderbilt
http://principaldatause.org/assets/files/presentations/Gates_Convening-Opening_Presentation-201405.pdf
Questions Study Examined
• How do principals perceive the quality of data
systems and data access that help them make
talent management decisions?
• How do principals use teacher effectiveness data
for talent management decisions?
• How can school systems support data use for
talent management decision making?
• What training and supports do principals value to
help them learn to use teacher effectiveness
data?
Who participated in this
nationwide study?
•
•
•
•
100 + central office people interviewed
78 principals interviewed
795 principal surveys, 82% return rate
4 in depth case studies
Findings
• Observation systems drive principals’ use of data
• Strong, ongoing calibration of observation scores seems to
increase use of observation data
• Value-added measures perceived as having many shortcoming
• Principals rarely use stakeholder feedback surveys or past
teacher evaluation measures
Problems Principals See with Value Added
• Timing (i.e., results are not available in real time when decisions are
made);
• Perceptions of validity (i.e., scores do not exist for untested
subjects, students are taught by multiple teachers);
• Specificity (i.e., the scores do not provide a window into what
teachers actually do that directly impacts students’ learning, the
measures are not fine-grained and actionable);
• Transparency and complexity (i.e., lack of understanding for both
school leaders and teachers about complicated statistical models
negates comfort with the results and their interpretation. Users
need clarity in how the measures were created).
“If you can’t explain it simply, you don’t understand it well enough.”
Albert Einstein, Physicist
UEN Response to Value Added Assessment Critique, April 24, 2013
Hiring
• Few principals use multiple teacher effectiveness
measures for hiring
• Individual principals—even within a single school
system—vary in their awareness of available data
that could point them to quality teacher
candidates or of data sources for hiring.
“I never guess. It is a capital mistake to theorize
before one has data. Insensibly one begins to twist
facts to suit theories, instead of theories to suit
facts.” Sir Arthur Conan Doyle
Recommendations:
• Screen applicants with a rubric aligned to
evaluation framework
• Provide principals with access to internal
transfers’ effectiveness data
• Require a demonstration lesson and feedback
aligned with observation rubric
• Maintain hiring information to compare with
future outcomes
Data Use for Teacher Assignment
• Some principals use data to place teachers where
they can be most effective.
• Many do not view assignments strategically or
think of data as informative in the assignment
process.
• Many also feel constrained in their ability to
effectively assign teachers.
• Principals use data to move less effective
teachers into grade levels that aren’t tested
“If you torture the data long enough, it will
confess.” Ronald Coase, Economist
Data Use for Teacher Support and PD
• Observation data help principals to have
constructive teacher feedback conversations.
• Principals rely almost entirely on observation
data for professional development decisions.
• Principals draw upon both school- and systemlevel resources to support teachers.
Recommendations
• Develop teacher professional development resources
that align with specific indicators on the instructional
framework or rubric.
• Monitor teachers’ use of professional development
resources, ask participants for feedback, and track
professional growth over time.
• Train and support principals to have honest, clear, and
sometimes difficult conversations about teacher
performance.
“Once we know something, we find it hard to imagine
what it was like not to know it.” Chip and Dan Heath,
Authors of Made to Stick, Switch
Data Use for Termination
• The late timing with which teacher effectiveness data become
available impedes principals’ use of these data for contract
decisions.
• An inability to gather enough documentation and lack of time
to do so are challenges.
• A lack of central/home office support is a major barrier to
dismissal.
Recommendations:
• Train principals on what constitutes effective
documentation, how to avoid potential pitfalls in the
process, and where to find resources to help them sort
through evidence as they create the file for dismissal.
• Support principals in gathering the appropriate
documentation that will hold up to official grievance
processes. Streamline principals’ efforts in gathering a
body of evidence for teacher dismissal by providing
direct support with the process.
• Train principals to use multiple years of teacher
effectiveness data in making renewal decisions.
Principals want….
Overall Recommendations…
Sum it Up
• “What gets measured gets managed.” Peter
Drucker
• “War is 90% information” Napoleon
Bonaparte
• “He uses statistics as a drunken man uses
lamp posts – for support rather than for
illumination.” Andrew Lang, Scottish Writer
Any insights related to data in how your district
supports principals?
Assignment Details for Recertification
Credit
It’s likely that you use some or many of the data practices
discussed in this Webinar today, and also probable that you
don’t use them all. (Warning - that’s just our guess, as we have
no data to back that up.)
• Write a paragraph identifying data practices in your district
that are useful and how they move your improvement work
forward.
• Then write another paragraph on data practices you heard
about today that your district doesn’t use. Could they help
move your district forward? Why or why not?
38
Finish with these:
“Statistics are like bikinis. What they reveal is suggestive,
but what they conceal is vital.” Aaron Levenstein,
Business Prof at Baruch College
“The alchemists in their search for gold discovered many
other things of greater value.” Arthur Schopenhauer,
German Philosopher
“The greatest value of a picture is when it forces us to
notice what we never expected to see.” John Tukey,
American Mathematician
“An intelligent person is never afraid or ashamed to find
errors in his understanding of things.” – Bryant H. McGill
Additional Resource
Without an effective system of curriculum and assessment, we are all a
little like Alice.
“Would you tell me, please, which way I ought to go from here?”
“That depends a good deal on where you want to get to.”
“I don’t much care where –”
“Then it doesn’t matter which way you go.”
-Lewis Carroll, Alice in Wonderland
Access the Booster on Assessment on the publications tab of the Skills Iowa
Web site here: http://www.skillsiowa.org/?q=publications
Questions or Comments?
Margaret Buckton , ISFIS – Partner
Cell: 515-201-3755
[email protected]
Susie Olesen, ISFIS School
Improvement Enthusiast
Cell: 641-745-5284
[email protected]
Iowa School Finance Information Services
1201 63rd Street
Des Moines, IA 50311
Office: 515-251-5970
www.isfis.net
41