Transcript Why did we choose these Direct Instruction programs?

What does evidence-based practice
mean?
Dr Kerry Hempenstall, RMIT University
What determines professional practices?
• Superstition?
• Folklore?
• Craft?
• Science?
Superstition
Unsubstantiated world views derived from
intuition or legend.
Galileo first observed moons around Jupiter in 1610.
His observations were derided.
Francesco Sizi’s refutation was: There are seven
windows in the head, two nostrils, two ears, two eyes
and a mouth.
So in the heavens there are seven - two favourable
stars, two unpropitious, two luminaries, and Mercury
alone undecided and indifferent.
From which, and many other similar phenomena of
nature, such as the seven metals, etc we gather that
the number of planets is necessarily seven...
We divide the week into seven days, and have named
them from the seven planets. Now if we increase the
number of planets, this whole system falls to the
ground...
Moreover, the satellites are invisible to the naked eye
and therefore can have no influence on the earth and
therefore would be useless and therefore do not
exist.
Folk beliefs, correlations, anecdotes, intuition
If a mother carries a child higher in
the abdomen, she will have a girl, and if
lower, a boy.
Spare the rod, spoil the child
Teachers are urged to attend to
children’s learning styles in their
instruction.
Craft
Traditionally, many disciplines were based on
observations, knowledge, and experience.
You practise the way you were trained, and adapt
practice according to clinical experience.
You may learn from colleagues, but an absence
of systematic questioning of efficacy.
Practices are time honoured.
Science
Until recent times, research findings have had
little impact on medical practice.
For example, in mid 19thC, Lister showed how
bacterial infection occurred. No impact on
surgical cleanliness for more than 50 years.
Doctor Spock‘s 1940’s advice to have infants
sleep face down was associated with 60,000
deaths from SIDS between 1974 and 1991. The
risk was first identified in 1970, but ignored
until the 1990’s.
History of evidence-based practice
In medicine, early 1990’s
EBM is “the conscientious, explicit and judicious
use of current best evidence in making
decisions about the care of individual patients.
The practice of evidence-based medicine
means integrating individual clinical experience
with the best available external clinical
evidence from systematic research (Sackett,
1996).
History of evidence-based practice
In medicine, psychology, and numerous other
disciplines, randomized controlled trials have
become the gold standard for evaluating an
intervention’s effectiveness.
This type of study is the least likely to suffer from
threats to validity and reliability
Training courses in these professions include a
strong emphasis on empirical research design.
Science impact on other professions
“A mature profession . . . is characterized by a shift
from judgments of individual experts to judgments
constrained by quantified data that can be inspected
by a broad audience, less emphasis on personal trust
and more on objectivity, and a greater role for
standardized measures and procedures informed by
scientific investigations that use control groups”
(Carnine, 2000).
“...if you think you know the truth without having to
collect any data, that saves a lot of time” (Stanovich,
2000).
EBP in many fields now strongly promoted.
“Those fields that have displayed unprecedented development
over the last century, such as medicine, technology,
transportation, and agriculture have been those embracing
research as the prime determinant of practice”
(Shavelson & Towne, 2002).
Education is moving in this direction:
The Australian Government’s Review of Funding for Schooling
Panel (2011) bemoaned the current lack of evidence-basis for
educational programs and the absence of evaluation of the
programs’ effects on learning
The 2005 Australian National Inquiry into the Teaching of
Literacy used the term evidence-based 48 times among its
recommendations
EBP now strongly promoted in many fields
“Those fields that have displayed unprecedented development
over the last century, such as medicine, technology,
transportation, and agriculture have been those embracing
research as the prime determinant of practice”
(Shavelson & Towne, 2002).
Education is moving in this direction:
The Australian Government’s Review of Funding for Schooling
Panel (2011) bemoaned the current lack of evidence-basis for
educational programs and the absence of evaluation of the
programs’ effects on learning
The 2005 Australian National Inquiry into the Teaching of
Literacy used the term evidence-based 48 times among its
recommendations
From the Australian Government’s very recent (Sep, 2012)
Response to recommendations of the Dyslexia Working Party
Report ‘Helping people with dyslexia: A national action
agenda’.
•
It should be ensured that appropriate teaching strategies,
shown through rigorous, evidence based research to be
effective in developing strong literacy skills, are used in all
Australian Junior Primary classrooms. This will assist in
reducing the impact of dyslexia significantly.
•
All teacher-training syllabi should include training in evidencebased reading instruction.
Recognition and Response (RnR)
“The (QLD) Department of Education and
Training (DET) is committed to ongoing
reviews of emerging evidence-based practice
to ensure that all students, including
students with disabilities, can access,
participate and succeed in education on the
same basis as other students”.
USA Federal edict
In the USA in May 2012, the Office of
Management and Budget sent out a memo that
could change history.
The memo requests the entire Executive Branch
of government to use every available means to
promote the use of rigorous evidence in
decision-making, program administration, and
planning across the nation.
Hot topic: Try Googling evidence-
based (your profession)
•
•
•
•
•
•
•
•
•
•
Evidence-based
Evidence-based
Evidence-based
Evidence-based
Evidence Based
Evidence-based
Evidence-based
Evidence-based
Evidence-based
Evidence-based
speech pathology
occupational therapy
psychology
design
Library and Information Practice
management
medicine
nursing
pharmacy
education?
How can practitioners access the
findings?
The greatest achievement of
evidence-based approaches has
been the development of
systematic reviews and
meta-analyses (aggregated studies).
These enable easier access to what works.
Popular beliefs negated by literacy research
• Learning to read is as natural as learning to
speak
• Children do not learn to read in order to be
able to read a book, they learn to read by
reading books
• Parents reading to children is sufficient to
evoke reading
Popular beliefs negated by literacy research
• Good readers skim over words rather than attend to
detail
• Skilled reading involves prediction from context
• English is too irregular for phonics to be helpful
• Accuracy is not necessary for effective reading
• Good spelling develops simply from the act of writing
Visible learning: A synthesis of over 800 metaanalyses relating to achievement (Hattie, 2009).
Over 50,000 studies analysed
Percentage of Achievement Variance
Teachers
> 30%
Students
~50%
Home
Peers
Schools
Principal
~5-10%
~5-10%
Variance in student achievement
Hattie (2009) ranks 138 variables on effectiveness –
Effect Size (ES) greater than 0.4 is desirable
Ineffective
• Mobility (shifting schools) -0.34
• Retention -0.16
• Television -0.14
• Summer holiday -0.09
• Ability grouping 0.11
• Diet 0.12
• Whole language 0.06
• Perceptual motor programs 0.08
• Team teaching 0.19
• Web based learning 0.18
• Problem based learning 0.15
• Class size 0.21
• Individualised instruction (learning styles) 0.22
Not a great effect
• Mainstreaming 0.28
• Exercise/relaxation programs 0.28
• Homework 0.29
• Computer assisted instruction 0.27
• Cooperative learning 0.41
• Social skills programs 0.40
Significant effects
• Expectations 0.43
• Self concept 0.43
• Early intervention 0.47
• Small group learning 0.49
• Classroom management 0.52
• Home environment 0.57
• Socioeconomic status 0.57
The strongest effects
• Cooperative vs. individualistic learning 0.59
• Study skills 0.59
• Direct Instruction 0.59
• Phonics instruction 0.58
• Comprehension programs 0.58
• Professional development 0.62
• Self-verbalization & Self-questioning 0.64
• Repeated reading programs 0.67
• Prior achievement 0.67
• Vocabulary programs 0.67
• Meta-cognitive strategies 0.69
• Spaced vs. Massed practice 0.71
• Teacher-Student relationships 0.72
• Feedback 0.73
• Reciprocal teaching 0.74
• Comprehensive interventions for learning
disabled students 0.77
• Classroom behaviour programs 0.80
• Acceleration 0.88
• Providing formative evaluation 0.90
• Response to Intervention model 1.07
• Self-reported grades 1.44
H
a
t
Controversial evidence-free treatments for LD
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Megavitamins
Allergy treatments
Chinese herbs
Evening Primrose oil, Vitamin A
Chiropractic, Osteopathy
Cranial Sacral Therapy
Stimulants
Tranquillizers
Anti-histamines
Anti-convulsants
Dolphin therapy
Coloured (Irlen) lenses
Brain-based programs, e.g., Dore, Arrowsmith, Brain Gym, Doman Delacato
Sensory Integration
Behavioural Optometry
Caveats on EBP
• EBP only effective if implemented with fidelity
• Not all questions have clear cut evidence-based
answers – and not all studies are well designed
• Absence of evidence is not evidence of absence
• Need to be prepared to surrender your professional
beliefs – challenging!
• Most practitioners require new training
• The larger the implementation the more difficult is
sustained success.
Where to find out what works?
Find Evidence-Based Practice Resources at
http://www.nectac.org/topics/evbased/evbased.asp
e.g., Best Evidence Encyclopedia, Promising Practices Network, What Works
Clearinghouse
•
•
•
•
•
•
•
•
•
•
Some of the Topics
Autism
Early Child Identification
Early Childhood Transition
Early Literacy
Home Visiting
Inclusion
Professional Development
School Readiness
Screening, Assessment, Evaluation Instruments
Social Emotional Intervention for Young Children
An example
Problem: Yr 7 struggling readers
Program chosen: Corrective Reading Program. Why?
Hattie’s (2009) meta-analysis finds:
Direct Instruction effect size: 0.59
Phonics effect size: 0.58
Program Components include:
Spaced vs. Mass Practice: 0.71
Repeated Reading: 0.67
Formative evaluation: 0.90
Teacher Clarity: 0.75
Feedback: 0.73
Micro Teaching: 0.88
My Initial Teaching Wishlist
1. Introduce RTI - early screening of beginning
students to identify those at-risk
2. Teach phonemic awareness informally in
Kindergarten (or even in preschool)
3. Provide intensive phonologically based
teaching to those identified, and to those not
progressing
4. Continue to monitor these students’ progress
5. Assess all students in school (primary and
secondary) with mass screening (e.g., PAT test)
6. Provide daily reading instruction for needy
students from Year 3 and above with Corrective
Reading program (Dec & Comprehension)