Transcript Thesis Prospectus

Thesis Defense
A Meta-Analysis of Adventure
Therapy Program Outcomes
Submitted in partial fulfillment of the degree
Master of Science: KinesiologyOutdoor Education
By
Norman Staunton
Committee Members
• Chair:
– Dr. Michael A. Gass, Ph.D.
• Members:
– Dr. Deborah Sugerman, Ph.D.
– Dr. Jason Bocarro, Ph.D.
• Advisor:
– Mr. James Neill
MANY THANKS TO THEM ALL!
Gillis (1992)
“Someone needs to conduct a meta-analysis
on [the] therapeutic aspect[s] of
adventure-challenge-outdoor-wilderness
that includes the criteria of clinical
significance along with traditional methods
of effect size”
Gillis (2000)
“To the best of my knowledge such a study
has not been done.”
As of December of 2002 no such study
had been completed, and it represented a
significant gap in the body of AT research.
This study fills that gap by conducting
such a meta-analysis.
Reasons for conducting this study:
• Not a duplication of previous research- it
provides new information to the field
• Represents a significant contribution to the
field- it fills a significant gap of needed
information
• Provides documentation of collective
efficacy- which is necessary for future
funding, program design, research design
Question I:
• What are the effects of Adventure
Therapy?
– How do the effects of Adventure Therapy
compare to Adventure Education?
– How do the effects of Adventure Therapy
compare to other forms of therapy?
Question II:
• What are the major correlates of
Adventure Therapy program outcomes?
Such as for:
–Programming (e.g. length, type, status)
–Group (e.g. social support, cohesion)
–Client (e.g. gender, age, diagnosis)
–Outcome (e.g. clinical, affective, attitude)
Definitions
• Adventure Therapy– Is programming aimed at changing [specified]
dysfunctional behavior patterns, using
adventure experiences as forms of habilitation
and rehabilitation." (Priest and Gass 1997).
For our purposes, this includes any adventure
experience with diagnosed clients or reporting
a specified therapeutic outcome.
Definitions
• Meta-analysis– A statistical technique for accumulating and
representing the research results reported in
multiple studies. Meta-analyses pool the
findings about a single research question from
many different sources and analyze the
overall effects.
Definitions
• Effect Size– Is a statistical standardized coefficient that
allows for comparisons of changes in means
over time across studies. ES is calculated by
the equation:
(M1-M2)/SDpooled
Practical Basis of Methodology
• Based on the previous OE meta-analyses
• Cason (1993)- AE outcomes on adolescents
• Hattie, Marsh, Neill & Richards (1997)- AE/ OB
outcomes for all populations
• Marsh (1999)- Outcomes of camp experiences
• Hans (2000)- AE effects on locus of control
Practical Basis of Methodology
This study departed from earlier OE metaanalyses by:
• Focusing specifically on AT and
therapeutic outcomes
• Looks outside of OE for applicable
research
• Being up to date
Methodology
• Study Methodology is divided into four
sections:
• Search Methods
• Selection Criteria
• Coding Methods
• Statistical Methods
Search Methods
• Searches were made in six categories:
• Electronic Databases
• Bibliographies
• Top AT/OE Web Sites
• Commonly Known Research
• Email Listserv Requests
• Existing Data
These methods returned over 2000 hits with
over 600 unique articles
Selection Criteria
Studies were included based on the
following criteria:
• Empirically-based and likely reporting
descriptive statistics
• Adventure-based intervention with
diagnosed populations
Studies meeting these criteria were actively
sought by the researcher
Selection Criteria
Studies were further included based on:
• Further investigation revealed the first
criteria were correct
• 75 articles passed the third criteria. 22
were unavailable, taking the effective
number of studies to 53.
• Analysis during coding eliminated 36
studies. 17 studies remained.
Coding Methodology
Primary Variable Codes:
Year of Study
Authors
Institution
Country
Publication Source
Type of Program
Outcomes Measure
Quality of Measure
Quality of Study
Pre M and SD
1st day M & SD
Last day M & SD
Follow up M & SD
(M & SD calculated as needed)
Secondary Variable Codes
Hierarchy
\PROGRAM/
Type, Modality, Delivery Format, Duration, Daily Duration
\GROUP/
Staff, Retention, Length of Program, Age, Gender
\INDIVIDUAL/
Age, Gender, Category, Diagnosis
\OUTCOME/
Behavioral, Attitudinal, Affective, Physical, Cognitive
Statistical Methods
• Effect Sizes (ES)
• ESs were calculated, then corrected to account for
their relative strength.
• Correlations
• Relationship between effect sizes and primary and
secondary variables were analyzed using
univariate analysis. 95% confidence intervals were
used to compare significant differences from zero
and between variables.
Statistical Methods
• Regression Analysis
– Regression analysis was conducted until all
returned correlates of variance were
significant at the p=.05 level
• File Drawer Effect
– Rosenthal’s File Drawer Effect was calculated
to estimate the number of negative or nonsignificant effects needed to negate the
findings.
Results
•
•
•
•
•
17 Studies were included
95 Effect sizes were calculated (91/4)
No negative effect sizes were found
Overall WES of .53 found
.53 for immediate effects, .35 for follow-up
Results
ID
Author(s)
Year
ES
W*ES
Total
ESs
Total N
Diagnosis
Publication Type
Program Type
10
Anderson
1995
.14
.14
12
33
Behavioral
Disertation
Wilderness
50
Bandoroff, & Scherer
1994
.74
.74
7
43
Behavioral
Journal
Wilderness
90
Castellano & Soderstrom,
1992
.49
.49
1
22
Behavioral
Journal
Wilderness
80
Chakravorty, et al.
1995
.36
.36
1
25
Psychological
Journal
ABC
540
Dahlen
1998
.72
.72
13
199
Behavioral
Unpublished
ABC
130
Fashimpar& Harris
1987
1.10
1.10
5
20
Psychological
Journal
ABC
150
Gillis, et al.
1995
.24
.24
6
134
Behavioral
Unpublished
Mixed
180
Herbert, J.
1998
1.19
1.19
4
36
Combined
Journal
Wilderness
200
Kelly & Baer
1969
.27
.27
20
60
Behavioral
Journal
Wilderness
390
Maizell
1988
.45
.45
4
19
Behavioral
Disertation
ABC
250
McPeake, et al.
1991
.64
.64
6
46
Behavioral
Journal
Mixed
470
Mossman
1997
.20
.20
2
12
Behavioral
Disertation
Mixed
600
Neill
2003
.49
.49
1
61
Behavioral
Unpublished
Wilderness
290
Pawlowski, et al.
1993
.41
.41
8
15
Psychological
Journal
Wilderness
480
Price & DeBever
1997
.46
.46
1
6
Behavioral
Unpublished
Wilderness
530
Russel
2002
1.11
1.12
4
144
Mixed
Unpublished
Wilderness
510
Witman
1987
.77
.77
1
5
Psychological
Journal
ABC
Results
• Univariate Analysis of Possible Predictors:
• Found significant positive results for:
– Dissertations (.17); Medium Quality (.74);
High Quality (.44); ABC Programs (.72);
Continuous Programs (.59); Intermittent
Programs (.37); Residential Programs (.55);
Single Sex Groups (.47); Coed Groups (.53);
Adolescents (.49); Adults (1.11); Self
Constructs (.25)
Results
• Univariate Analysis of Possible Predictors:
• All other variables were found to be
significantly different from zero, but not
between variables.
• Only Outpatient Programs had a nonsignificant ES (.19)
Results
• File Drawer Effect: 19 studies/ 648 ES
would be needed to negate the findings of
this study
• Regression returned Dissertations (-.28);
ABC Programs (.38); Mixed Diagnoses
(.74) as the three predictors significant at
p=.05
• Regressed Predictors account for 35.9%
of variance in ES
Discussion
• Sample Size: small and some sampling
error likely to exist. However, Quality of
Studies and File Drawer Effect suggest
some strength to the sample.
• Dahlen (2002) & Russell (2000) deserve
special comment. ES drops to .35 without
these studies.
Discussion
Author
Studies
ESs
ES
Hattie et al (1997)
91
1728
.34
Hans (2000)
24
30
.36
Marsh (1999)
22
37
.25
Cason (1993)
43
147
.31
Bunting & Donley (2003)
10
n/a
.75
Staunton (2003)
17
95
.53
Hattie (1992)
Innovative Classroom
Interv.
.28
Lipsey & Wilson (1993)
Psych. Training
.47
Smith et al (1980)
Psychotherapy adults
.68
Casey & Berman (1985)
Psychotherapy child
.71
In other words...
ES Comparison
Innovative Interv.
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Outdoor Education
Psychological
Training
AT
Psychotherapy
ES
Discussion
• Why ABC Programs may be more
effective:
– More Developed Theory; Clinical
Associations; More Educated Practitioners;
Flexibility
• Why Mixed Diagnoses may be more
effective:
– Holistic Nature of AT Programs; Role of
Processing and Behavior
Discussion
• Why Dissertations may be least effective:
– Rigors of Publishing; Self-designed
Measures; Least Experienced Researchers
Conclusions
• AT is effective (.53) and has lasting
(increasing?) affects (.37).
• As a whole, the body of research is
overwhelmingly positive.
• However, the overall quality of research
and reporting is low
• AT compares favorably to other treatment
modalities.
Conclusions
• ABC Methodologies and Mixed Diagnoses
appear to be correlates of large ES.
• Further research is needed to test the
causes and correlates of AT program
effects.
• Further Meta-analysis is recommended.
• AT should become a respected Tx
Modality.
THE END
Thanks For Coming.
Any Questions?