Transcript Slide 1

USING OBSERVATION-ORIENTED MODELING TO
EXAMINE DAILY PATTERNS AND PREDICTORS OF POSTTRAUMATIC STRESS SYMPTOMATOLOGY
IN A SAMPLE OF FEMALE RAPE VICTIMS
Amy M. Cohn, James Grice, Brett Hagman,
and Liz Schlimgen
November 17, 2012
ABCT
Aim of Presentation


Test a mediation model with daily diary data
Compare multi-level modeling (MM) to Observation
Orientated Modeling (OOM)
Daily
negative
affect
Daily posttraumatic stress
symptoms
Daily alcohol
involvement
Assumptions of MM have limitations

Homogeneity between individuals


Within-person fluctuations in behavior represented as
aggregated, over-time association (slope)
Linear monotonic changes in behavior over time

Associations between variables are additive, not (necessarily)
dynamic
Random sampling
 Normal population distribution (for differences)

Abstract population parameters that have little (or no) empirical basis
 The population is completely theoretical

What’s wrong with the p-value?


Not a new argument
Relies on population statistics that may not represent
the data
 The
sample is different, the way you collect the data is
different, the questions you ask are different….

Creates a “false belief” in the validity and
generalizability of findings
 Many
study results cannot be replicated
Observation Orientated Modeling
“Why is it that the patterns of phenomena are the way they are?” (Harre,
1986)
“Fundamentally incompatible with prevailing research tradition in Psychology”
(Grice, 2012)
OOM Incompatible with MM

Variable-based approach (such as MM) is linear,
causal, and based on aggregate statistics such as
betas and variances
Independent
variable

Dependent
variable
OOM approach is integrative and focused at the
level of the individual
OOM




Non-parametric, idiographic
 Examines qualitative pattern in the data
 Rooted in Aristotle’s notion that most things in nature are not
produced by people
 The researcher does not control everything in a study
Eschews null hypothesis significance testing (NHST)
Results based on probabilities found within the data, not
comparison to population distribution
Variables not described in a cause-effect format
 OOM describes how the effect conforms to the cause
OOM

To Repeat…….
 EFFECTS

SHOULD CONFORM TO THEIR CAUSES
What the @#*&$?
 We
do not always know why participants do what they
do
 Effects are never truly “causal”
 Unmeasured pieces of “error” or “garbage” in the data
collection process

With OOM, patterns of observations reveal what
are in the data – The EFFECTS
Study 1 Hypotheses


NA will be greater on days characterized by greater PTSD
Craving and consumption will be greater on days characterized
by more intense PTSD and NA
a
Daily PTSD
symptoms
Daily NA
c (c’)
b
Alcohol
Involvement
Sample characteristics (n = 54)
 54
untreated female rape victims who completed at least one
day of daily interactive voice response (IVR) monitoring
Characteristic
Statistic
Age
26 (SD = 9.08)
Some college or post high school
education
70% (n = 38)
Employed (full or part-time)
25% ( n = 15)
Single
60% (n = 34)
Caucasian
70% ( n = 38)
Income (Median)
$9,000 (SD = $20,032)
IVR Assessment
1x a day in the evening (6pm to 12am)
 Alcohol use, negative affect intensity, craving
intensity, and PTSD symptoms (presence/absence)
 Since previous phone call
 93% compliance rate
 13/14 calls were completed

HLM Analysis
DV’s = Number of drinks consumed and intensity
of craving (850 observations)
 Controlled for day of week
 Poisson distribution with log link function for
drinking
 Examined relationship of one variable EACH DAY
to the outcome variable ON THAT SAME DAY

Figure 1. Mediation of NA on the PTSD-alcohol link.
0.13***
Daily PTSD
symptoms
Daily NA
-0.14 (-0.02)
*** p < .001
In Cohn, Hagman, Moore, Mitchell, Ehlke, and Bramm (under review)
0.42***
Number of
standard
drinks/day
Figure 2. Mediation of NA of the PTSD-craving link.
0.13***
Daily PTSD
symptoms
Daily NA
-0.10 (-0.12)
0.39***
Daily craving
intensity
Note. Covariates included day of the week, baseline PTSD symptom severity, baseline
alcohol use.
*** p < .001
In Cohn, Hagman, Moore, Mitchell, Ehlke, and Bramm (under review)
OOM Analysis: Mediation Steps
Daily NA
Daily PTSD
symptoms
Number of
standard
drinks/day
Step 1: Because the effect conforms to the cause, we first examine the probability
that number of standard drinks consumed each day conforms to daily ratings of
NA intensity
OOM Results




Accuracy rate: % observations correctly classified out of total
number of observations
 Missing data is not a problem
Randomization test
 Out of1000 trials of randomized versions of the same
observations, what number of instances do we obtain a result
high or higher than percent correct classification?
Binomial p-value or chance value should be small (less than .01)
 Indicates pattern is unique
Results for individual and group-level patterns
Perfect Ordinal Matches for 14 Occasions
Proportion of Matches = 1.00; Binomial p-value = .00012
Weak Ordinal Matches for 14 Occasions
Proportion of Matches = .15; p-value = .99
Aggregate Results for all 54 Women
Overall Results (n = 54 women) :
Number of Matches :
Number of Observations :
Proportion of Matches :
123
399
0.31
Randomization Results :
Observed Proportion of Matches
Number of Randomized Trials
Minimum Random Proportion of Matches
Maximum Random Proportion of Matches
Values >= Observed Proportion
Matching c-value
Proportion
C-value
:
0.31
: 5000.00
:
0.24
:
0.36
: 1758.00
:
0.35
of matches is unimpressive at .31
of the Randomization Test indicates that .31 is not an
unusual aggregate outcome compared to randomized versions of
the same observations
Aggregate Results for all 54 Women
1. Proportion of Matches > .50 for only 9 women
(5 of these women had 7 or fewer data points)
2. Fourteen women (26%) showed no variability in their
drinking across the 14 days
3. An additional 6 women drank on only one day
Conclusions


Women showing no variability in drinking and those
who did not drink across 14 days are “swept” into
HLM aggregates
 Should this disturb us?
OOM recognizes women with no variability in their
drinking
 Since OOM not based on means and variances,
impact of these women does not adversely effect
the overall percent matches
Conclusions


OOM “effect sizes” are proportions of matches that are
readily interpretable and linkable to individual women
 No need for interpretations- such as Cohen’s effect sizes
 Idealized p-values are primary in HLM, even over effect
sizes
 Even if effect is small, if p < .05 we say “YES”!
Proportions of matches consistent with causal hypotheses are
primary in OOM
 Distribution free p-values (from binomial and randomization
tests) are secondary
Summary
 Erroneously
enticed to posit a mediation mechanism that
operates successfully for every woman with HLM

OOM treats the women and their individual
observations as primary
 Does not rely on p-values, means, or variances
estimated from a theoretical population
 OOM develops integrated models
 More accurately explains patterns of observations
Acknowledgments



Participants who dedicated their time and effort
Research assistants: Jessica Mitchell, Stephanie
Bramm, Sarah Ehlke, Ruschelle Leone, Joanne Wang
Grants: NIDA P30DA028807; USF 582000 /
MHBCSG
Thank you!
Questions?
Dr. James Grice
Department of Psychology
Oklahoma State University
Stillwater, OK 74078
[email protected]
Deep Structure Transformation
Cause
Observations
1. Observations are
transformed into their
“deep structure”
2. Rotate deep structure
effect observations into
“conformity” with deep
structure observations
Effect
Observations
M F
0 1 2 3 4 5 6 7 8 9 10
0
0
1
1
1
1
0
0
00000000001
00000000010
00100000000
00100000000
0
0
1
1
1
1
0
0
0
0
1
1
1
1
0
0
00000000001
00000000010
00100000000
00100000000
Conformed
Effect
Observations
Effect
Observations
3. Accuracy is our central judgment (not statistical significance) and shows how many
observations were correctly classified by the algorithm, or how many observations
match the pattern.