Transcript Write-Up Section for Experimental Design

Data Analysis for Experimental
Design Method Section & Data Analysis
Components of the Research
Method of Experimental Design





Method (Overview)
Sample
Design and experimental stimuli
(manipulations)
Pre-manipulation and post-manipulation
measures
Administration procedure
Independent Variables


Setting the stage
 informed consent
 brief explanation of what is expected
Types of manipulations
 straightforward
 staged
 to create a psychological state
 to simulate a real world situation
 use confederates
Dependent Measures

Types of measures
 self-report
 rating scales
 behavioral
 reaction time
 error rate
 physiological
 GSR, heart rate
Dependent Measures (cont.)


Sensitivity
 ceiling effect - too easy
 floor effect - too hard
 no effect
Multiple Measures
 e.g. time perception

Ethics of measures (e.g. privacy)

Cost
Data Defects



Missing data
 some statistics allow for missing data
 can replace with averaging techniques
 SPSS have several missing data options
Extreme score or outliers
 techniques for discarding
 replace with averaging
Appropriate Statistics!!!!!!!!!!!!!!!!!
Missing Data


A Simple Example of a Missing Data
Analysis
Understanding the Reasons Leading to
Missing Data




Ignorable Missing Data
Other Types of Missing Data Processes
Examining the Patterns of Missing Data
Diagnosing the Randomness of the
Missing Data Process
Missing Data (Cont.)

Approaches for Dealing with Missing
Data


Use of Only Observations with Complete
Data
Delete Case(s) and/or Variable(s)
Outliers

Detecting Outliers





Univariate Detection
Bivariate Detection
Outlier Designation
Outlier Description and Profiling
Retention or Deletion of the Outlier
Measurement Scales

Nominal (categorical)

Ordinal

Interval

Ratio
Nominal Scales




Mutually exclusive categories
Labels only, no arithmetic properties
Examples: Sex, location, marital status
Statistical operations: Mode,
frequencies, crosstabs
Ordinal Scales



Implies more or less, but not how much
more or less
Can use median or mode, but not
average
Crosstabs
Interval/Ratio Scales



Numbers in scale represent equal
increments
Scales have the most desirable
statistical properties
(The ratio scale is an interval scale with
a fixed zero point, e. g. age and
temperature)
The Multiple Choice Question





Simple, versatile
Single or multiple response
No more than 8-10 so categories
Mutually exclusive categories
Pre-testing should show labeled
alternatives cover 90% of answers
Single Response, Multiple
Choice Questions


Choice criterion must be clearly stated
Choice criterion must define a single
answer
Forced Ranking Scales
Forced Ranking Scales




Options judged relative to one another
Mimics actual decision making
Disadvantage: interval between items
not measured
Limited number of items can be ranked
Guidelines for Using Forced
Ranking Scales




10 Items or less
Relative standing is focus
Use single judgment criterion
Analysis confined to frequencies, special
procedures for ranked data
The Likert Scale





Widely used scale
5 points
Degree of agreement with statements
Use several statements to measure a
“construct”
e.g. “customer satisfaction,” “loyalty,”
“attitudes towards technology”
Developing Constructs
Curiosity
1. I think learning about things is interesting and exciting.
2. I am curious about things.
3. My spare time is filled with interesting activities.
4. I like to solve problems that puzzle me.
5. I want to probe deeply into things.
6. I enjoy exploring new places.
7. New situations capture my attention.
8. I feel inquisitive.
9. The prospect of learning new things excites me.
10. I feel like searching for answers.
Attitudes towards Technology
4.
I can usually figure out new hi-tech products and services
without help from others.
New technologies are often too complicated to be useful.
I get overwhelmed by how much I need to know to use the
latest technology.
Technology gives people more control over their daily lives.
5.
People get too dependent on technology to do things for them.
1.
2.
3.
6.
7.
8.
9.
New technologies are mentally stimulating.
Learning about new technologies can be as rewarding as the
technology itself.
The products and services with the newest technologies are
more convenient to use.
New technologies are not designed for use by people like me.
Customer Satisfaction
(tailored for online purchases)
1.
2.
3.
4.
5.
6.
If I had it to do over again, I’d make my most
recent purchase at this website.
I am sure it was the right thing to make my most
recent online purchase at this website.
I have truly enjoyed purchasing from this website.
My choice to purchase from this website was a wise
one.
I am satisfied with my most recent decision to
purchase from this website.
I am happy I made my most recent online
purchase at this website.
Loyalty (intentions)
1.
2.
3.
4.
5.
I encourage friends and relatives to do business
with this website.
I say positive things about the website to other
people.
I will do more business with the website in the next
few years.
I would recommend the website to someone who
seeks my advice.
I consider this website to be my first choice to buy
the kind of product I most recently purchased
online.
The Likert Scale




Easy to use
Easy to code
Useful in statistical analyses
Usually treated as interval level
The Likert Scale
Keep in mind:
 Only use for statements that ask for
degree of agreement
 Use for several items to use space well
 Items should be diverse enough to
cover the issue
The Likert Scale


Items for a construct can be summed
Recode reverse scored items before
summing
Likert-like Scales
(NOT Likert-Scales)
Verbal Frequency Scale


Indicates how often
Likert-like, but usually not interval level
The Horizontal Numeric Scale


Items judged on a single dimension
Intermediate values are not labeled
The Semantic Differential
Scale



7 point scale
Bipolar adjectives/statements
Can give quick comparative profile of
objects
Purchase Intent Scales


Often of interest in marketing
Write them carefully
The Fixed Scale

Rs allocate a fixed number of points
(usually 10 or 100) to choices
Univariate Statistical Analysis




What is hypothesis?
Null and alternative hypotheses
Significance level
Chi-square test (test of a proportion)


“Is the difference between the expected
(hypothetical) distribution and the
observed distribution statistically
significant?”
Illustration of an Example
Bivariate Statistical Analysis:
Tests of Differences

Chi-square test (a test of differences
among groups)



“Is the difference between the distribution
statistically different?”
Illustration of an Example
T-test for Difference of Means

“Is the difference between the means of
men and women statistically different?”
ANOVA (Analysis of Variance)
Determines if mean group scores are far
apart relative to our uncertainty about
the actual value of the means
Example
Green
Yellow
Blue
Store 1
14
8
8
Store 2
10
14
6
Store 3
11
3
5
Store 4
9
7
1
Average
Sales
11
8
5
Grand Mean= 11+8+5/3 = 8
Example
There seems to be an overall average
difference
Questions:
 Is this difference statistically significant?
 If so, is the size of the difference
managerially significant?

ANOVA (Analysis of Variance)
Conceptually, ANOVA compares:
difference among means/uncertainty OR
 explained variance/unexplained
variance OR
 between treatment variance/within
treatment variance
Analysis of Variance (ANOVA)

F-test



The larger F-value, the better result
Determines whether there is more
variability in the scores of one sample than
in the scores of another sample)
“Are the two sample variances different
from each other or are they are from
the same population?”
Logic of ANOVA




Each observation is different from the
Grand (total sample) Mean by some
amount
There are two sources of variance from
the mean
1) That due to the treatment or
independent variable
2) That which is unexplained by our
treatment
Logic of ANOVA (cont.)
The statistical test of significance is based
on the relative size of the variance caused
by the IV relative to unexplained variance
Three quantities are calculated:
 Total variance
 Between treatment sum of squares
(explained)
 Within treatment sum of squares
(unexplained)

The F-ratio
F



MS = mean square
bg = between groups
wg = within groups
MSbg
MS wg


Numerator is the “effect”
and denominator is the
“error”
df = # of categories – 1
(k-1)
F-Statistic





Used in Factorial Designs
Is an extension of the t-test.
It is an analysis of variance that is a more general
procedure than the t-test.
When a study has only one independent variable
and only two groups using an F or a t makes no
difference.
However analysis of variance (ANOVA) is conducted
when there are more than two levels of an
independent variable and when a factorial design
with two or more independent variables is used.
One-Way ANOVA
Partitions Total Variation
Total variation
Variation due to
treatment




Sum of Squares Among
Sum of Squares Between
Sum of Squares
Treatment (SST)
Among Groups Variation
Variation due to
random sampling



Sum of Squares Within
Sum of Squares Error
(SSE)
Within Groups Variation
Post Hoc Tests Vs Planned
Comparisons



Post hoc tests allow us to make all pairwise
comparisons. These tests trawl the data,
searching for significant differences and so
are not very sensitive.
Planned comparisons enable us to address
specific questions with a higher degree of
sensitivity.
It is not appropriate to use both in order to
answer the same question.
Statistical Issues

Assumptions of the test




are your data normally distributed
is the sd about equal in each group
Statistically significant difference vs. “real”
difference
Hypotheses


null vs. research
never “prove” anything
Statistical Issues (cont.)

Scale attenuation effects


floor and ceiling effects
restricted range

Regression to the mean

Criterion choice


the .05 “rule”
Type I and Type II errors
Statistical Issues (cont.)

Interpreting non-significant results





poor design
poor procedures
poor instructions
poor operational definitions
power of the test

N and effect size