Write-Up Section for Experimental Design
Download
Report
Transcript Write-Up Section for Experimental Design
Data Analysis for Experimental
Design Method Section & Data Analysis
Components of the Research
Method of Experimental Design
Method (Overview)
Sample
Design and experimental stimuli
(manipulations)
Pre-manipulation and post-manipulation
measures
Administration procedure
Independent Variables
Setting the stage
informed consent
brief explanation of what is expected
Types of manipulations
straightforward
staged
to create a psychological state
to simulate a real world situation
use confederates
Dependent Measures
Types of measures
self-report
rating scales
behavioral
reaction time
error rate
physiological
GSR, heart rate
Dependent Measures (cont.)
Sensitivity
ceiling effect - too easy
floor effect - too hard
no effect
Multiple Measures
e.g. time perception
Ethics of measures (e.g. privacy)
Cost
Data Defects
Missing data
some statistics allow for missing data
can replace with averaging techniques
SPSS have several missing data options
Extreme score or outliers
techniques for discarding
replace with averaging
Appropriate Statistics!!!!!!!!!!!!!!!!!
Missing Data
A Simple Example of a Missing Data
Analysis
Understanding the Reasons Leading to
Missing Data
Ignorable Missing Data
Other Types of Missing Data Processes
Examining the Patterns of Missing Data
Diagnosing the Randomness of the
Missing Data Process
Missing Data (Cont.)
Approaches for Dealing with Missing
Data
Use of Only Observations with Complete
Data
Delete Case(s) and/or Variable(s)
Outliers
Detecting Outliers
Univariate Detection
Bivariate Detection
Outlier Designation
Outlier Description and Profiling
Retention or Deletion of the Outlier
Measurement Scales
Nominal (categorical)
Ordinal
Interval
Ratio
Nominal Scales
Mutually exclusive categories
Labels only, no arithmetic properties
Examples: Sex, location, marital status
Statistical operations: Mode,
frequencies, crosstabs
Ordinal Scales
Implies more or less, but not how much
more or less
Can use median or mode, but not
average
Crosstabs
Interval/Ratio Scales
Numbers in scale represent equal
increments
Scales have the most desirable
statistical properties
(The ratio scale is an interval scale with
a fixed zero point, e. g. age and
temperature)
The Multiple Choice Question
Simple, versatile
Single or multiple response
No more than 8-10 so categories
Mutually exclusive categories
Pre-testing should show labeled
alternatives cover 90% of answers
Single Response, Multiple
Choice Questions
Choice criterion must be clearly stated
Choice criterion must define a single
answer
Forced Ranking Scales
Forced Ranking Scales
Options judged relative to one another
Mimics actual decision making
Disadvantage: interval between items
not measured
Limited number of items can be ranked
Guidelines for Using Forced
Ranking Scales
10 Items or less
Relative standing is focus
Use single judgment criterion
Analysis confined to frequencies, special
procedures for ranked data
The Likert Scale
Widely used scale
5 points
Degree of agreement with statements
Use several statements to measure a
“construct”
e.g. “customer satisfaction,” “loyalty,”
“attitudes towards technology”
Developing Constructs
Curiosity
1. I think learning about things is interesting and exciting.
2. I am curious about things.
3. My spare time is filled with interesting activities.
4. I like to solve problems that puzzle me.
5. I want to probe deeply into things.
6. I enjoy exploring new places.
7. New situations capture my attention.
8. I feel inquisitive.
9. The prospect of learning new things excites me.
10. I feel like searching for answers.
Attitudes towards Technology
4.
I can usually figure out new hi-tech products and services
without help from others.
New technologies are often too complicated to be useful.
I get overwhelmed by how much I need to know to use the
latest technology.
Technology gives people more control over their daily lives.
5.
People get too dependent on technology to do things for them.
1.
2.
3.
6.
7.
8.
9.
New technologies are mentally stimulating.
Learning about new technologies can be as rewarding as the
technology itself.
The products and services with the newest technologies are
more convenient to use.
New technologies are not designed for use by people like me.
Customer Satisfaction
(tailored for online purchases)
1.
2.
3.
4.
5.
6.
If I had it to do over again, I’d make my most
recent purchase at this website.
I am sure it was the right thing to make my most
recent online purchase at this website.
I have truly enjoyed purchasing from this website.
My choice to purchase from this website was a wise
one.
I am satisfied with my most recent decision to
purchase from this website.
I am happy I made my most recent online
purchase at this website.
Loyalty (intentions)
1.
2.
3.
4.
5.
I encourage friends and relatives to do business
with this website.
I say positive things about the website to other
people.
I will do more business with the website in the next
few years.
I would recommend the website to someone who
seeks my advice.
I consider this website to be my first choice to buy
the kind of product I most recently purchased
online.
The Likert Scale
Easy to use
Easy to code
Useful in statistical analyses
Usually treated as interval level
The Likert Scale
Keep in mind:
Only use for statements that ask for
degree of agreement
Use for several items to use space well
Items should be diverse enough to
cover the issue
The Likert Scale
Items for a construct can be summed
Recode reverse scored items before
summing
Likert-like Scales
(NOT Likert-Scales)
Verbal Frequency Scale
Indicates how often
Likert-like, but usually not interval level
The Horizontal Numeric Scale
Items judged on a single dimension
Intermediate values are not labeled
The Semantic Differential
Scale
7 point scale
Bipolar adjectives/statements
Can give quick comparative profile of
objects
Purchase Intent Scales
Often of interest in marketing
Write them carefully
The Fixed Scale
Rs allocate a fixed number of points
(usually 10 or 100) to choices
Univariate Statistical Analysis
What is hypothesis?
Null and alternative hypotheses
Significance level
Chi-square test (test of a proportion)
“Is the difference between the expected
(hypothetical) distribution and the
observed distribution statistically
significant?”
Illustration of an Example
Bivariate Statistical Analysis:
Tests of Differences
Chi-square test (a test of differences
among groups)
“Is the difference between the distribution
statistically different?”
Illustration of an Example
T-test for Difference of Means
“Is the difference between the means of
men and women statistically different?”
ANOVA (Analysis of Variance)
Determines if mean group scores are far
apart relative to our uncertainty about
the actual value of the means
Example
Green
Yellow
Blue
Store 1
14
8
8
Store 2
10
14
6
Store 3
11
3
5
Store 4
9
7
1
Average
Sales
11
8
5
Grand Mean= 11+8+5/3 = 8
Example
There seems to be an overall average
difference
Questions:
Is this difference statistically significant?
If so, is the size of the difference
managerially significant?
ANOVA (Analysis of Variance)
Conceptually, ANOVA compares:
difference among means/uncertainty OR
explained variance/unexplained
variance OR
between treatment variance/within
treatment variance
Analysis of Variance (ANOVA)
F-test
The larger F-value, the better result
Determines whether there is more
variability in the scores of one sample than
in the scores of another sample)
“Are the two sample variances different
from each other or are they are from
the same population?”
Logic of ANOVA
Each observation is different from the
Grand (total sample) Mean by some
amount
There are two sources of variance from
the mean
1) That due to the treatment or
independent variable
2) That which is unexplained by our
treatment
Logic of ANOVA (cont.)
The statistical test of significance is based
on the relative size of the variance caused
by the IV relative to unexplained variance
Three quantities are calculated:
Total variance
Between treatment sum of squares
(explained)
Within treatment sum of squares
(unexplained)
The F-ratio
F
MS = mean square
bg = between groups
wg = within groups
MSbg
MS wg
Numerator is the “effect”
and denominator is the
“error”
df = # of categories – 1
(k-1)
F-Statistic
Used in Factorial Designs
Is an extension of the t-test.
It is an analysis of variance that is a more general
procedure than the t-test.
When a study has only one independent variable
and only two groups using an F or a t makes no
difference.
However analysis of variance (ANOVA) is conducted
when there are more than two levels of an
independent variable and when a factorial design
with two or more independent variables is used.
One-Way ANOVA
Partitions Total Variation
Total variation
Variation due to
treatment
Sum of Squares Among
Sum of Squares Between
Sum of Squares
Treatment (SST)
Among Groups Variation
Variation due to
random sampling
Sum of Squares Within
Sum of Squares Error
(SSE)
Within Groups Variation
Post Hoc Tests Vs Planned
Comparisons
Post hoc tests allow us to make all pairwise
comparisons. These tests trawl the data,
searching for significant differences and so
are not very sensitive.
Planned comparisons enable us to address
specific questions with a higher degree of
sensitivity.
It is not appropriate to use both in order to
answer the same question.
Statistical Issues
Assumptions of the test
are your data normally distributed
is the sd about equal in each group
Statistically significant difference vs. “real”
difference
Hypotheses
null vs. research
never “prove” anything
Statistical Issues (cont.)
Scale attenuation effects
floor and ceiling effects
restricted range
Regression to the mean
Criterion choice
the .05 “rule”
Type I and Type II errors
Statistical Issues (cont.)
Interpreting non-significant results
poor design
poor procedures
poor instructions
poor operational definitions
power of the test
N and effect size