Transcript Business Research Methods William G. Zikmund

BUS 332
Scientific Research Techniques
Main Textbook:
William G. Zikmund’s
Business Research Methods
Lectured by
Prof. Dr. Lütfihak Alpkan
Gebze Institute of Technology
WEEK DATE
TEACHING PLAN
1
20. 02 Introduction
2
27. 02 Ch. 9: Survey Research: An Overview
3
05.03 Ch. 10: Survey Research: Communication with Respondents
4
12.03 Ch. 11: Observation Methods
5
19.03 Ch. 12: Experimental Research
6
26.03 General overview
7
02.04
8
09.04 Ch. 13: Measurement and Scaling
9
16.04 Ch. 14: Attitute Measurement
11
30.04 Ch. 15: Questionnaire Design 1
12
07.05 Ch. 15: Questionnaire Design 2
13
14. 05 General overview
14
21.05
MIDTERM EXAM 1
MIDTERM EXAM 2
Business
Research Methods
William G. Zikmund
Chapter 9:
Survey Research
Chapter 9:
Survey Research
1. Basic Definitions for surveys
2. Errors in Surveys
3. Classification of Survey Methods
1. Basic Definitions for surveys
Survey: a research technique in which information
(primary data) is gathered from a sample of
people to make generalizations.
Primary data: data gathered and assembled
specifically for the project at hand.
Sample of the survey: respondents who are asked to
provide information, assuming that they can
represent (possess same features with) a target
population.
Selecting a Sample
Sample:
Subset of a larger population
SAMPLE
Sampling:
POPULATION
• Who is to be sampled?
• How large a sample?
• How will sample units be selected?
Basic Definitions for sampling
(http://www.stats.gla.ac.uk/steps/glossary/sampling.html)
Target population: the group about which the
researcher wishes to draw conclusions and make
generalizations
Random sampling: selecting a sample from a
larger target population where each respondent is
chosen entirely by chance and each member of
the population has a known, but possibly nonequal, chance of being included in the sample.
Basic Definitions for data collection
Surveys ask respondents (who are the subjects of the
research) questions by use of a questionnaire.
Respondent: The person who provides information
(primary data) by answering a questionnaire or an
interviewer’s questions.
Questionnaire: a list of structured questions
designed by the researchers for the purpose of
codifying and analyzing the respondents’ answers
scientifically.
Advantages of Surveys: Quick, Inexpensive, Efficient,
Accurate, Flexible way of gathering information.
2. Errors in Surveys
2.1. Random Sampling Error
2.2. Systematic Error (sample bias)
2.2.1. Respondent error
* Nonresponse bias
* Response bias
2.2.2. Administrative error
* Data processing error
* Sample selection error
* Interviewer error
* Interviewer cheating
2.1. Random Sampling Error
• Even if randomly selected, samples may possess
different characteristics than the target population
(the likelihood of bias is reduced but still exists)
• This is a statistical fluctuation due to chance
variation.
• Then, an important difference occurs between the
findings obtained from this sample and the
findings obtained from a possible census of the
whole target population.
• Consider the hypothetic case in which a study
sample could be increased until it was infinitely
large; chance variation of the mean, or random
error, would be reduced toward zero. These are
random errors.
• Systematic errors would not be diminished by
increasing sample size.
(Bias in Research Studies,
http://radiology.rsna.org/content/238/3/780.full)
2.2. Systematic Error
• Systematic error results from some mistake(s)
done in the design and/or execution of the
research.
• All types of error -except random sampling error,
are included in this definition,
• Sample bias: a persistent tendency for the results
of a sample to deviate in one direction from the
true value of the population parameter.
• Sample bias can arise when the intended sample
does not adequately reflect the spectrum of
characteristics in the target population.
2.2.1. Respondent Bias
• A classification of sample bias resulting
from some respondent action or inaction
• Nonresponse bias
• Response bias
Nonresponse Error
• Nonrespondents: in almost every survey information
from a small or large portion of the sample cannot be
collected. These are those people who refuse to
respond, or who can not be contacted (not-at-homes)
• Self-selection bias: only those people who are
interested strongly with topic of the survey may
respond while those who are still within the same
sample but indeferent or afraid avoid participating.
• This leads to the over-representation of some extreme
positions, but under-representation of others.
Response Bias
• A bias that occurs when respondents tend to
answer questions with a certain inclination
or viewpoint that consciously (deliberate
falsification) or unconsciously
(unconscious misinterpretation)
misrepresents the truth.
Reasons of response bias
• Knowingly or unknowingly people who
answer questions of the interviewer may
feel unconfortable about the truth that they
share with others, and change it in their
responses.
• They may desire to show themselves as
more intelligent, wealthy, sensitive, etc.
than they really are.
Types of Response Bias
Deliberate falsification (consciously false answers)
Acquiescence bias (positive answers)
Extremity bias (exaggerated answers)
Interviewer bias (answers acceptable by the interviewer)
Auspices bias (answers acceptable by the organization)
Social desirability bias (answers creating a favorable impression)
2.2.2. Administrative Error
• Unadvertently or carelessly improper
administration and execution of the research
task
• Blunders are:
• Confusion
• Neglect
• Omission
Types of Administrative Errors
Data processing error: incorrect data entry, computer
programming, or other procedural errors during the
analysis stage.
Sample selection error: improper sample design (e.g.
based on incomplete databases) or sampling
procedure execution (e.g. executed in daytime while
most of the target population are working)
Interviewer error: mistakes done by the interviewer
(e.g. taking wrong or incomplete notes about the
answers of the respondents.
Interviewer cheating: filling in fake or false answers
indeed not given by the respondents.
3. Classification of Survey Methods
3.1. Structure of the questionnaire:
* whether standardized questions with a limited
number of allowable answer -multiple choices
* or unstandardized open ended questions with the
possibility of being answered in numerious ways.
3.2. Level of Directness of the questions:
* whether direct/undisguised questions
* or indirect/disguised questions to hide the real
purpose of the survey
Classification of Survey Methods
3.3. Time basis of the Survey:
Cross-Sectional Study: data on various segments of a
target population are collected at a single moment in
time to make comparisons among segments.
Longitudinal Study: data are collected at different times
from the similar respondents to compare trends and
identify changes.
Panel Study: A longitudinal survey of exactly the same
respondents to record (in a diary) their attitudes,
behaviors, or purchasing habits over time.
Business
Research Methods
William G. Zikmund
Chapter 10:
Survey Research: Basic
Communication Methods
Chapter 10:
Survey Research: Basic
Communication Methods
* Comparison of Basic Communication Methods in
Surveys:
* Questionnaires administered by an interviewer
1. Door-to door interviews
2. Mall intercepts
3. Telephone interviews
* Self-administered questionnaires
4. Questionnaires sent by mail, fax, or e-mail
5. Internet questionnaires
1. Door-to-Door Personal
Interview
• Speed of data collection • Questionnaire length
– Moderate to fast
– Long
• Geographical flexibility • Item non-response
– Limited to moderate
– Low
• Respondent cooperation • Possibility of
respondent
– Excellent
misunderstanding
• Versatility of
– Lowest
questioning
– Quite versatile
Door-to-Door Personal Interview
• Degree of interviewer influence of answer: High
• Supervision of interviewers: Moderate
• Anonymity of respondent: Low
• Ease of call back or follow-up: Difficult
• Cost: Highest
• Special features: Visual materials may be shown
or demonstrated; extended probing possible
1. Mall Intercept Personal
Interview
• Speed of data collection: Fast
• Geographical flexibility: Confined, urban bias
• Respondent cooperation: Moderate to low
• Versatility of questioning: Extremely versatile
• Questionnaire length: Moderate to long
• Item non-response: Medium
• Possibility of respondent misunderstanding: Lowest
Mall Intercept Personal Interview
• Degree of interviewer influence of answers: Highest
• Supervision of interviewers: Moderate to high
• Anonymity of respondent: Low
• Ease of call back or follow-up: Difficult
• Cost: Moderate to high
• Special features: Taste test, viewing of TV
commercials possible
3. Telephone Surveys
• Speed of Data Collection: Very fast
• Geographical Flexibility: High
• Respondent Cooperation: Good
• Versatility of Questioning: Moderate
• Questionnaire Length: Moderate
• Item Non-response: Medium
• Possibility of Respondent Misunderstanding: Average
• Degree of Interviewer Influence of Answer: Moderate
Telephone Surveys
• Supervision of interviewers: High, especially with central
location WATS (Wide Area Telecommunications Service)
interviewing
• Anonymity of respondent: Moderate
• Ease of call back or follow-up: Easy
• Cost: Low to moderate
• Special features: Fieldwork and supervision of data
collection are simplified; quite adaptable to computer
technology (e.g. Central location interviewing, Computerassisted telephone interviewing, Computerized voiceactivated interviews)
Self-Administered
Questionnaires
SELF-ADMINISTERED
QUESTIONNAIRES
PAPER
QUESTIONNAIRES
MAIL
IN-PERSON
DROP-OFF
INSERTS
ELECTRONIC
QUESTIONNAIRES
FAX
E-MAIL
INTERNET
WEB SITE
KIOSK
4. Mail Surveys
• Speed of data collection: Researcher has no control over
return of questionnaire; slow
• Geographical flexibility: High
• Respondent cooperation: Moderate – but, poorly
designed questionnaire will have low response rate
• Versatility of questioning: Highly standardized format
• Questionnaire length: Varies depending on incentive
• Item non-response: High
Mail Surveys
• Possibility of respondent misunderstanding:
Highest--no interviewer present for clarification
• Degree of interviewer influence of answer:
None - interviewer absent
• Supervision of interviewers: Not applicable
• Anonymity of respondent: High
• Ease of call back or follow-up: Easy, but takes
time
• Cost: Lowest
5. E-Mail Questionnaire Surveys
•
•
•
•
Speed of data collection: Instantaneous
Geographic flexibility: worldwide
Cheaper distribution and processing costs
Flexible, but
– Extensive differences in the capabilities of
respondents’ computers and e-mail software limit
the types of questions and the layout
• E-mails are not secure and “eavesdropping”
can possibly occur
• Respondent cooperation
– Varies depending if e-mail is seen as “spam”
6. Internet Surveys
• A self-administered questionnaire posted on a Web site.
• Respondents provide answers to questions displayed
online by highlighting a phrase, clicking an icon, or
keying in an answer.
Internet Surveys
•
•
•
•
Speed of data collection: Instantaneous
Geographic flexibility: worldwide
Cost effective, visual and interactive
Respondent cooperation
– Varies depending on web site
– Varies depending on type of sample
– When user does not opt-in or expect a voluntary survey
cooperation is low.
– Self-selection problems in web site visitation surveys participants tend to be more deeply involved than the
average person.
Internet Surveys
• Versatility of questioning: Extremely versatile
• Questionnaire length: varying according to the answers of
each respondent
• Item non-response: Software can assure none
• Possibility for respondent misunderstanding: High
• Interviewer influence of answers: None
• Supervision of interviewers: not required
• Anonymity of Respondent: Respondent can be anonymous
or known
• Ease of Callback or Follow-up: difficult unless e-mail
address is known
• Special Features: allows graphics and streaming media
Business
Research Methods
William G. Zikmund
Chapter 11:
Observation Methods
Chapter 11:
Observation Methods
1. Types of Observed Phenomena
2. Advantages and Disadvantages of
Observation
3. Types of Observation Techniques
1. Types of Observed Phenomena
•
•
•
•
•
•
Physical actions
Verbal behavior
Expressive behavior
Spatial relations and locations
Temporal patterns
Verbal and pictorial records
Examples for Observed Phenomena
Phenomena
Example
Human behavior or physical Shoppers (buyers) movement
action
pattern in a store
Verbal behavior
Statements made by
airline travelers who wait
in line
Expressive behavior
Facial expressions, tone of
voice, and other form of
body language
Examples for Observed Phenomena
Phenomena
Example
Spatial relations
and locations
How close visitors at an
art museum stand to paintings
Temporal patterns
How long fast-food customers
wait for their order to be served
Physical objects
What brand name items are
stored in consumers’ pantries
Verbal and Pictorial
Records
Bar codes on product packages
2. Advantages and Disadvantages
of Observation
“YOU SEE, BUT YOU
DO NOT OBSERVE.”
Sherlock Holmes
2.1. Benefits of Observing Human Behavior
• Communication with respondent is not
necessary
• Data without distortions due to self-report
(e.g.: without social desirability) Bias
• No need to rely on respondents memory
• Nonverbal behavior data may be obtained
Benefits of Observing Human Behavior
• Certain data may be obtained more quickly
• Environmental conditions may be recorded
• May be combined with survey to provide
supplemental evidence
2.2. Limitations of Observing
Human Behavior
• Cognitive phenomena cannot be observed
• Interpretation of data may be a problem
(e.g. misinterpretation)
• Not all activity can be recorded
• Only short periods can be observed
• Observer bias possible (e.g. selective
perception)
• Possible invasion of privacy
3. Types of Observation Techniques
• Natural versus Contrived Observation
• Direct versus Indirect Observation
• Disguised versus Nondisguised Observation
• Physical-trace evidence Observation
• Mechanical Observation
3.1.Natural versus Contrived Observation
Natural Observation:
• Reactions and behavior observed as they
occur naturally in real-life situations
• A wide variety of companies are sending
researchers to the field to observe
consumers in their natural environment.
• Natural observation is also suited for
ethnographic research on foreign cultures.
Contrived Observation:
• Environment artificially set up by the researcher.
• Researchers are increasingly relying on
computers to conduct simulated market testing.
• Offers a greater degree of control
– Speedy
– Efficient
– Less expensive
• However, it may be questionable as to whether or
not the data collected does truly reflect a "real
life" situation.
3.2. Direct versus Indirect Observation
Direct observation captures actual behavior or
phenomenon of interest
Indirect observation consists of examining the
results of the phenomenon.
• can give only relatively crude or imprecise
indications of a phenomenon
• More efficient use of research time
• More efficient use of research budget
• May be the only way to get data from situations
impractical to observe directly.
3.3. Disguised versus Nondisguised Observation
Nondisguised observation:
• Respondents are aware that they are being
observed
• Data may be contaminated by respondentinduced errors.
• Data gathered through using disguised
observation might not be as rich as those
from nondisguised observation.
Disguised Observation
• Respondents are unaware they are being
observed
• Allows for monitoring of the true reactions of
individuals.
• Unethical if disguised observation monitors
– Normally private behaviors
– Behaviors that may not be voluntarily revealed to
researchers.
• Mystery shopping
– popular disguised observational technique
– Mystery shopper
• Unknown to the retail establishment
• Visits the store
• Uses a structured script
• Observes and records the shopping
experience.
3.4. Physical-trace evidence
Observation
• Wear and tear of a book indicates
how often it has been read
• garbology - looking for traces of purchase patterns
in garbage
• detecting store traffic patterns by observing the
wear in the floor (long term) or the dirt on the
floor (short term)
3.5. Types of Mechanical Observation
•
•
•
•
•
•
Eye-Tracking
Response Latency
Voice Pitch Analysis
People Meter
Psychogalvanometer
Monitoring Web Site Traffic
Eye Tracking
Measures unconscious eye movements
Records how the subject actually reads or views
an advertisement, product packaging,
promotional displays, websites, etc.
Measures which sections attract customers'
attention and how much time they spend
looking at those sections
• Oculometers - what the subject is looking at
• Pupilometers - how interested is the viewer (This
device observes and records changes in the diameter
of the subject’s pupils)
Voice Pitch Analysis
• Measures emotional reactions through
physiological changes in a person’s voice
• Used to determine
– how strongly a respondent feels about an answer
– how much emotional commitment is attached to an
answer.
• Variations from normal voice pitch is considered
a measure of emotional commitment to the
question's answer.
Response Latency
• It measures the speed with which a respondent
gives a decision about a choice between
alternatives
• It records the decision time necessary to make
this choice.
• For instance: it can measure the effectiveness
of an advertisement on brand preferences.
• It assumes that a quick expression of brand
preference indicates a stronger preference.
People Meter
• Electronic device to monitor television
viewing behavior
– who is watching
– what shows are being watched.
Psychogalvanometer
• Measures galvanic skin response
• Involuntary changes in the electrical
resistance of the skin
• Assumption: physiological changes
accompany emotional reactions
Business
Research Methods
Donald R. Cooper and Pamela S. Schindler
Chapter 12.1.:
Basics of Experimental Research
Chapter 12.1.:
Experimental Research
1. Basics of Experiment & Causality
2. Advantages and disadvantages of the
experimental method
3. Steps of a well-planned experiment
4. Validity in experiments
1.Basics of Experiment & Causality
1.1. Definition of Experiment:
An experiment is a study involving intervention by
the researcher beyond that required for
measurement.
The usual intervention is to manipulate some
variable in a setting and observe how it affects the
participants or subjects being studied.
There is at least one independent variable and one
dependent variable in a causal relationship.
1.2. Causal Evidence
There are three types of evidence necessary to
support causality.
Agreement between
Independent and Dependent Variables
Time order of occurrence
Extraneous variables
did not influence Dependent Variables
1.2.1. Agreement between
Independent and Dependent Variables
First, there must be an agreement between
independent and dependent variables.
The presence or absence of one is associated with
the presence or absence of the other.
1.2.2. Time order of occurrence
Second, beyond the correlation of independent and
dependent variables, we consider the time order of the
occurrence of the variables.
The effect on the dependent variable should not precede
the manipulation of the independent variable.
The effect and manipulation may occur simultaneously
or the manipulation may occur before the effect.
1.2.3. Extraneous variables
did not influence Dependent Variables
The third source of support comes when researchers are
confident that other extraneous variables did not
influence the dependent variable.
To ensure that these other variables are not the source of
influence, researchers control their ability to confound
the planned comparison.
2. Advantages and disadvantages of
the experimental method
Advantages
• Ability to manipulate
Independent Variable
• Use of control group
• Control of extraneous
variables
• Replication possible
• Field experiments
possible
Disadvantages
• Artificiality of labs
• Non-representative
sample
• Expensive
• Focus on present and
immediate future
• Ethical limitations
2.1.Explanation of Some Advantages of
Experiments
• Replication: is the process of repeating an
experiment with different participant groups and
conditions to determine the average effect of the
Independent Variables across people, situations,
and times.
• A field experiment: is a study of the dependent
variable in actual environmental conditions.
2.2.Explanation of Some Disadvantages of
Experiments
• The artificiality of a lab is possibly the greatest
disadvantage of experiments.
• Also, experiments typically use small convenience
samples which cannot be generalized to a larger
population.
• Compared to surveys, they are expensive.
• They also cannot deal with past events or predict
events in the far-off future.
• Finally, marketing research is often concerned with
the study of people and there are limits to the types of
manipulation and controls that are ethical.
3. Steps of a well-planned
experiment
Specify treatment variables
Specify treatment levels
Control environment
Choose experimental design
Select and assign participants
Pilot-test, revise, and test
Collect data
Analyze data
Steps of a well-planned experiment
The activities the researcher must accomplish to make
an experiment a success:
3.1. Specify treatment variables:
a) select variables that are the best operational
definitions of the original concepts,
b) determine how many variables to test,
c) select or design appropriate measures for the chosen
variables.
The selection of measures for testing requires a
thorough review of the available literature and
instruments.
3.2. Specify treatment levels:
In an experiment, participants experience a
manipulation of the independent variable, called the
experimental treatment.
The treatment levels are the arbitrary or natural groups
the researcher makes within the independent variable.
A control group is a group of participants that is
measured but not exposed the independent variable
being studied.
A control group can provide a base level for
comparison.
3.3. Control environment:
Environmental control means holding the physical
environment of the experiment constant. When
participants do not know if they are receiving the
experimental treatment, they are said to be blind.
When neither the participant nor the researcher knows,
the experiment is said to be double-blind.
3.4. Choose experimental design:
The design is then selected. Several designs are discussed
on the next several slides.
3.5. Select and assign participants:
The participants selected for the experiment should be
representative of the population to which the researcher
wishes to generalize the study’s results.
Random assignment is required to make the groups as
comparable as possible.
Random assignment uses a randomized sample frame for
assigning participants to experimental and control groups.
Matching is an equalizing process for assigning
participants to experimental and control groups.
• 3.5.1. Random assignment :
• The sampling frame is often small for experiments
and the participants may be self-selected.
• However, if randomization is used, those assigned to
the experimental group are likely to be similar to
those assigned to the control group.
• Random assignment allows one to make the groups as
comparable as possible.
• It means that participants have an equal and known
chance of being assigned to any of the groups in the
experiment.
3.5.2. Matching :
Matching is a control procedure to ensure that
experimental and control groups are equated on one or
more variables before the experiment.
The object of matching is to have each experimental and
control participant matched on every characteristic used
in the research. Matching employs a nonprobability
quota sampling approach.
Quota matrix is a means of visualizing the matching
process. If matching does not alleviate assignment
problems, a combination of matching, randomization,
and increasing the sample size may be useful.
Quota Matrix
Example
Exhibit 10-3 presents an
example of a quota
matrix.
One-third of the
participants from each
cell of the matrix would
be assigned to each of
the tree groups.
4. Validity in Experimentation
Internal validity exists when the conclusions
drawn about a demonstrated experimental
relationship truly implies cause.
External validity exists when an observed causal
relationship can be generalized across persons,
settings, and times.
4.1.Threats to Internal Validity
There are twelve possible threats to internal validity:
•History
•Maturation
•Testing
•Instrumentation
•Selection
•Statistical regression
•Experimental mortality
•Diffusion or imitation of treatment
•Compensatory equalization
•Compensatory rivalry
•Resentful Demoralization of the disadvantaged
•Local history
http://cde.annauniv.edu/CourseMat/mba/sem2/dba1657/val.html
Threats to internal validity
•History: In the experimental designs a control
measurement (O1) of dependent variable is taken before
introducing the manipulation (X).
After the manipulation an after measurement (O2) of the
dependent variable is taken. Then the difference between
O1 and O2 is attributed to the manipulation. (See also One
Group Pretest-Posttest Design)
However some events may occur during the course of the
experimental study, which will affect the relationship
between the variables under the study.
Threats to internal validity
•Maturation: Changes may also occur within the
participant that are a function of the passage of time
and are not specific to any particular event.
•A participant may become hungry, bored, or tired and
these conditions can affect response results.
•Testing: The process of taking a test can affect the
scores of a second test. For instance, repeatedly taking
(the same or similar) intelligence tests usually leads to
score gains.
Threats to internal validity
•Instrumentation: This threat to internal validity
results from changes between observations in either the
measuring instrument or the observer.
•Selection: Differential selection of subjects for
experimental and control groups affects the validity.
Validity considerations require the groups to be
equivalent in every aspect.
The problem can be overcome by randomly assigning
the subjects to experimental and control groups. In
addition matching can be done. Matching the members
of the groups on key factors also enhances the
equivalence of the groups.
Threats to internal validity
•Statistical regression: This factor operates especially
when groups have been selected by their extreme
scores.
•For example, when children with the worst reading
scores are selected to participate in a reading course,
improvements at the end of the course might not be
due to the course's effectiveness.
•Experimental mortality: This occurs when the
composition of the study groups changes during the
test. Some participants may drop out the experiment.
Threats to internal validity
•Diffusion or imitation of treatment: If people in
the experimental and control groups talk, then those
in the control group may learn of the treatment. This
eliminates the difference between the groups.
•Compensatory equalization: Where the
experimental treatment is much more desirable for
the experimental group, there may be an
administrative reluctance to deprive the control
group members. Actions to compensate the control
group may confound the experiment.
Threats to internal validity
•Compensatory rivalry: This may occur when
members of the control group know they are in
the control group. This may generate competitive
pressures, causing the control group members to
try harder. (e.g. Hawthorne effect )
•Resentful demoralization of the
disadvantaged: When the treatment is desirable
and the experiment is conspicuous, control group
members may become resentful that they are
deprived and lower their cooperation and output.
Threats to internal validity
•Local history: The regular history effect
already mentioned impacts both experimental
and control groups alike.
When one assigns all experimental persons to
one group session and all control group people
to another, there is a chance for some peculiar
event to confound results.
4.2.Threats to External Validity
External validity is concerned with the interaction of
the experimental treatment (X) with other factors and
the resulting impact on the ability to generalize to (and
across) times, settings, or persons.
External validity is high when the results of an
experiment are applicable to a larger population.
Three major threats to external validity are as follows:
Reactivity of testing on X
Interaction of selection and X
Other reactive factors
Reactivity of testing on X
• The reactive effect refers to sensitizing participants
via a pretest so that they respond to the experimental
stimulus (X) in a different way.
•For instance, people who participate in a web survey
may then be sensitized to store displays and
organization.
Interaction of selection and X
The process by which test participants are selected for
an experiment may be a threat to external validity.
The population from which one selects participants
may not be the same as the population to which one
wishes to generalize the results.
It limits the generalizability of the findings.
Other reactive factors
•The experimental settings themselves may have a
biasing effect on a participant’s response to X.
•An artificial setting can produce results that are not
representative of larger populations.
•If participants know they are participating in an
experiment, there may be a tendency to role-play in a
way that distorts the effects of X.
•Another reactive effect is the possible interaction
between X and participant characteristics.
Business
Research Methods
Donald R. Cooper and Pamela S. Schindler
Chapter 12.2.:
Types of Experimental
Research Designs
Chapter 12.2.:
Types of Experimental
Research Designs
1. Pre-experiments
2. True experiments
3. Field experiments
•X refers to the treatment or manipulation of the
independent variable (more than one X refers to a
different level of treatment).
•O refers to the observation or measurement of the
dependent variable.
•Experimental designs vary widely in their power to
control contamination of the relationship between the
independent and dependent variables.
•Experiments can be categorized as pre-experiments,
true experiments, and field experiments based on the
characteristic of control.
1. Pre-experiment
Pre-experimental research designs are research
designs that are characterized by a lack of random
selection and assignment.
Types of Pre-experiments:
•After-Only Case Study
•One Group Pretest-Posttest Design
•Static Group Comparison
1.1. After-Only Case Study
X
O
• In this type of experimental design only one
treatment (X) or manipulation is done on the
independent variable.
• Then, the dependent variable is measured.
X
O
An example is a media campaign about a product’s
features without a prior measurement of consumer
knowledge.
Results would reveal only how much target consumers
know after the media campaign, but there is no way to
judge the effectiveness of the campaign.
The lack of a pretest and control group makes this
design inadequate for establishing causality.
1.2. One Group Pretest-Posttest Design
O1
X
O2
This design meets the threats to internal validity better
than the one-shot case study, but it is still a weak design.
For example, a researcher examining the effect of a
commercial on brand liking would begin by taking a
pre-test to determine current levels of brand liking
among the participants.
O1
X
O2
The commercial would be shown.
Then a post-test would measure brand liking after the
commercial.
A comparison between the post-test and the pre-test
shows the change in liking.
However, any changes in liking are not necessarily due
to the commercial.
The act of giving a pre-test could have influenced liking
(testing effect).
1.3. Static Group Comparison
Experimental Group:
Control Group:
X
O1
O2
This design provides for two groups, one of
which receives the experimental stimulus while
the other serves as a control.
For example, imagine that a new type of cheeseburger
is being introduced, and an advertisement campaign is
run.
After the ad airs, those who remember seeing it would
be in the experimental group (X). Those who have no
recall of the ad would be in the control group.
The intent of each group to purchase the cheeseburger
would be measured.
The main weakness of this design is that there is no
way to be certain that the two groups are equivalent or
that the individuals are representative.
2. True experiment
• A true experiment is a method of social research in
which there are two kinds of variables. The
independent variable is manipulated by the
experimenter, and the dependent variable is
measured.
• The signifying characteristic of a true experiment is
that it randomly allocates the subjects in order to
neutralize the potential to ensure equivalence.
• There is also a control group for comparison.
Types of True experiments:
• Pretest-Posttest Control Group Design
• Posttest-Only Control Group Design
2.1.Pretest-Posttest Control Group Design
Experimental Group:
Control Group:
R
R
O1 X
O3
O2
O4
• The symbol R means that the true experimental designs
use randomly assigned groups to ensure equivalence.
• The effect of the experimental is: E = (O2-O1) – (O4-O3).
• This design deals with many of the threats to internal
validity, but local history, maturation, and
communication among groups can still lead to problems.
• External validity is threatened because there is a chance
for a reactive effect from testing.
2.2. Posttest-Only Control Group Design
Experimental Group:
Control Group:
R
R
X
O1
O2
• In this design, the pretest measurements are omitted.
• Pretests are well established in classical research design
but are not really necessary when it is possible to
randomize.
• The experimental effect is measured by the difference
between O1 and O2.
• Internal validity threats from history, maturation,
selection, and statistical regression are controlled
adequately by the random assignment.
• Different mortality rates could cause a problem.
Example for Posttest-Only Control Group Design
• Buick dealerships wish to determine the effectiveness
of a special “test-drive” incentive.
• Buick dealerships nationwide are randomly assigned
to either the control group or the experimental group.
• Those in the experimental group use a promotion to
encourage test drives.
• The control group does not use any such promotions.
• The number of test drives throughout are measured
and compared to determine if the promotion resulted
in significantly more test drives.
3. Field experiment
• Experiment conducted in a natural setting
(e.g. on a sports field during play). The
conditions of field experiments are usually
very difficult to replicate.
Types of Field experiments:
• Nonequivalent Control Group Design
• Separate Sample Pretest-Posttest Design
• Group Time Series Design
3.1. Nonequivalent Control Group Design
Experimental Group: O1
Control Group:
O3
X
O2
O4
• This is a strong and widely used quasiexperimental design.
• It differs from the pretest and posttest control
group design because the test and control
groups are not randomly assigned.
• There are two varieties: intact equivalent
design and self-selected experimental group
design.
Nonequivalent Control Group Design
• In the intact equivalent design, the membership of
the experimental and control groups is naturally
assembled.
• The self-selected experimental group design is
weaker because volunteers are recruited to form the
experimental group, while non-volunteer
participants are used for control.
• A comparison of the pretest results for each group
is one indicator of the degree of equivalence
between test and control groups.
Example for Nonequivalent Control Group Design
• For example, children from two different classes in
school may be asked to test a toy.
• Participants are pre-tested on their interest in the toy.
• The experimental group spends time playing with the
toy while the control group is not exposed to the toy.
• A post-test then measures interest in the toy.
3.2.Separate Sample Pretest-Posttest Design
Experimental Group:
Control Group:
R
R
O1 (X)
X
O2
• This design is most applicable when we cannot know
when and to whom to introduce the treatment but we
can decide when and whom to measure.
• The parenthesized treatment (X) means that the
experimenter cannot control exposure to the treatment.
• This is not a strong design because several threats to
internal validity are not handled adequately.
• History can confound the results.
Example for Separate Sample PretestPosttest Design
• For example, an new advertising campaign for a
prescription drug is introduced on television.
• Awareness of the brand name is measured prior to
the campaign introduction. After the campaign
ends, awareness is measured again.
3.3. Group Time Series Design
R
R
O1 O2 O3 X O4 O5 O6
O7 O8 O9 O10 O11 O12
A time series design introduces repeated observations
before and after treatment and allows participants to act
as their own controls.
The single treatment group design has before-after
measurements as the only controls.
There is also a multiple design with two or more
comparison groups as well as the repeated
measurements in each treatment group.
• This format is especially useful where regularly kept
records are a natural part of the environment and are
unlikely to be reactive.
• The time series approach is also good way to study
unplanned events in an ex post facto manner.
• The internal validity problem for this design is
history. To reduce this risk, we keep a record of
possible extraneous factors and attempt to adjust the
result to reflect their influence.
• For example, if the government were to begin price
controls, we could still study the effects of this action
on gasoline prices later if we had regularly collected
records for the period before and after the advent of
price control.