Transcript Response rate

Research methodology in
management sciences
DBA IMI / GOLESTAN
1
RM1
Research Methodology (1)
2
INTRODUCTION
• What is research?
• Ways of knowing
• Characteristics of research
• What is “good” research?
• Approaches to research
• Research process overview
3
What is research?
• The systematic, controlled, empirical, and
critical examination of hypothetical propositions
about a phenomenon in order to enhance
knowledge
4
Ways of knowing
• Authority
• Intuition
• Experience
• Research
5
Characteristics of common sense
• Unless we decide not to, we usually observe
inaccurately
• We usually generalize from only a few cases
• We observe selectively to see what we’re
looking for
• We make things up to fill in the gaps
• We believe in luck and fate
• We get personally and emotionally involved
6
Characteristics of research
• We consciously decide how to observe
• We explicitly sample for generalizing
• We consciously decide what to observe
• We base conclusions only on the evidence
• We believe in probability
• We have to respect scientific norms regardless
of opinions
7
In summary
Research is more conscious and more careful
than knowing through authority, intuition or
experience
8
What is “good” research?
• Thoughtful
• Carefully planed
• Theoretically grounded
• Carefully conducted
9
Deductive approach
• Theory generates…
• Hypotheses, which are then subjected to…
• Observation and possibly lead to…
• Confirmation
10
Inductive approach
• Observation generates…
• Patterns leading to…
• Tentative hypothesis, that is next integrated
within a…
• Theory
11
Positivism
• The world does exist
• It is possible to study it objectively
12
Constructivism/Interpretivism
• The world does not exist
• It is impossible to study it objectively
13
Other “isms”
• Post-positivism
• Relativism
• Post-modernism
• Critical realism
14
The research process (1)
• A. Plan the research
– Define the research question(s)
– Define the population and sampling method
– Determine variables of interest and measures
– Determine model /set of hypothesis
– Select research design and statistical tests
– Write proposal
15
The research process (2)
• B. Conduct the research
– Identify sample
– Conduct experiment/ collect data
– Analyze data
– Test hypotheses / research questions
• C. Report the results
– Write research report
– Write academic paper
16
RM2
Research Methodology (2)
17
18
19
20
21
22
23
24
25
26
27
28
29
Ad. Cost
Independent
variable
Customer
ability
Intervening variable
Price
Moderator variable
Sell quantity
Dependent variable
Media
Control variable
30
31
32
33
34
35
36
37
38
39
40
41
42
2
n=
Z pq
d
2
43
44
Response rate is…
# that answered
# you contacted
The proportion of people who return the survey
questionnaire.
It is calculated by dividing the number of returned
surveys by the total number of surveys distributed.
Example: If you distribute 250 questionnaires and
you get 85 questionnaires back, your response rate
is 34%.
45
Low response to our surveys
We often send out surveys and find that few are
returned. Response rates of 30% and lower are
common in Extension. Often, the number of
returned surveys is too small to aggregate in a
meaningful way or make any comparisons.
Best practice says that a response rate under 70%
should be a warning.
46
Why is response rate important?
It’s the only way to know if your survey results
are representative.
 High response rate promotes confidence in
results.
 Low response rate increases the probability of
biased results.
 a higher response rate is preferable because
the missing data is not random
47
Comparing Early, Late, and NonRespondents
• Identify subjects who responded to the first mailing within the
deadline date, and label them as early. Similarly, identify all other
subjects who responded to subsequent mailings, and label them as
late. After the data collection is complete, identify and label the
non-respondents. According to Miller and Smith (1983), nonrespondents tend to be similar to late respondents in responding to
surveys. Therefore, compare the early and late respondent groups
on key variables (Figure 1). If you find no significant differences
between early and late respondents, you can statistically conclude
that non-respondents are perhaps similar to late respondents and
thus generalize the findings to the population. The other accepted
procedure is to follow-up with a telephone call to 15-20% of the
non-respondents, and collect data from them on key variables.
Then do a comparison between early and late, early and nonrespondents, and late and non-respondents.
48
Logic of Comparing Early, Late, and Non-Respondents
49
• If the comparison indicates no differences between
these three groups of respondents, then you can
generalize the findings to the population. On the other
hand, if you find significant differences, you cannot
generalize the findings to the population. Therefore,
use your judgment as to whether or not to include the
differed variables for final analysis. Normally, the
differed variables are eliminated from further analysis.
Explain why the subjects differed on the key variables.
Otherwise, provide justification for including the
differed variables in the final analysis.
• Use independent t-test to compare early and late
respondents; early and non-respondents; and late and
non-respondents. Use ANOVA if you want to compare
all three (early, late, and non respondents) response
types and conduct a post-hoc analysis to determine
group (early, late, and non respondents) differences.
50
Response rate (RR)
•How to achieve a high response rate?
•Getting a high response rate (>80%) from a small,
random sample is considered preferable to a low
response rate from a large sample.
51
There is no standard response rate
“The higher, the better.”
While it is not actually the % that matters but
WHO responds, a higher response rate means
that you can be more sure that the answers
reflect the population.
So, we want to remove barriers and
motivate as many people as possible to complete
and return the questionnaire.
How can we do that?
52
Most important things that
influence response rate:
 Importance of the topic – interest in the topic of
the survey
 Personalized request and communications
 Multiple follow-up contacts
 Sponsor of the survey is respected, trusted
 Questionnaire is brief and easy to complete
53
Checking Representativeness
• Early vs. late Respondents
• Respondents vs. non- Respondents
54
55
10 Ways to increase response rate
1. Generate positive publicity for your survey.
2. Appeal to people’s helping tendencies – ask
them to help by providing their input.
3. Make the survey topic salient – important
•
•
Ensure that respondents see the value of the
survey and their response.
Point out their personal connection to the topic
4. Tailor, personalize communications
5. Make the questionnaire attractive and easy to
complete AND easy to return
56
10 ways to increase response rate,
cont.
6. Provide incentives (token of appreciation)
7. Show positive regard; Say thank you
8. For mail survey, provide 1st class
postage/return postage
9. Make (multiple) follow-up contacts
– by mail, email, telephone, in person…
10. Use a combination of survey modes –
telephone plus mail; internet plus mail.
57
RM3
Research Methodology (3)
58
59
60
61
62
1.
2.
3.
4.
5.
Temperature in Celsius degrees (from 10°C to 20°C) ?
Age (from 0 to 99 years) ?
Date (from 1457 BC to AD 2013) ?
beautiful vs. ugly?
male vs. female?
63
64
The term "level of analysis" points to the location, size, or scale of a research target
65
66
67
True Score Theory
68
True Score Theory
is a theory about measurement. Like all theories, you
need to recognize that it is not proven it is postulated
as a model of how the world operates. Like many
very powerful model, the true score theory is a very
simple one. Essentially, true score theory maintains
that every measurement is an additive composite of
two components :true ability (or the true level) of
the respondent on that measure; and random error.
69
True Score Theory
• We observe the measurement -- the score on the test,
the total for a self-esteem instrument, the scale value
for a person's weight. We don't observe what's on the
right side of the equation (only God knows what those
values are!), we assume that there are two
components to the right side.
• The simple equation of X = T + eX has a parallel
equation at the level of the variance or variability of a
measure. That is, across a set of scores, we assume
that:
Var (X) = var (T) + var (eX)
70
71
Respondent error
• In survey sampling, respondent error refers to any error
introduced into the survey results due to respondents
providing untrue or incorrect information. It is a type of
systemic bias .
• Several factors can lead to respondent error :
• Misunderstanding (Language and educational )
• Recall bias can lead to misinformation (misrecalling the facts
in question)
• Social desirability bias(he or she thinks is correct or better or
less embarrassing, rather than providing true and honest
responses)
72
Administrative error
• Improper administration or execution of a survey results
in administrative errors. Such errors can be caused by
carelessness, confusion, neglect, omission or another
blunder. There are four types of administrative errors.
73
1- Data-processing error
A category of administrative error that occurs in data
processing because of incorrect data entry, incorrect computer
programming or other error during data analysis.
2- Interviewer error
This type of administrative error is caused by failure of an
interviewer to correctly pose questions or record responses.
Interviewer error generally leads to biased results, and perhaps
to an increase in variability
3- Interviewer cheating
The practice of filling in fake answer or falsifying questionnaire while
working as an interviewer.
4- Sample selection error
Selection bias : an administrative error caused by improper
selection of a sample during a survey , resulting in accidental
bias in the results.
74
75
76
Same time : consistency
Different time : stability/consistency 77
78
79
80
81
82
83
84
Construct validity
•
•
•
•
•
What is construct?
Construct is a trait or attribute
You cant measure it.
But you can evaluate it by theories bakground
Intelligence , creativity , love
85
86
• Construct validity seeks agreement between
a theoretical concept and a specific
measuring device or procedure.
Construct validity can be broken down into
two sub-categories:
1- Convergent validity
2- Discriminate validity
87
88
Construct validity
• Convergent validity
• Discriminate validity
89
1. Predictive
2. Concurrent
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
Issues with unstructured format
•
•
•
•
Can generate unpredictable responses
Dependent on number of respondent
Requires content analysis
Places a greater load on the respondent –
important issues not occur to them
112
113
114
115
116
117
RM5
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184