Quantitative Research Methods
Download
Report
Transcript Quantitative Research Methods
QUANTITATIVE RESEARCH
METHODS
Irina Shklovski
Quantitative Research Methods
Include a wide variety of laboratory and nonlaboratory procedures
Involve measurement…
Quantitative Research Methods
Measurement
Populations
and Sampling
Random Assignment
Generalizability
Quantitative Research Methods
Measurement
Populations and Sampling
Random Assignment
Generalizability
Time
Cross-sectional studies & single experiments
Longitudinal studies & repeated measures
Quantitative Research Methods
Method
Experiments
& Quasi-experiments
Behavioral Measures
Questionnaires & Surveys
Social Network Analysis
Archival and Meta-Analysis
What we will talk about today
Measurement
Population
& Sampling
Random Assignment
Generalizability
Method
Experiments
& Quasi-experiments
Questionnaires & Surveys
Measurement – Sampling
Specify your population of concern
Sampling
Selecting
respondents from population of concern
Random sampling
Systematic selection
Stratified sampling
Convenience sampling
Snowball sampling
Sampling Biases
Non-response bias
Be
persistent
Offer incentives and rewards
Make it look important
Volunteer bias
Some
people volunteer reliably more than others for
a variety of tasks
Random assignment
Different from random sampling
Mostly used for experiments or quazi-experiments
Protects against unsuspected sources of bias
Does NOT guarantee to balance out the differences
between participants
Chance is LUMPY
Generalizability
How do you know that what you found in your
research study is, in fact, a general trend?
Does A really, always cause B?
If A happens, is B really as likely to happen as you
claim? Always? Under certain conditions?
Association vs. Causality
Thanks to
Sara Kiesler
for these graphs!
Experiments & Quasi-experiments
ex·per·i·ment
Pronunciation: \ik-ˈsper-ə-mənt also -ˈspir-\
Function: noun
Etymology: Middle English, from Anglo-French esperiment,
from Latin experimentum, from experiri
Date: 14th century
An operation or procedure carried out under controlled
conditions to discover an unknown effect or law, to test
or establish a hypothesis, or to illustrate a known law
Experiments & Quasi-experiments
Key feature common to all experiments:
To
deliberately vary something in order to discover
what happens to something else later
To seek the effects of presumed causes
An Experiment is
A controlled empirical test of a hypothesis.
Hypotheses include:
A causes B
A is bigger, faster, better than B
A changes more than B when we do X
Two requirements:
Independent variable that can be manipulated
Dependent variable that can be measured
Experiments in Research
Comparing one design or process to another
Deciding on the importance of a particular feature
in a user interface
Evaluating a technology or a social intervention in a
controlled environment
Finding out what really causes an effect
Finding out if an effect really exists
Remember
Experiments explore the effects of things that can
be MANIPULATED
(but there is a caveat)
Types of Experiments
Randomized – units/participants assigned to receive
treatment or alternative condition randomly
Quazi – no random assignment
Natural – contrasting a naturally occurring event
(i.e. disaster) with a comparison condition
If your study involves experiments
Experimental design:
Shadish W.R., Cook T.D. & Campbell P.T. (2002) Experimental
and Quasi-Experimental Design for Generalized Causal
Inference. Boston, Mass: Houghton Mifflin
Experimental data analysis:
Bruning, J. L. & Kintz, B. L. (1997). Computational handbook of
statistics (4th ed.). New York: Longman.
Questionnaires & Surveys
Self-report measures
Questionnaires
Interviews
Diaries
Types
Structured
Open-ended
& surveys
Questionnaires & Surveys
Advantages
Sample
large populations (cheap on materials & effort)
Efficiently ask a lot of questions
Disadvantages
Self-report
is fallible
Response biases are unavoidable
Response biases
Relying on people’s memory of events & behaviors
Emotional
states can “prime” memory
Recency effects
Routines are deceiving
Social desirability
Solution:
none that are simple
Yea-saying
Solution:
vary the direction of response alternatives
General Survey Biases
Sampling – are respondents representative of
population of interest? How were they selected?
Coverage – do all persons in the population have
an equal change of getting selected?
Measurement – question wording & ordering can
obstruct interpretation
Non-response – people who respond differ from
those that do not
Design is KEY
Format – booklet, printed vertical, one-sided
Question ordering – earlier questions can prime
answers to later questions
Page layout – group similar items & use consistent
fonts and response categories
Pre-testing – conduct think-alouds with volunteers
demographically similar to expected participants
Common Problems
Avoid complicated & double-barrel questions
Complexity
increases errors & non-response
Navigation is paramount – make sure the survey is
EASY to follow
Open-ended questions
The
size of the field allotted will determine the number
of words
Incentive is key
BUT
amount differences have little impact
If your study involves surveys
Designing surveys:
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet,
mail, and mixed-mode surveys : the tailored design method (3rd
ed.). Hoboken, N.J.: Wiley & Sons.
Fowler, F. J. (1995). Improving survey questions : design and
evaluation. Thousand Oaks: Sage Publications.
Analyzing data:
Cohen, J., Cohen, P., West, S., & Aiken, L. (2003). Applied
multiple regression/correlation analysis for the behavioral
sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum
Associates.
So… what?
Difference between quantitative methods is in the
questions they can answer
There are a LOT of methods and even more
statistical techniques
Regardless of the method, if it’s not an experiment,
you CAN NOT prove causation
Things we did NOT talk about
Reliability assessments
Validity assessments
Statistical analysis of data
Interpretation of results