9 Quantitative Research.ppt
Download
Report
Transcript 9 Quantitative Research.ppt
Quantitative
Research
Experimental Research
Laboratory Experiments
Artificial – low realism
Few extraneous variables
High control
Low cost
Short Duration
Subjects aware of
participation
Field Experiments
Natural – high realism
Many extraneous variables
Low control
High cost
Long Duration
Subjects unaware of
participation
Quantitative Research
Emotional Intelligence Survey
http://quiz.ivillage.co.uk/uk_work/tests/eqtest.htm
Validity
Extent to which a measurement instrument
actually measures the attribute it was
intended to measure.
Validity can be examined from a number of
different perspectives, including:
Face,
content, criterion-related, and construct
validity
Assessing Validity
Face validity
Researchers judge the degree to which a measurement
instrument seems to measure what it is supposed to.
Content validity
The degree to which the instrument items represent
the universe of the concept under study.
Criterion-related
validity
The degree to which a measurement instrument can
predict a variable that is designated a criterion.
(a) Predictive ability; (b) Concurrent validity
Construct validity
The degree to which a measure confirm a hypothesis
created from a theory based upon the concepts under
study.
(a) Convergent validity; (b) Discriminate validity
Reliability
Reliability
Degree
to which measures are free from
random error and, therefore, provide
consistent data.
There are three ways to assess reliability
Test-retest,
equivalent forms, and internal
consistency
Assessing the Reliability of a
Measurement Instrument
Test-retest reliability:
Use the same instrument a second time under
as nearly the same conditions as possible.
Equivalent form reliability:
Use two instruments that are as similar as
possible to measure the same object during the
same time period.
Internal consistency reliability:
Compare different samples of items being
used to measure a phenomenon during the
same time period.
Reliability and Validity Situations
in Measurement
Situation 1
.
.
.
.
.
.
Situation 2
.
Situation 3
.........
. .
.
.. ..
..
.
.
Neither reliable
nor Valid
Highly reliable
but not valid
Highly reliable
and valid
Error
Types of Error in Survey
Research
Random Sampling Error (Random error)
Error
that results from chance variation
Impact can be decreased by increasing
sample size and through statistical
estimation (confidence interval) or “rule of
thumb”
Systematic Error (non sampling error)
Error
that results for the research design or
execution.
Total Error
Random Error
Systematic Error
Respondent error
Administrative error
Non response error
Response bias
Deliberate Falsification
Acquiescence
Extremity bias
Unconscious misrepresentation
Interviewer bias
Auspices bias
social desirability
Data Processing
Sample selection
Interviewer cheating
Interviewer error
Types of Systematic Error
1. Administrative Error
Error
that results from improper execution.
Data Processing Error
Quality
of data depends on quality of data
entry.
Use of verification procedures can
minimize
Sample Selection Error
Systematic
error resulting from improper
sampling techniques either in design or
execution.
Interviewer Error
Data
recorded incorrectly (error or
selective perception).
Interviewer Cheating
Mitigate
by random checks
2. Respondent Error
Humans
interviewing humans...
Non-response error
Statistical
difference between a survey that
includes only those who responded and a
survey that also includes those who failed to
respond.
Non-respondent: person not contacted or who
refuses to participate
Self selection bias: extreme positions
represented
Response bias
Errors
that result from tendency to answer in
“a certain direction”.
Conscious or unconscious misrepresentation
Types:
1.
Deliberate falsification (why?)
Why would people deliberately falsify data
Appear
to be what they are not
Don’t trust confidentiality
Protect
To end the interviewer quicker
“Average man effects”
Types of response bias continued:
1.
Deliberate falsification
2. Unconscious misrepresentation
Reasons for unconscious
misrepresentation:
Question
format
Question content
Misunderstanding of question leading to
biased answer
Lack of time to consider answer fully
Communication or semantic confusion
other
Types of response bias
Acquiescence bias: individuals have a
tendency to agree or disagree with all
questions or to indicate a positive/negative
connotation
Extremity bias: results for response styles
varying from person to person; some
people tend to use extremes when
responding to questions
Types of response bias continued...
Interviewer bias: Bias in the responses of
the subject due to the influence of the
interviewer
Auspices bias: respondents being
influenced by the organization conducting
the study
Social desirability bias: caused by
respondents’ desire, either consciously or
unconsciously to gain prestige or to
appear in a different social role
Measurement
The Concept of Measurement
and Measurement Scales
• Measurement
• Process of assigning numbers or labels to things
in accordance with specific rules to represent
quantities or qualities of attributes.
• Rule: A guide, method, or command that tells a
researcher what to do.
• Scale: A set of symbols or numbers constructed to
be assigned by a rule to the individuals (or their
behaviors or attitudes) to whom the scale is applied.
Types of Measurement Scales
• Nominal Scales
• Scales that partition data into mutually
exclusive and collectively exhaustive
categories.
• Ordinal Scales
• Nominal scales that can order data.
• Interval Scales
• Ordinal scales with equal intervals between
points to show relative amounts; may include
an arbitrary zero point.
• Ratio Scales
• Interval scales with a meaningful zero point so
that magnitudes can be compared
arithmetically.
Nominal
Win
Place
Show
Ordinal
Interval
1 length
2 lengths
Ratio
40 to 1 long-shot pays $40
Type of Scale
Nominal
Numerical
Operation
Counting
Descriptive
Statistics
Frequency;
Percentage;
mode
(plus…)Median
Range; Percentile
Ordinal
Rank ordering
Interval
Arithmetic
operations on
intervals bet
numbers
(plus…) Mean;
Standard
deviation;
variance
Ratio
Arithmetic
operations on
actual quantities
(plus…)
Geometric mean;
Co-efficent of
variation
Selecting appropriate univariate
statistical method
Scale
Nominal
Scale
Business
Problem
Identify sex
of key
executives
Statistical
question to
be asked
Possible test
of statistical
significance
Is the number Chi-square
of female
test
executives
equal to the
number of
males
executives?
Scale
Nominal
Scale
Business
Problem
Indicate
percentage of
key
executives
who are male
Statistical
question to
be asked
Possible test
of statistical
significance
Is the
T-test
proportion of
male
executives
the same as
the
hypothesized
proportion?
Scale
Ordinal
scale
Business
Problem
Compare
actual and
expected
evaluations
Statistical
question to
be asked
Possible test
of statistical
significance
Does the
Chi-square
distribution
test
of scores for
a scale with
categories of
poor,good,
excellent
differ from
an expected
distribution?
Scale
Interval or
Ratio scale
Business
Problem
Statistical
question to
be asked
Compare
actual and
hypothetical
values of
average
salary
Is the
sample
mean
significantly
different
from the
hypothesize
d population
mean?
Possible test
of statistical
significance
Z-test
(sample is
large)
T-test
(sample is
small)
Surveys
Survey data
Survey data are obtained when individuals
respond to questions asked by
interviewers or when the individual
responds to questions that he has read
Quantitative and qualitative information
Census or sample?
Survey Sampling
Sample Survey
Sample
of the population
(e.g., sample of Alberta CEOs)
Census Survey
Complete
population
(e.g., all CEOs in Lethbridge)
Functions
Descriptive
Causal (limited function)
Exploratory (limited function)
Information Provided by
Surveys
respondent’s knowledge of facts
how many times a month do you buy cookies?
respondent’s attitudes
“chocolate
May describe processes undertaken by
respondent
Stages
chip is my favorite variety of cookie.”
in the purchase decision, for example
usually self-report data
The type of information gathered
depends on a survey’s objectives.
Surveys typically have multiple
objectives
Classifications
Method of Communication
Telephone
Mail
Personal
interview
Degree of Structure
Unstructured
structured
Degree of Disguise
Limitations:
Structure and disguise
are not clear categories;
most surveys are hybrids
Temporal classifications
Cross
sectional studies
Longitudinal studies
Trend Studies
Cohort studies (tracking studies)
Samples general population at each point; complete
turnover in who is actually sampled
Samples from one group over time; e.g., sample of a
graduating class
Panel studies
sample cohort, same specific respondents each sample
Method of Communication
Media
Personal Interviews
Telephone Interviews
Self administered Questionnaires
Questionnaire
design
A survey is only as good as
the questions it asks
What should you ask?
The questions asked are a function of
previous decisions
The questions asked are a function of
future decisions (such as statistical
analysis)
Key criteria
Questionnaire relevancy
No
unnecessary information is collected and
only information needed to solve the problem
is obtained. Be specific about your data
needs; tie each question to an objective
Questionnaire accuracy
Information
is both reliable and valid
Phrasing Questions
Open ended response versus fixed
alternative questions
“?”
Decision criteria: type of research; time;
method of delivery; budget; concerns
regarding researcher bias
Avoid
Leading questions
Overly complex questions
Use of jargon
Loaded questions (can use a counterbiasing
statement)
Ambiguity
Double barreled questions
Making assumptions
Order?
Order bias results from an alternative
answer’s position in a set of answers or
from the sequencing of questions
Funneling
technique: general to specific helps
understand the frame of reference first
Anchoring effect: the first concept
measured tends to become a comparison
point from which subsequent evaluations
are made
Types of questions
Types of fixed alternative questions…
• Single dichotomy or dichotomous-alternative
questions
“Are you currently registered in a course at the
University of Lethbridge?
Yes____ No____”
• Respondent chooses one of two alternatives
(yes/no; male/female)
• What scale would this data create?
Types of fixed alternative questions…
Multi-choice
alternative
Respondent
chooses from several
alternatives
Many types…
Multi-choice alternative questions…
Determinant choice
Choose
only one from several possible responses
“Which faculty are you currently registered in at
the University of Lethbridge?
Management ___
Education ____
Arts/Science____
Health sciences____
Combined degree____
What type of scale would these data create?
Frequency determination
Asks for an answer about frequency
of occurrence
In a typical week, how often do you
purchase chocolate chip cookies?
__never
__ once
__ 2 or more times
What type of scale would these data create?
Check list
Provide
multiple answers to a single
question
Should be mutually exclusive and
exhaustive
“What brands of chocolate chip cookies have
you, to the best of your memory, purchased
in the past month (check all that apply?)”
__ Dare
__ Chips A’hoy
__ Presidents Choice Decadent etc. etc.
What type of scale would these data create?
Unidimensional
Scaling
Multidimensional
Scaling
Procedures
designed to
measure only one
attribute of a
respondent or
object
Procedures
designed to measure
several dimensions
of a respondent or
object
Attitude
rating scales
Attitude:
An enduring disposition to consistently
respond to various aspect of the
world, including persons, events and
objects
Typically seen as having three
components:
Cognitive
Affective
Behavioural
Affective
The feelings or
emotions toward an
object
Cognitive
Knowledge and beliefs
Behavioral
Predisposition to action
Intentions
Behavioral expectations
Attitude measuring process
Ranking
Rating
Sorting
Choice
Types of attitude scales
Simple attitude scales
Most basic form – respondent responds to a
single question
Do not allow for fine distinctions or placement on
continua
You
are at a company party and are feeling nervous,
but you are obligated to be there. Do you:
__ find someone you know to buddy up with
__ take it as an opportunity to meet new people
What type of scale would these data create?
Category scales
More
sensitive; provides more information
Overall, how satisfied are you with the high speed
performance of your Mercedes:
__ very satisfied
__ somewhat satisfied
__ neither satisfied nor dissatisfied
__ somewhat dissatisfied
__ very dissatisfied
If you could choose, how long would each term be?
___26 weeks __ 13 weeks __ 6 weeks ___4
weeks
What type of scale would these data create?
Summated rating scales – the Likert
scale
Respondents
indicate their attitudes by
checking how strongly they agree or
disagree with statements
Chocolate
chip cookies are my preferred
variety of cookie
Strongly disagree Disagree Uncertain Agree Strongly Agree
(1)
(2)
(3)
(4)
(5)
What type of scale would these data create?
Semantic Differential Rating scale
An
attitude measure consisting of a series
of seven-point bipolar rating scales
allowing response to a “concept”
Think of your favorite type of cookie. Rate it on each
of the following continua:
Hard------------------------------------------------------Soft
Lots of chips---------------------------------------Fewer
chips
Crispy---------------------------------------------------chewy
What type of scale would these data create?
Numerical Rating scale
Similar
to a semantic differential except that it
uses numbers as response options to identify
response positions instead of verbal descriptions
Think of your favorite type of cookie. Rate it on each of the
following continua:
Hard------------------------------------------------------------------------Soft
8
7
6
5
4
3
2
1
This scale is called an 8 point numerical scale, why?
What type of scale would these data create?
Constant Sum Scales
Attributes
based on their importance to the
person. Respondents are asked to divide a
constant sum to indicate the relative importance of
attributes
Example: Suppose the photocopy budget per
professor was $100 per month. How much should
be allocated to the following. Divide the $100
according to your preference:
____ photocopying for student needs;
____ photocopying for research needs;
____ photocopying for committee needs.
====
$100 TOTAL
Stapel Scales
An
attitude measure that places a single adjective
in the center of an even-number range of
numerical values
Example:
Research Methodology
+3
+2
+1
Exciting
-1
-2
-3
Graphic Rating Scales
An
attitude measure consisting of a graphic
continuum that allows respondents to rate
an object by choosing any point on the
continuum
Rank-Order Scales
Scales
in which the respondent compares
one item with another or a group of items
against each other and ranks them.
Example: handout
Most important skills
Adaptability to change
Problem identification
Listening skills
Written communication
Leadership
Informal Oral communication
Analytical thinking/problem solving
Time management
Coping with stress/job pressures
Interpersonal relations
Formal oral presentations
Most important skills
Adaptability to change
Problem identification
Listening skills
Written communication
Leadership
Informal Oral communication
Analytical thinking/problem solving
Time management
Coping with stress/job pressures
Interpersonal relations
Formal oral presentations
8
6
1
2
4
3
5
7
11
9
10
9
6
1
4
2
3
5
10
7
8
11
Paired Comparison Scales
Respondent
is presented with two objects
and is asked to pick the preferred.
Example: Which type of cookie do you prefer
__ chocolate chip
__ oatmeal
__ I do not have a preference between these two
Sorting
Respondent
indicates their attitudes or
beliefs by arranging items.
Example: Please sort the following cards with
pictures of cookies into the following categories
Like
Dislike
Neither like nor dislike