Transcript GSSR Research Methodology and Methods of Social Inquiry www.socialinquiry.wordpress.com December 20, 2010 Survey Research (cont.

GSSR
Research Methodology and Methods of Social Inquiry
www.socialinquiry.wordpress.com
December 20, 2010
Survey Research (cont. De 13)
Research Using Available Data
II. Survey Research (continued Dec. 13)
Survey Modes
• Questionnaires;
• Interviews;
• Mixed-mode Surveys
Questionnaires:
- the respondent completes himself/herself & returns by a
specified deadline;
Mail questionnaire (pen-and-pencil; internet)
Group-administered questionnaire
Household drop-off surveys (home/business)
Interviews:
- completed by the interviewer based on what the
respondent says;
Telephone, Internet, Face-to-Face, Focus Groups
CATI
CAPI
Mixed-mode Surveys:
- combination of modes to sample and/or collect the data.
Constructing the Survey
The Survey instrument should:
- ensure effective two-way communication between
respondents and the researcher;
- assist respondents in recalling & clarifying their
experiences, attitudes and thoughts;
- keep the respondent interested and motivated.
Types of Questions
A. Open-ended & closed-ended questions
B. Direct and Indirect questions
C. Funnel sequence questions
D. Reason Analysis
E. Filter or Contingency Questions
See www.socialresearchmethods.net/kb/questype.php
A. Open-ended questions:
How would you rate the President’s performance in office
so far?
Closed-ended questions:
How would you rate the President’s performance in office
so far?
() Poor
() Below average
() Average
() Above average
() Excellent
See Singleton & Straits, p. 269-270 for considerations on when to
choose open vs. closed-end questions.
B.
Direct and Indirect questions
“Would you mind having a woman as supervisor?”
vs.
“Do you believe your co-workers would mind having a
woman as supervisor?”
C.
-
Funnel sequence (questions):
move from a very general question to progressively
more specific questions.
Inverted funnel sequence: begins with the most specific
questions, and ends with the most general
D. Reason Analysis:
- well-devised serious of questions to get at R’s decision
process, instead of asking just “Why?”
Understanding students’ choice of going to UW:
1. Decision to go to college
“When did you first consider seriously going to college? How did you
reach this decision?
2. Selection of schools applied to
“Did you apply to any other colleges and univ? Which ones?”
3. Final choice of UW
“Did your parents, friends, teachers or other persons help you come to
this decision? Who? How much influence did this have on you?”
E. Filter or Contingency Questions
- to determine whether the respondent is qualified to
answer your question of interest.
Response Formats
Structured & Unstructured
Structured
Dichotomous: the question has 2 possible responses;
Yes/No, True/False or Agree/Disagree response.
Questions using Rating Scales
Survey questions that attempt to measure on an interval level:
Likert response scale
Cumulative or Guttman scale
R check each item with which they agree. The items themselves
are constructed so that they are cumulative
The semantic differential rating approach: an object is assessed
by the respondent on a set of bipolar adjective pairs (using 5point, or 7-point rating scale):
General Issues on Questions & their Content
Is the question necessary/useful?
Are several questions needed?
Does it cover all possibilities?
Double-barreled question (a question in which 2 separate
ideas are represented together as a unit)
“What are your feelings towards African-Americans and
Hispanic-Americans?”
- “and” “or”
Does the question need to be more specific?
“How well did you like the book?” on scale ranging from
"Not At All" to "Extremely Well."
What does it mean to say you liked a book very well?
Instead:
“Did you recommend the book to others?”
or
“Did you look for other books by that author?”
Avoid Biased/emotionally loaded questions
Leading questions
- suggest a possible answer/ make some responses seem
more acceptable than others
“How often do you smoke marijuana?”
(for certain underreported behavior, this type of questions
may be necessary)
“Do you agree that ….”
Are the questions personal or sensitive (i.e. will
respondents answer truthfully)?
Can the question be misunderstood?
What assumptions does the question make?
Specify the time frame
“Do you think the government will increase taxes?”
Research Using Available Data
(Secondary Analysis)
- use of already existing sources of data;
- generally refers to re-analysis of quantitative data;
However: Content analysis
Content Analysis
- analysis of text documents;
- typically, the major purpose = identification of patterns in
text;
- unobtrusive;
- can be a relatively rapid method for analyzing large
amounts of text.
Involves:
- selecting and defining a set of content categories;
- defining & then sampling the elements of the text that
are described by the categories;
- quantifying the categories (ex. frequency of occurrence)
- analyzing the data: relating content categories to one
another, or to other variables.
Thematic analysis of text
The identification of themes or major ideas in a document
or set of documents
(field notes, newspaper articles, technical papers,
organizational memos, …)
Indexing
- wide variety of automated methods for rapidly indexing
text documents.
Ex: Key Words in Context (KWIC) analysis = computer
analysis of text data.
Key word = any term in the text that is not included in an
exception dictionary (exception dictionary includes all
non-essential words like "is", "and", and "of").
All key words are alphabetized & are listed with the text
that precedes and follows it, so the researcher can see
the word in the context in which it occurred in the text.
Quantitative descriptive analysis
- describe features of the text quantitatively
Basic systems of enumeration
1. Time/space measures;
Ex: space devoted to certain topics in newspaper articles;
2. Appearance; when does a category appear in the text;
3. Frequency;
Assumptions:
a) frequency of a category is a valid indicator of its importance, value,
intensity;
b) each individual count is of equal importance, value or intensity;
4. Intensity
-devise mechanisms to judge the intensity of the category
(similar to constructing scales and indexes).
Likely Methodological Problems
a) the types of information available in text form are limited
b) Bias due to sampling
c) Bias due to (mis)interpreting results of automated
content analyses