Survey Research

What is survey?

The survey is a non-experimental, descriptive research method Survey can be useful when a researcher wants to collect data on phenomena that cannot be directly observed (such as opinions and behaviors)

Surveys are Everywhere

Pop-up quick polls Evaluations On-line Product Registrations Email Initiated Surveys Product Downloads requiring survey Election Exit Polls

Types of surveys

Surveys can be divided into two broad categories: the

questionnaire

and the

interview

. Questionnaires are usually paper-and-pencil instruments that the respondent completes Interviews remind conversations Sometimes, it's hard to tell the difference between a questionnaire and an interview because you will see questionnaires with open-ended questions (although they do tend to be shorter than in interviews) and there will often be a series of closed-ended questions asked in an interview

Questionnaires: mail survey

When most people think of questionnaires, they think of the

mail survey

There are many advantages to mail surveys They are relatively inexpensive to administer You can send the exact same instrument to a wide number of people They allow the respondent to fill it out at their own convenience

Mail survey

But there are some disadvantages as well Response rates from mail surveys are often very low And, mail questionnaires are not the best vehicles for asking for detailed written responses

Group administered questionnaire

A sample of respondents is brought together and asked to respond to a structured sequence of questions Traditionally, questionnaires were administered in group settings for convenience High response rate If the respondents have questions about the meaning of a question they can ask for clarification

Focus group…

What's the difference between a group administered questionnaire and a group interview or focus group?

In the group interview or focus group, the interviewer facilitates the session People work as a group, listening to each other's comments and answering the questions Someone takes notes for the entire group -- people don't complete an interview individually.

Household drop-off survey

A researcher goes to the respondent's home and hands the respondent the instrument In some cases, the respondent is asked to mail it back or the interview returns to pick it up Like the mail survey, the respondent can work on the instrument in private, when it's convenient Like the group administered questionnaire, the interviewer makes personal contact with the respondent -- they don't just send an impersonal survey instrument And, the respondent can ask questions about the study and get clarification on what is to be done Generally, this would be expected to increase the percent of people who are willing to respond.

Interviews

Interviews are a far more personal form of research than questionnaires In the

personal interview

, the interviewer works directly with the respondent Unlike with mail surveys, the interviewer has the opportunity to probe or ask follow-up questions And, interviews are generally easier for the respondent, especially if what is sought is opinions or impressions

Telephone interview

Enable a researcher to gather information rapidly Allow for some personal contact between the interviewer and the respondent Allow the interviewer to ask follow-up questions Many people don't have publicly-listed telephone numbers.

Some don't have telephones People often don't like the intrusion of a call to their homes Telephone interviews have to be relatively short or people will feel imposed upon.

Survey

Survey is a series of questions asked of a number of people

Face-to-face (interview) Self-administered (mail, web) Phone Provide different answers

Reasons for differences

Social desirability and sensitive questions Acquiescence (culturally-based tendency to agree)

Social desirability

During fall semester, how often did you use drugs other than alcohol or marijuana, for example cocaine, speed, LSD, etc.?

 No Response  Never  Less than once a month but at least once in the past year  One to three times a month  One to two times a week  More than twice a week

Sensitive question

Since the beginning of the school year, have you been forced or coerced to engage in unwanted sexual activity by...someone you someone you know very well  Never  Once or twice  Once a month  Once a week  2-3 times a week  Every day

Acquiescence

Please indicate the extent to which you agree that lying is okay if keeps friends out of trouble.   Strongly agree Agree    Neither agree nor disagree Disagree Strongly disagree

Surveys

The goal of writing questions for self administered questionnaire is to develop a query that every potential respondent will interpret in the same way, be able to respond accurately, and be willing to answer

Types Of Questions

Dichotomous Questions -

When a question has two possible responses, we consider it

dichotomous

Surveys often use dichotomous questions that ask for a Yes/No, True/False or Agree/Disagree response There are a variety of ways to lay these questions out on a questionnaire:

Nominal question

Ordinal question

Likert scale

Cumulative or Guttman scale

Filter or Contingency Questions

Response Format

The response format is how you collect the answer from the respondent:

unstructured response formats structured response formats

Structured formats

Structured formats help the respondent to respond more easily and help the researcher to accumulate and summarize responses more efficiently But, they can also constrain the respondent and limit the researcher's ability to understand what the respondent really means There are many different structured response formats, each with its own strengths and weaknesses. We'll review the major ones here.

Fill-In-The-Blank

One of the simplest response formats is a blank line. A blank line can be used for a number of different response types. For instance:

Please enter your gender: _____ _____

Fill-In-The-Blank

NAME: ________________________ NAME:

Check The Answer

The respondent places a check next to the response(s).

Sometimes, we supply a box that the person can fill in with an 'X' (which is sort of a variation on the check mark.

Check The Answer

Please check if you have the following item on the computer you use most:  modem  Printer  CD-ROM  Drive  Joystick  scanner

Circle The Answer

Sometimes the respondent is asked to circle an item to indicate their response. Usually we are asking them to circle a number. For instance, we might have the following:

Unstructured Response Formats

While there is a wide variety of structured response formats, there are relatively few unstructured ones What is an unstructured response format? Generally, it's written text.

These can vary from short comment boxes to the transcript of an interview.

Unstructured Response Formats

In almost every short questionnaire, there's one or more short text field questions. One of the most frequent goes something like this:

Please add any other comments:

Writing Questions

One methodologist described the question writing task as similar to a driving in freeway traffic while drinking a cup of hot coffee and answering an emergency call on his cell phone

How answer categories might impact results of the study

How many hours per day do you typically study?

 Less than 0.5 hours  0.6 -1 hour  1.1- 2 hours  2.1 – 2.5 hours  More than 2.5 hours How many hours per day do you typically study?

 Less than 2.5 hours  2.6 – 3 hours  3.1 – 4 hours  4.1 – 4.5 hours  More than 4.5 hours

Reported Hours of studying

Reported Hours/Day 2.5 hours or more Less than 2.5 hours Low version Mail Telephone 23% 42% High version Mail 69% Telephone 70% 77% 58% 31% 30%

Rules for writing questions

Rule 1:

Each respondent has to have a chance to answer a question

What is wrong with the following question?

If you fixed dinner at home last night, did you eat meat as part of that meal?

 Yes  No

Rule 2.1

Do not use ambiguous questions

What is wrong with the following question?

“Have you had an opportunity to make use of the suggestions for handling obscene calls from the phone company?”

Revision

“Has the phone company offered suggestions for handling obscene phone calls? Have you made use of them?”

Is anything wrong with this question?

“Do you agree that pornographic videos are harmful to your children and that you must get rid of them?”

Revision

“Do you agree or disagree that pornographic videos should be banned?”

Rule 2.1

Do not use ambiguous answer categories

What is wrong with the following question?

How often did you attend religious services during the past year?

 Never    Rarely Occasionally Regularly

Revision

How often did you attend religious services during the past year?

  Not at all A few times   About once a month Two to three times per month   About once a week More than once a week

Rule 3: Use equal numbers of positive and negative categories for scalar questions

How satisfied were you with the service you received when you bought air conditioner?

 Completely satisfied  Mostly satisfied  Somewhat satisfied  Neither satisfied nor dissatisfied  Dissatisfied How satisfied were you with the service you received when you bought air conditioner?

  Completely satisfied Somewhat satisfied  Neither satisfied nor dissatisfied   Somewhat dissatisfied Dissatisfied

Rule 3.1: avoid overlapping categories

What is your age ?

 Less than 10  10-15  15-20  20-30  30-40  40 and older

Rule 4: Avoid double-barreled questions

Should the city build a new swimming pool that includes lanes for swimming laps that is not enclosed for winter use?

  Yes No

Revision

Should the city build a new swimming pool that includes lanes for swimming laps?

 Yes  No Should the city build a new swimming pool that is not enclosed for winter use?

 Yes  No

Rule 5

Avoid biased or leading questions: “Do you still beat your spouse?”  Yes  No

Rule 6:Use appropriate language

Avoid complex phrases ( do not show off your vocabulary) “Occupants of this household” vs “people who live here” “Subnational region” vs “area of the country

Rule 7: Avoid jargon and slang

“Do you agree or disagree that out county prosecutors regularly kink their cases?”

Rule 8: Write questions in full sentences

Sex______________ Revision: What is your sex?_____________

Rule 9: Avoid negative questions

“Do you agree or disagree that taverns ought not to be prohibited from opening on non-weekends-not including holidays?”

Sensitive Questions

Barton (1958, p. 67) provides humorous examples of many of the techniques that have been used as means of asking threatening questions

The Causal Approach

: “

Do you happen to have murdered your wife?”

The “Other People” approach: “Do you know

any people who have murdered their wives? How about yourself?”

Sensitive Questions

The Numbered Card

: “Would you please read off the number on this card which corresponds to what became of your wife?” (Hand card to respondent). 1.Natural Death 2.I killed her 3.Other (What?) (Get the card back from respondent before proceeding)

Elements of visual design

In self-administered questionnaires, information is communicated through four distinct languages, each of which may independently influence question meaning Verbal language (the words) Numeric language (numbers) Symbolic language (symbols defined by one’s culture, used to convey meaning independent of any words) Graphical language (variations in the display of words, numbers, and/or symbols)

Eye Movement Patterns

Under conditions of visually “equal” information, we tend to follow culturally prescribed patterns

Eye Movement Pattern Equal Information

Eye Movement Patterns

When information is visually “unequal” what we see and comprehend is often influenced by visual qualities of the information

Eye Movement Pattern; Unequal Information

Eye Movement Pattern Unequal Information

Gestalt Psychology: Principles of Visual Perception

Principle of Proximity: We tend to group together things that are close to one another and more distant from other (similar) objects.

Principle of Proximity: Similar figures in closer proximity will be seen as a group)

Principle of Proximity

Principle of Proximity

Principle of Similarity : Similar figures will be seen as a group

Do you see groups?

Principle of Similarity: Similar figures will be seen as a group

Do you see groups?

Figure/Ground Variations Can Also Facilitate Grouping

It is difficult to switch from one grouping to the other that this sentence is difficult to read? somewhat disagree or strongly disagree

Figure and Ground