Transcript Survey Design 101 - Humboldt State University

SURVEY DESIGN 101
Gay Hylton
Institutional Research and Planning
WHAT YOU’RE TRYING TO FIND
OUT
Often there is the temptation to skip on preparation in
order to move to the field too rapidly. This temptation
should be avoided.
Ghislaine Delaine
“The Social Dimensions of
Adjustment Integrated Survey”
INFORMATION WANTED:

What information needs to be obtained to meet
the objectives of the survey
Strategize with stakeholders
 Questions that need to be answered
 Focus groups
 Has the information been gathered in the past
 What are the outcomes desired
 Will basic demographical data be required/confidential
 How will the information be obtained?
 Electronic survey
 Mail
 Telephone Interview
 Personal Interview
 Paper

INFORMATION WANTED: CONTINUED

Determine what correlations of the data you
want to see.
What will the answers to the questions allow you to
determine
 If X then Y

WHO YOU ASK
One common misconception is that the adequacy of a
sample depends heavily on the fraction of the
population included in a sample.
Floyd J. Fowler
Survey Research Methods
CHOOSING THE POPULATION
All or representative sample
 http://www.surveysystem.com/sscalc.htm
 Is a comparative sample needed
 Response rate



# of responses/ # surveyed
If survey needs to reflect the entire student body
population of HSU, complete a survey request
form at http://www.humboldt.edu/irp/survey.html
HOW YOU ASK THE QUESTION
“The goal is to have differences in answers reflect
differences in where people stand on the issues, rather than
differences on their interpretations of the questions.”
Floyd Fowler,
Improving Survey Questionnaires:
Design and Evaluation
BASICS
Be aware of your own biases
 Use language that survey participants will
understand
 Develop neutral questions
 Ask enough to cover topic adequately while
keeping the survey as short as possible
 Pay attention to the order of questions
 Provide exhaustive range of response categories
 Write clear unbiased instructions

BASICS - CONTINUED

Order
Matters
 Start with easy - proceed to complex
 Keep respondents interested
 Vary the question type


Simple is better
Avoid technical jargons or concepts
 Use the same definitions and scales throughout the form
 Watch for “double-barreled” questions
 Be specific

TYPES OF QUESTIONS

Multiple Choice

Allow multiple responses?
Ranking
 Scale
 Open Ended

SPECIFICS

Multiple Choice

Lists
Should be exhaustive while not being too long
 Categories should be mutually exclusive
 Allow respondents to provide multiple answers when
relevant
 When appropriate – use Other


Scales
Odd or Even
 Likert or Numeric
 Provide clear, concise instructions on scale meaning
 Order matters – Positive to Negative vs Negative to Positive
 Use the same scale throughout the survey

SPECIFICS - CONTINUED

Open-Ended
Allows for spontaneous responses
 Use when you don’t know the answer
 Analyzing the responses can be difficult and timeconsuming


Question Logic

Will all participants answer all questions?
HOW AND WHEN TO ASK
“The questionnaire is only one element of a welldone survey.”
Don A. Dillman
Mail and Internet Surveys
– The Tailored Design Method
BEFORE YOU SURVEY

Design your communication to participants
Personal or authoritarian
 Confidentiality?
 Explain the nature and reasons for the survey
 How it benefits them
 Include timeframe to answer survey

BEFORE YOU SURVEY - CONTINUED

TEST
Pilot the survey with colleagues or a small sample of the
population
 Include all communications they will receive with the
survey instrument
 Helps ensures the reliability and the validity of your survey
 For electronic surveys check for the flow of logic questions


TRAIN (If survey is to be administered via
telephone or personal interviewer)
All interviewers need to conduct the survey in the same
manner, using the same language
 Design response sheets that include instructions to
interviewers

INCREASING RESPONSE RATES
“Sending questionnaires out is one thing; getting
them back is quite another.”
Bill Gillham
Developing a Questionnaire
TIMING MATTERS
What are your target population’s habits
 Ask again but don’t ask too often
 Choose a different time to ask for each additional
request

COMMUNICATION
Say please and thank you for your help
 Show positive regard – provide the reasons for
the survey and what the results will achieve
 Asking for advice – “We need your input on this
matter.”
 Subject line of email – does it look like spam
 Does it need to come from an authoritative source
or a personal approach

TOOLS
All tools have intrinsic politics and technology is
the tool of now.
Godfrey Reggio
FREE OR NOT

Software
Survey Monkey
 Zoomerang
 Surveygizmo
 And many more


Design help
Library – reference section “Survey Kit” , ebooks and books
on survey design
 IRP website has some links to reference material

QUESTIONS?
“Who questions much, shall learn much, and
retain much.”
Francis Bacon