Transcript Introduction to Research Methods

Questionnaire Design
Jenny Driscoll
2013
Surveys: uses
 Best suited for:
 large-scale research (large numbers)
 quantitative data – can be counted or measured
 description, knowledge, comparison, attitude,
behaviour
 need for standardisation
 [large budget]
 competent respondents
 Rough and ready rule for statistical analysis is minimum
of 30 counts in each subgroup
 Can adapt for smaller-scale research BUT
 Care and ingenuity required
Questionnaires and research process








Identify research aims, objectives and research
questions
Identify population and sample
Decide on means of response (postal, email,
online, face-to-face)
Design questionnaire
Run a pilot survey
Carry out main survey
Analyse the data
Report findings and dissemination
Surveys : Advantages
 Can access large numbers of participants
 Can cover wide geographical spread
 Low cost per respondent
 Low pressure on participants
 Data can be easily compared
 Pre-coded answers allow ease of analysis
 No interviewer effect?
Surveys: Disadvantages
Often low response rate (introduces bias)
Pre-coded questions may introduce bias
No opportunity to clarify or explore
answers
No way to assess truthfulness of answers
Unsuitable for some groups
Eg visually impaired, young children,
poor literacy
Terms
 Population: all members of the group you are
researching eg children in year 11
 Sample: subset of population selected to
participate in research eg children in year 11 in
one London borough
 Respondents: subset of sample that actually
participate
 Try to ensure sample is representative of
population eg do children in sample schools
match population statistics for socio-economic
groups, ethnicity, religion, gender etc?
Sampling
 Random (gold standard: randomly selected
subset of population - large numbers
required)
 Systematic (every nth case)
 Stratified (proportions with known variations
selected from within population)
 Quota (ensure eg balance of genders, boost
ethnic minorities: implications for bias)
 Cluster (eg school)
Sampling, response rate and bias
 How does your sample differ from the
population you wish to study?
 What is your response rate?
 Are you able to ascertain how
respondents differ from non-respondents?
 To what extent might your sampling
strategy and response rate have
introduced bias into your survey?
Administration of questionnaires
 Postal
 Email
 Face to face
 Online eg SurveyMonkey
Consider how administration mode affects
 Response rate
 Anonymity
 Bias
How could you maximise response rate?
Structure of questionnaire

Preamble



Often included in main body, not as a separate
sheet. Check kcl ethics requirements for contents:
anonymity, confidentiality, use of data, right to
withdraw etc
Questionnaire schedule
Concluding statement

Thanks; how to return questionnaire; may include
invitation to participate in next stage by leaving
contact details
Questionnaire content
Content informed by
 Place of questionnaire in research design
 Scholarly research relating to your topic area
Account of the way you have used the research
literature will figure in write-up
Make sure you include appropriate basics eg
 Age
 Socio-economic status
 Gender
Questionnaire Design
 Keep it short!
 Plan overall structure in advance
 Use logical sequence
 Group questions in topics
 Use transitional phrases between topics
 Start with easy questions
 Start with relevant questions
 Put sensitive questions at end
 Ensure clear routing instructions
 Check for ordering – might an answer be
influenced by earlier questions?
Questionnaires: Question Writing
Keep questions as short and simple as possible BUT
 Meaning must be crystal clear
 Use simple language – not jargon, slang or
acronyms
 Think about memory issues
 Missing categories – consider including ‘other’/
‘don’t know’/‘not applicable’
See http://surveynet.ac.uk/sqb/ for sample questions,
questionnaires and variables
Question writing: things to avoid
ambiguous terms
leading questions
double-barrelled questions
double negatives
sensitive questions (as far as possible)
suggestions of social desirability
Types of questions


Closed (are you male/female?)
Partially closed





Inventories (list of options)(tick all that apply)
Ratings (how much did you enjoy the art
gallery…on scale of 1-5, where 1 is ‘not at all’
and 5 is ‘enjoyed it very much’)
Rankings (list in order of importance to you)
Likert scales (attitudes) (my teacher makes
Maths fun: all of the time/most of the
time/sometimes/not much of the time /never)
Open (how do you think we could improve
your experience of the health clinic?)
Open and closed questions (from
Oppenheim)
Open
 Greater response
freedom
 Opportunity to probe
 Can test ideas or
awareness
 Time-consuming
 Coding more problematic
 More effort for
participants
Closed










Requires little time
No extended writing
Low costs
Easy to process
Group comparisons easy
Can test specific hypothesis
No spontaneous responses
Bias in answer categories
Sometimes too crude
May irritate respondents
Questionnaires: Piloting
 Always pilot questionnaire on small
group
 Test for:





Consistent interpretation of question
Understanding of instructions
“other” categories to include in main list
Successful routing
Time to complete questionnaire
 Cognitive interview
Checklist
 Explanation of purpose
 Sponsor
 Voluntary completion
 Confidentiality and Data Protection Act
 Return address
 No omissions
 No duplication
 Thanks to respondents
Surveying children
See Scott (2000) in Christiansen and James
Research with Children
 Until recently researchers have tended to
take proxy data from parents/teachers
Concerns about cognitive abilities of children
 Accreditation of adults with superior knowledge
and experience: ‘adults know what is best for
children’
 Practical and ethical issues
But validity?

Research with children under 11
 Under 7, hypothetical questions unlikely to be
understood by children
 Once children are able to process and
respond to standard questions they are also
adept at selecting what they choose to reveal
 Young children may be very suggestible
 Under 11, visual stimuli particularly useful
 Questions may need to be broken down eg
do you agree or disagree? (Agree) Do you
agree a lot, quite a lot, or only a little?
Surveying children 11+



Standardized questionnaires can be used at
11+ provided researcher takes into account:
 Literacy levels may vary
 Confidentiality issues
 Context: adolescents may not accord same
meaning to questions as adults
 Boredom and flippancy
Careful pre-testing with cognitive interviews
Use focus groups to develop response
options
Reliability of children
 Children are used to adults expecting a
response – include ‘don’t know’ options?
 Children may be more prone to the
primacy effect
 But there is no reason to suppose
children are inherently unreliable
compared to adults
Reading










Bryman, A. (2012, 4th ed) Social Research Methods Oxford: Oxford
University Press, ch 10
Christensen, P. and James, A. (eds) (2000) Research with Children:
Perspectives and Practices London: Falmer Press
De Vaus, D.A. (2002, 5th ed) Surveys in Social Research Routledge
Fink, A. (2006, 3rd ed) How to Conduct Surveys Sage
Foddy, W. (1994) Constructing Questions for Interviews and Questionnaires
Cambridge: Cambridge University Press
Gilbert, N. (2001, 2nd ed) Researching Social Life London: Sage
Gorard, S. (2001) Quantitative Methods in Educational Research London:
Continuum
Grove, R. (2009, 2nd ed) Survey Methodology Hoboken, N.J.: Wiley, chs 7 &
8
Moser, C. and Kalton, G. (2001) Survey Methods in Social Investigation
Aldershot: Ashgate
Oppenheim, A.N., (2000) Questionnaire Design London: Continuum