Questionnaire Design

Facilitated by Elizabeth Wiredu 1

Course Aim

The aim of this course is to facilitate the design of valid questionnaire to support good data collection and the subsequent analysis.

2

Learning Outcomes

By the end of the course, you will be able to:  Identify the principles that will guide the design of a good questionnaire.

 Learn how to put a good design together.

 Examine how response types determine the choice of summaries and statistics you could generate on your data.

 Critique a poorly designed questionnaire.

 Create a sample good questionnaire from scratch.

3

Questionnaire

 What is questionnaire?

 When to use questionnaire?

 Is validated questionnaire the solution?

4

The Framework

   

Pose the research question.

Develop and write the questionnaire, and do validity checks. Summarise and analyse data.

Interpret and report findings.

Stage 1- Pose the Research Question

 Funnel the research question down into specific key questions or objectives which will then be grouped as themes.

 Write down the hypotheses and the conclusions you want to arrive at.

 Establish the relationships, comparisons, trends, patterns or summaries you wish to make. 6

Stage 2- Develop and Write the Questions

 Formulate each key question into a theme and choose which variables you must collect data on.

 Decide how each variable is to be measured i.e. how to frame the question (open/closed) and its response set.  Get help from your literature reviews.

Stage 2- Questions Designs

 Factual Information   Simple dichotomy Simple numeric  Simple choice  Opinion and Attitude Information    Likert categorical choice Anchor scale Rank order  Mixed Designs   Multiple choice Matrix design

Examples of Closed Questions Design

Dichotomous

Questions???

Response:   Yes  Male  No Female

Likert Scale

Questions???

Response :   Very satisfied Somewhat satisfied  Undecided   Somewhat dissatisfied Very dissatisfied

Simple Choice

Questions???

Response:  Exceeded expectations   Met expectations Fell short of expectations

Anchor Scale

Questions???

1-directional design: On a scale of 1 to 10, where 1 is virtually no pain and 10 is excruciating pain. 2-directional design: On a scale of 1 to 5, where 1 is highly unsatisfied and 5, highly satisfied. 9

Examples of Closed Questions Design

Multiple Choice

Which of the following products do you own? (select all that applies)

Matrices

Please rate your purchase decision based on the following characteristics  Pen drive     Computer Printer Fax machine Scanner  Digital camera V. Important Will Consider Not Important Reliability    Price    Features The look Functionality          10

Opened Question Design Example

Open ended question…???

Do you think smoking and wound-healing are linked, and if so, how? Please write your response in the box below.

11

Stage 2- Questions Designs

 Wording  Phrase your questions in positive tone  Logical Arrangement, Appearance, and Layout  Ambiguity and Imprecision  Acronyms, Jargons, Technical Language  Assumption  Memory and Knowledge  Double Questions or complex and heavy loaded questions  Leading Questions  Wording Answers before the Questions  Subjective Questions  Vague Questions

Reliability and Validity

Reliability.

A questionnaire is considered to be reliable if the results of a study can be reproduced at different times using the same questionnaire under a similar methodology. Use test-retest method to check reliability

Validity

refers to whether the questions (questionnaire) are measuring what they are supposed to measure. Validity checking: use pre-test 13

Stage 2- Sampling

 How much sample do you need?.  Decide on the appropriate Sampling Method(s)  Simple Random  Stratified Random  Systematic Sampling  Cluster Sampling  Quota Sampling.

14

Pay attention to the following during the planning stage

 Have you budgeted for non-response by sending (10-40%) more to protect your specified sample size for your power calculation?

 Have you carefully considered the ethics of the research, e.g. questions you should not ask? The protection of the respondents’ data?

 Have you cleared access to your respondents?

 Are you going to use software to summarise and analyse the responses? If so, do you know how to use the software or have you got access to support?

15

Pay attention to the following design issues:

  Themes and their logical arrangement Place a text box under each theme to ask for an overall opinion about the theme  Avoid ‘Any comment/suggestion’ question at the very end. It does not work!  Clear instructions to reduce response error  Navigation – adopt question numbering and goto jumps, etc.

 Appearance, layout and sequence  Logical, inclusive/complete response sets 16

Piloting the Questionnaire

 Proofreading and Editing  Pre-test piloting  Feedback prompts  Do you believe other people may have difficulty in answering this questionnaire? If so, which question in particular may pose a problem?

 Given the purpose of my study outlined in the introduction section, would you suggest a question, may perhaps be added or removed to enhance the study?  How long did it take you to complete the questionnaire?

 Did you read the instructions?

 Were the instructions clear and neatly emphasised?

 Did you object to answering any of the questions?

 Was the layout of the questionnaire clear, logical and attractive?

 Is there any question that the wording could be enhanced or clarify?

17

Stage 3- Summarising and Analysing the data

 Summarise and organise the data  Look for common and unusual patterns, trends, differences, features in the data  Relate the data to the research question  Undertake appropriate statistical analyses of the data  Clearly communicate the relevant findings  Access the limitations of the study 18

Stage 4- Interpretation and Reporting of Findings

 Relate the findings to the research question  Draw appropriate conclusions  Compare and contrast findings with those of other relevant research  Suggest future research question leading on from here.

Introduction to the Questionnaire



A short introduction explaining:

 The purpose of your research.

 How this study will benefit the respondent directly or indirectly.

 State official backing or sponsorship if any.

 A note of appreciation to the respondent for their contribution to your study.

 Confidentiality if applicable and Data protection statement.

 Contacting information if respondent wish to make a contact.

 The introduction letter MUST be part of the questionnaire and also the separate covering letter.

Concluding Note



A short concluding note containing:

 A note of appreciation to the respondent for their contribution to your study  Returns address  Returns deadline 21

Frequently Asked Questions

   

Question

: Can I use tick boxes for all the questions?

Answer

: It depends on your intended statistical analysis. E.g. if you intend to do a comparative testing (Ttest/ANOVA) you will need the

dependent variable

to be a continuous data (scale or ordinal measure) and the factor variable, categorical data (choice response).

Question

: After the data is collected, can it be manipulated to allow certain statistics?

Answer

: It is possible to regroup continuous variables into categorical, or even reduce categories of a categorical variable to take advantage of certain statistics. However, it is not possible to convert categorical variable to continuous variable. 22

What Next? - Practical Session

 Evaluate a sample questionnaire  Group Feedback and Discussion of the sample questionnaire  Create a questionnaire from scratch  Examine the design of your questions and the summaries or statistical analysis opportunities available.

23

How can I help you?

We provide:  Questionnaire design feedback.  A guide to summary statistics and inferential statistical analysis.

 Assistance with reporting the statistics for publication.

24