Transcript Chapter 8
Chapter 8 Designing the Survey Questionnaire McGraw-Hill/Irwin Copyright © 2013 by The McGraw-Hill Companies, Inc. All rights reserved. Questionnaires in Marketing Research • Questionnaire: A formal, structured response framework consisting of a set of questions and scales designed to generate primary data. • Can be administered orally or textually (i.e. “written”) 8-2 Questionnaires in Marketing Research – Two Main Kinds • Descriptive survey questionnaires collect data that can be turned into knowledge about consumers, companies, markets, etc. • Predictive survey questionnaires predict changes in attitudes and behaviors and are used to test research hypotheses. 8-3 Steps in Questionnaire Design 8-4 Step 1: Confirm Research Objectives • Example of Research Objectives: – To collect data on selected customer demographic characteristics and lifestyle dimensions of our different market segments – To identify preferred banking services, as well as attitudes and feelings toward those services 8-5 Step 2: Select Appropriate Data Collection Method • We will develop and administer an internet survey with the following sections: – Section I: Banking services – Section II: Lifestyle dimensions – Section III: Banking relationships – Section IV: Demographic characteristics 8-6 Step 3: Develop Questions and Scaling • Question formats – Unstructured questions: Open-ended questions allow respondents to reply in their own words – no pre-determined choices imposed by the researcher (like an essay test!) – Structured questions: Closed-ended questions that require the respondent to choose from a predetermined set of responses or scale points (pretty much any scale we have looked at so far) 8-7 Step 3: Develop Questions and Scaling • Bad questions: Any questions that prevent or distort the fundamental communication between researchers and respondents • A question is bad when it is: – Unanswerable – Leading (loaded) – Double-barreled 8-8 Step 3: Develop Questions and Scaling • Other considerations: – Use simple words; avoid technical words or jargon – Make questions as concise as possible – Avoid using qualifying phrases (always, sometimes, never, etc.) in question stems, but they’re generally ok in response scales – Ensure response categories are mutually exclusive – Ensure question and corresponding scale descriptors are meaningful to respondents and “jive” with each other. – Grammar, diction and spelling must be flawless!!! 8-9 Step 3: Develop Questions and Scaling • Sensitive questions: Questions about income, sexual beliefs or behaviors, medical conditions, financial difficulties, drug/alcohol consumption, or other questions that respondents might skip, answer falsely or otherwise take offense to should appear near the end of the survey. 8-10 Step 3: Develop Questions and Scaling • Skip questions: Used if the next question (or set of questions) should be responded to only by respondents who meet a previous condition • Much easier to implement in web/digital surveys 8-11 Step 4: Determine Layout and Evaluate Questionnaire • Introductory section: Gives the respondent an overview of the research • Re-mention the purpose of the research (if indicated) • Set any necessary ground rules • Provide general instructions • Screening questions: Used to disqualify inappropriate respondents • This should usually/mostly be done before the formal survey is administered • Research questions section: Focuses on the main research questions of interest 8-12 Step 4: Determine Layout and Evaluate Questionnaire • Ensure instructions are clear for each section • Arrange questions in a logical order. – Keep questions using the same response formats in close proximity if possible. – Start with easy questions and ask the more difficult questions later in the survey. • Be mindful of coding needs when choosing response formats, especially if manual entry of data will occur. • The entire survey should be as short as possible without sacrificing the integrity and amount of data collected – it’s ultimately a judgment call! • End with a thank-you statement. 8-13 Step 4: Determine Layout and Evaluate Questionnaire • Response order bias: Occurs when the order of the questions, or of the closed-end responses to a particular question, influences the answer given • Common methods variance (CMV): Falsely correlated answers due to a subject’s need to appear consistent across a series of related questions. 8-14 Step 5: Obtain Initial Client Approval • Provide copies of the questionnaire to all parties involved in the project • Client may identify any overlooked items or other problematic aspects of the survey (i.e. inaccurate representations, offensive questions, etc.) • What if client makes a stupid suggestion!? 8-15 Step 6: Pretest, Revise, and Finalize the Questionnaire • Final evaluation of the questionnaire is obtained from a pretest – Helps the researcher determine: • How much time respondents will need to complete the survey (Is survey too long?!) • Whether to add, delete or revise instructions, questions and scales that are biased or poorly written. • Where did subjects get confused or misled? • Helps with how to frame aspects of the cover letter (see subsequent slides) 8-16 Step 7: Implement the Survey • Administer the survey to the “real” (i.e. nonpretest) sample. • Keep correcting any survey shortcomings / errors that are revealed during implementation without changing the fundamental nature of the survey so that results remain comparable across the changes. • Good + Bad Data is usually better than All Bad Data 8-17 The Cover Letter / Invitation to Participate • Cover letter / Invitation: • Designed to enhance respondent willingness to accurately and faithfully complete and/or return the survey in a timely manner 8-18 Guidelines for Developing Effective Cover Letters 8-19 Collecting Data • Interviewer instructions: Used to train interviewers how to: – Select prospective respondents – Screen them for eligibility – Conduct the actual interview • Screening questions • Quotas: A tracking system that collects data from respondents and helps ensure that subgroups are represented in the sample as specified 8-20 Collecting Data: Interviewer Instructions • Used to train interviewers how to: – Select prospective respondents – Screen them for eligibility – Conduct the actual interview 8-21 Collecting Data: Contact Records • Contact Record: Gathers basic summary information about an interviewer’s performance efficiency 8-22