Transcript Slide 1

Chapter 13
Survey Designs
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
By the end of this chapter,
you should be able to:
Identify the intent and use of survey research
Identify types of survey designs
Describe the key characteristics of a survey
Describe how to construct survey questionnaires
Describe how to design and conduct survey
interviews
List the steps in the process of conducting
survey research
Identify how to evaluate survey research
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.2
What Is Survey Research?
Survey research designs are procedures in
quantitative research in which investigators
administer a survey to a sample or to the entire
population of people in order to describe the
attitudes, opinions, behaviors, or characteristics
of the population.
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.3
When to Use a Survey
To assess trends
To assess opinions, beliefs, and attitudes
For follow-up analyses
For evaluations
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.4
The Development of Survey Research
1817 International Survey of Education Systems
1890 Stanley Hall survey of children
1907 The Pittsburgh Survey examined social
problems
WWI and WWII sampling techniques improved
Applications began to vary from social science to
media
Universities established social science research
centers
Private organizations such as Gallup, Rand
Corporation, and Roper formed
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.5
The Types of Survey Designs
Time of Data Collection
Study Over Time
Longitudinal
Study at One Point in Time
Cross-sectional
Changes
in a
Changes
Trends in subpopulation
in the
the same group identified
same
population by a common
people
over time characteristic
over time
over time
Trend
Cohort
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Panel
Attitudes
and
Practices
Community Program
Needs
Evaluation
Group
Comparisons
National
Assessment
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.6
Key Characteristics of Survey
Research
Sampling from a population
Collecting data through questionnaires or
interviews
Designing instruments for data collection
Obtaining a high response rate
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.7
Population, Target Population,
Sampling Frame, and Sample
The Population
The group of individuals having one characteristic
that distinguishes them from other groups.
The Target Population or Sampling Frame
The actual list of sampling units from which the
sample is selected.
The Sample
The group of participants in a study selected
from the target population from which the
researcher generalizes to the target population.
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.8
Reducing Error in Survey Research
Reducing coverage error: Have a good
sampling frame list on which to select
individuals
Reducing sampling error: Select as large a
sample from the population as possible
Reducing measurement error: Use a good
instrument with clear, unambiguous questions
and response options
Reducing nonresponse error: Use rigorous
administration procedures to achieve as large
a return rate as possible
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.9
Forms of Data Collection Based on
Who Completes or Records the Data
Who completes or records the data?
Participant
Mailed
Questionnaire
Electronic
Questionnaire
Researcher
One on To a
Group
One
Over
Telephone
Individual Focus Group Telephone
Interview Interview
Interview
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.10
Aspects of Survey Instrument Design
Personal
Attitudinal
Behavioral
Sensitive
Scale
Open-ended
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.11
Question Construction Problems
Question is unclear
Double-barreled questions
Question is wordy
Question is negatively worded
Overlapping responses
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.12
Question Construction Problems (cont’d)
Questions contain jargon
Unbalanced response options
Mismatch between questions and answers
Question includes overly technical language
Not all questions are applicable to all the
participants
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.13
Pilot Testing
Test on a small number of individuals in the
sample
Ask for written feedback on the questions
Revise the survey based on the written
comments
Exclude the pilot participants from the final
sample for the study
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.14
Response Rate
Mail out the original questionnaire with cover
letter
Follow it in two weeks with a second mailing of
the questionnaire to those who have not
responded
Follow again in two weeks with a postcard to
nonrespondents reminding them to complete the
questionnaire
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.15
Three-Phase Survey Administration
Procedure
Step 1:
First Mailing
of Survey
2 Weeks
Step 2:
Second Mailing
of Survey
2 Weeks
Start
Step 3:
Postcard
Mailing
2 Weeks
6 Weeks
Time
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.16
Planning and Conducting a Mailed
Questionnaire Survey
Write a cover letter to invite the participants to
complete the questionnaire
Form and construct the questionnaire
Identify what statistical procedures will be used
to analyze data from the mailed questionnaire
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.17
Planning and Designing an Interview
Survey
The interviewer should maintain a neutral
stance during the interview
Train the interviewers prior to the interview
Take good notes of responses or use a tape
recorder
For telephone interviews, develop a telephone
interview guide prior to the interview
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.18
Steps in Conducting
Survey Research
1. Decide if a survey is the best design to use
2. Identify the research questions or hypotheses
3. Identify the population, the sampling frame,
and the sample
4. Determine the survey design and data
collection procedures
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.19
Steps in Conducting
Survey Research (cont’d)
5. Develop or locate an instrument
6. Administer the instrument
7. Analyze the data to address the research
questions or hypotheses
8. Write the report
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.20
Evaluating Survey Research
Was the target population or sampling frame
clearly specified?
Was the sampling procedure specified? If a
random sample was not used, were modifications
from it explained?
Was the sample clearly identified and the basis on
which it was chosen specified?
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.21
Evaluating Survey Research (cont’d)
Did the type of survey match the questions or
hypotheses advanced by the author?
Was it clear whether a questionnaire or an
interview survey comprised the form of data
collection? Did the researcher identify the basis
for selecting or developing the instrument?
Was information reported on the reliability and
validity of scores from past use of the
questionnaire or interview?
Did the author mention the dates on which the
questionnaire or interview was administered?
Were the administration procedures clearly
identified?
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.22
Evaluating Survey Research (cont’d)
If a questionnaire was used, were the procedures
for obtaining responses identified? Were followup procedures used to ensure a high response
rate?
Did the data analysis match the research
questions or hypothesis to be answered in the
study?
If an interview was used, were the field
procedures for collecting this information
specified? Were demographic characteristics,
previous experience, training, and monitoring
discussed for the interviewer(s)?
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.23
Evaluating Survey Research (cont’d)
Did the researcher check on the reliability and
validity of scores from the data collection in
the study?
Was the study written scientifically and
ethically?
John W. Creswell
Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research, third edition
Copyright © 2008 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
13.24