Transcript Online surveys as data collection instruments in education
Online Surveys as Data Collection Instruments in Education Research: A Feasible Option?
EMASA 2012 - Dr Lorinda Minnaar
Introduction
Study Title
School Governance and Leadership for Sustainable Quality Education for All
Study Theme
School governance and leadership
Key Purpose of the Study
To explore and establish the ability of parent members of school governing bodies to govern and manage schools’ finances
Research Design
Approach
Large-scale, exploratory, quantitative study
Sample
All (1433) schools in the Western Cape (WCED)
Participants
School principals
Data Collection Instrument
Online survey
Motivation for Choice of an Online Survey
• • • • Large sample – 1433 schools Financial and human resource constraints An online survey is a convenient and cost-effective data collection method Access to“Checkbox”
Review of Literature on Online Surveys Problem
Despite an extensive search, existing literature documenting the feasibility of this data collection method in an education research context, seems deficient
Solution
Adapt existing literature to the purpose and design of the study, located within a South African educational research context
Types of Online Surveys
E-mail an introductory letter or invitation to respondents with a Uniform Resource Locator (URL) hyperlink to a web-based survey Send respondents a survey embedded in an e-mail message with or without an attachment Place a general request for respondents in an electronic communication environment such as a web page to complete a survey
Advantages of Online Surveys
Global reach Large samples easily accessed Time-efficient and convenient Low preparation and administration costs Environmentally friendly Question diversity Question sequencing capabilities Question branching capabilities Ease of data entry and analysis
Disadvantages of Online Surveys
Perceptions as Spam Lack of participant online experience and expertise Computer configuration Impersonal Security and confidentiality
Planning the Online Survey
• • • • Planning the online survey entailed: Drafting the hard copy Placing instructions, questions and answer options in simple tables Continually revising instructions and questions to ensure they did not contain design errors such as confusing ambiguities, double, split and negative questions and value-laden concepts Consulting an expert in the field of school governance to seek confirmation that questions were valid, meaningful, grammatically correct and appropriate to the research theme
Designing the Online Survey
• • • Designing the online survey entailed the following methodological considerations: Selecting a visual layout comprising design elements such as background colour, font face, size and colour Uploading Stellenbosch University’s oak leaf logo, which not only lent colour and formality to the survey but immediately informed participants of the survey’s origin Avoiding and eliminating complex, vague and ambiguous instructions that may perplex and frustrate participants to such an extent that they exit the survey before completing it
•
Designing the Online Survey (Cont).
Checking the survey’s length to ensure it was not too long and that participants would not spend too much time completing it Long surveys with matrix type questions that have numerous items and alternatives, increase respondent fatigue and boredom and frequently result in respondents adopting a “satisficing” behaviour where they select the same scale-point to rate all items without paying them much attention (Mora 2010)
Question Formats
Radio Buttons
A list from which a participant selects one option
Question Formats (Cont).
Checkboxes
A list from which the participant may select multiple options
Question Formats (Cont).
Drop-Down Lists
A space-saving format
Question Formats (Cont).
Open-Ended Single-Line Text
Allows freeform text with formatting rules
Question Formats (Cont).
Open-Ended Multi-Line Text
Allows freeform text within a box with a specified number of rows and columns
Question Formats (Cont).
Matrixes and Rating Scales
Explore the frequency of participants’ behaviour or their attitudes toward certain phenomena
Activating the Online Survey
• • • • • • Activating the online survey included the following aspects: Sourcing the e-mail addresses of schools in the sample Checking e-mail addresses for errors Deliberating on which day, week and month to activate the survey Determining the period permitted for participants to respond and the final date for submission The frequency of reminders to be sent to participants Compiling an appropriate invitation to principals to participate in the survey
Activating the Online Survey (Cont).
The invitation to participate in the online survey
Response Rates and Results
Activation and Reminders
1st 2nd 3rd 4th
Number of Surveys sent
1 433 1 433 1 433 1 433
Number of Surveys Completed and Returned
64 44 37 220
Response Rate
4% 3% 2% 15%
24%
Overall Response Rate
Low response rates appear typical of online surveys
Response Rates and Results (Cont).
20 15 10 5 0 40 35 30 25
Participants’ Responses to the Quality of the Survey
Bar/Column Plot of Count of 1 Spreadsheet14 3v*6c 45 3 Relevant 2 Interesting 1 Quick Other Quality of survey 4 Long 5 Complicated Count of 1
Conclusion
“How feasible are Internet-based online surveys used as data collection instruments for educational research?” • Denscombe (2009, 281) asserts “There are still several unanswered questions concerning the use of online surveys as data collection instruments for research purposes”. • Despite implementation of salient methodological considerations, the relatively low response rate alerts educational researchers to use online surveys as data collection instruments circumspectly.