Transcript ADVANCED RESEARCH TECHNIQUES
QUESTIONNAIRES
Introduction Questionnaires in language research
What are questionnaires What do they measure Advantages and disadvantages of questionnaires Constructing the questionnaires
General features The main parts of a questionnaire Questionnaire content and multi-item scales Close-ended questionnaire items Open-ended questions
Introduction
One of the most common methods of data collection in research is to use various kinds of questionnaires. (Dörnyei, 2003) A survey is a system for collecting information to describe, compare, or explain knowledge, attitudes, and practices or behaviour. (Fink, 1995) The essential point in research is that it cannot be built on poorly collected data. (Gillham, 2004)
What are questionnaires?
Brown (cited in Dörnyei, 2003) expresses that questionnaires are any written instruments that presents the respondents with a series of questions or statements to which they are to react either by writing out their answers or selecting from among existing answers.
What are questionnaires?
Dörnyei (2003) states that although the term ‘questionnaire’ is familiar, it’s hard to provide a precise definition for it. He reasons that many of the questionnaires do not contain any or many real questions that end with a question mark
.
What are questionnaires?
He adds that the other reason is that the rubric of ‘questionnaire’ has been used by researchers in at least two broad senses:
Interview schedules
Self-administered pencil and paper questionnaires
What are questionnaires?
Aiken (cited in Dörnyei, 2003) explains that questionnaires are often referred to as ‘forms, opinnionaires, tests, batteries, checklists, scales, surveys, schedules, studies, profiles, indexes, indicators, or even simply sheets.’ (p. 5)
What do questionnaires measure?
Factual questions- classification questions or subject descriptors are used to find out about who the respondents are.
Behavioral questions what the respondents are doing or have done in the past.
are used to find out Attitudinal questions are used to find out what people think. (Dörnyei, 2003)
Advantages
The main attraction of questionnaires is their unprecedented efficiency in terms of: researcher time researcher effort financial resources. (Dörnyei,2003)
Advantages
Questionnaires are: easy to conduct extremely versatile uniquely capable of gathering a large amount of information quickly in a form that is readily processible. (Dörnyei,2003)
Advantages
Low cost in time and money.
Easy to get information from a lot of people very quickly.
Respondents can complete the questionnaire when it suits them. Analysis of answers to closed questions is straightforward.
Advantages
Less pressure for an immediate response.
Respondents’ anonymity.
Lack of interviewer bias.
Standardization of questions.
Can provide suggestive data for testing an hypothesis. (Gillham, 2004)
Disadvantages
Simplicity and superficiality of answers Unreliable and unmotivared respondents Respondent literacy problems Little or no opportunity to correct the respondents’ mistakes Social desirability (or prestige) bias Self-deception
Disadvantages
Acquiescence bias Halo effect Fatigue effects Problems of data quality (completeness and accuracy).
Typically low response rate unless sample «captive».
Disadvantages
Questionnaire development is often poor.
Seek information just by asking questions.
Assumes respondents have answers available in an organized fashion.
Disadvantages
Lack of control over order and context of answering questions. Question wording can have a major effect on answers.
People talk more easily than they write.
Respondent uncertainty as to what happens to data. (Dörnyei, 2003; Gillham, 2004)
QUESTIONNAIRES
Constructing the questionnaires
The steps required to design and administer a questionnaire include:
Defining the objectives of the survey Determining the sampling group Deciding on the general features of the questionnaire Writing effective items/questions and drawing up an item pool (Dörnyei.2003)
Constructing the questionnaires
Selecting and sequencing the items
Writing appropriate instructions and examples
Piloting the questionnaire
Conducting item analysis
General features
Length Dörnyei (2003) suggests that to be effective a questionnaire should not be more than 4 pages and it shouldn’t take more than 30 minutes for the respondents to answer all the questions.
General features
Layout: An attractive and proffessional design involves:
Booklet format Appropriate density
Orderly layout Paper quality Sequence marking
General features
Sensitive topics and anonymity Dörnyei (2003) claims that these are not only confined to explicitly illegal or embarrassing subjects but also include basic demographic items such as age and marital status.
He also states that to diffuse sensitive items we make the questionnaire anonymous.
General features
Sensitive topics and anonymity Oppenheim (cited in Dörnyei, 2003) suggests that to overcome the issues concerning sensitive topics and anonymity, something along the following line should be displayed prominently on the front of the questionnaire: «The contents of this form are absolutely confidential. Information identifying the respondent will not be diclosed under any circumstances.»
The main parts of a questionnaire
Title 1.
2.
Instructions General instructions Specific instructions Questionnaire items Additional information Final «Thank you!»
Questionnaire content and multi-item scales
Appropriate sampling of the content «The temptation is always to cover too much, to ask everything that might turn out to be interesting. This must be resisted.» (Moser & Kalton, cited in Dörnyei, 2003 p.32)
The research problem should be clarified.
What critical concepts need to be addressed by the questionnaire should be identified.
Questionnaire content and multi-item scales
Using multi-item scales
Close-ended questionnaire items
Rating scales
Likert scales Ex. strongly agree
5
1 strongly disagree;
Close-ended questionnaire items
Semantic differential scales Ex. Listening comprehension tasks are: difficult__:__:__:_x_:__:__:__ easy
useless__:__:__:__:__:_x_:__ useful Numerical rating scales True- False items
Close-ended questionnaire items
Multiple choice items
Response options should be short.
Responses should be put in a natural order or in random or alphabetical order.
All the options should be grammatically correct with respect to the stem.
Using negative expressions in both the stem and the response options should be avoided.
Close-ended questionnaire items
Rank order items Numeric items Checklists
Open-ended questions
Disadvantages
They take up more time and restrict the range of topics the questionnaire can contain.
They are difficult to code in reliable manner.
Open-ended questions
Specific open questions Clarification questions Sentence completion items Short answer questions
References
Aiken, L. (1997). Questionnaires and inventories:
Surveying opinions and assessing
personality. New York: John Wiley Brown, J. D. (2001). Using surveys in language programs. Cambridge, UK: Cambridge University Press Dörnyei, Z. (2003). Questionnaires in second language research. Mahwah, New Jersey: Lawrence Erlbaum Associates Fink, A. (1995). How to ask surveys questions. Thousand Oaks, California: Sage
Gillham, B. (2004). Developing a questionnaire. London: Continuum Hopkins, K.D., Stanley, J.C. & Hopkins, B.R.(1990). Educational and psychological measurement and evaluation (7 TH Ed.). Englewood Cliffs, NJ: Prentice Hall Low, G.(1999).What respondent do with questionnaires: Accounting for incongruity and fluidity. Applied Linguistics, 20, 503-533
Moser, C. A. & Kalton, G. (1971). Survey methods in social investigation. London: Heinemann Newell, R. (1993). Questionnaires. In N. Gilbert (Ed.) Researching social life. (pp 94-115) London: Sage Oppenheim, A. N. (1992). Questionnaire design,
interviewing and attitude measurement
(New Edition). London: Pinter
Robinson, J.P., Shaver, P.R. &Wrightson, L.S. (1991). Criteria for scale selection and evaluation. In J.P. Robinson, P.R. Shaver & L.S. Wrightson (Eds.) Measures of personality and social psychological attitudes (pp1-16). San Diego, CA: Academic Press Robson, C. (1993). Real world research: A source for
social scientists and practitioner-researchers.
Oxford: Blackwell
Sanchez, M. E. (1992). Effects of questionnaire design on the quality of survey design. Public Opinion Quarterly, 56,216-217 Wilson, N., McClean, S. (1994). Questionnaire design. Newtownabbey, Northern Ireland: University of Ulster
How would you rate this presentation?
Good Very Good Excellent
Thank you for your attention.
Cemal ESKİCİ