Transcript Developing your questions

Questionnaires
Questionnaires
• Quantitative and Qualitative Research
• Questionnaires (quantitative) – do the
Carbon Footprint questionnaire
• Creating questionnaires
• Tutorial on lit review assignment
Next week
• Creating a sample
• List potential participants
Reality is the hardware
of a computer
programme
What is real and
knowable depends on
the software
Five
assumptions
1. The
Ontological
Assumption
What is the nature of reality?
What is real and knowable?
Qualitative vs Quantitative
Assumption
Question
Ontological
Assumption
What is the nature
of reality?
Quantitative
Qualitative
Reality is objective
and singular,
Reality is subjective
and multiple,
apart from the
researcher
as seen by participants
in a study
2. Epistemological assumption
The researcher is all powerful
In order to know I must win “I
want to be a Millionnaire”
What is the relationship of the researcher to
that researched?
What we must do in order to know?
Assumption
Question
Epistemological
assumption
What is the relationship of
the researcher to that
researched?
Quantitative
Qualitative
Researcher is
independent from that
being researched
Researcher interacts
with that being
researched
3. Axiological Assumption
What is the role of
values?
What is worth
knowing and
why?
Survival values – nature red in
tooth and claw
Social Values – fitting in with
society
Transcendent values –
beyond self, for the greater
good
Our values determine what
we want to research
Assumption
Question
Axiological Assumption
What is the role of
values?
Quantitative
Qualitative
Value-free and
unbiased
Value-laden and
biased
4. Rhetorical Assumption
What is the language of research?
How best to share our research with others.
Dramatic
Appealing to the personal
Stretching the truth
Large group of believers present
Assumption
Rhetorical
assumption
Question
What is the language
of research?
Quantitative
Qualitative
Formal
Based on set definitions
Impersonal voice
Use of accepted
quantitative words
Informal
Evolving decisions
Personal voice
Use of accepted
qualitative words
Research is complex
There is no single approach
Before you use any
particular method you have
to understand the
assumptions behind it
Methodology deals with the
assumptions behind specific
methods of research
5. Methodological
assumption
What is the
process of
research?
What
techniques
will lead to
fruitful
inquiries?
Assumption
Methodological
assumption
Question
What is the process of
research?
Quantitative
Deductive process
Cause and effect
Qualitative
Inductive process
Mutual simultaneous
shaping of factors
Emerging design, categories
isolated during research
Context-bound
Patterns, theories developed
for understanding
Accurate and reliable
through verification
Static design, categories
identified before study
Context-free
Generalisations leading to
prediction and explanation
Accurate and reliable
through validity and
reliability
Qualitative Methods descriptive, theory
building, meaning making
•
•
•
•
•
•
•
Focus Groups
Interviews
Case Studies
Ethnographic Inquiry
Collaborative Inquiry
Grounded Theory
Qualitative analysis of documents, video,
observation, etc
What are they?
Quantitative Methods measuring,
theory confirmation, defining
•
•
•
•
Experiments
Questionnaires and surveys
Psychometric Scaling and Tests
Quantitative analysis of
documents, video, observation, etc
• Audits
What are they?
Carbon footprint questionnaire
• Do the questionnaire
• Tick the answers closest to
your understanding of the
way you live
• Add up your score by
adding up the numbers in
each box you ticked
• Discuss the outcome
• Critique the questionnaire
Earths
Scores required
32-62
1.2-1.6
62-93
1.6-2.3
93-124 2.3-5.8
124-155 5.8-15.3
The importance of enhancing whānaungatanga
(the feeling of close connection as between kin)
between students at Unitec Waitakere
Question Issues
• What types of
questions can be
asked and how?
• How complex will
the questions be?
• How long will the
questionnaire/
survey be?
Developing your questions - 1
Managing bias.
• The researchers – how easy is it to disprove
your hypothesis, theory, bias?
• The participants – will the subject and the
questions put off the participant?
Opening questions.
• Simple
• Key open ended questions to get
uninfluenced “off the top of my head”
answers.
Developing your questions - 2
Response option lists. (forced choice)
7. Would you like to help with any of the
following activities?
yes
being involved in support group for your
community
setting up a support group for your community
being a driver on field trips
helping with education activities (e.g.. cooking,
sewing, computer skills)
helping Shakti with policy development
being involved in activities aimed at changing
government policy
no
Developing your questions - 3
Rating questions (likert scales)
Did your mental health change because of the
help you got from the hospital or the service(s)?
1=much better; 2=better; 3=didn’t change/not
sure; 4=worse; 5=much worse
• What to do about don’t know/not sure/did not
answer responses
• Rating option lists – putting in an “other”
option space
• Question Matrices
Developing your questions - 3
The “other” option
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
GP or Doctor
Visiting nurse (Community, District, Plunket)
Community mental health service
Community drug and alcohol (D&A) service
Psychiatrist*
Psychologist*
Counsellor (include psychotherapists)*
Social worker
Deaf Association Service Coordinator
Employment or occupational service (eg.
work experience, Workbridge, NZES)
Accommodation or housing service (eg.
Housing New Zealand, rest homes)
Maori health service (eg. marae health
service, Tohunga, spiritual healer)
Alternative health service (eg. use of herbal
and natural medicines, acupuncturist,
chiropractor, spiritual healer)
Interpreter or communicator on their own for
advice or discussion of problems (do not
include social or friendship meetings)
Other health professional (eg. audiologist
physiotherapist, optician, obstetrician,
ear, nose and throat specialist)
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
If>0
(please circle which professionals you have been to or write their professions down if they are not in the list____________________________________________)
*Differences between a psychiatrist, a psychologist and a counsellor are 1) psychiatrists prescribe medication for mental health problems, the others do not; 2) Sessions with psychologists and counsellors are
usually longer than with psychiatrists (one hour compared with less than half an hour). 3) Psychologists use more cognitive and behavioural techniques to help you control your thoughts and behaviours
much worse
worse
did not change
better
much better
3e. Did your mental health
improve because of the
help you got?
never
occasionally
half the time
mostly
3d. Was there an
interpreter or
communicator
present?
always
In your home
3c. Where did the mental
health consultation(s) mainly
take place?
place =
clinic
surgery,
office, or
rooms
Hospital or A & E
3b. When you saw
a health
professional, were
there times when
you were feeling
these things?
(Show illness list)
How many times?
At another's person's place
If you have seen a mental health or D&A
professional while using a community
mental health or D&A service, do not
count them again under their specific
profession (eg nurse, social worker,
psychiatrist).
3a. Over the past
12 months how
many times have
you consulted with
the following
people or services
about your health?
At their place
3 Community health
professionals/services
(other than overnight
services).
Developing your questions - 4
•
“Why” questions sparingly used after key
options questions
• Mopping up questions. Solutions, answers,
ideas, “anything else you want to say”
• Reliability checks.
• Check response patterns for rigidity
• Vary direction of rating responses
• Set up correlating questions
• Set up some very unlikely options
Demographic questions
The big three:
age,
gender,
ethnicity/culture.
Socio-economic status:
• Income (tax brackets/income support levels),
• Occupation
1=factory, unskilled;
2=clerical, fishing, farming, trades;
3=administrator, professional, technician
Demographic questions - 2
Socio-economic status (continued):
• Employment (full-time, part-time, student,
not working, on the benefit),
• Education (SC, UE/bursary, tertiary cert,
diploma, degree)
• Family: partners, dependants,
responsibilities
• Health: disabling conditions
Beliefs: religion, political affiliation, sexual
orientation
Strengths of Questionnaires
• Good for measuring attitudes and eliciting other content
from research participants
• Inexpensive (especially mail questionnaires and groupadministered questionnaires)
• Can be administered to large probability samples
• Quick turnaround from data collection to report
• Can be administered to groups
• Perceived anonymity by respondents is high
• Moderately high measurement validity for wellconstructed and well-tested questionnaires
• Low dross rate for closed-ended questionnaires
• Ease of data analysis for closed-ended items
Weaknesses of Questionnaires
• May need validation and may have poor reliability
• Must be kept short
• Often has missing data, particularly to open ended
questions
• Possible reactive effects (e.g. response sets, social
desirability, dislike of questionnaires)
• Failure to reach certain groups – low SES, low literacy
• Response rates generally low for mail questionnaires
• Open-ended answers are vague or reflect differences in
verbal ability, obscuring the issues of interest
• Data analysis sometimes time-consuming for openended items
Sampling
Population issues
• Can the population be counted? What data is
available
• Are response rates likely to be a problem?
– Is the population literate?
– Are there language issues?
– Will the population cooperate?
– What are the geographical restrictions?
• Generalisation.
– How small is the effect size we are seeking?
– What sub-groups do we want to compare?
– How do we represent the people who won’t
answer?
Probability sampling
• Based on statistics which can describe the
similarity of a sample to the population that it
is supposed represent.
• The simplest form of random sampling is
called simple random sampling. In this we
select participants from a given population
such that each person in the population has
an equal chance of being selected.
Stratified Random Sampling
Involves dividing a population into homogeneous
subgroups based on one factor and then taking
a simple random sample in each subgroup (e.g.
culture). Participants are randomly selected
within each group.
Subgroups might be based several factors (e.g.
culture (two), gender and age (two groups))
which would give 8 (2x2x2) equal subgroups
within which participants are randomly
selected.
Quota Sample
The entire population is first divided into
homogeneous sub-groups with respect to the
given characteristic such as culture. A
specified number of people are recruited from
each strata as you come across them rather
than selecting them through random
procedure. The resulting samples are called
quota samples.
Purposive sampling
• Purposive sampling starts with a purpose in
mind and the sample is thus selected to
include people of interest and exclude those
who do not suit the purpose. Subjects are
selected on the basis of some characteristic/s.
Purposive sampling is popular in qualitative
research.
Examples of Purposive Sampling
• Extreme or Deviant Case - Learning from highly
unusual manifestations of the area of interest
• Intensity - Information-rich cases that demonstrate the
area of interest very clearly. Critical incident case,
Politically Important Case
• Maximum Variation - Purposefully picking a wide
range of variation within a group
• Homogeneous – Reducing variation to get a clear
group or sub-group perspective
• Typical Case - Illustrates or highlights what is typical,
normal or average. (e.g. the average student).
Criterion based or Theory based selection
Examples of Purposive Sampling
• Random Purposeful – random selection form a pool
of participants.
• Purposive Quota Sampling Interviewers are given
quota of particular groups of people to interview
and the quota are organised so that final sample is
representative of population. If we want our
sample to represent the age of our population and
20% are between 20 and 30, and sample is to be 20
then 4 of the sample (20%) must between 20 and
30. Complex quotas can be developed so that
several characteristics (e.g. age, sex, marital status)
are used simultaneously.
Waitakere City 2006 Census Data (People
16 years and over)
Occupation
Managers
Professionals
Technicians and Trades Workers
Community and Personal Service Workers
Clerical and Administrative Workers
Sales Workers
Machinery Operators and Drivers
Labourers
Emloyed Not Elsewhere Included
Unemployed
Full-time and part-time Tertiary Students
Not in the Labour Force (house persons,
retired, disability)
Status Unidentifiable
N
12,645
16,167
13,035
6,798
12,570
9,258
5,622
7,362
4,689
5,349
13987
27494
7,311
Total 142,287
% of pop- participants rounded
ulation
in each participants
category
9%
2.84
3
11%
3.64
4
9%
2.93
3
5%
1.53
2
9%
2.83
3
7%
2.08
2
4%
1.26
1
5%
1.66
2
3%
1.05
1
4%
1.20
1
10%
3.15
3
19%
6.18
6
5%
1.64
2
100%
32
33
Purposive Quota sample
Examples of Purposive Sampling
• Snowball or Chain - Initially contact a few
potential respondents and then ask them
whether they know of anybody with the same
characteristics that you are looking for in your
research.
• Opportunistic –Following new leads during
fieldwork, taking advantage of the unexpected.
• Convenience - The sample comprises subjects
who are available in a convenient way to the
researcher.