Transcript Welcome to PSY206F - University of Cape Town

Design Into Practice
• These lectures tie into Terre Blanche
chapter 4 and 5
• Now you have a design – how do you run
the study?
• Many practical issues involved in converting a
design into a study to run
Conceptualisation
• We want to speak of abstract things
• “Intelligence”, “ability to cope”, “life
satisfaction”
• We cannot research these things until we
know exactly what they are
• Conceptualisation os the process of defining
terms before research
• Once a thing has been conceptualised it is a
“Construct”
Making conceptual definitions
• Begin with the lay understanding of the
definition
• This will be understood by the subjects
• Then consult the experts (literature)
• Can be confusing, contradictory
• Create a preliminary definition
• “Test it” hypothetically
• Use thought experiments
The danger of reification
• You must not try to make constructs out of
things that don’t exist – reification
• Careful grounding of the construct in
extablished theory will prevent this
• Eg. Is Homophobia a construct? (is it not
just prejudice)?
• Using reified constructs leads to empty,
disconnected research
Operationalising variables
• Your design specifies the variables
• How do you measure the variables?
• How do you put a number to “intelligence”?
• How do you put a number to “capacity to
cope”?
• We need to convert abstract variables into
things which we can measure in the real
world - operationalisation
Operationalisation (2)
• Turn your variable into a directly
measureable thing
• Eg: How would you operationalise “success at
university?”
• Often there are developed scales available
• If you operationalise badly, you end up not
studying what you want
• Eg. Operationalising “success in career” by
looking at the paycheque only
Measuring variables
• The operationalisation implies what to
measure variable – how do you do it?
• If at all possible, use an established scale
• If no scale exists, construct one
• Scales must be valid and reliable
• The more of each of these properties, the better
the scale
• Validity and reliability need to be sorted out
before you run your study
Reliability in scales
• Reliability: stability of a measure over time
• If I measure you now and then in half an hour,
do I get the same reading?
• Max reliability depends on the construct
• Some construct are unstable (eg. heart rate)
• Low reliability implies that other variables
(“noise variables”) are being measured also
• Speaks of the “accuracy” of the scale
Ensuring reliability
• Reliability suffers when subjects have to
interpret
• Everyone’s interpretation is slightly different
• Objective scales are always more reliable
• Allow little interpretation
• Using a fixed response format helps
• Eg. Multiple choice, Likert type
• Researcher does not have to interpret what the
subject meant
Examples of response types
• Open ended item:
Briefly describe your most frightening experience
• MCQ:
The most frightening for me is
A) Dogs
B) Snakes
C) Spiders
D) None of the above
More examples
• Likert type:
Circle the one which best describes your
experience.
I find dogs to be
1
2
3
Not
frightening
at all
4
5
6
7
Terrifyingly
frightening
Validity in scales
• Validity: the degree to which a scales
measures what it is supposed to
• Validity is subdivided into many types
• We will look at 2 most important
• Criterion Realted Validity
• Construct Validity
Criterion Related Validity
• The degree to which this scales matches
other established scales
• By comparing to a scale known to be valid,
you can be sure yours is valid
• Why make a new scale if one already
exists?
• Maybe yours is quicker to do
• Maybe the established is not for group testing
How to check for criterion related
validity
• This is done through a set of studies
• Run a sub-study in which you give the subjects
your scale and the established one
• Run a correlation between the two scales
• If the correlation is statistically significant, your
scale compares well to the established one.
• It is better to run several of these validity
studies rather than just one.
Example: intelligence test
• An accepted test is the WAIS-R
• Very long to run (3 hours)
• You need something quicker (20 minutes),
create the QIQ
• Create a test, select a group of subjects
• Make them take the WAIS-R and then the QIQ
• Compare the results (correlation)
• If they correlate well, your test is measureing
intelligence
Construct Validity
• Construct validity: Does the scale actually
measure the construct?
• Eg: measuring cranial circumference to
measure intelligence
• Closely tied into the theory of the construct
• Most difficult to achieve, most important
• Measures lacking in construct validity are
almost useless
How to check for construct validity
Think abou it for a minute:
How can you show that a scale truly measures
what it claims to?
How would you show that your depression scale has
construct validity?
Hint: Compare it not to scales of the same thing, but
to similar and dissimilar things
The strategy
• Similar procedure to criterion related
validity:
• Before your actual study, run a set of substudies to check your measure
• You will need 2 sets of studies
• Concurrent construct Validity
• Discriminant construct validity
Quick aside: direction of correlations
• Correlation: the degree of relationship between
two variables, A and B
• Positive correlation: when A has a high value, B
has a high value. When A has a low value, B has a
low value
• Negative correlation: when A has a high value, B
has a low value. When A has a low value, B has a
high value
Correlations example
• Positive correlation: the relations ship
between amount smoked and probability of
heart disease
• Negative correlation: the relationship
between amount of daily exercise and
probability of heart disease
• No correlation: the relationship between
whether you drink tea or coffee and the
probability of heart disease
Concurrent validity
• Show that your scale relates positively to
related concepts
• People who do are depressed will have many
sad thoughts (mood conguency effect)
• Establish concurrent validity against several
other constructs
Discriminant validity
• Show that your scale relates negatively to
opposite concepts
• People who are depressed will have very low
energy
• Establish discriminant validity against several
other constructs
Ensuring construct validity
• Best way: be an expert on that construct
• Theory should tell you what things to include
• BUT: only if the theory is well-established!
• Second way: consult the experts/literature
closely
• Stay with the uncontroversial aspects of that
construct
Validity & reliability summary
• Aim: make sure that your variables are
correctly operationalised
• Reliability: scale is stable over time/place
• Validity: scale is truly measuring the
construct not something else
Validity & Reliability summary (2)
• Ensuring reliability: require verly little
interpretation / increase objectivity
• Ensuring validity: base the measure closely on
current understanding of construct
• Measuring validity: positive correlations with
related scales, negative correlations with opposite
scales