Transcript Chapter Two


Ch
5: Measurement Concepts
Ch 6: Observing Behavior
Reliability

Reliability refers to the consistency or stability of a
measure of behavior [p92]
 If you weighed yourself now and then at the end of class
and you weighed the same both times, you would say the
scale is reliable.

True score: The real score on the variable

Measurement error: occurs when a measure yields
inconsistent results; the greater the inconsistency, the
greater the measurement error
Reliability of Measures
 How can we assess reliability?
• Correlation coefficients tell us how strongly two variables
are related.
- Pearson Product-Moment Correlation Coefficient
(noted as r in text) [p93]
 Coefficients range from 0.00 to - 1.00 and 0.00 to +1.00
 Sign of the coefficient indicates direction
 Value of the coefficient indicates the strength
Reliability of Measures
Variables covary in
opposite directions
- 1.00
Variables covary in
the same direction
0.00
+ 1.00
Methods of assessing reliability
[pp 94-96]
 Test-retest reliability: Assesses reliability of a score by
measuring the same individuals at 2 points in time
 Internal consistency reliability: Assesses reliability of the
assessment tool (e.g. test) at one point in time
 Questions should yield consistent results
 Split-half reliability – individual’s total score on one half
of the test is correlated with the total score on the other
half of the test
Methods of assessing reliability:
Interobserver (Interrater) Reliability

A measure of how  How we assess interobserver reliability:
often two or more
Number of times two
observers agree (are
observers agree
consistent) in their
observations. [p96]
____________________ X 100

Nominal scale:
percentage
agreement
Number of opportunities
to agree
Construct Validity of Measures [pp 97-100]
 Refers to the adequacy of the operational definition of variables
 Is the measure that is used actually assessing what it is
supposed to assess?
If so, it has face validity.
Example: Are facial expressions an adequate measure of
happiness?
Indicators of construct validity


Face Validity: The content of the measure
appears to reflect the construct being measured.
Content Validity: The content of the measure is
linked to the universe of content that defines the
construct
•
Facial expressions are part of a set of behaviors related
to happiness, such as body posture, thoughts, etc.)
Example: Are facial expressions an adequate measure of
happiness?
Indicators of construct validity

Predictive Validity: Scores on the measure
predict behavior on a criterion measured at a time
in the future
•

SAT (Scholastic Aptitude Test predicts future success at a
university)
Concurrent Validity: Scores on the measure are
related to a criterion measured at the same time
•
•
People who score high on a verbal anxiety test
experience increased sweating at the same time
People who have happy facial expressions concurrently
report feeling happy
Indicators of construct validity

Convergent Validity: Scores on the measure
are related to other measures of the same
construct.
•

A score for happy facial expression is related to a
score for body posture or mood or optimism
Discriminant Validity: Scores on the measure
are not related to other measures that are
theoretically different.
•
A score for happy facial expression is not related to
one for intelligence or cleanliness
Measurement Scales

Four levels for quantifying behavior: nominal,
ordinal, interval, and ratio [p102-105]
Nominal scale



Classifies behaviors,
events, and
characteristics into
different categories
[p102]
No numerical or
quantitative
properties
Independent
variables are often
nominal or a
categorical variable
Ordinal
Scale

Measurement scale in which events and
behaviors can be rank ordered (i.e, first,
second, third, etc.) [p103]
• Allows categories to be ordered first to last,
highest to lowest, biggest to smallest, etc.
• Quantitative but no values attached to the
intervals
Interval Scale

Measurement scale
that allows researcher
to specify how far apart
two observations are
on a given dimension
[p104]

Difference between the
numbers is meaningful
•
•
Intervals are equal in size
Quantitative but no
meaningful zero reference
point
Ratio Scale

Measurement scale that is quantitative,
with all numerical properties including an
absolute zero reference point [p104]
Let’s practice! Identify the measurement
scale for the following data:
Circle your marital status: Married Single Divorced Engaged
Let’s practice! Identify the measurement
scale for the following data:
Circle your marital status: Married Single Divorced Engaged
NOMINAL
Let’s practice! Identify the measurement
scale for the following data:
Do you go to work? Yes
No
Let’s practice! Identify the measurement
scale for the following data:
Do you go to work? Yes
No
NOMINAL
Let’s practice! Identify the measurement
scale for the following data:
If you work, how many hours a week do you work? _______
Let’s practice! Identify the measurement
scale for the following data:
If you work, how many hours a week do you work? _______
RATIO
Let’s practice! Identify the measurement
scale for the following data:
Rate your enjoyment of college on the scale below.
1
Not Very Much
2
3
4
5
Very Much
Let’s practice! Identify the measurement
scale for the following data:
Rate your enjoyment of college on the scale below.
1
2
3
4
Not Very Much
5
Very Much
INTERVAL
Let’s practice! Identify the measurement
scale for the following data:
What is your class standing?
Freshman
Sophomore
Junior
Senior
Let’s practice! Identify the measurement
scale for the following data:
What is your class standing?
Freshman
Sophomore
Junior
ORDINAL
Senior
Places to Observe on Campus
(Spring 2010)













1. Area in the middle of campus, by the Info Trolley.
2. By the food place on campus which is near the gym/pools.
3. By the statue of the boy overlooking the water feature.
4. Entrance of the SE parking structure (people exiting structure)
5. Entrance of the SE parking structure (people entering structure)
6. Entrance of the SW parking structure (people exiting structure)
7. Entrance of the SW parking structure (people entering structure)
8. By the entrance of the bookstore.
9 In front of C Building, by mirror pools
10. Shuttle stop by U Building
11. Bus stop on Colorado in front of library
12. Passenger drop off/pick up (Colorado B. in front of L Bldg)
13. Starbucks
Reactivity

Occurs when individuals change their
usual behavior, when they know they are
being observed [p101]

Minimize reactivity by:
• Allowing time for individuals to become
used to the presence of an observer or
the recording equipment
Sampling Behavior [p116]

Time sampling: Researchers choose
time intervals for making observations
• Systematic: Schedule observations to occur at a
regular time
• Random: Use some random means for identifying
times for observations
• Event sampling is used for rare events

Situation sampling: Researchers choose
different settings, circumstances, and
conditions for their observations
Classification of Observational
Methods

Two categories of observational
methods:
• Observation without Intervention
• Observation with Intervention
Observation without Intervention

Naturalistic Observation (AKA: field work or
field observations): Observation in natural
(“real-world”) settings without an attempt to
intervene or change the situation [p110]
Participant observation

Observer is an active participant in the natural
setting he or she observes [p112-113]
•
•
Undisguised
(unconcealed): people
in the setting know
they are being
observed
Disguised (concealed):
people don’t know
they are being
observed
Norah Vincent
“My Life as a Man”
Observation without
Intervention


Physical traces: The
remnants, fragments,
and products of past
behavior
Products: Creations,
constructions, or
other artifacts of
earlier behavior
Natural-use traces

Produced without any
intervention by the
investigator
Controlled-use traces

Produced with some
degree of
intervention or
manipulation by the
investigator
Natural use or controlled use
trace?
Natural use or controlled use
trace?
Natural use or controlled use
trace?
Archival records

Public and private
documents that describe
the activities of individuals,
institutions, governments,
and other groups [p204]
Archival Research
[118-121]
A non-reactive measure
(or indirect method) for
collecting data—when the
individual who did the
behavior is no longer
present
Archival research involves
using previously compiled
information to answer
research questions
Running records & records of
specific events

Running records are
continuously kept and
updated (e.g., check
book)

Records of specific
events (e.g., diploma)
Archival data are used to:




test hypotheses as part of a multimethod
approach
test the external validity of laboratory
findings
test hypotheses about past behavior
assess the effect of a natural treatment
Multimethod approach

Researchers use a variety of measures
to examine a research question
Natural treatments

Naturally occurring events that impact
society and individuals
Selective deposit

Occurs when some
information is
selected to be
included in the
archival record, but
other information is
not
Selective survival

Occurs when
information is lost or
missing from an
archival source
Content analysis

The process of making inferences based
on objective coding of archival data
[p120]
Quantitative analysis

Classifying events and behaviors into categories
to count their frequency of occurrence [p109]
•
Assign numerical values to responses and measures and
then subject the data to quantitative statistical analyses
Ex: Count the number of
times genderstereotypical jobs were
assigned to characters
in the story.
Qualitative analysis

Subjective judgments about the content in an
archival record [p109]
•
•
Describe behavior or findings based
on themes that emerge from the data.
Data are nonnumerical and expressed
in language and images
Ex: Watch the tape of
Osama Bin Laden and
tell me if you think he is
being deceptive.
Three Steps of Content Analysis
1)
2)
3)
Identify a relevant source
Sample selections from the source
Code units of analysis
Observation with Intervention
[p114]

Systematic observation: The careful
observation of one or more behaviors in
a particular setting. Use when:
• interest is in only a few very specific
•
•
behaviors
observations are quantifiable
researcher has developed prior hypothesis
Field experiment

Researcher
manipulates an
independent variable
in a natural setting
and observes
behavior (dependent
variable) [pp 83, 113]
Coding systems for systematic
observation… [p115]
 are either (1) developed to fit the needs of the
particular study or (2) “borrowed” systems
developed by others
 should be as simple as possible
 must allow researchers to easily categorize
behaviors
Coding systems can involve:


Comprehensive records of people’s
behavior (e.g., complete records, such as
video tapes)—a qualitative record
Selecting specific behaviors to record—a
quantitative record [pp115-116]
Equipment & Narrative Records
Used when researchers want a complete (comprehensive)
reproduction of people’s behavior [pp 115-116]
• Examples: video and audio recordings, field notes
Systematic Observation: Methodological Issues
[pp115-116]
1. Equipment: can fail
2. Reactivity: the probability that the presence of the
observer will affect behavior
3. Reliability: refers to how stable/consistent the measure is
over time or between observers
4. Sampling: refers to how participants and behaviors are
chosen to be studied
1. Larger samples of participants and multiple samples of behavior
can increase both internal and external validity.
Case Studies [pp117-118]
 A case study provides a description of an individual.
• Usually the individual is a person, but may also be
a setting.
• A psychobiography is a type of case study in which
a researcher applies psychological theory to explain
the life of an individual.
Case Studies…
 are done when an individual possesses a particularly
rare, unusual, or noteworthy condition.
 provide unique data about some psychological
phenomenon
 and the insights gained from them may lead to other
research using other methods