Transcript Unobtrusive Measures

Unobtrusive Methods for
Social Science Research
A Neglected Methodological Approach in
the Social Sciences
Yoram M Kalman
Resources
Methodological Weaknesses of
Interviews & Questionnaires

Responders provide answers that are in line:
 with
 with


self-image
researcher expectations
We only get answers from those who are
accessible and willing to respond
Hawthorne effect, etc.
Justifications for Unobtrusive
Measures



Complementary to direct elicitation
techniques
Adaptable in situations where eliciting is
difficult and/or dangerous
Approach that promotes creative
methodologies
Categories of Unobtrusive Data



Found data
Captured data
Retrieved data
 Running records
 Personal and episodic
records
Found Data – Erosion







Floor tiles around chick exhibit
Popularity of gym apparati by chalk
consumed
Popularity of library books by smudges,
finger marks etc.
Leaflets in different languages in tourist sites
Postcards in museum exhibits
Paper tissue and cough medicine in campus
store correlated with class attendance
records
Fun, but relatively rare in research
Found Data – Accretion


Before and after electric waste disposal units:
change in fly population measured on car grilles
Graffiti:




Racial tensions in Hawaii
Youth relationships in high schools
Correctional facility for male juveniles
Garbology:



Survey data on alcohol consumption vs. number of empty
alcohol containers appearing in garbage cans. The survey
data underestimated consumption due to high refusal rate
and exclusion of teenage drinkers
Others: Condom wrappers1976-1984, and beyond; Beer
tabs vs. beer cans as evidence for recycling.
In US, 4th amendment issues
Found Data – Disadvantages




Conservative estimate
Socially dependent
Takes time to accumulate
Inferentially weak
Captured Data





Exterior physical signs: head and facial hair,
tattooing, clothing and adornments
Expressive movement: demeanor, eye gaze,
touching, verbal latency
Physical location: proxemics, spatial
arrangement
In-situ conversation: pronunciation
Time related behavior: often neglected.
Duration as proxy for importance, time of day
influencing behavior
Retrieved Data – Running Records







Running records vs. episodic and private
records: e.g.: mass media, reference works,
records of proceedings
Lengthy periods of time
Ubiquitous
Lower cost
Less amenable to self report
Allow the exploration of trends and temporal
patterns
Limitation: collected by others
Retrieved Data – Running Records



Mass media: news stories, advertisements,
photographs, obituaries/weeding
announcements
Reference works: directories, almanacs,
yearbooks
Records of proceedings: discussions and
decisions
Retrieved Data – Running Records


Actuarial records: births, deaths, marriages
Personal ads:
(1998) and gender conceptions in 1st and
2nd halves of 20th century
 Race relations
 HIV status
 Jagger

Marriage announcements and obituaries:
 Class vs. church denomination
 Gender vs. occupation
 Gender vs. length of obituary

Others: job ads, book lists, phone books, etc.
Retrieved Data – Disadvantages





Often can’t be used “as is”. Should consult
with those who produced them (reactivity!)
Quality issues, especially when more
extensive
Selectivity: exclusion and inclusion criteria
Statistics might reflect more the organization
collecting the data, than the sources of the
data
Confidentiality
Retrieved Data – Personal and
Episodic Records

Best example is personal documents:
 Letters
 Diaries/ daily journals
 Autobiographies
 CV’s
 Wills
 Photo albums
Key Principles of Unobtrusive
Measurement






Construct and impose multiple indices that
converge
Assume noise is rare
Investigators believe in amortization
Find foolishness functional
Ponder the variance rather than the mean
Investigators use expectancy as a control
Webb & Weick, 1983 in Lee, 2000
Example: What email Latency
Constitutes Silence?






Response latencies in Enron emails
Research on CRM in hotel industry
Chain letters
OOO messages
Published corporate policies
Blog postings discussing online
responsiveness
+


Questionnaires
EVT experiments
Key Challenges



Unconventional measures stand out and
could receive less respect
Often, unchartered ground
Privacy:
 When does observation intrude on privacy
 Informed consent, etc. are reactive
 Challenge is multiplied in online settings

Falling in love with methodology
Unobtrusive Measures Online







Plenty of sources
Searchable
Digitized, and ready for processing
Logs
Sharing with other researchers
Demographics of online users are no longer
unrepresentative
Less limited geographically
Conclusion



Originality and innovation
Complementary, triangulation
Do the impossible:
 Sensitive issues
 Limited resources


ICT revolution a significant influence
Beware of:
 Ethical challenges
 Methodological pitfalls
Yoram Kalman
www.kalmans.com
Center for the research of the Information
Society
[email protected]