Transcript Protecting Science and Human Subjects in Online Research

Protecting Science and
Human Subjects in
Online Research
José A. Bauermeister, MPH, PhD
John G Searle Assistant Professor
Health Behavior & Health Education
University of Michigan School of Public Health
Outline

Online methods
 Transitioning




into Online Data Collection
When?
Why?
For What?
Issues to Tackle
 Scientific
Integrity
 Human Subjects Regulation

Other Considerations
HIV/AIDS Prevention Online

90% of Americans use the Internet

Broadened discourse of sexuality

HIV/AIDS e-prevention needed

Online networks are a part of every day life…
Pequegnat, W., et al. (2007). Conducting Internet-based HIV/STD Prevention Survey Research: Considerations in Design and
Evaluation, AIDS & Behavior, 11(4), 505-521.
Online Data Collection
Benefits:
 Incredible reach of internet



90% of YA use internet
85% of YAs have broadband at home
Speed and ease of survey administration
Challenges:
 Response rates
 Generalizability
 Digital Divide
The Family Tree…
Interviewer-administered surveys
Self-administered surveys
Paper and pencil surveys
Mail/postal surveys
Telephone interviews
Disk-by-mail (DBM)
Computer-assisted interviewing (CAI)
E-mail surveys
Computer-assisted self-interviewing
(CASI/ACASI)
Phone Reporting Systems (PRS)
Online Surveys
Web-surveys
SMS Surveys
Dynamic/Interactive
Why?

Replacement technology
 Limitations
of mail or telephone surveys
 Individual burden
 Data collection costs
 Systematic data collection
 Adaptive
 Complex skip patterns
Couper, M.P. (2008). Designing Effective Web Surveys. New York: Cambridge University Press.
Why? (cont)

Supplemental technology
 Mixed-method
data collection
 Visual/interactive elements
 Additional tool to acquire data
 Social validity
Couper, M.P. (2008). Designing Effective Web Surveys. New York: Cambridge University Press.
Collecting Sensitive Information
Sensitive Information

Questions are contingent on the study goal.

Online questions…

Avoid social desirability regarding sensitive questions

Improve recall

Ensure data consistency

Tailor content to participants


Remove irrelevant content
Program “change” into survey
Approaches

Avoid interviewer-based social desirability
regarding sensitive questions
 Pregnancy
 Substance Use
 Sexual
Behavior
 Mental
Health ***

Symptoms vs. Diagnosis
Approaches

Improve recall
 Use
of prior data entries
 Use
of event calendars
Approaches

Ensure data consistency
Approaches

Tailor content to participants
 Remove
irrelevant content
 Use formal vs street language
Approaches

Tailor content to participants
Language Preference
“The following questions refer to your sexual behavior
during the past 30 days. Our focus will be exclusively on
anal, vaginal, and oral sex. Therefore, do not include in
your answers references to partners with whom you did
not engage in anal, vaginal, and oral sex.
Would you like to see these questions in formal language
or street language?”
Formal language  60(45.5%)
Street language  72 (54.5%)
Online Data Collection

Considerations re: Human Subjects
 Participant
Understanding of Study
 Participant
Burden
 Data
Security
 Collection
of (non-consented) third party data
Standard recommendations

Piecemeal consent by sections

Certificate of Confidentiality

“In-house” survey administration and data management

128-bit SSL Encryption

Use of well-validated psychometric measures
“In-house” survey administration

Collect through a third-party?

Buy licensed software?
Data Repository

What strategies are in place to offset a breach?
 Where
are the data?
 Who
accesses the data?
 How
are the data files (en)coded?
 When
do you download/clean the data?
DATA QUALITY
Why does data quality matter?

Increasing adoption of web-based data collection in sex
research.

Important to make sure data collected through this
modality are valid and reliable, and that conclusions are
accurate.
 Duplication
 Falsification

Decrease research costs.
Bauermeister, J.A., Pingel, E., Zimmerman, M.A., Couper, M., Carballo-Diéguez, A., & Strecher, V.J. (2012). Data quality in
web-based HIV/AIDS research: Handling Invalid and Suspicious Data. Field Methods, 24(3), 272-291.
How common are these issues?

Invalid entries occur commonly in web-based research.

Konstan and colleagues (2005) found that 11% of entries in their
MSM sample (N=1,150) were duplicate entries from
participants.

Bowen and colleagues (2008) found that approximately 1/3 of
the 1,900 total submissions among MSM were multiple entries.

Bauermeister and colleagues (2012) found that 16% of entries in
their YA sample (N = 3,448) were falsified.

Bauermeister and colleagues (2014) found that 15% of entries in
their YMSM sample (N = 2,329) were falsified.
Preventing falsified data…

Automated Procedures
 Eligibility screeners
 Restriction of one submission per
 Reverse IP lookup
 Creating statistical algorithms

IP address
Manual Procedures
 Cross-checking
 Flagging cases
 Geocoding
entries from similar IPs
spatial data (if available)
Dear participant,
We appreciate your interest and willingness to complete our survey on young men who
have sex with men’s (YMSM) online dating behavior. Unfortunately, we have noticed
irregularities during data collection and have had to stop our study. Specifically, a few
individuals have chosen to provide false data and/or create multiple entries so that they may
receive one or more $15 iTunes incentives.
We cannot underscore how disappointing this has been for us. As public health
practitioners, we strive to collect quality and robust data through research that will inform
HIV/AIDS and sex education programs for young men. False data diminishes our ability
and actually harms the population that we seek to help through science and social services.
We hope that similar events will not occur in future efforts. It is only through the
honesty, integrity, and willingness of participants that we can help to contribute to the
eradication of HIV and other sexually transmitted diseases from our communities.
Recommendations

Pre and post hoc decisions regarding how to handle
suspicious data are warranted.

We encourage researchers using web-surveys to:
 Mention whether data exclusion criteria are set.
 Explicitly state whether the presence of invalid data
will be
examined and how it will be handled.

Quality criteria for web-survey research may be an
important covariate in meta-analyses.
Questions?