Transcript The ACHA-National College Health Assessment

Implementing the ACHA-NCHA: What
users need to know
to get started
Mary Hoban, Ph.D., E. Victor Leino, Ph.D.,
Theresa Jackson Hughes, MPH
American College Health Association
What will we cover today?
• Overview of the ACHA-NCHA surveying process
–
–
–
–
–
–
–
Steps in the surveying process
Paper vs. Web versions
Incentives
Sampling
Scheduling
Reporting
Much more!
• Additional Resources
• Question and Answer Session
Summary of steps involved in
the surveying process
• Determine the purpose of surveying and target
population
• Identify feasibility and sources of
support/campus resources
• Determine cost
• Specify the method for conducting the survey
• Determine the incentives to be used, if any
• Determine sampling procedures and level of
over-sampling needed
Summary of steps involved in
the surveying process
• Specify the schedule for conducting the survey
and the amount and schedule of follow-up
contacts
• Determine if you want to add extra questions or
automatically downloaded variables
• Obtain Institutional Review Board and/or
administrative approval
• Send ACHA all required information
• Determine how you will report results to other
campus officials
Purpose of Surveying and
Target Population
• Purpose of Surveying
– Pre-test and/or Posttest
– General Health
Assessment
– Needs Assessment
– Comparison to
National Sample
– Program Planning
• In what population are
you interested?
–
–
–
–
–
–
Freshmen
Athletes
Graduate Students
Commuters
All Students
Other?
Feasibility, sources of support,
and campus resources
• Is it feasible to conduct the ACHA-NCHA on your
campus?
– Depends on sources of support and campus resources available
• Sources of support
–
–
–
–
–
Administrators
Health Center staff
Campus Recreation staff
Other Student Affairs staff
Information Technology staff
• Campus resources
– Graduate students
– Academic departments
Cost
• Cost
– ACHA costs available at:
www.acha.org/projects_p
rograms/NCHA_Pricing.
pdf.
– Other costs may include
personnel costs for your
institution, postage, and
incentives
Cost
Institutional Member
Participation Fee
Paper-Based
Web-Based
Paper-Based
Web-Based
$0.40 each
$0.20 each
$0.75 each
$0.40 each
Non-Respondent
Recontact Fee
Processing Fee
Non-Institutional Member
$0.10 each
$0.20 each
$0.50 each
$0.25 each
$0.90 each
$0.50 each
Reports Package
$300
$300
$300
$300
User’s Manual
Free
Free
Free
Free
$60/hour of labor
$60/hour of labor
$60/hour of labor
$60/hour of labor
Customized
Analysis
Custom
Questions
Varies
Varies
Estimate
ACHA Institutional Member Price
Based on 3,000 Contacts
Methods of surveying
• Paper Survey Options
– All students
– Randomized mailed survey
– Randomized classroom survey
– Campus assessment day
– New student orientation
• Web-based Survey Options
– All students
– Randomized NCHA-Web survey
Paper Survey Options
Randomized Classroom Survey
• Cooperation from instructors is
critical
• Selection of classes
– Random
– Should be representative of the universe
from which you are drawing your sample
– Seek assistance of registrar’s office
• Approximately 35 minutes to
complete
• Complete steps available in the
User’s Manual
Randomized Classroom Survey
• Advantages
–
–
–
–
Convenient
Cost-effective
High response rate
Possible to collect
information from a
wide coverage of
majors if classes
selected randomly
• Disadvantages
– Instructors may be
reluctant to give up
class time for survey
purposes
– Classes may not be as
representative of
various majors and
campus demographics
as other methods
Randomized Mailed Survey
• Randomized selection of
students (or all students)
• Send survey with an
addressed stamped envelope
• May send thank you letters
and re-contact letters
Randomized Mailed Survey
• Advantages
– Ease of randomization
– Greater
generalizability of
results
– Follow-up notices
possible, which may
increase response rate
– Well documented
procedures and
policies
• Disadvantages
– Expense (costs of
mailing, tracking, and
re-contacting)
– Require more
administrative work
– Generally has a
response rate below
50%
How to avoid pitfalls of paperbased surveying
• Get organized early!
• Classroom based survey
– Secure approval of instructors as soon as possible
and remind them of when you will be coming
– Bring plenty of pencils or pens
• Mailed surveys
– Mail to campus address, not permanent address
– Send follow-ups to increase response rates
– Pay for all return postage!
Web-based Survey Options
Randomized NCHA-Web
Survey
• Hosted through a secure server
• Students are generally invited to
participate via e-mail
– IHE sends a list of e-mail addresses
ACHA
– Can be randomized by registrar,
randomized by ACHA, or can invite
entire population of students
• Less work on the surveyor
Randomized NCHA-Web
Survey
• Advantages
– Cost effective
– Non-responders can be
contacted easily
– Respondent may scroll
through survey
– Possible for the respondent
to leave and come back to
the survey
– Easy to select those who
will receive incentives
– Ease of administration
– Easy to add extra
questions
• Disadvantages
– Students may not trust web
security and may choose
not to participate
– Low response rates
• Typically between 10-50%
• Average: 25%
– Issues with spam blockers,
faulty e-mail addresses
How to avoid pitfalls of webbased surveying
• Obtaining student e-mail
addresses
• Spam filters and rate
control systems
Incentives
• Used to increase response rates and
thank students for participating
• Methods of giving incentives
– Randomly select one participant to receive a
large incentive
– Randomly select multiple participants to
receive several smaller incentives
– Give incentives to all participants
Incentives
• Example incentives
– iPod
– Gift certificates
• Bookstore
• Local restaurants or stores
–
–
–
–
–
–
–
Cash
University parking permit
Airline tickets
DVD player
Tuition
$2.50 Dining Services Coupon
Extra Credit
Sampling procedures
• Goal is to select a representative sample
of your institution
– Usually not necessary to sample entire
institution
• Important to use random sampling
• Randomized classrooms
– Want to sample variety of academic subjects
and levels
Sample Size
Size of Campus
<600
600-2,999
3,000-9,999
10,000-19,999
20,000-29,000
≥30,000
Desired Sample
All students
600
700
800
900
1,000
Sample Size and Response
Rates
• Mailed and web surveys have lower return
rates (approximately 25%)
• This affects the number of students you
should invite to participate
– For example, if the desired sample is 600 for
a mailed survey, 2400 names should be
randomly drawn to receive the survey
• Better to oversample than undersample!
Scheduling
• Pre-survey publicity
• Spring version
– Recommended to survey between the last week of
March and the first week in May
– More than 30 days after spring break and winter
break
• Fall version
– Possible to survey at almost anytime
– More than 30 days after fall break
• Recontacts
• Survey close
Extra Questions
• Web version
– Possible to add up to five additional questions for a surcharge of
$700
– Added to end of the online survey
– Reposes automatically downloaded with other ACHA-NCHA data
– Must be single questions
• Each item on a “Select all that apply” question is considered as a
separate question
• May not have more than 10 response categories
• Paper version
– Tracked by lithocodes on paper ACHA-NCHA
– No cost nor limits
– Campus does all the work
IRB/Administrative Approval
and Informed Consent
• The ACHA-NCHA is research, and as such, all participants
must be protected
• Institutional Review Board
– Initiated to protect human subjects from potential harm due to
the nature of the research
– If your school does not have an IRB, you must secure human
subject protection and approval to administer the survey from an
administrator who is familiar with these procedures
• Informed consent
– Each IHE should conform to the institutional requirements for
informed consent and human subject’s protection
– Sample consent forms are provided in the User’s Manual
• Each IHE must provide IRB/administrative approval to NCHA
prior to administration of the survey
What do I need to send to ACHA to
implement the NCHA paper version?
• Before Implementation
– The ACHA-NCHA Order form
– Initial payments for both the total number of
student contacts and report package
– IRB/Administrative approval letter
• After Implementation
– Completed surveys (do not bend)
– Demographic information sheet
– Final payments
What do I need to send to ACHA to
implement the ACHA-NCHA-Web?
• Before Implementation
– The ACHA-NCHA Order form
– Initial payments for both the total number of student contacts and
report package
– Spreadsheet file of student e-mail addresses in the first column
of the file and any personalization/download variables in
columns thereafter
– The letter of invitation/consent and subject line
– The reminder letter of invitation/consent and subject line to be
sent to non-responders
– Dates for initial contacts, recontacts, and survey close
– IRB/Administrative approval letter
• After Implementation
– Demographic information sheet
– Final payments
Reporting Results
• ACHA provides the
following products:
– Institutional data file
on CD
– Institutional data report
– Institutional executive
summary
– Reference group data
report
– Reference group
executive summary
Data Report Example
Suggested Resources
• Neuman, W. Lawrence (2000). Social Research
Methods: Qualitative and Quantitative Approaches, 4th
ed. Boston: Allyn and Bacon. Chapter 10.
• Babbie, Earl (1998). The Practice of Social Research.
Belmont: Wadsworth. Chapter 10.
• ACHA-NCHA User’s Manual
– Available at
www.acha.org/projects_programs/ACHA-NCHA_USERS_
MANUAL_ver2-004.pdf
• ACHA-NCHA Frequently Asked Questions
– Available at
www.acha.org/projects_programs/ncha_webfaqs.cfm
Questions about the
ACHA-NCHA?
• Contact:
Mary T. Hoban, Ph.D., CHES
American College Health Association
410.859.1500
[email protected]