Transcript Improving the Online Evaluation Process and Response Rates

Improving the Online Evaluation Process
and Response Rates
SAIR 2011
WCU
•
•
•
•
9,352 students
Master’s Comprehensive
Mountain location
Resident and Distance
History of Evaluations at WCU
• No university-wide policy on administration,
instruments, courses covered, and use of results
• Scanned, paper forms administered during class time
• Long wait for results
• No flexibility in types of reports
• Poor security– esp. comments
Implementation Timeline
2002 through 2006
• Faculty Senate establish ‘teaching evaluation
committee’
• Provost mandated online evaluations (PACE initiative)
• Developed SAI instruments based on course types
• Developed guidelines for results usage
• Researched available options for administering evals
and selected ConnectEdu’s CoursEval
Faculty “own” the evaluation process!
Timeline cont.: Course Evaluation Pilot – Spring 2007
•
•
•
•
5 departments
328 courses (5689 surveys)
45% response rate
Problems: log-in and email issues
lack of experience
Timeline cont.: Establishing Procedures (2007-2008)
Faculty Senate
•
•
•
•
•
Scheduling guidelines
Low-enrolled courses
Student incentives
Refined SAI instruments
Planned validity study
Implementation Committee
•
•
•
•
•
•
Advertising/incentives
Faculty training
Summer parts-of-term
Instrument defaults
Student email policy
Login issues
Lessons Learned
• Lesson 1: Involve everyone with a stake in the
evaluation procedures and results; everyone with
experience in processing & administering evaluations
• Lesson 2: Consider all issues before implementing
• Lesson 3: Build a strong communication system
between faculty and survey administrators
IR Responsibilities
• Set-up & administer each semester’s evaluations
• “Help desk” for faculty and students
• Learn about new tools and related software, reporting,
and response rates
• Communicate to Faculty Senate any issues arising
affecting senate policies and survey administration
Overview of Current Policies
•
•
•
•
•
•
•
•
•
•
•
All course sections, all terms
Open period set by course length
University-wide incentives allowed
Low-enrolled courses
Crosslisted courses
Periodic response rates & student reminders
Faculty evaluation (not course content)
Results availability
Use in faculty AFE/TPR process
IR office responsible for administration
Department heads have final say in instrument used
Response Rates over time
80%
Term
Rate
70%
Spr 07
44.7%
60%
Fall 07
43.5%
Spr 08
32.8%
Fall 08
46.5%
40%
Spr 09
47.3%
30%
Fall 09
43.4%
Spr 10
49.6%
Fall 10
73.4%
Spr 11
73.4%
50%
20%
10%
0%
Spr 07
Fall 07
Spr 08
Fall 08
Spr 09
Fall 09
Spr 10
Fall 10
Spr 11
Marketing
Strategy
Did it work?
Large Banners
Yes
Postcards
Maybe
Emails
Yes
Flyers around campus
Maybe
Radio announcement
Maybe
Giveaways
No
“Early” grade release
YES!
“Early” Grade Release
• Students are encouraged to complete evaluations in
order to receive their final grades “early”
• A list of all students who complete evaluations is sent
to the Registrar
• Emails go out to students as faculty members submit
grades
• Those who do not complete evaluations get final
grades after exam week is over
Overall, what works for us
•
•
•
•
Faculty Senate owns the process
Changes are made slowly, with much deliberation
Small, centralized survey administration
SaaS – few technical issues on our end, backed up
data, no questions about data integrity
Features of CoursEval that we like
• Students and Faculty – Find all associated courses
and surveys with one click
• Consolidation of people without deleting anything
• Active Directory login
• Automated emails, open/close, results release
• Response rates – real time, easy to locate and email
• Easy to use administrative tools
• Customizable emails and log-in site
Features we would like
• More flexibility with reports
• Advance notice of outages and problems
• Ability to access individual student responses (for
research purposes)
• Faculty member attached to course cannot be
changed
• More stream-lined set-up
Contact Information
WCU’s Course Evaluation website:
http://www.wcu.edu/8356.asp
“Guidelines & Procedures for Administration &
Oversight of Student Assessment of Instruction”
Institutional Planning & Effectiveness:
http://www.wcu.edu/12829.asp 828-227-7239
Elizabeth Snyder [email protected] 828-227-2570
Alison Joseph
[email protected] 828-227-3042
ConnectEdu website:
http://www.connectedu.com/