Transcript Slide 1

Evaluating Academic Success Effectively with “EASE”:
A New Survey of Academic Engagement
The University of North Carolina at Charlotte
4th Annual Retention-Graduation Conference
UNC General Administration
October 24, 2007
Dr. Theodore W. Elling
Associate Vice Chancellor for Student Affairs
[email protected]
Dr. Cynthia Wolf Johnson
Associate Provost for Academic Services
[email protected]
Dr. John Smail
Dean of University College
[email protected]
Overview
• Introductions
• Setting the Context: UNC Charlotte and Institutional
Climate
• Related Research & UNC Charlotte Modified Retention
Model
• EASE: Evaluating Academic Success Effectively
• Next Steps & Applicability to Your Institution
Setting the Context: UNC Charlotte
• Fall 2007 Enrollment: 22,407 (4.1% increase)
– New Freshmen:
2,950
– New Transfers:
2,040
– Graduate Students: 4,781 (21.3% total enrollment)
– Residing on Campus: 4,739
• Retention Rate/Goal:
77.4% (by 2012: 83.0%)
• Graduation Rate/Goal: 24.2% (by 2012: 28.8%) – 4 yr
49.8% (by 2012: 51.6%) – 6 yr
Institutional Climate: Spring 2005
UNC Charlotte
– AASCU – Graduation Rate Outcomes Study
– Internal Review of Support Services
• Loosely connected islands & data silos
• Support often arrives too late for maximum benefit
– Need for better coordination of services
– Need to develop a process for early-alert/early intervention with atrisk students
– Need to enable coordinated longitudinal research
– Need to begin studying new transfer students
~ Continued Concern: Retention & Graduation ~
Related Research
• Theoretical origins of survey development
– Tinto’s model of student departure (1975; 1987; 1993)
• Attrition occurs when students are not successfully integrated into the
college environment.
– Academic
– Social
– Astin’s theory of student involvement (1984; 1996)
• Involvement leads to a variety of outcomes, including increased
satisfaction and retention
UNC Charlotte Modified Retention Model
EASE
Survey with
Data
Integration
EASE – A Survey & Data Integration Process
Target: New Freshmen and New Transfers, end of their first 6-8 weeks
Pilot: Fall 2005
Full “Program”: 2006, 2007
Goals:
1)
Identify and contact students who identify that they
need help during their first term
2)
Combine survey information with first semester academic
performance and retention information to develop a detailed
comparison of students who perform at different levels
academically
3)
Enable longitudinal research addressing issues of new student
retention, academic performance and time to degree studies
EASE - Who is using it?
• What offices can benefit from EASE?
- New Student Programs/Orientation
- Student Success Offices
- First-year Programs
- Learning Centers
- Tutoring/Supplemental Instruction
- Counseling/Health/Wellness
- Residence Life
- Judicial Affairs
- Student Activities/Student Involvement Offices
- Academic Departments
- And many more….
EASE Elements
•
Pre-Survey Demographics
–
–
–
–
–
•
The EASE Survey
–
–
•
Student demographics
College and major
Predicted grade point average
Financial aid data
First year structured programs
Administered during weeks 6-8 of first semester
All new Freshmen and Transfer students
Post-Survey Performance Data (added as available)
–
–
–
–
–
Unsatisfactory grades at mid-semester
Grade point average
Academic standing (probation status)
Withdrawal codes
Subsequent semester enrollment/attrition
EASE - Question Architecture – 54 items
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Academic preparedness
Place of initial residence
Peer group interactions
Interactions with faculty
Interactions with staff
Involvement in structured first year programs
Academic goals
Academic performance self-appraisal
Academic advising
Academic effort
Academic integration
Social integration
Student employment
Family support
Financial resources
EASE Outcomes – Immediate Early Alerts
New Feature for Fall 2007
• At the end of the survey experience
– Customized online “Tips Sheet” displayed to at-risk populations
• Response to final question authorizes contact about programs and services
• Referred to Academic Advising Resources Page via URL link:
www.advising.uncc.edu/resources.htm
– URL links to appropriate University support service displayed for self-help use
– Six Early Alert Triggers:
•
•
•
•
•
•
Do not know who their academic advisor is
Experiencing difficulty in one or more courses
Do not have a career goal or related major
Is not involved in a student club or organization
Report inadequate high school preparation in mathematics
Report they are working
EASE Outcomes – Intervention Example #1
University Center for Academic Excellence
– At-risk populations:
•
•
•
•
•
•
•
•
Experiencing difficulty in one or more courses
Have not received feedback in one or more courses
Low study hours per week (ten or less)
Do not prepare adequately for class
Do not study with other students
Inadequate HS preparation (math, chemistry, biology, writing papers)
Miss classes
Have not met with faculty or staff about academic difficulties
– Custom emails sent to students alerting them to
• Coping Strategies
• Services
• Upcoming workshops
EASE Outcomes – Intervention Example #2
Housing and Residence Life
– On Campus At-risk populations:
•
•
•
•
•
•
Being lonely
Not establishing friends
Feeling they don’t belong
Anticipate not returning for next term
Poor academic fit
Low end of term grade estimate
– Personal Contact by Residence Coordinator or Resident Advisor
• Stop Light notification system – multiple data sources & connection
points
– Green Light – all is well – normal RA follow-up contact
– Yellow Light – RA contact – more direct conversation and assistance
– Red Light – RA & RC contact – higher level conversations and assistance
EASE Outcomes – Actionable Evidence
Example #1: Simple Messages
•
Relationship between behavior (attendance) and outcome (grades)
• Go To Class !!!
Note: Scale changed in Fall 2007 survey
EASE Outcomes – Actionable Evidence
Example #2: Extrapolation
• What would be a good question for an academic advisor to ask?
UNC Charlotte - EASE Survey - Fall 2007
Analysis By Engagement & Anticipated Retention
EASE Transfers (n=585)
Do you think you are coming back to this institution next semester?
Yes
I haven't decided yet
No
I feel this institution is a good fit for me academically
n = 537
n = 40
n=8
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
38%
5%
0%
47%
28%
0%
14%
48%
38%
1%
15%
13%
0%
5%
50%
EASE Outcomes – Actionable Evidence
Example #3: Academic Advising
•
•
•
•
2005, 2006: Too many students did not know who their advisor was
Institutional change: new University Advising Center for undeclared and
transitional students; emphasis on identifying advisor during summer
orientation
2007: Significant increase in student awareness
Satisfaction still needs improvement
Do you know who your academic
advisor is?
Percent Yes
Freshmen
Transfers
Fall 2007
Fall 2006
Fall 2005
71%
50%
64%
76%
50%
61%
80%
77%
n/a
73%
75%
n/a
I am satisfied with the assistance I
received from the academic advisor.
Percent Strongly Agree or Agree
Freshmen
Transfers
EASE Outcomes – Actionable Evidence
Example #4: Mid-term grade feedback
•
2005: Too many students not getting early feedback.
•
Emphasis by Provost and Academic Deans on the importance of mid-term
unsatisfactory progress reports, focus on courses with high
freshman/sophomore enrollments.
•
2006: Improvement in students’ perceptions, improvement in mid-term
deficiency grade reporting.
Number of Unsatisfactory Grades
Fall 2006
Received by Mid-Term
Freshmen
2406
Transfers
1398
Fall 2005
1036
631
EASE Outcomes – Next Steps
Using Data to Develop Retention Plans
1st Semester
Risk Factor Description: Freshmen Fall 05 - EASE
Unmet Financial Need (mean $ amount)
Predicted Grade Point Average Index
Math Preparation - self report
HS Grade Average - self report
Retention
x
x
Ethnicity (not Asian or Caucasian)
2nd Semester
Academic
Probation
Retention
x
x
x
x
x
x
x
Academic
Probation
x
x
x
Studying with Other Students
Early Course Feedback on Grades
Intend to Return
Hours Studying Per Week
GPA Estimate - self report
How Often Classes Are Missed Weekly
Coming to Class Prepared
Satisfaction with Current Major
# of Classes in Academic Difficulty - self report
Unsatisfactory Grade Letters Received
Attended SOAR
Participated in a Learning Community
x
x
x
x
x
x
x
Feeling Like They Belong Here
Having Friends
Feeling Lonely
Club & Organization Involvement
x
x
x
Methods of Paying for College: Employment
Methods of Paying for College: Pay Tuition-Fees-Books
Methods of Paying for College: Monthly Bills
Hours Worked Per Week
Family: Understanding Academic Commitments
Methods of Paying for College: Scholarship
Employed
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Higher Score Positive - Higher Score Negative
1st term Academic Probation - 2nd term Attrition Factors
x
x
x
x
x
x
EASE Outcomes – Next Steps
Using Data to Develop Retention Plans
The Top Six Factors Associated with
First Semester GPA for Freshmen
•
•
•
•
•
•
Predicted Grade Point Index Score
How Often Classes Are Missed Each Week ***
# Unsatisfactory Grade Notices Received ***
Enrollment in Freshman Seminar
Grade Point Average Self-Estimate
# Hours Worked Per Week
*** Factors that can be monitored by faculty
These factors account for 49% of the variance in first semester GPA scores
Green – Increase GPA Red – Decrease GPA
Factors Listed in Decreasing Order of Importance
EASE Outcomes – Next Steps
Using Data to Develop Retention Plans
Undeclared vs. Declared Students and Academic Fit
Undeclared Freshmen
vs.
Declared Freshmen
EASE Outcomes – Next Steps
Using Data to Develop Retention Plans
Mid-term grades
• Continue efforts to get faculty to report mid-term grades
• Actively distribute mid-term grade information to advisors
Attending class
• Develop ‘watch list’ of students from institutional and Ease
data; request faculty to report issues to advisors during the
first 5 weeks of class
EASE Outcomes – Next Steps
Using Data to Develop Retention Plans
University College
•
Created to provide a home for undeclared students and those in transition; a home
for General Education; and to increase faculty involvement in retention efforts
Possible Initiatives:
•
Freshman Seminars: Increase the scale of the freshman seminar program (and
•
Course Clusters: Develop more meaningful and coherent first year curriculum to
develop parallel transfer seminars)
boost students’ feelings about their “academic fit” with UNC Charlotte
EASE Online
Using the Online Dataset Locally
StudentVoice Reporting
•
•
Simple Login and Navigation
Ability to Filter Multiple Levels
•
Multiple Result Displays
•
Multiple Report Export Formats
•
•
Ability to Save Custom Views
Three Click Download of Data from Report to Desktop
– Drilling down into your data
– Frequency analysis
– Graphs
– Crosstabs
– Word
– Excel
– PDF
References
•
•
•
•
•
•
•
Astin, A. W. (1996). Involvement in learning revisited: Lessons we have learned. Journal of College
Student Development, 37(2), 123-133.
Astin, A. W. (1984). Student involvement: A developmental theory for higher education. Journal of
College Student Personnel, 25(4), 297-307.
Barefoot, B. O., & Fidler, P. P. (1996). The 1994 national survey of freshman seminar programs:
Continuing innovations in the collegiate curriculum. the freshman year experience monograph series no.
20. U.S.; South Carolina: October 11, 2005, from ERIC database.
Barefoot, B. O., & Fidler, P. P. (1992). National survey of freshman seminar programming, 1991. helping
first year college students climb the academic ladder. the freshman year experience: Monograph series
number 10. U.S.; South Carolina: National Resource Center for the Freshman Year Experience. October
11, 2005, from ERIC database.
Schnell, C. A., & Doetkott, C. D. (2003). First year seminars produce long-term impact. Journal of College
Student Retention, 4(4), 377-391. Retrieved October 11, 2005, from ERIC database.
Soldner, L., Lee, Y., & Duby, P. (1999). Welcome to the block: Developing freshman learning communities
that work. Journal of College Student Retention, 1(2), 115-129. Retrieved October 11, 2005, from ERIC
database.
Tinto, V. (1993). Leaving college: Rethinking the causes and curses of student attrition (2nd ed.).
Chicago: The University of Chicago Press.
Questions and Discussion
For more information about the UNC Charlotte Implementation:
Dr. Theodore W. Elling
• [email protected]
Dr. Cynthia Wolf Johnson
• [email protected]
Dr. John Smail
• [email protected]
For more information about the EASE Survey & Survey Engine:
StudentVoice
• http://www.studentvoice.com/ease.htm