Transcript Retention Modeling Project Overview (PPT)

Retention Modeling Project Overview 2016.04.08
Ruth Vater, Ellenor Anderbyrne, and Hilary Walker’16
July: incoming students who may be at risk
January: continuing students who may be at risk
Pitfalls
Pitfall I
RELYING TOO HEAVILY ON THE
NATIONAL LITERATURE
Testing Retention Parameters from the
National Literature
•
•
•
•
•
•
•
•
•
•
•
Race is white, non-Hispanic
Whether the student transferred into Beloit from another college or university
Whether the student is a first generation college student
Whether either of the student’s parents did not attend college
Whether either of the student’s parents did not complete middle school
Distance between student’s home and Beloit
Whether the student is low-income
Whether the student has a documented disability
Scores from the 12-question HOPE instrument
Scores from the 8-question grit instrument
A number of measures of high school honors and activities, from the Common
Application
Useful Parameters
•
•
•
•
Total tuition and fees
Pathways score
Students’ sense of belonging
Difference between Beloit GPA and high
school GPA
• First generation college student status
• Gender/athlete
Useful Parameters…Not So Useful
•
•
•
•
Total tuition and fees
Pathways score
Students’ sense of belonging
Difference between Beloit GPA and high
school GPA
• First generation college student status
• Gender/athlete
Pitfall I
HOW WE RESPONDED TO PITFALL I
As An Intermediate Step
• Flagged entering students with low GPAs,
adjusting to give extra attention to some
demographics
• Experimental design
– Calculated a predicted retention rate but-for
intervention
– An alternative would have been to divide these
students into two groups and only give intervention to
one group
Improving Persistence 2015-16
• Inclusive Success Coordinator
• Persistence Study Group
• Persistence Implementation Group
Making New Models
• Listen to parameter ideas from colleagues at
similar institutions: thank you, Grinnell College
– Alert slips
• Received before fall mid-term
• More than 3
• Received yes/no flag
– CIRP Constructs
• Social self-concept
Listen to Experienced Faculty and Staff
Freshman
Seminar
Grade
Retained to
Sophomore
Year
No Credit/Withdrawn
52%
78%
83%
86%
93%
93%
91%
93%
C
C+
BB
B+
AA
Freshman Seminar Grade Correlated
with H.S. GPA
Pitfall II
USING AVAILABLE DATA RATHER THAN
HAVING AN INTELLECTUAL FRAMEWORK
Schreiner’s Reasons for Thriving
•
•
•
•
•
Positive Emotion
Engagement
Meaning
Accomplishment
Positive Relationships
Schreiner, Laurie A., “Thriving in College,” New Directions for Student Services, no 143, Fall 2013 p. 42.
Tinto’s Reasons for Departure
•
•
•
•
•
•
•
•
Intention
Commitment
Adjustment
Difficulty
Congruence
Isolation
Obligations
Finances
Tinto, Vincent, Leaving College: Rethinking the Causes and Cures of Student Attrition, University of Chicago Press, 1993, p. 81.
Tinto’s Reasons for Departure
•
•
•
•
•
•
•
•
Intention
Commitment
Adjustment
Difficulty
Congruence
Isolation
Obligations
Finances
Dispositions upon entry
Experiences after entry
External forces
Health issues (e.g. schizophrenia) developed in early adulthood
Tinto, Vincent, Leaving College: Rethinking the Causes and Cures of Student Attrition, University of Chicago Press, 1993, p. 81.
Intention: The goals (career, academic, completion)
of individual action
•
•
CIRP College Reputation – gauges how students measure future career potential as
reason for coming to institution
CIRP Social Agency – measures the extent to which students value social and
political involvement as a personal goal
Commitment: Willingness to work toward the
attainment of a goal; motivation, drive, or effort
•
•
•
•
•
•
•
•
•
CIRP Question 15 - “Is this college your first, second, third, or less than third
choice?”
Large Family
Grit
Hope Score
Hope Q1 – “I can think of many ways to get out of a jam.”
Hope Q8 – “Even when others get discouraged, I know I can find a way to solve the
problem.”
Hope Q10 – “I’ve been pretty successful in life.”
Hope Agency
Hope Pathways
Note: Significance of Hope Q1, Hope Q10, and Pathways depended upon when students took survey
Adjustment: Ability to adjust both socially and
intellectually to the new college environment
•
•
•
•
•
•
CIRP Habits of Mind
CIRP Pluralistic Orientation
CIRP Social Self-Concept
First-generation status
From China
Distance between student’s home and Beloit
Difficulty: Student cannot meet
minimum academic standards
•
•
•
•
•
•
•
•
•
•
•
•
•
Received Alert slip
Alert slip before fall Midterm
Alert slip before spring Midterm
Alert slip before either Midterm
More than three Alert slips
First Semester GPA
Freshman Seminar Grade
Received Freshman Seminar Credit
HS GPA
Difference between Beloit and High School GPA
SAT Score
ACT Composite/Concordance
CIRP Academic Self-Concept
Congruence: Level of fit between the needs,
interests, etc. of the student and the institution that
she/he is attending
•
•
•
Documented Disability
Athlete
Athlete/gender interaction
Isolation: Absence of sufficient contact between
individual and members of college community
•
•
•
•
•
Domestic Minority
NSF Domestic Minority
YFCY Sense of Belonging
CIRP Civic Engagement
CIRP Likelihood of College Involvement
Obligations: External demands that limit the ability
of the student to meet the college’s demands
•
[Exit interview]
Finances
•
•
•
•
•
•
•
•
•
•
•
•
Family expected family contribution
Parent expected contribution
Student expected contribution
Parent adjusted gross income
Student adjusted gross income
Parent net worth
Student net worth
Primary expected family contribution
Original need
Filed FAFSA
Financial Aid Gap
Low income status
Pitfall III
ASSUMING CAMPUS CONSTITUENTS
UNDERSTAND STATISTICAL TERMS
Jargon Examples
• Statistically significant
– Doesn’t tell us anything about magnitude
• Correlation
– Still does not equal causation
– Preventing stereotype threat
• As predominately white institutions or historically
predominately white institutions we need to step up on
inclusivity
Steele, Claude, Whistling Vivaldi: and other clues to how stereotypes affect us, New York: W.W. Norton & Company, 2010.
Pitfall IIII
USING QUANTITATIVE WITHOUT
QUALITATIVE
Need for Mixed Methods
• We’ve found a pattern, but we don’t
understand the cause
– Without understanding cause, difficult to select
the right intervention
FYI grade correlated with H.S. GPA
NEXT STEPS
Address Insufficient Data
• Frequency: We collect three-yearly, but would
need yearly to intervene
• Timing: Most of our surveys are end-of-year,
but we may want to know sooner
• Coverage: Most surveys are freshman or
seniors, but we are concerned about midclassman, too
• Topics: We don’t have any data about much of
the content we want
Look for Patterns by Departure Type
• Non-voluntary academic dismissal—22% of cases
– Inability/unwillingness to meet the minimum
academic requirements
– Possible to have the intellectual capacity but unable to
apply it to the daily tasks of college work?
• Voluntary withdrawal—78% of cases
– “For most departures, leaving has little to do with
inability to meet formal academic requirements.”
• External Forces
Tinto, Vincent, Leaving College: Rethinking the Causes and Cures of Student Attrition, University of Chicago Press, 1993, pp. 81-82.
OPPORTUNITIES FOR
COLLABORATION
Parameter Sharing
• Exchange parameters we’ve tested with similar institutions
• Discuss what isn’t working
– Grit may not do as much to boost creative achievement as it
does for academics. The areas studied by Duckworth “are welldefined areas and the rules for achievement are well-defined in
those areas”; not helpful to predict a student’s success in
individual and performing art, writing, scientific ingenuity, or
even creativeness in everyday problem-solving (Sparks)
– Creative achievement and grit, intellectual creativity and grit,
everyday creativity and grit: no effects (Grohman) (Sparks)
– Like stubbornness, too much grit can keep us sticking to goals
that should be abandoned (Lucas) (NPR)
Sparks, Sarah, D., “‘Grit’ May Not Spur Creative Success, Scholars Say, Education Week, August 19, 2014.
NRP Staff, NPR Digital Media, “The Power and Problem of Grit,” April 5, 2016.
Collaborate on Data Collection
• Optimistic about micro surveys – frequent,
short surveys
– Joint question design & methodological testing
– Joint surveying
Parameter sharing, collaborative question design,
simultaneous surveying and data sharing