Transcript ppt

ACT — Cognitive and NonCognitive Factors that Determine
Student College and Career
Readiness
April Hansen, Postsecondary Director
George Schlott, Senior Consultant for Program Solutions
ACT, Inc. – Midwest Region-Lincolnshire IL
© 2012 by ACT, Inc. All rights reserved.
A Question for All
What is College and Career
Readiness?
Ensuring that all students, graduating from high
school, have acquired the skills in English,
mathematics, reading, and science that they need to
be ready for entry-level college courses without
remediation
ACT College & Career Readiness System
Measuring Student Progress Toward
College and Career Readiness
Improving
Course Rigor
Planning for
School
Improvement
College & Career Readiness Information System
EXPLORE
PLAN
8th & 9th
grade
curriculumbased
educational
and career
planning
program
10th grade
curriculumbased
educational
and career
planning
program
The ACT
11th
12th
&
grade
curriculumbased
measureme
nt for
learning
outcomes
WorkKeys
ENGAGE
QualityCore
CoreWork
Diagnostics
Job skill
assessments
and portable
credentials
necessary
for career
pathways
Middle and
High School
assessment
that measures
factors of
academic
success
Researchdriven
solutions for
strengthening
curriculum &
instruction
Online service to
diagnose and
improve content
and practice areas
Core Practice
Audit
ACT Core
Course
Standards
ACT College Readiness Benchmarks
ACT College Readiness Standards
Research
Online service to
diagnose and
improve content
and practice areas
ACT College Readiness Benchmarks
Through collaborative research with postsecondary institutions nationwide, ACT has established
the following college readiness benchmarks*:
Test
College Course
Test
English
English Composition
English
Math
College Algebra
Math
Reading
Science
EXPLORE 8
EXPLORE 9
PLAN 10
College Course
13
14
15
18
69
22
65
21
88
24
NA
18
18
19
22
College Algebra
Reading
Social Science
15
16
17
Biology
Science
20Biology
20
21
Social Science
COMPASS
ACT
English Composition
17
ACT
21
24
* Minimum score needed on an ACT subject-area test to indicate a 50% chance of obtaining a B or higher or a 75% chance of
obtaining a C or higher in the corresponding first-year credit-bearing college course.
Value of College Readiness
Students who are college/career ready when they
leave high school have a significantly higher
likelihood of:
– Enrolling in a postsecondary program,
– Enrolling in credit bearing courses without the need for
remediation,
– Succeeding in entry level postsecondary course work,
– Persisting in their postsecondary education,
– Completing a postsecondary degree or training program,
and
– Entering the job market with significantly higher lifetime
earning potential.
Regardless of ethnicity and SES
Leaky Educational Pipeline
Leaky Educational Pipeline
• Many students are not prepared to meet the hurdles they
face throughout the academic pipeline, and they don’t persist
and succeed.
~100%
Enter 9th
grade
HS
Graduate
Enter
College
College Graduate
(Bachelor’s)
96.9%
74.9%
52.5%
29.0%
NCES 2010
• What can we do to improve student success and
persistence?
© 2012 by ACT, Inc. All rights reserved.
6
Leaky Educational Pipeline
Pop Quiz
What’s the number of American high school
students who drop out of school, every day,
bored, frustrated, or so far behind that they’ve
given up?
6,000
Attrition Costs Money
Two-year schools -- 54 %
Four-year private colleges -- 73%
Four-year public -- 71% *
*Wes Habley, ACT, January 2009
By 2020:
Cost of the average private--$60-70,000 per year
Cost of the average public--$17,500-27,500 per year
In 2008:
Out of approx. 1.8 million first year college students, 450,000 did not return to
the college or university where they began their college career.* That’s 25%.
*Bryan Matthews, Retention Matters, Inside Higher Ed, Nov. 2, 2009
Do the Math!
If you have an incoming class of 1000, and lose even 10% of
the class (100 students), that’s
100 x $30,000 = $3 million
$3 million x 4 years = $12 million
Retaining students allows you to maintain revenue streams
while keeping tuition increases down.
It’s the low hanging fruit.
What if you had an assessment that would:
 evaluate students’ personal, behavioral, and academic
skills critical to high school and college achievement
 determine their levels of academic risk
 apply specific interventions to help them persist in their
studies and achieve academic success
 identify student strengths and areas for improvement in
student motivation, social engagement, and self-regulation
predict college retention for each incoming freshman
Research is the Foundation
1. The strongest predictors of college persistence and degree
completion are: prior academic achievement and course
selection (rigorous high school classes).
2. Prior academic achievement and cognitive ability surpass
all other factors in their influence on student performance.
3. Non-academic factors can influence academic
performance, retention and persistence, but cannot
substitute for it.
You Do!
Grades 6-9
Grades 10-12
College
Teacher
The Challenges
• Create a comprehensive assessment system that
predicts success in education and work
• Tailor assessments for critical transition points
and context
• Measure characteristics amenable to change
• Help educators connect students’ needs to
interventions
© 2012 by ACT, Inc. All rights reserved.
14
ACT’s Academic Achievement, Behavior &
Career Planning Solutions
6
Achievement
Behavior
Career
7
Grades
8
9
10
EXPLORE ® PLAN ®
11
12
ACT ®
ENGAGE and ENGAGE Teacher Edition
College
COMPASS ® WorkKeys ®
ENGAGE
ACT Interest Inventory & Map of College Majors
© 2012 by ACT, Inc. All rights reserved.
Work
15
Talent
Fit
ACT’s Academic Behavior Assessments
• Students’ personal characteristics and psychosocial
development influence their ability to stay in school
and be successful
• ACT research shows that psychosocial/behavioral
development can be structured into 3 broad domains:
– Motivation
– Social Engagement
– Self-Regulation
• ACT’s academic behavior assessments measure these
domains
© 2012 by ACT, Inc. All rights reserved.
16
ACT’s Academic Behavior Assessments
Student Grade Levels
6
Student reported
(surveys)
Teacher reported
(teacher ratings)
7
8
9
10
11
12
ENGAGE
Grades 6-9
ENGAGE
Grades 10-12
ENGAGE Teacher Ed.
Grades 6-9
ENGAGE Teacher
Ed. Grades 10-12
© 2012 by ACT, Inc. All rights reserved.
College
ENGAGE
College
17
ACT’s Academic Behavior Assessments
• Surveys (Student reported)
– ENGAGE Grades 6-9 (typically in 6th & 9th grade)
– ENGAGE Grades 10-12 (typically in 11th grade)
– ENGAGE College (typically early in 1st semester)
• Behavioral Measures (Teacher reported) COMING SOON!
– ENGAGE Teacher Edition for Grades 6-9 and 10-12
(completed by teachers 2-3x per year for each student)
© 2012 by ACT, Inc. All rights reserved.
18
ACT’s Academic Behavior Assessments
ENGAGE Grades 10-12
& College
Domain
ENGAGE Grades 6-9
Motivation
(Getting work done)
 Academic Discipline
 Commitment to School
 Optimism






Social Engagement
(Getting along)




 Social Activity
 Social Connection
Self-Regulation
(Keeping your cool)
 Managing Feelings
 Thinking Before Acting
 Orderly Conduct
Family Attitude toward Education
Family Involvement
Relationships w/ School Personnel
School Safety Climate
Academic Discipline
Commitment to College
Goal Striving
General Determination
Study Skills
Communication Skills
 Academic Self-Confidence
 Steadiness
© 2012 by ACT, Inc. All rights reserved.
19
ENGAGE: Grades 6-9
• Measures students’ perceptions of themselves,
their families’ commitment to education, schoolrelated factors, and important behavioral data
• Developed to predict academic success and
high school graduation
• Low stakes, self-report instrument, 4th-grade
reading level
• 10 scales
• Online administration
© 2012 by ACT, Inc. All rights reserved.
22
ENGAGE 10-12 & College
• Measures students’ psychosocial attributes, determines
level of risk, and helps identify interventions to
promote successful transition to postsecondary studies
• Developed to predict academic success and retention
in the first year of college
• Low stakes, self-report instrument, 6th-grade reading
level
• 10 scales
• Online administration
© 2012 by ACT, Inc. All rights reserved.
23
ENGAGE: Reports
• Academic Success and Graduation/Retention Indices
• Student profiles can be used to:
– Identify relative strengths and needs
– Inform student advising
– Match students’ needs to interventions
• Aggregate reports
• Roster reports
• All reports available online 24/7
© 2012 by ACT, Inc. All rights reserved.
24
ENGAGE: Sample Advisor Report
Success Indices
(only on Advisor
Report)
Profile
of scores
Interpretive
feedback, sorted
from strengths to
needs
© 2012 by ACT, Inc. All rights reserved.
25
ENGAGE: Sample Profile Report
© 2012 by ACT, Inc. All rights reserved.
26
ENGAGE: Sample Intepretative Reports
© 2012 by ACT, Inc. All rights reserved.
27
ENGAGE: School/Institution Aggregate Report
© 2012 by ACT, Inc. All rights reserved.
28
ENGAGE: Roster Report
© 2012 by ACT, Inc. All rights reserved.
29
Why ENGAGE?
• Grounded in research
– Developed based on meta-analysis of predictors of
academic performance and persistence (Robbins et al., 2004)
– Validated across many studies (e.g., Allen et al., 2010; Casillas et
al., in press; Robbins et al., 2006, 2008)
• Helps educators to consider the whole student
– Provides profile of relative strengths and needs
• Practical and flexible
–
–
–
–
Administer online at any time
Reports available online 24/7
Identify areas of challenge for students
Create a crosswalk of ENGAGE scales to existing services
© 2012 by ACT, Inc. All rights reserved.
30
Percentage of students accurately identified
as having a 9th-grade GPA <2.0
Selection Method
Hit Rate
Random
24%
EXPLORE Composite only
69%
ENGAGE Grades 6-9 only
80%
EXPLORE & ENGAGE
83%
© 2012 by ACT, Inc. All rights reserved.
31
Average Early High School GPA, by EXPLORE
and ENGAGE Grades 6-9 Scores
Average Early High School GPA, by EXPLORE and Average ENGAGE Grades 6-9 Scores
Low ENGAGE
Moderate ENGAGE
High ENGAGE
4.0
3.5
3.63
3.34
Average Early HS GPA
3.0
3.13
2.89
2.5
2.62
2.41
2.0
1.96
1.5
1.99
1.77
1.0
0.5
0.0
Bottom 25%
Middle 50%
Top 25%
EXPLORE Composite Score
© 2012 by ACT, Inc. All rights reserved.
32
Percentages of Students Accurately Identified
as Being at High Risk for Dropping Out
Selection Method
Hit Rate
Random
17%
EXPLORE Composite only
37%
ENGAGE Grades 6-9 Grad. Index only
42%
EXPLORE & ENGAGE Grad. Index
52%
© 2012 by ACT, Inc. All rights reserved.
33
Persistence Rates, by EXPLORE and ENGAGE
Grades 6-9 Graduation Index Levels
Low Graduation Index
Moderate Graduation Index
High Graduation Index
100
96
92
96
89
80
72
Persistence Rate
69
60
62
48
40
20
0
Bottom 25%
Middle 50%
Top 25%
EXPLORE Composite Score
© 2012 by ACT, Inc. All rights reserved.
34
98
Use of ENGAGE College to Identify
Students Academically at Risk
Accuracy of Identification
Selection Method
Drop Out
Academic Difficulty
Random
10%
20%
ENGAGE
Success Indices
24%
46%
Note. Students scoring in the bottom 5% of each of these populations were flagged.
© 2012 by ACT, Inc. All rights reserved.
35
ENGAGE: Teacher
• Designed to assess and monitor students’ development
• Students are rated by teachers who know them well
• Requires each student to be rated by at least one
teacher
• Used to 2-3 times per year
• Scales are “anchored” with behavioral statements to
enhance accuracy and reliability
• Same dimensions for Grades 6-9 and 10-12, but
rating scales are made up of different items.
© 2012 by ACT, Inc. All rights reserved.
36
ENGAGE: Teacher
Domain
Scales
Motivation
(Getting work done)




Social Engagement
(Getting along)
 Communication
 Working with Others
Self-Regulation
(Keeping your cool)
 Managing Feelings
 Orderly Conduct
Initiative
Planning & Organizing
Sustained Effort
Performance
© 2012 by ACT, Inc. All rights reserved.
37
Combining Assessment
and Interventions
to Promote Student Success
ENGAGE
ENGAGE
Teacher
Edition
Support
Resources
&
Interventions
Reduced
Risk
Continued
Evaluation
& Feedback
© 2012 by ACT, Inc. All rights reserved.
38
Example Crosswalk between
Interventions and Areas of Need
Interventions
Areas of
Need
Goal Setting
& Time
Management
Motivation
x
Mentoring
Cross-Age
Tutoring
Community
Service
x
x
x
Social
Engagement
x
x
x
SelfRegulation
x
© 2012 by ACT, Inc. All rights reserved.
Group Work &
Communication
Skills
SelfConfidence
Workshop
How to
Follow
Rules
x
x
x
39
Behavioral Education Solutions
• Grade 6-12 curriculum
COMING SOON!
– Units designed to help students build behavioral skills
– Based on carefully developed Behavioral Learning
Objectives
– Fully developed and structured lesson plans and activities
• ENGAGE Tool Shop (Postsecondary)
– Several tools available for each ENGAGE scales to address
students’ needs
– Can be completed by students on their own or as part of
FYE courses or other group settings
© 2012 by ACT, Inc. All rights reserved.
40
40
ENGAGE Tool Shop (College)
© 2012 by ACT, Inc. All rights reserved.
41
Intervention Research
• University of North Texas
– Large university (36,000 students)
– Administers ENGAGE to all incoming 1st-year students
– Identify at-risk students and follow up with immediate
individualized intervention
• Northern Arizona University
– 4-year public institution serving 13,000 students
– Use ENGAGE to guide intrusive interventions with
those students who need the most help
© 2012 by ACT, Inc. All rights reserved.
42
UNT Face-to-Face Meeting
• Advisor or student affairs staff meet with
student during first 5-6 weeks of semester
• Present ENGAGE results by focusing first on
strengths, then needs
• Focus resources to aid student in:
– building on one strength and addressing one area
for improvement
– using crosswalk of ENGAGE to campus resources
© 2012 by ACT, Inc. All rights reserved.
43
Sample Crosswalk of Resources
SRI Scale
Definition
Resources
Academic Discipline
Effort put into school work and
the degree to which students
see themselves as hardworking
and conscientious.
• Learning Center
• Office of Exploring Majors
• UNT Math Tutor Lab
• UNT Student Writing Lab
Social Connection
Feelings of connection and
involvement with school and
community.
•
•
•
•
Academic SelfConfidence
Belief in ability to perform well
in school.
• Learning Center
• Counseling and Testing
• Office of Exploring Majors
© 2012 by ACT, Inc. All rights reserved.
Off-Campus Student Services
Transfer Center
Residence Life
Recreation Center
44
www.act.org/engage/index.html
© 2012 by ACT, Inc. All rights reserved.
45
www.act.org/engage/index.html
For each version of ENGAGE, website contains:
–
–
–
–
–
Features and Benefits
Testing Timeline
Program Materials
Research
Results and Reports
© 2012 by ACT, Inc. All rights reserved.
46