Transcript Student Success:

Promising pathways
Understanding and predicting preparation and
success and its potential as a model for change for
the California Community Colleges and beyond
Donald Berz, Executive Vice President, Academic Affairs
John J. Hetts, Director of Institutional Research
Long Beach City College
Critical aspects of this research were completed by our Office of Institutional
Effectiveness, including our Associate Dean of Institutional Effectiveness , Eva
Bagg and our two research analysts, Andrew Fuenmayor and Karen
Rothstein, with key additional support from Terrence Willett, Nick Wade, and
the research team at Cal-PASS
Overview
1) Background on the research
2) Placement in English at Long Beach City
College
3) Predicting placement in English at LBCC
4) Predicting performance in English at LBCC
5) Alignment of placement and performance
a) Building a better pathway
b) Effects of misalignment
6) Recommendations
2
1) Background: Data development
 Worked with Cal-PASS to generate 5 year cohort
(05-06 to 09-10)
 All courses taken at Long Beach Unified School District
(LBUSD) and Long Beach City College (LBCC),
California Standards Tests (CSTs), more…
 Key aspects of cohort
 Students coming directly from LBUSD
 Examining first English course taken at LBCC
 Key variables
 Junior & senior year English courses taken in high
school and spring semester grades in those courses
 11th grade CST-ELA scores at LBUSD
 English courses at LBCC and grades in those courses
3
2) Fall 2010 LBUSD Cohort
 What does LBUSD’s pattern of English
placement look like?
English One level Two
1
below
levels
below
Three
Levels
Below
No
Assessment
Total
Total
171
581
0
480
442
1674
Percent
10.2%
34.7%
N/A
28.7%
26.4%
100%
4
3) Predicting Preparation/Placement
 Primary method of assessing level of preparation has
been:
 student’s performance on standardized assessment
instrument (Descriptive Test of Language Skills (DTLS)
and, more recently, Accuplacer)
 combination of assessment & essay
 What predicts assessed preparation?
 Predicting how students will score on the assessment
 I.e., where they will be placed in our English course
sequence.
5
Predicting Placement via DTLS/Accuplacer
 Outcome: Whether or not students are placed into
Basic Skills
 Key predictors:
 11th Grade – Grade in 11th grade English
 12th Grade – Grade in 12th grade English
 CST - 11th Grade California Standards Test score in English
(CST-ELA)
 Course taken in 12th grade

Using LBUSD’s 12th grade English course Rhetoric & Composition
as baseline for comparison
 Comparing 5 major groups of courses to that baseline
 AP English, Multicultural Literature, Film Analysis, Remedial
English, Other Electives
6
Predicting Placement in English 1 (vs. any
remedial English): (relative size of influence - standardized beta;
LBUSD Course
(compared to Rhetoric and Comp)
i.e., how big, relatively, the impact of the variable is on placement)
15.3†
11th Grade CST
LBUSD Course
0.9
Other Elective
1.9
Multicult Lit
1.1
Film Analysis
-0.4
Remedial
4.2***
AP Lit
2.5*
Grade, 11th
2.2*
Grade, 12th
-5
0
5
10
* p<.05, *** p <.0001, † = p <1 x 10-52
15
20
7
11th Grade CST
Accuplacer
 What matters most for predicting
placement/Accuplacer is: 11th grade CST scores
 I.e., performance on and placement using
Accuplacer is almost entirely predicted by CST
score in 11th grade

AP increases placement in English 1
 Students 11th grade CST score could be used
instead of Accuplacer to place students

effectively disregarding last 3 semesters of high school
8
4) Predicting Performance
 People jump prematurely to conclusions about quality of
the instruction high schools are providing
 But that data just speaks to how students are placed into
Basic Skills
 Need to verify which factors best predict performance
 Same data but using successful completion (C or
better) of English courses
 Rhetoric & Composition again used as comparison for
courses at LBUSD
 English course taken at LBCC
 English 1 used as baseline for comparison against


One level below: Fundamentals of Writing
Three levels below: College English Skills I
9
Predicting success in English at LBCC
(relative size of influence - standardized beta; i.e., how big,
relatively, the impact of the variable is on placement
4.0***
LBUSD Course
(vs. Rhetoric and Comp)
11th grade CST
LBUSD Course
Other Elective
2.2*
Multicultural Lit
2.3*
Film Analysis
Remedial
1.1
-3.0**
4.0****
AP Lit
5.2******
Grade, 11th
11.1††
Grade, 12th
LBCC course
(vs. English 1)
LBCC Course
3.7***
1 level below transfer
6.9******
3 levels below transfer
-5.0
0.0
5.0
10.0
15.0
20.0
* p<.05, **, p< .01, *** p <.0001, **** p<.0001, ******, p<.000001 † † = p <1 x 10-27
25.0
10
Summary: Performance
 What most strongly predicts performance in our
English courses is: grade in 12th grade English
 Grade in 11th grade also matters
 CST/Accuplacer matters less than either grade
 Taking AP at least as important as CST/Accuplacer

Taking remedial English predicts poorer performance
11
5) Misaligned Assessment and
Performance
 11th grade CST strongly predicts Accuplacer-based placement
 weakly predicts performance in courses.
 Grade in 12th grade English strongly predicts performance
 Only weakly related to placement in those courses
 Implication: current method of placement is not well aligned with
what leads to success in our classrooms
 Placing students based on 12th grade course performance holds
potential to more strongly align placement with likelihood of
success
 Place students currently likely to fail in more appropriate courses
 Place students likely to succeed in courses more suitable to
preparedness
12
Building a better pathway
 Based on research done for on California Leadership
Alliance for Student Success (CLASS) initiative and other
best practices





Prescriptive course load
Emphasis on Basic Skills courses
Priority enrollment
Emphasis on full-time course work
Student Success Course

Modeled on Counseling 1: Orientation for College Success
 Results: Higher persistence, better course success rates,
higher achievement rates, smaller equity gaps
 Direct placement in Basic Skills
 A’s and B’s more likely to successfully complete in English 1
than current rate, C’s to 1 level below.
13
What this will mean at LBCC:
Promise Pathways, Semester 1
60%
52%
52%
50%
40%
Current 1st
semester
enrollment
29%
30%
21%
20%
Promise
Pathways
model
17% 19%
11%
10%
0%
0%
English 1
One level below
Three levels
below
Did not take
English
14
More broadly what this means…
Current Placement Model
Promise Pathways Model
70%
62%
59% 58%
60%
54%
53%
50%
49%
45%
40%
40%
30%
20%
34% 32%
28%
27%
26%
19% 20%
19%
13% 14%
20% 21%
8%
9%
8%
10%
18%
26%
19%
11%
8%
0%
English 1
One level
below
Three levels
below
Did not take
English
Asian
Black
P2: English 1
Hispanic
White
P2: One level
below
P2: Three
levels below
15
Why this matters
Projected completion of Transfer Level English in One Year
(cohort of ~1700)
1000
900
800
700
600
500
400
300
200
100
0
Current
Current + P2
Direct Placement
Direct Placement + P2
Total
Asian
Black
Hispanic
White
101
289
366
914
16
45
67
168
9
25
38
94
45
128
154
384
22
63
70
175
Current
Current + P2
Direct Placement
Direct Placement + P2
16
Why this will work
 Promise Pathways built on
Placement using
current method
emerging new best practices
 Data not previously available
now is, thanks to Cal-PASS
 Fits principles that most
educators know well based
on decades of experience
 Solves two problems at once:
 Students placed in courses
where they are likely to fail
(red in table)
 Students placed in courses
far below that they are
capable of (green in table)
LBUSD
Grade
Transfer
Level
1 level
below
3 levels
below
A
36%
23%
18%
B
30%
35%
33%
C
27%
29%
33%
D
6%
11%
15%
F
1%
1%
1%
17
Additional evidence
 Working model nearby at Grossmont-
Cuyamaca
 95% of their A & B students in one of their large
high schools were placed into Basic Skills in English

Our rate for LBUSD seniors is ~81%
 A & B students now placed into English 1
 86% of A & B students successfully complete
English 1 on their first try

Only 5% would have been allowed to even try.
18
Additional implications
 Helps explain why acceleration and alignment work
so well whenever and wherever they’re tried*
 Many students are prepared but are being
misclassified.
 Acceleration works well because it places students
closer to where they are fully prepared to succeed
 Why instructors complain about students being
unable to succeed in English 1
 Students placed there based on mechanism that
doesn’t relate to performance
 30% of them are not actually prepared despite
Accuplacer judging them so and are likely to fail
19
6) Recommendations:
 There is nothing you could do that would do more to increase all your
student progress and achievement rates than:
 1) Determining the actual predictors of performance in your Basic
Skills sequences.
 2) Aligning placement with those predictors*
 If you have one, call your IE/IR office right after this meeting
 Start building your local database with the help of Cal-PASS**
 Support the efforts of Cal-PASS, the Institute for Evidence-Based
Change, and the Data Quality Campaign to make K-16 data
available to all community colleges in California and nationally.
 Might we suggest that you consider John Lennon’s advice from one
of his later, albeit more controversial, songs….
 All we are saying… is give B’s a chance
20