Transcript Document

Transitioning Math
Transcript Research in Support of TMP
Dave Pavelchek
Washington State University
Social & Economic Sciences Research Center
Puget Sound Division
360-586-4583
http://www.sesrc.wsu.edu/k12
[email protected]
December 2007
1
Background

SESRC has been providing technical
assistance to the TMP project(s) on
evaluation issues

SESRC has also been conducting
transcript studies for districts, analyzing
the relationship between courses and
grades and post high school outcomes
2
The Shoreline Transition
Math Partnership

Shoreline School District and Shoreline
Community College

Interested in the patterns of students
currently making the post-secondary
transition

For insight into current dynamics

As a potential baseline to measure change
3
Study Sample
 Graduated
from Shoreline District in
2003, 2004 or 2005
 Enrolled
 SPED
in district at least 3 years
& ELL students excluded
4
Data

High School transcripts for 1,785 students

Community college math placement
scores, courses, and grades for the 713
enrolled at Shoreline Community College

Post-secondary followup ranged from 2 to
4 years for the three classes
5
Definitions of “College
Ready”

A student who enrolled in a college-level
course at a community college without
first taking a remedial course.
OR

A student whose placement test score
placed them at a college math level
6
Coding Courses from
Transcript to Ordinal Scale
Math Level
Math Course
0
1
2
3
Pre-high school math course or variable content
Integrated Math 1 - "Typical” freshman math class
Integrated Math 2 - "Typical” sophomore math class
Integrated Math 3 -"Typical” junior math class
Math Analysis at Shorecrest
(some review of Math 3 and some Pre-Calculus content)
Pre-calculus, AP Statistics, Math Analysis at Shorewood
Calculus, AP Calculus
3.5
4
5
7
What the Graduates Do
College Type of SSD Graduates
% students
100%
80%
60%
40%
20%
25%
33%
15%
16%
7%
4%
0%
SCC Only SCC and
4 year
2 year
(other than
SCC)
4 year
2 and 4
year
No College
8
College Type by Gender
College Type by Gender
% of students
100%
80%
60%
40%
26% 24%
20%
30%
12% 17%
37%
20%
8% 7%
4% 4%
11%
0%
SCC Only
SCC and 4 2 year (other
year
than SCC)
Male
4 year
2 and 4 year No College
Female
9
College Type by Ethnicity
College Type by Ethnicity
35%
40%
20%
10%
31%
29%
30% 25%
15%
14%
7%
16%
11% 11%
4%
2%
White or Asian (n=1,666)
Co
lle
ge
yr
4
yr
N
o
an
d
4
yr
2
SC
C
O
2
S
CC nl y
yr
(O
an
th
d
er
t h 4yr
an
SC
C)
yr
Co
lle
ge
o
N
2
yr
an
d
4
4
yr
0%
SC
C
2
SC Onl
yr
y
(O C a
nd
th
er
t h 4yr
an
SC
C)
% of students
50%
Non-white/non-Asian (n=119)
10
Math Grades and College Attendance
(Note - percentages are of each grade level)
% of students
College Type for Each High School Math GPA Level
100%
80%
60%
40%
20%
0%
40%
14%
51%
46%
19%
11%
4%
4%11%
15%
SCC Only SCC and 4
2 year
year
(other than
SCC)
A or B
C
13%
2%
4 year
4% 4% 1%
2 and 4
year
32%
21%
8%
No College
D or F
11
Math Preparation
Highest High School Math Level
(for all SSD Grads)
Math Anal.Calc.
52%
Math 0.5 Math 2
26%
Int. Math 3
22%
12
Math Preparation & College Type
(Note – the percents are of each Prep Level)
College Type for Each Highest High School Math Level
% of students
100%
80%
56%
60%
40%
20%
39% 38%
13%
36%
17% 19% 17%
8%
5%
2%
19%
14%
5%
1% 5% 5%
2%
0%
SCC Only SCC and 4
2 year
year
(other than
SCC)
Math 0.5 - 2
Int. Math 3
4 year
2 and 4
year
No College
Math Anal.- Calc.
13
Composition of Math Preparation for
Shoreline Grads at Different College Types
Highest Level High School Math for Each College Type
% of students
100%
80%
60%
18%
17%
27%
66%
23%
86%
67%
21%
33%
40%
60%
20%
0%
40%
62%
25%
8%
12%
2%
SCC Only SCC and 4
2 year
year
(other than
SCC)
4 year
Math 0.5 - 2
Int. Math 3
26%
7%
2 and 4
year
No College
Math Anal.- Calc.
14
The Traditional College ‘Gateway’
Grade in Integrated 3 Math
(for all SSD Grads)
D or F
12%
C
23%
A or B
65%
15
Int 3 Grade & College Type
(Note – the percents are of each Grade Level)
College Type for Each Integrated Math 3 Grade Category
% of students
100%
80%
56%
60%
40%
20%
48%
36%
11%
24%
20%
18% 17%
2%
0%
8% 7%
SCC Only SCC and 4
2 year
year
(other than
SCC)
A or B
C
11%
4 year
5%5% 3%
2 and 4
year
6%
14%
10%
No
College
D or F
16
Composition of Int 3 Grades for Shoreline
Grads at Different College Types
Grade in Integrated Math 3 for Each College Type
% of students
100%
80%
60%
26%
21%
3%
13%
8%
22%
84%
68%
35%
20%
30%
46%
39%
40%
20%
10%
22%
33%
70%
50%
0%
SCC
Only
A or B
SCC
and 4
year
2 year
(other
than
SCC)
C
4 year
2 and 4
No
year
College
D or F
17
The Senior Math Question
Math in Senior Year
(for all SSD Grads)
No
41%
Yes
59%
18
Senior Math Composition for
Shoreline Grads at Different College Types
% of students
Percent of Students in Each College Type
Who Took Math Senior Year
100%
80%
60%
40%
20%
0%
18%
33%
55%
45%
30%
66%
65%
82%
67%
70%
34%
35%
SCC Only SCC and
2 year
4 year (other than
SCC)
Math Senior Year
4 year
2 and 4
year
No
College
No Math Senior Year
19
What Level of Math?
Level of Math Taken at Shoreline Community College
No Math
Taken
43%
College Math
Only
22%
Remedial and
College Level
Math
14%
Remedial
Math Only
21%
20
Delayed Math in College?
Math in 1st Year at SCC
Math in 1st
Year
38%
No Math in
College
44%
No Math in
1st Year
18%
21
Grades in College Math
College Level Math GPA
D or F
23%
A or B
48%
C
29%
22
Grades in Remedial Math
Remedial Level Math GPA
A or B
32%
D or F
31%
C
37%
23
Remedial Grades & Further Progress
Remedial Math Grade by College Level Math Grade
% of Students
100%
15%
80%
40%
60%
40%
80%
85%
60%
20%
20%
0%
A or B
C
D or F
Remedial Math Grade
C or higher in College Level Math
Lower than C in College Level Math
24
Adaptive Placement Test
Components
Highest Level Placement Test Taken
College
Algebra
19%
Inter.
Algebra
48%
Numeric
20%
Elem.
Algebra
13%
25
Placement Results
Course Placement from Placement Test Score
Math090/
097
58%
College Math
9%
Math060/
ABE
13%
Math080
15%
Math070/
Bus175
5%
26
Is There Math After Placement?
% of students
Place m e nt Te st S core Math Le ve l by
Subse qu e nt Math C ou rse s Take n
100%
80%
60%
40%
20%
0%
77%
20%
38%
29%
13%
7%
16%
0%
Remedial Remedial College No Math
&
Math
Math
College
Only
Only
Math
Remedial Remedial College No Math
&
Math
Math
College
Only
Only
Math
Remedial Level Placement Math Score
College Level Placement Math Score
27
Empirical Evidence for Placement
Process
Next Math Enrollment by Placement Test Results
400
College Level
Remedial and College Level
300
Remedial
No Math
200
100
0
Didn't take test
<90
90-99
college level
Placement Test Results
28
Complexity and the
Colinearity Problem

What we would like to know is causation – what ‘gives’
students better math skills and retention

If not that, predictive value – what tells us how what a
student should tackle next, and how hard it will be

Unfortunately, the usual suspects travel in packs. The
same students who get higher math grades in Int 3,
tend to get higher math grades overall, and take more
and more advanced math, and are more likely take math
in senior year

This is called “colinearity” & it is why we resort to
multivariate regression
29
Research Questions on
Remedial Enrollments

How well are Placement Test Scores predicted
by high school math preparation?

How well is Enrollment in Remedial Math
predicted by high school math preparation?

How well is Enrollment in College Level Math
predicted by high school math preparation?

How well are Grades in College Level Math
predicted by high school math preparation?
30
College-Readiness Rates
36% directly into College-Level
Of Graduate s Enrolle d in Math at CTC, Pe rce ntage Enrolle d Dire ctly in
Colle ge Le ve l Math
100%
% of students
80%
54%
60%
46%
40%
20%
43%
37%
17%
9%
0%
Int. Math 1 or
lower (N=70)
Int. Math 2
(N=133)
Int. Math 3
(N=212)
Math. Anal.
(N=35)
Pre-Calc.
(N=201)
Calc. (N=60)
Highe s t Le ve l HIgh School Math Cours e
31
College-Readiness Rates and
Preparation
Placement Test Results of Graduates by Highest Math Completed
100%
10%
30%
% students
80%
60%
7%
42%
52%
79%
80%
40%
75%
68%
58%
45%
20%
21%
3%
2%
10%
18%
Int. Math 1 or
lower (N=470)
Int. Math 2
(N=77)
Int. Math 3
(N=107)
Math. Anal.
(N=10)
Pre-Calc.
(N=79)
0%
College Level
0%
Math Level 097-099
Calc. (N=19)
Math Level <090
32
Remedial Rates and Four Year Colleges
(Northshore Data)
Number and Percentage of Graduates who are College
Ready in Math by Type of College
1,500
Remedial M ath
1,200
900
48%
567
31%
600
300
College-Level M ath
407
521
183
0
NSD Total
(N=1,088)
Attended 2-year
(N=590)
69%
152
346
Attended 4-year
(N=498)
33
Untangling Complexity, What Factors
Predict Best
R-square estimates Highest Math
GPA in
Cox & Nagelker Course HS
Snell
ke
Placement Test Results
Placed below Math 97
Taking Remedial
Going on to College Level
Going on to College
Level** w/ remedial grade
College Level Course w/wo remedial
College-Level Math grades
Grade in
Math in
Remedial
Gender Highest
12th
Ethnicity Course
(female) Math
grade
Grade
Course
0.233
0.516
4
0.224
0.307
4
-1*
~ 0.365
~ 0.500
4
-3
0.109
0.147
4
0.25
0.337
0.265
0.363
0.225
4
-3
4
-3
4
3
3
2
4
-black *
3
* 0.10 Significance
**Forward and Backward Regression significantly disagree about some factors - show best fit (R-square)
34
Factors in the Remedial Process
Grade in
Math
R-square estimates
Highest
Gender Highest
GPA in
Course
(female) Math
HS
Cox & Snell Nagelkerke
Course
Placed at Remedial Level
Placed below Math 97
Taking Remedial
0.233
0.516
4
0.224
0.307
4
-1*
~ 0.365
~ 0.500
4
-3
4
-3
4
* 0.10 Significance
35
College Performance Factors
R-square
estimates
Cox & Nagel
Snell kerke
Going on to College Level
Going on to College
Level** w/ remedial grade
College Level Course w/wo remedial
College-Level Math grades
0.109 0.147
Grade in
Math
Math in
Remedial
Highest
Gender Highest
GPA in
12th
Ethnicity Course
Course
(female) Math
HS
grade
Grade
Course
4
-3
0.25 0.337
0.265 0.363
0.225
4
3
3
2
4
-black *
3
* 0.10 Significance
**Forward and Backward Regression significantly disagree about some factors - show best fit (R-square)
36
Findings from Similar Projects

Different districts are interested in different
questions.

Data availability changes

Methods improve (hopefully).
37
“Math Atrophy” is Hard to Find
Northshore SD data
Percent of Graduates who were College Ready
by Highest Math Successfully Completed
100%
78% 79%
75%
50%
32%
25%
11%
37%
16%
10%
6%
0%
Algebra or Lower
(N=53) (N=42)
Geometry
(N=85) (N=51)
Took 1 year off
Algebra II/Trig
(N=246) (N=173)
Did not take a year off
Pre-Calculus
(N=41) (N=264)
38
Examples of Other Uses of
Transcript Research

Investigating Remedial English

Patterns of enrollment & persistence by type of
college

Role of CTE in college selection, enrollment &
persistence
39
Transcript Research on Remedial
English (Not Shoreline SD Data)
Percentage of Graduates who are "College Ready" by Highest
Level of English Taken
93%
100%
75%
69%
67%
Junior Level
(N=164)
Senior Level
(N=314)
47%
50%
25%
25%
0%
Freshman Level
(N=4)
Sophomore
Level (N=19)
AP/Honors
(N=184)
40
Findings from Regression on
English College Readiness
(Pooled Data from 2 Districts)

Logistic regression with an R2 value of approximately 20%
(N=717)

Highest level English course taken strongest

English GPA was second-most important

ELL, & Ethnicity were also significant

Non-significant variables: gender and taking any English in the
senior year
41
However, remedial English is not
just a problem for recent immigrants
(Not Shoreline SD Data)
Enrollments in Remedial English
by Race and ELL Status
Minority & ELL
Eligible
21%
White & ELL
Eligible
2%
Minority & Not
ELL
33%
White & Not ELL
44%
42
Persistence from First to Second
College Year by Level of HS
Coursework (Not Shoreline Data)
100%
94%
92% 93%
86% 85% 87%
83%
78%
75%
75%
88%
86% 86%
86%
84%
94%
70%
50%
25%
0%
RS (66)
5 (124)
4 (546)
3 (93)
2 (7)
Highest Level Science
RS (32)
5 (149)
4 (411)
3 (222)
2 (27)
RS (21)
5 (138)
4 (311)
3 (279)
2 (73)
1 (20)
Highest Level Math
Highest Level English
43
Percent Transferring from a Local 2Year to 4-Year College by HS Grades
(Not Shoreline SD Data)
100%
75%
46%
50%
43%
31%
25%
39%
29%
24%
17%
14%
9%
0%
3 to 4.0 (66)
2 to 2.9 (80)
Cum GPA Science
0 to 1.9 (11)
3 to 4.0 (49)
2 to 2.9 (70)
0 to 1.9 (37)
3 to 4.0 (28)
2 to 2.9 (81)
0 to 1.9 (48)
Cum GPA Math
Cum GPA English
44
For more information
To view other WSU-SESRC research reports on
education topics please see our website at:
http://www.sesrc.wsu.edu/k12/
Email:
[email protected]
[email protected]
45