Transcript Creating an Accurate Picture of Student Success in ESL and

Measuring and Reporting Basic
Skills Success; You Need a Valid
Instrument - CB 21 Coding
1
CAROLE BOGUE-FEINOUR, RETIRED VICE CHANCELLOR OF
ACADEMIC AFFAIRS, CCCCO
MYRNA HUFFMAN, DIRECTOR MIS, CCCCO
JANET FULKS, ASCCC CURRICULUM CHAIR
JULY 2009
CURRICULUM INSTITUTE
Background
2
 BSI directed attention to the ARCC data
 Some measures did not make sense
 Investigating the coding revealed inaccuracies
 Particularly the
 Basic Skills Progress Outcome
 ESL Progress Outcome
 Progress for courses prior to transfer
 Needed to define the Supplemental Basic Skills
Report for accountability on the BSI funding and
noncredit enhanced funding
“CB 21-Course Prior to Transfer Course Level”
3
 CB21 is code for the course “level”, in terms of
number of levels below the transferrable level
How many levels below transfer level is this course?
 It is used primarily for basic skills
but can be used for non-basic skills, degree-applicable
courses
 It is used only for English, writing, ESL, reading,
or mathematics (TOP codes)
 We are reporting credit courses; noncredit in
progress
MIS Data Element CB21
4
 Is used for a lot of accountability reporting
 Which in turn is used to justify investments and expenditures
in basic skills
 ARCC Technical Advisory Group: defines metrics for
mandated reports
 Is necessary to show student progress through basic
skills curriculum

4…3…2…1…transferrable
Tracking Progress
5
CB21 Course Progress to Transfer
Transfer Level Courses
Courses One Level Prior to
Transfer
Courses Two Levels Prior to
Transfer
Courses Three Levels Prior to
Transfer
Courses Four Levels Prior to
Transfer for Mathematics, English
and Reading. ESL has Five & Six levels
prior to transfer.
Old Coding Instructions for Math
6
 Currently, CB21:
 A=
1 prereq. for transfer math (Intermediate
Algebra)
 B= 2 prereq./prep. for “A” (Algebra I/Elem.
Algebra)
 C= 3 prereq./prep. For “A/B” (Arithmetic)
 Y= 4+ >3 levels below transfer level (N/A)
Old Coding Instructions for English
7
 Currently, CB21:
 A=
1 prereq. for transfer Eng. Comp. (Subject A)
 B= 2 prereq./prep. for “A” (Not available)
 C= 3 prereq./prep. For “A/B” (Not available)
 Y= 4+ >3 levels below transfer level (Not
available)
Old Coding Instructions for
Reading and ESL
8
 Not addressed at all
Coding problems
9
 No colleges had a common beginning; college level
was different for every college – even colleges within
a district
 Some college coding had no progress – all courses
one level
 Some courses were in the wrong order of progression
 Some courses were incorrectly identified as transfer
Sample Coding – All are Credit Courses
10
ElementaryGeometry
IntermediateAlgebra
ElementaryAlgebra
Pre‐Algebra
DegApplicable
DegApplicable
DegApplicable
NOT DegApplicable
Not Basic Skills
Not Basic Skills
Not Basic Skills
Basic Skills
1
1
2
3
ElementaryAlgebra
Arithmetic
DegApplicable
Not Basic Skills
NOT DegApplicable Basic Skills
1
2
ElementaryAlgebra
ElementaryAlgebra‐1st Half
ElementaryAlgebra‐2nd Half
Intermediate Algebra
DegApplicable
DegApplicable
DegApplicable
DegApplicable
1
1
1
1
Not Basic Skills
Not Basic Skills
Not Basic Skills
Not Basic Skills
Sample Data – All are Credit Courses
11
IntermediateAlgebra DegApplicable
Not Basic Skills
Basic Mathematics NOT DegApplicable Basic Skills
Beginning Algebra
NOT DegApplicable Basic Skills
1
3
4+
Sample Data – All are Credit Courses
12
Course Id
ESLV33
ESLV01
ESLV40
ESLV34A
ESLV15
ESLV06
ESLV05
ESLV03
ESLV14
ESLV12
ESLV02
ESLV34B
ESLV07
ESLV04
ESLV08
ESLV16
Title
Intermed Reading Com prehension
Low-Beginning ESL
Vocab&Com m unication: Nonnative
Low-Begin Read Com prehension
Inter ESL Listening & Speaking
High-Advanced ESL
Low-Advanced ESL
Low-Intermediate ESL
Begin ESL Listening & Speaking
English Proficiency: Citizen
High-Beginning ESL
High-Begin Read Com prehension
Reading Skills LEP
High-Intermediate ESL
Writing Skills LEP
Adv ESL Listening & Speaking
Basic
Skills
Status
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
Transfer
Status
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
Level
B
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
Sample Data
13
Title
ELEMENTARY ALGEBRA
ELEMENTARY ALGEBRA II
ELEMENTARY ALGEBRA I
INTERMEDIATE ALGEBRA
MATH: OPERATIONS WITH WHOLE NUMBERS
MATH: OPERATIONS WITH DECIMALS
MATH: OPERATIONS WITH RATIONAL NUMBERS
OPERATIONS WITH FRACTIONS, DECIMALS AND %
PRE-ALGEBRA
BASIC MATHEMATICS SKILLS
Credit
Status
D
D
D
D
C
C
C
C
C
C
Basic
Skills Transfer
Status Status Level
N
C
Y
N
C
Y
N
C
Y
N
C
Y
B
C
Y
B
C
Y
B
C
Y
B
C
Y
B
C
Y
B
C
Y
CCC MIS Database
EOPS
DSPS
VTEA
PBS
Emp.
Assign.
Matric.
Student
Demographics
(SB)
Fin.
Aid
Emp.
Demo.
CalWORKs
Assess.
Enrollments
(SX)
Pgm.
Awds.
Calendar
Assignments
Sessions
Sections
Courses
Why do we code courses?
What are the data uses?
15
Research Questions
Accountability Reporting
Justification & Funding
• Legislative Analyst Office
• Department of Finance
• California Postsecondary
•
•
•
•
•
•
•
• Matriculation
• EOPS
• DSPS
Education Commission
California Student Aid
Commission
Public Policy Institute
UC/CSU
Legislature – Committees and
individual members
Community College Organizations
Newspapers
Labor Unions
Career Technical Education
Perkins Core Indicator Reports
Perkins Allocations
BOGW Administrative Funding
Federal Integrated Postsecondary
Education Data System
(IPEDS) Reporting
CCC Data Mart
Annual Staffing Report
Data Matches
• Transfer to UC/CSU/NSC match
• Dept. of Social Services
• EDD/UI Match/Wage Study
16
MIS Data Element CB21
17
 Chancellor’s Office MIS system collects all course
information each term
 Courses are coded for identification purposes
 TOP
code, credit status, transfer status, units,
basic skills status, SAM/voc code, etc.
 CB21=Course Prior to College Level (Current)
MIS Data Element CB21
18
 Last changed in 1994
 Defined
number of “codeable” levels at 5
(xfer + 4 below)
 Is used across
math/English/reading/writing/ESL
 Has little curricular definition of levels
ARCC Reporting
19
2009 Statewide ARCC Data
What CB21 is used for
20
 Basic Skills Improvement Rate (ARCC)
◦
◦
◦
Credit courses only
Completed (A,B,C,CR, P) any math/Eng basic skills
course at 2 or more levels below
Within 3 years, successfully completed a higher level
basic skills course of same discipline
 Anywhere in the system where SSN’s are reported
Current data range: 24%-62%,
◦ avg 49%.
◦
What CB21 is used for
21
 ESL Improvement Rate (ARCC)
 Credit
ESL courses only
 Completed (A,B,C,CR, P) any ESL course at 2 or
more levels below
 Within 3 years, successfully completed a higher
level ESL course
Anywhere in the system where SSN’s are
reported Current
81%, avg. 42%
data range: 0% to
The Strategy - Establishing a Rubric
22
 Is not standardization
 Does not drive curricular changes
 Is not common course numbering or articulation
 IS a mapping exercise designed to maximize our
ability to show student progress AND your good
work
Newly Designed CB 21 – Attached to Rubrics
23
 Each college retains their own curriculum
 Does not affect degree applicability
 Allows for uniformity throughout state
 Will enhance collection of accurate and comparable
data
 Can be used to collect assessment and placement
data
The Process
24
 140 Faculty
 Research for standards, outcomes and exit skills
 Divided by Discipline
 Developed rubrics based on need and curriculum
 Universally acceptable, not comprehensive
The results
25
 Rubrics created using 4 levels below transfer
◦ Reading
◦ Writing
◦ Math,
 ESL created 3 rubrics and needed 6 levels below
transfer
◦
◦
◦
◦
ESL Writing
ESL Reading
ESL Listening and Speaking
**ESL Integrated
Vetting the Rubrics
26
Demographics of Discipline Faculty Taking Part in the Vetting Process
Faculty experience teaching basic skills
Courses taught transfer and below
(select all that apply)
Full load
76
32.6%
transfer
153
65.7%
> 50% of load
71
30.5%
1 level below
204
87.6%
25-50% load
67
28.8%
2 levels below
207
88.8%
9
3.8%
3 levels below
151
64.8%
no basic skills
*10
4.3%
4 levels below
82
35.2%
totals
233
100%
5 levels below
30
12.9%
21
9.0%
<25%
coding issues -teach geometry but not classified as basic skills etc
6 levels below
assignment issues: adjuncts have varying loads, senate pres
7 levels below
10
4.3%
*some are chairs this semester
8 levels below
3
1.3%
Vetting the Rubrics
CB 21 Rubric-Vetting Data Results
27 as of February 18, 2009
Discipline
Number of
respondents
Approval Contingent Doesn’t
work
Approval
other
4 levels
6 levels
comments
adequate adequate on levels
Math
79
59
(74.7%)
18
(22.8%)
1
(1.3%)
1
(1.3%)
86.1%
none
Reading
29
12
(42.9%)
14
(50%)
1
(3.6%)
2
(7.1%)
50%
few
ESL
35
3
(14.3%)
ESL writing
21
9
(42.9%)
10
(47.6%)
1
(4.8%)
1
(4.8%)
ESL listening &
speaking
10
5
(50%)
5
(50%)
0
0
ESL reading
4
3(75%)
1(25%)
0
0
0
75%
38
(55.1%)
27
(39.1%)
3
(4.3%)
1
(1.4%)
30
(43.5%)
3
(4.5%)
English
69
other
21
total
233
16
(76.2%)
6.3% wanted
3 levels, 5.1%
five levels
25% want
only 3 levels
& 10.7%
more than 4
some
decisions
based on
local course
offerings
other
comments
need to deal
with geometry
1 or 2 levels
below
78.6% (22)
want grade
levels
included
including
word count
61.9% yes
38.1% no
100%
25% (1) 5
levels
much relates
to ESL &
student
population
40.6% (28)
3 levels
These people answered some but not all of the questions within specific disciplines
word counts
52.2% helpful;
47.8% not
helpful
Moving on to noncredit
28
 ABE/ASE
 Math
 English
 ESL integrated
Next Steps
29
 What will this mean?
 How will it change ARCC reporting?
 Will the data suddenly show a big
difference due to recoding?
 How will you proceed?
Next Steps & Timeline
30
 Coding instructions
 CIOs and Researchers meetings and listservs
 Possible web training
 Contact people for questions
 All recoding must be done by November 30
Questions
31