Transcript Noncredit Progress Indicators Pilot

Innovation and Value in Basic
Skills and ESL: Got Noncredit?
Host: Wheeler North, Basic Skills/Noncredit Committee Chair
Karen Dennis, Santa Ana College
Janet Fulks, Noncredit Ad Hoc Task Force
Candace Lynch-Thompson, School of Continuing Education
0 Information from a two-year pilot measuring student
success in ESL and basic skills with a variety of
instructional methods (e.g. self-paced, open
entry/open exit, managed enrollment).
0 Examined faculty norming (with primarily adjuncts),
assessment using multiple measures, and hours of
study.
0 Do some basic skills and ESL courses belong as
noncredit rather than credit?
0 Basic skills units are limited to 30 and this credit is
not transferable or degree applicable.
0 Over 70% of our students needing basic skills
0 Would students benefit by teaching initial basic skills
and ESL courses in noncredit?
Assessment Levels of Incoming
Students
Overall (N=23)
Overall
(N=23)
Average
based
Overall
(N=23)
Average based
onCounts
Countsbased
onAverage
on Counts
Transfer
level
Transfer
level
Transfer
level
One
level below
below
One
level
One level below
transfer
transfer
transfer
Two
levels
Two
levels
Two
levels
below
transfer
below
transfer
belowor
transfer
Three
more
Three
or
more
Three
or
more
levels below
levelsbelow
below
levels
Total
Total
Total
English (N = 23)
Math (N = 23)
English (N = 23)
English (N = 23)
Math (N = 23)
Math (N = 23)
Reading (N= 11)
Reading (N= 11)
Reading (N= 11)
ESL (N = 15)
ESL (N = 15)
ESL (N = 15)
%
Placements
% Placements %% Placements
% Placements
%
Placements % Placements
Placements %
Placements
%
Placements
%
Placements
%
Placements
%
Placements
26%
20,083
16%
12,539
33%
12,782
71 71
26%
20,083
16%
12,539
33%
12,782 1%1%
26%
20,083
16%
13%
13%
13%
100%
100%
100%
9,814
39%
30,080
7%
2,895
70%
3,964
9,814
39%
30,080
7%
2,895
70%
3,964
9,814
39%
7%
2,895
76,138
100% 30,080
77,231
100%
38,733 70%
100% 3,964
5,629
76,138 100%
100% 77,231
77,231 100%
100% 38,733
38,733 100%100% 5,6295,629
76,138
36%
13,921
9%9%
71
22,142
25%
19,626
24%
9,136 20%
20%
1,102
22,142
22,142 25%
25% 19,626
19,626 24%24% 9,136
9,136
20% 1,1021,102
14,986
13,921
13,921
1%
29%
29%
29%
19%
36%
36%
12,782
24,100
24,100
24,100
14,986
14,986
33%
32%
32%
19%
19%
12,539
9%
493493
493
Total Enrollment (2006-2007 headcount)
Basic Skills & ESL
Credit
326,478
Noncredit
*
393,004
* supervised tutoring
Course Alignment Prior to TRANSFER
Discipline
Credit
Noncredit
Likely bridge
noncredit to credit
Math
Four levels CB 21
A, B, C, D
Six levels CB 21
A, B, C, D, E, F
Levels C & D
English
Four levels CB 21
A, B, C, D
Seven levels CB 21 A, B,
C, D, E, F, G
Level B or C
Reading
Four levels CB 21
A, B, C, D
Five levels CB 21
A, B, C, D, E
Level A or B
ESL
6 levels ESL Reading CB 21 8 levels ESL Integrated
A, B, C, D, E, F
CB 21
6 levels ESL Writing CB 21 A,B,C,D,E, F, G, H
A, B, C, D, E, F
Includes vocational and
6 levels ESL Speaking &
Cultural skills
Listening CB 21
Student Success Conference 2009
A, B, C, D, E, F
Most noncredit end
2 levels prior to
English 1 A at Level
B
6
What is Noncredit?
 Fact: Noncredit serves over 350,000 FTES in our system
and represents about half of the basic skills work in the
CCCs.
 Fact: Students are significantly more diverse, represent
students with greatest need and least likely to succeed
Link to data
 Fact: Noncredit offers flexible schedules, increased contact
hours, self paced learning
ALL AT NO COST to the STUDENT
Traditional Educational Expectations
and Accountability
Percent Successful
Percent Successful
A
B
C
D
Grades for a specified semester
Time is the independent variable
Success is dependent on the time
F
Adult Ed & Noncredit Education
Time
Percent Successful
Success is the independent variable and time is
dependent on success
Why Accountability?
Enhanced Noncredit Funding
0 Noncredit gets funded less per FTES
than credit ($4,565)
0 SB 361 increased noncredit funding
from $2745per FTES to $3232 per FTES
0 Applies to students enrolled in a
sequence of courses leading to career
development or college preparation
(CDCP certificates)
The Puzzle of Noncredit
Accountability
Current statewide data
 Only 2.3 – 5.1% of noncredit students transition to credit
 All noncredit courses without grades report zero success.
 Wage data is incomplete because of SSN#s
 CDCP data is incomplete or programs are undefined.
This is not the noncredit story?
This is not really the noncredit story.
And we have data to prove it!
2007-08 SCE Award Data from MIS
Award hours
SCE Actual Data
Program Type
Counts
9
Unknown (Top code 99)
192 – to fewer
than 288
Business and Management
288 – to fewer
than 480
Family and Consumer Sciences
288 – to fewer
than 480
Health
960 or more
Interdisciplinary Studies
Total
Data Link
Program Type
Counts
Administrative Assistant
62
Management
10
Early Childhood Education
21
20
Pharmacy Technician
50
214
High School Diploma
322
27
9
279
Total
465
Credit students start in Noncredit
Statewide –
1 of every 4
AA/AS degree-earners
started in Noncredit
Source: Leslie Smith, 2006. Noncredit: The
Education Gateway. City College of San Francisco
CDCP Wage Reporting
Data Collection Strategies
0 SCE’s “You Count!” Campaign
0 Collecting more SSN’s
0 DREAM team efforts
0 Program improvement
0 Tracking student progress
0 Benefits of Banner
0 Assessment scores
0 Enrollment trends
0 Certificates earned
Accountability
Background & Rationale
How progress has been measured
The interplay of proof of progress and funding
The role of CB21 coding
Enhanced funding for noncredit Career Development and College
Preparation (CDCP) courses
0 The need for progress indicators in noncredit
0 Review and Evaluate current mandated noncredit metrics –
further meetings and discussion to take place in November
0
0
0
0
0 Pilot project to allow the use of noncredit progress indicators
– Fall pilot to begin this semester and another group
beginning in Spring
Academic Senate Resolution
13.04 S10
Improve Noncredit Accountability Reporting through
Progress Indicators
• Task force of primarily noncredit faculty and
administrators representing all noncredit areas and
other representative.
Pilot Project Goals
0 Develop a set of working progress indicators to
use in the pilot project
0 Establish clear communication between
institution MIS reporting and noncredit
programs
0 Collect a pilot set of accountability data based
on these indicators
0 Evaluate the ability for noncredit programs to
work with these indicators
0 Evaluate the effectiveness of these indicators
for use as accountability requirements
Grades and Title 5
 55021: not required for noncredit
 55023: currently accepted symbols
 Currently some noncredit classes are graded in order to
qualify for federal funding. But the CCCCO only accepts UG
for noncredit. Every grade submitted by a faculty member
is changed to UG and all student success data reads as
zero.
Pilot Progress Indicators
0 Pass (P)
0 Satisfactory Progress (SP)
0 No Pass (NP)
0A–B–C–D-F
Timeline & Guidelines
0 First Cohort - Fall 2010
0 Collection of First
Cohort data - February
2011, July 2011,
February 2012
0 Participant Evaluation of
Usability of the
Indicators - Fall 2011,
Spring 2012
0 Data can still be
submitted
0 Pilot Project Work is risk
free
0 Data will not be shared
with anyone else.
0 Data will be available to
individual institutions
about their own college
0 Information will be
analyzed as aggregate
anonymous data
outside of the
institutions
Measuring Learning Gains
Multiple Measures Scoring by Rob Jenkins for Santa Ana
College School of Continuing Education
0 Factors to consider include :
0 tests scores
0 school experience
0 speaking and writing ability.
Example of Mt Sac. Rubric for Instructors
Progress
Indicator
Level Courses
Skills Courses
P
completed all necessary
requirements (written
& oral evaluation and SLO or
Measurable Assessment),
good attendance and
participation, scored 70% or
higher on the final exam.
priority attendance, good
participation, ability
at level, passed SLO or
Measurable Assessment
NP
poor attendance and
participation, didn’t
complete necessary
requirements, scored below
70% on the final exam.
poor attendance and participation,
ability
below level, didn’t pass SLO or
Measurable Assessment.
SP
Added class too late to make
decision of P / NP
Added class to late to make
decision of P / NP
Noncredit Grading (Progress
Indicators) Pilot Final Survey
Please provide your demographic information.
Answer Options
Response Response
Percent
Count
99.5%
186
Name: (Mostly Faculty)
College: Over 14 Institutions
(some responding for groups of
100.0%
faculty e.g. Mira Costa 1 response
25 faculty)
answered question
187
187
Noncredit Grading (Progress Indicators) Pilot Final
Survey
Q2. What was your assessment and grading practice in your
noncredit classes prior to your participation in this pilot?
Answer Options
Response Response
Percent Count
I assessed and
submitted grades
39.1%
(progress indicators)
before this pilot
I assessed in my class
but did not need to
43.2%
submit grades
(progress indicators)
I did not assess or
8.9%
grade my students
Other (Please specify
8.9%
below)
Other:
answered question
skipped question
66
What was your assessment and grading
practice in your noncredit classes prior
to your participation in this pilot?
8.9%
73
8.9%
39.1%
15
15
19
169
26
I assessed and
submitted grades
(progress
indicators) before
this pilot
I assessed in my
class but did not
need to submit
grades (progress
indicators)
I did not assess or
grade my students
43.2%
Other (Please
specify below)
Noncredit Grading (Progress Indicators) Pilot Final Survey
Q3. How practical was the assignment of P/SP/NP in measuring
learning progress for your student population?
Answer
Options
Very practical
Somewhat
practical
Response Response
Percent
Count
45.5%
76
33.5%
56
10.2%
Neutral
Somewhat
7.2%
impractical
Very
3.6%
impractical
Please explain your
choice:
answered question
How practical was the assignment of
P/SP/NP in measuring learning
progress for your student
population?
3.6%
7.2%
10.2%
17
45.5%
12
33.5%
6
72
167
Very
practical
Somewhat
practical
Neutral
Somewhat
impractical
Very
impractical
Noncredit Grading (Progress Indicators) Pilot Final Survey
Q8. Would you support an ASCCC resolution to implement
progress indicator reporting for noncredit areas with the caveat
that some areas (e.g. older adults, parenting) may need more
time to adequately explore and implement what indicators work
best?
Answer
Options
Response Response
Percent
Count
Yes
72.5%
No
I would
support it if
(please add
comment)
8.8%
Q8. Would you support an ASCCC resolution to
implement progress indicator reporting for noncredit
areas with the caveat that some areas (e.g. older
adults, parenting) may need more time to
adequately explore and implement what indicators
Yes
work best?
116
18.8%
No
14
8.8%
18.8%
answered question
30
160
72.5%
I would
support it if
(please
add
comment)
Statewide ARCC Data 2008-2011
ARCC DATA
Indicator
2008
2009
Statewide Rates
2010
2011
Student Progress &
Achievement
Completed 30 or more
units
Fall to Fall persistence
51.2% 51.8%
52.3%
Change since
inception
53.6%
2.4%
70.4% 71.2%
72.4%
72.8%
2.4%
68.3% 69.2%
68.7%
67.6%
0.7%
Vocational Ed Course
Completion
Basic Skills Course
Completion
Basic Skills Course
Improvement
ESL Course
Improvement
78.2% 77.7%
77.6%
77.0%
1.2%
60.5% 60.5%
61.5%
61.4%
0.9%
50.0% 51.2%
53.8%
58.6%
8.6%
44.7% 50.1%
50.2%
54.6%
9.9%
What have we learned about noncredit?
How much time is required for Success?
Median
Median
Median
Median
NG
Course Students %P P hours %SP SP hours %NP NP Hours %NG hours
ESL 410
2111 16.4
75 42.9
40 16.4
12 24.3
21
ESL 480
735 35.6 112.25 21.8
47.5 20.8
12 21.8
35
Cost over time for SP
Open Entry Open Exit Costs less
over the years.
Cost over time for SP
Open Entry Open Exit Costs
less over the years.
Typical class: 30 students, 17
weeks, 10 hrs per week
Total
Cost
3
terms
1
Term
ESL Beginning
-410
ESL 1
ESL Beginning
-410
ESL 1
Total
Hours
%P
SP
NP
MEDIAN
MEDIAN NP
P_Cost P_Hrs %SP
HRS SP_Hrs %NP
HRS
_Hrs
30
$31,397 5100
7.44 $14,192 2305.2 9.36
$11,743 1907.4 12.9
$ 5,149 836.4
30
$31,397 5100
5.58 $11,177 1815.6 12.1
$13,469 2187.9 12.3
$ 6,719 1091
Santa Ana College School of Continuing Education ESL Program
2011SPN (19 weeks)
Total
Median
Studen
FTES
Hours
ts
CB21
Title
G
Beginning
ESL 1
1,894
F
Beginning
ESL 2
1,739
E
Beginning
ESL 3
1,303
D
Intermediate
ESL 1
1,040
C
Intermediate
ESL 2
941
B
Intermediate
ESL 3
652
%P
%P Hrs
P Median
%SP
Hrs
%SP SP Median
Hrs
Hrs
%NP
NP
%NP
Median
Hrs
Hrs
45.0 230.8 17.4 32.7
129.0 43.6 45.4
48.0
39.0 21.9 20.0
59.5 247.7 24.3 40.8
120.0 32.7 34.4
57.5
43.0 24.8 24.0
65.0 178.9 25.9 48.2
138.0 26.9 34.9
80.5
47.2 17.0 21.0
60.0 135.9 26.8 48.3
131.0 30.7 31.4
57.0
42.5 20.3 22.8
72.8 160.4 24.5 36.9
136.5 24.2 23.8
77.0
51.2 39.3 44.0
73.5 90.3 39.4 68.9
135.0 22.9 22.4
55.0
37.7
8.6
14.0
Santa Ana College School of Continuing Education ESL Program
2011SPN
Beginning ESL 3
P
SP
NP
25.9
26.9
47.2
48.2
34.9
17.0
Students (%)
Hours (%)
100%
90%
25.9
80%
48.2
70%
60%
26.9
P
50%
SP
40%
34.9
30%
20%
NP
47.2
10%
17.0
0%
Students (%)
Hours (%)
Due to the at-risk situation of non-credit students, almost one half of them
are either not graded or the instructor didn’t have enough elements to
assess progress. However, those students consume only 17% of the hours.
Santa Ana College School of Continuing Education ESL Program
2011SPN (19 weeks) Vs.
Three Terms (2010Fall, 2011 Spring and 2011 Fall)
51 weeks of instruction and skipping summer (5 weeks)
Beginning ESL 3
P
25.9
Students 1 Term (%)
SP
26.9
NP
47.2
Students 3 Terms(%)
30.6
26.6
42.8
Beginning ESL 3
Hours 1 Term (%)
Hours 3 Terms (%)
P
48.2
SP
34.9
NP
17.0
55.6
30.8
13.6
100%
90%
100%
25.9
80%
90%
30.6
80%
70%
60%
70%
26.9
50%
26.6
SP
40%
NP
30%
20%
P
42.8
40%
SP
34.9
NP
30.8
20%
10%
0%
0%
Students 3 Terms(%)
P
50%
10%
Students 1 Term (%)
55.6
60%
30%
47.2
48.2
17.0
13.6
Hours 1 Term (%)
Hours 3 Terms (%)
The metrics improve when scope is increased from a single term to multiple
terms.
Santa Ana College School of Continuing Education ESL Program
2011SPN (19 weeks) Vs.
Three Terms (2010Fall, 2011 Spring and 2011 Fall)
51 weeks of instruction and skipping summer (5 weeks)
Typical class: 30 students, 17 weeks, 10 hrs per week
5100 attendance hours = 9.71 FTES
9.71 x $ 3,232 = $ 31,397
Beginning ESL 3
Students (out of 30)
Cost (out of $31,397)
P
9.18
$ 17,456
SP
7.98
$ 9,670
NP
12.84
$ 4,270
100%
90%
80%
9.18
$17,456
70%
60%
50%
7.98
P
SP
40%
NP
30%
20%
$9,670
12.84
10%
$4,270
0%
Students (out of 30)
Cost (out of $31,397)
Noncredit is successful &
efficient
0 We need more time to finalize the info on Parenting, OA
and DSPS
0 ESL Pass rate 64-80% (lowest ESL less & middle more
successful, highest less successful)
0 HS good Pass rate
0 Parenting high pass
0 Number of hours necessary to Pass & SP
0 ESL
0 CTE
0 Cost for success and non-success
0 Cost over three semesters
Considering Accountability
and the Resolutions
Healthy accountability should:
0 Address higher level learning outcomes
0 Report on authentic student proficiencies
0 Indicate potential interventions and improvement
0 Target improved practice not just reporting
Noncredit has piloted and examined training fro
progress indicators and the results of student data.
0 Fully support resolutions
0 Urgency due to the link of funding and reporting
Questions
0 What are the factors in basic skills classes that
represent barriers for students at your college?
0 Do you see anything here that would benefit your
institution?
0 In addition to innovative and flexible scheduling,
what other ideas do you have?
CCC General Student Ethnicity 2008-2009 in the General, Credit and Noncredit
Population Compared to California’s Current and Projected Population
Ethnicity
ETHNICITY
% Total
Enrollment
% Total
Credit
Basic
Skills/ESL
% Total
Noncredit
Basic
Skills/ESL
California
Population
2010
California
Population
Ethnicity
Projection
2050
AFRICANAMERICAN
7%
11.3%
3.5%
6%
5%
ASIAN
12%
17%
15.5%
12%
13%
HISPANIC/
LATINO
30%
41.3%
52.1%
37%
52%
1%
0.9%
0.3%
1%
1%
1%
.9%
.3%
0%
1%
35%
21.8%
12.3%
42%
26%
NATIVE
AMERICAN
PAC
ISLANDER
WHITE
MiraCosta Noncredit ESL Data 2008 2009
Term I
Persistence %
Promotion %
Morning Classes
80%
(10% Perfect Attendance)
43%
Evening Classes
80%
(7% Perfect Attendance)
47%
Morning Classes
79%
(9% Perfect Attendance)
50%
Evening Classes
79%
(7% Perfect Attendance)
56%
Morning Classes
81%
(8% Perfect Attendance)
30%
Evening Classes
76%
(5% Perfect Attendance)
54%
Morning Classes
78%
(10% Perfect Attendance)
63%
Evening Classes
74%
(8% Perfect Attendance)
46%
Term II
Term III
Term IV
Brain Anatomy overlaid with
Kolb’s Learning Cycle Zull p 18
Active Learning:
Engages all of the Brain
Epilepsy Foundation of America
http://www.epilepsyfoundation.org/about/science/functions.cfm
Note that as we go down
the pyramid, we are
engaging additional areas
of the brain, creating
deeper learning.
OVERVIEW and Background
1. Issues with Accountability in Higher Education
2. Mandated reporting– ARCC, CDCP and Basic Skills legislated reports – some
does not include noncredit due to lack of success and progress indicators
3. CDCP certificates – current state of noncredit and CDCP
4. BSI and the attempt to capture progress CB 21
a. Noncredit and credit aligned
b. Progress only by subsequent enrollment
c. inability to count success
5. In noncredit progress all indicators and grades turned in are converted to UG
ungraded at the state level
See Background document and Noncredit Accountability Documents
for more information
43
Persistence Indicators
Is this the noncredit story? Link to Mira Costa Data