Transcript nrs changes

Overview
and Discussion
of NRS Changes
NRS Changes for FY13
AGENDA
Review changes on follow-up outcomes and
new data requirements
Review criteria for cohorts
Review changes to reporting timeline and
reporting tables
Review questions and answers posted on the
ACLS website
Respond to participant questions
Massachusetts Department of Elementary and Secondary Education
2
Time Period Definitions
Program Year (PY)– Twelve month period from
July 1 through June 30 defined by the calendar
year at start of period. (federal terminology)
For example, July 1, 2012 through June 30, 2013 is
the time period corresponding to PY12.
Fiscal Year (FY)– Twelve month period from July
1 through June 30 defined by the calendar year
at end of period. (state terminology)
For example, July 1, 2012 through June 30, 2013 is
the time period corresponding to FY13.
Massachusetts Department of Elementary and Secondary Education
3
Changes to Follow-up Outcomes:
Automatic Cohort Designation
Program Year (FY2012): Participants identified
based on self-reported goals (program – level)
Program Year (FY2013): Participants
automatically identified based on “cohort” which
are based on demographic variables in the MIS
system (state – level)
4
The definition for “cohort” varies by outcome.
Massachusetts Department of Elementary and Secondary Education
Implications of “Automatic Cohort
Designation”
 Students belong to a group to be followed for NRS
post-program outcomes based on pre-defined
characteristics.
 Following students for post program outcomes will
no longer be goal dependent.
 As has always been the case, we are only
concerned with students having at least 12 hours
of attendance.
Massachusetts Department of Elementary and Secondary Education
5
Do Student Goals Still Matter?
 YES!!
 Local staff should continue to set goals with
students
 Goal setting is an essential component of a good
program and good instruction but….
 Students goals will no longer be used as a way to
identify students for the NRS follow-up measures 6
Massachusetts Department of Elementary and Secondary Education
7
Massachusetts Department of Elementary and Secondary Education
Secondary Credential Cohort
Obtain Secondary Credential Cohort
 Learners who take all GED tests, or
 Are enrolled in adult high school at the high ASE
level, or
 In the assessment phase of the EDP, and
 Who exit during the program year
Entry into cohort occurs only for those who take all 5 GED tests by
the end of reporting period (December 31).
8
Massachusetts Department of Elementary and Secondary Education
Question: Cohort for Exited
Students and GED Tests
Q: Do students have to still be enrolled
when they take the tests?
 No, students are included in the cohort if they take
all five test before the end of the reporting period
for the program year in which they exit (December
31)
Q: Must a student pass all GED tests to be
included as achieving the outcome?
 Yes, students must pass all GED tests to be
included as achieving the outcome
Massachusetts Department of Elementary and Secondary Education
9
Passing GED Tests: Examples



Example 1: A student who enrolls in fall of 2012, exits in
March 2013, takes 3 tests then, takes 2 more in September
2013, would be included in the cohort for 2012-13
Example 2: A student who enrolls in October 2012, takes 2
GED tests in May 2013, exits in November 2013 and takes
3 GED tests would be included in the cohort for 2013-14
Example 3: A student who enrolls in December 2012, exits
in March 2013, takes 3 tests then, takes 2 more in January
2014, would not be included in any cohort
10
Massachusetts Department of Elementary and Secondary Education
Postsecondary Cohort
Postsecondary Cohort
All learners who
 have earned a secondary credential while enrolled
 hold a secondary credential
 or are enrolled in a class specifically designed for
transitioning to community college
and who exit during the program year
11
Massachusetts Department of Elementary and Secondary Education
Questions: Foreign Credential
and Transition class
Q: Are students with a secondary credential, earned in
another country, in this cohort? Are ESOL students to be
included?
 Yes, these students are included in the cohort
Q: How is a transition class defined ?
 A class that has a specific purpose to prepare students
for entry into postsecondary education, training or an
apprenticeship program.
 The rate based class focus would be Bridge to College/Math
or Bridge to College/Writing. The non rate based class focus
would be Bridge to College/Math, Bridge to College/Writing,
or College and Career Readiness.
Massachusetts Department of Elementary and Secondary Education
12
Reporting of Entry into
Postsecondary--Timing
Q: Please clarify multi-year reporting of entry into
postsecondary education
 Students who exit during program year achieve this
outcome if they enter postsecondary any time from
exit through the end of the following program year
(June 30)
 Follow students and report for two years, in two
rows on Table 5 (unduplicated)
 First report (2012-13): report same as of now
 Future years: Update prior year—keep cohorts separate
 First two year reporting on 2013 report
(due December 31, 2014)
Massachusetts Department of Elementary and Secondary Education
13
Employment Cohorts
 Entered Employment Cohort
 Learners who are not employed at time of entry
and in the labor force who exit during the
program year
Employment Status in SMARTT:
 Employed
 Unemployed and Looking for Work
 Unemployed and Not Looking for Work
 Retired or Otherwise Not Looking for Work
 Homemaker
Programs should exit students who are no longer in attendance by filling out
the “Exit Student” screen in SMARTT.
Massachusetts Department of Elementary and Secondary Education
14
Employment Cohorts
 Retained Employment Cohort
 Learners who were not employed at time of entry
and in the labor force, who are employed by the
first quarter after exit quarter
 Learners employed at entry who exit during the
program year
Employment Status in SMARTT:
 Employed
 Unemployed and Looking for Work
 Unemployed and Not Looking for Work
 Retired or Otherwise Not Looking for Work
 Homemaker
Programs should exit students who are no longer in attendance by filling
out the “Exit Student” screen in SMARTT.
Massachusetts Department of Elementary and Secondary Education
15
Reminder on Retained
Employment
 Students, in entered employment cohort, that
are employed the first quarter after exit
are also in the retained employment cohort
 The timeframe is the same as always–it has
not been changed
 States must use students’ employment status
at entry to determine cohort status for
employment outcomes.
Massachusetts Department of Elementary and Secondary Education
16
Question: Definition of
Unemployed in Labor Force
Q: How is “unemployed, in the labor force
defined?”

NRS definition (Unchanged): Learners who are not
employed but are seeking employment, make
specific efforts to find a job and are available for
work
Employment Status in SMARTT:
 Unemployed and Looking for Work
17
Massachusetts Department of Elementary and Secondary Education
Timing of Employment Measure
 Entered Employment
 Employed in the first quarter after exit.
 Retained Employment
 Employed in the third quarter after exit.
18
Massachusetts Department of Elementary and Secondary Education
Review: Employment Data
Collection and Reporting
 Collection: One quarter after exit for entered
employment, three quarters after exit for
retained employment
 Reporting lag (previous year) to
accommodate lag in Unemployment Insurance
(UI) wage records
19
Massachusetts Department of Elementary and Secondary Education
Exit Quarter
Measure
Report
Entered Employment
October 1–December 31, 2012
(Second Quarter FY 2013)
FY 2013
Retained Employment
April 1–June 30, 2013
(Fourth Quarter FY 2013)
FY 2013
Entered Employment
January 1–March 31, 2013
(Third Quarter FY 2013)
FY 2013
Retained Employment
July 1–September 30, 2013
(First Quarter FY 2014)
FY 2014
Entered Employment
April 1–June 30, 2013
(Fourth Quarter FY 2013)
FY 2013
Retained Employment
October 1–December 31, 2013
(Second Quarter FY 2014)
FY 2013
Entered Employment
July 1–September 30, 2013
(First Quarter FY 2014)
FY 2014
Retained Employment
January 1–March 31, 2014
(Third Quarter FY 2014)
FY 2014
July 1–September 30, 2012
(First Quarter)
October 1–December 31,
2012
(Second Quarter)
Collect
January 1–March 31, 2013
(Third Quarter)
April 1–June 30, 2013
(Fourth Quarter)
Massachusetts Department of Elementary and Secondary Education
20
Data Matching
Department of Revenue
 Enter Employment/Get a Job -- Those with
SSN and signed release form. For all others,
survey method must be used.
 Retain Employment -- Those with SSN
and signed release form. For all others, survey
method must be used.
Survey form may be accessed at:
http://www.doe.mass.edu/acls/smartt/nrssurvey
Massachusetts Department of Elementary and Secondary Education
_followup.pdf
21
Data Matching
Oklahoma Scoring Service, and National Student Clearinghouse
 Obtain GED -- All students (regardless if SSN
provided and release form signed since match
will be with the Oklahoma Scoring Service)
 Enter Postsecondary Education -- -- All
students (regardless if SSN provided and
release form signed since match will be with
the National Student Clearinghouse)
Massachusetts Department of Elementary and Secondary Education
22
Follow-up Outcome Timing
Secondary Credential
Obtained credential before the end of the
reporting period (December 31)
Postsecondary Education
Entered any time from exit until the end of the
following program year (June 30)
 Report for two years, in two rows on Table 5
(unduplicated)
Programs should exit students who are no longer in attendance by
filling out the screen found by clicking on the “Exit Student” menu
option within SMARTT .
Massachusetts Department of Elementary and Secondary Education
23
Reporting Changes: Table 5
 Table 5 revised to include instructions for
automatic cohort designation
 New row added for postsecondary
transition: Report two years (unduplicated)
 Update prior year
 First two year reporting on 2013 report
(Due December 31, 2014)
24
Massachusetts Department of Elementary and Secondary Education
Reporting Postsecondary
Example in 2013-14
 In 2013-14, 125 students in cohort exit, state
matches 100 and finds 50 have entered
postsecondary
 State runs match on 2012-13 cohort again by
June 30, 2014 and finds another 20 students
have entered postsecondary in addition to the
40 from previous year
25
Massachusetts Department of Elementary and Secondary Education
Example of Table 5: 2013-14
Core Follow-up
Outcome
Measures
(A)
Entered
Postsecondary
Education or
Training -- current
program year
Entered
Postsecondary
Education or
Training – prior
program year
Number of
Participants in
Cohort
(B)
120
Number of
Participants
Included in
Survey Sample
(C)
Number of
Participants
Responding
to Survey or
Used for
Data
Matching
(D)
100
Response
Rate or
Number of
Percent Participants Percent
Available Achieving Achieving
for Match
Outcome
Outcome
(E)
(F)
(G)
83%
(100/120)
40
40%
(40/100)
Leave entire
row blank
26
Massachusetts Department of Elementary and Secondary Education
Example of Table 5: 2014-15
Core Follow-up
Number of
Outcome
Participants in
Measures
Cohort
(A)
(B)
Entered
Postsecondary
125
Education or
Training -- current
program year
Entered
Postsecondary
120
Education or
Training – prior
program year
Number of
Participants
Included in
Survey Sample
(C)
Number of
Participants
Responding
to Survey or
Used for
Data
Matching
(D)
Response
Rate or
Number of
Percent Participants Percent
Available Achieving Achieving
for Match
Outcome
Outcome
(E)
(F)
(G)
100
80%
50
50%
100
83%
60
60%
(prior year (60/100)
was 40,
27
now
40+20)
Massachusetts Department of Elementary and Secondary Education
Implications for Implementation
 The number of learners to follow will
increase.
 Obtaining SSN’s and signed release forms
will lessen the follow-up burden.
 Local staff will need to follow-up by survey
method for cases where SSN’s and signed
release forms were not obtained for those
within any of the employment cohorts.
28
Massachusetts Department of Elementary and Secondary Education
Questions: Reporting Student
Education
Q: How should we determine years of
student education? Do we need to
check?
 Self-reported by student, NRS does not require
verification but the state may choose to do so
Q: How do we count education in other
countries that is different than our
system?
 Use years of schooling
 Use in-country definition for secondary credential
Massachusetts Department of Elementary and Secondary Education
29
Question: Multiple Cohorts
Q: Can students belong to multiple
cohorts?
 Yes, for example, an unemployed student in the
labor force, with secondary credential who exits,
would be in two cohorts
30
Massachusetts Department of Elementary and Secondary Education
Question: Continuing Students
Q: Do we have to re-identify cohort
membership for students continuing the
next year?
 Yes, for example, employment status or secondary
credential status may change
 But only collect measure at exit
31
Massachusetts Department of Elementary and Secondary Education
ADDITIONAL
QUESTIONS??
32
Massachusetts Department of Elementary and Secondary Education