Transcript Virginia’s Employer Follow-up: An Examination of Response and NonResponse Patterns Presenters: Eric Lichtenberger Jim Washington.

Virginia’s Employer Follow-up: An
Examination of Response and NonResponse Patterns
Presenters:
Eric Lichtenberger
Jim Washington
Organization of Presentation
●
●
●
●
Introduction
Description of follow-up procedures
Adaptations over the past few years to
improve response rates
Going beyond reporting requirements
(providing useful information to stakeholders)
Brief History of Our Research
Center
●
●
●
●
Center for Assessment, Evaluation, and Educational
Programming (CAEEP)
Virginia Tech’s School of Education’ Office of
Education Research and Outreach
Conducting the annual follow-up for the Virginia
Office of Career and Technical Education Services
for 30 years
We also perform similar work for the Virginia
Department of Adult Education and Literacy and the
Virginia Department of Correctional Education
Overview of the Follow-up
●
●
●
●
●
Local school divisions are highly involved in
the follow-up process
Several years ago we moved towards a
flexible on-line reporting system that blends
paper based, on-line, and telephone data
collection.
Currently we have 5 years of parallel data
30,000 CTE completers (concentrators) and
4,500 employers
Description of survey
Improvements to the Process
●
●
●
●
Developed the Address Checker Tool
Used Division Letters, Division Envelopes,
and Allowed Divisions to Conduct their Own
Employer Follow-up
Obtaining Employer Phone Numbers
Establishing 75% as the standard response
rate
Summary Numbers from the
2006 Survey
●
State Numbers: Divisions and Regional
Centers may obtain specifics for their area.
–
–
–
–
41.15% of 4,578 employers responded.
30.23% of 4,578 employers provided usable
responses.
89.52% of 1,384 employers were either Satisfied
or Very Satisfied with the Employee's high school
preparation
These numbers have steadily increased since
2002
Summary Numbers from the
2006 Survey
●
●
●
35.48% of 1,384 employers rated the
employee's high school preparation as
Equally Prepared
44.29% of 1,384 employers rated the
employee's high school preparation as Better
Prepared
79.77% of 1,384 employers were either
Satisfied or Very Satisfied with the
employee's Technical Skills as they related to
the job
Employer Comments
●
●
●
●
●
Survey question asks for specific comments
in reference to the Workplace Readiness
Skills, categorized by “Satisfied” and
“Unsatisfied”
About half of the respondents provide
comments
Average of three comments per survey that
has comments
Comments tend to be 4-to-1 favorable
We let the computer do the summarizing
Comments Summary
●
Satisfied
–
–
–
–
–
–
–
–
–
Attitude
Work Ethic
Computer Skills
Attendance
Teamwork
Working with Others
Math
Reading
Quick Learner
●
Dissatisfied
–
–
–
–
–
–
–
–
–
–
Attendance
Writing
Decision-making
Initiative
Math
Spelling
Work Ethic
Listening
Attitude
Working with Others
Satisfaction Vs. Applicability
●
●
●
●
“Not Applicable”
Low N/A = High
Applicability
High N/A = Low
Applicability
Satisfaction Where
Applicable, e.g.,
(368+364)/(893-140)
Satisfaction Where Applicable
●
●
●
●
●
●
●
●
97% Reading
93% Math
93% Computer
92% Technical Skills
91% Writing
90% Teamwork
89% Attendance
89% Pos. Attitude
●
●
●
●
●
●
89% Speaking &
Listening
86% Work Ethic
86% SelfPresentation Skills
85% Reasoning and
Problem-solving
83% Independence
and Initiative
81% Big Picture
Applicability
●
●
●
●
●
●
●
94% Attendance
93% Work Ethic
92% Pos. Attitude
92% Teamwork
90% Independence
& Initiative
90% Speaking &
Listening
87% The Big Picture
●
●
●
●
●
●
87% Reasoning &
Problem-Solving
85% Technical Skills
84% Reading
79% Math
79% Writing
62% Computer
Address Checker Tool
●
●
●
●
Local school divisions are required to verify
employer addresses
Employer response rate has improved since
implementing this tool
Once an employer address is entered the
information is compared against a list of
deliverable addresses
There are numerous outcomes based on that
comparison
EMPLOYER MAIL ADDRESS STATUS - STATE REPORT
Address Status
Bad state abbreviation
Number
Responses
Response Rate
1
1
100%
605
63
10%
7
4
57%
119
31
26%
8
3
38%
37
12
32%
3294
1559
47%
31
13
42%
Street not found
117
33
28%
Unknown Status
51
36
71%
444
130
29%
4714
1885
40%
Insufficient address
Missing street directional
No street address number
Not found
Out-of-state
Possibly deliverable
Probably not deliverable
Verified OK
Statewide
2006 Employer Response Rates by Follow-up
Procedure
Procedure
Traditional
Division Envelope
Division Letter
Division Conducted
Employers
Response rate
3147
36%
469
29%
42
62%
1056
57%
Employers that Traditionally Do
Not Respond
●
●
●
●
●
●
Created a database of employers who due to
company policy, do not respond
Categorized the information by year and
branch
Some companies are consistent and with
others it depends on the branch or the year
Wal-Mart and Lowe’s have the greatest
number of such responses
Lowe’s
School divisions could use the information to
contact those who do not respond to further
explain the purpose of the follow-up
Other Things we do with the
Follow-up Data
●
●
●
●
●
●
We’ve been able to do some other things with
the data that did not require an employer
response
In Virginia, students who are employed fulltime provide their employer’s address
That address is used in the employer followup, but not all employers respond
We are still able to use that address even if
the employers do not respond
Applicable to states who perform data
matching
Providing school division with this information
makes the 75% response rate standard more
bearable
Distance
●
●
●
Distance between the completer’s school and their
employer
Within the same county or independent city as the
completer’s school
Keep in mind, the distance field is only a proxy
variable
Major weakness: do not know if the completer
moved
Distance Trends for CTE
Completers Working Full-Time
Graduation
Year
Number
Employed
Average
Distance
14.25
Number
Employed
within Same
Area (%)
2,303 (58%)
Number
Employed
within 25 Miles
(%)
3,647 (92%)
2001
3,962
2002
3,540
14.26
2,005 (57%)
3,258 (92%)
2003
3,439
17.29
2,031 (59%)
3,164 (92%)
2004
3,985
19.21
2,327 (58%)
3,642 (91%)
2005
4,011
17.59
2,253 (56%)
3,670 (92%)
Distance and Gender (Full-Time
Only)
Number
Employed
Average
Distance
Number Employed
within Same Area (%)
Number Employed
within 25 Miles (%)
Male
10,634
17.61
5,960 (56%)
9,653 (91%)
Female
8,303
15.22
4,959 (60%)
7,728 (93%)
Distance and Relatedness for CTE
Completers Employed Full-Time
Relatedness
to CTE
Program
Number
Employed
Full-Time
Average
Distance
15.02
Number
Employed
within Same
Area (%)
3,746 (58%)
Number
Employed
within 25
Miles (%)
6,005 (92%)
Closely
6,499
Somewhat
5,486
17.81
3,186 (58%)
5,010 (91%)
Unrelated
6,405
17.11
3,661 (57%)
5,860 (91%)
Distance and Skill Application for CTE
Completers Employed Full-Time
Skill
Application
Number
Employed
Full-Time
Average
Distance
Number
Employed
within Same
Area (%)
Number
Employed
within 25
Miles (%)
Most
6,317
15.41
3,617 (57%)
5,813 (92%)
Some
5,706
17.03
3,332 (58%)
5,251 (92%)
Little
3,250
16.88
1,866 (57%)
2,971 (91%)
None
3,006
17.87
1,720 (57%)
2,734 (91%)
Distance and Earnings for CTE
Completers Employed Full-Time
Hourly
Wages
Number
Employed
Full-Time
Average
Distance
28.51
Number
Employed
within Same
Area (%)
686 (47%)
Number
Employed
within 25
Miles (%)
1,229 (84%)
More than $12
1,466
$9 to $12
5,501
17.54
2,889 (53%)
4,958 (90%)
$6 to $8.99
9,541
14.52
5,781 (61%)
8,922 (94%)
Less the $6
1,118
13.79
774 (69%)
1,059 (95%)
Credentials by Service Area
Service Area
Number of
Completers
Number with
Credentials/ License
Percent
Agriculture
Business
FCS
Health
Marketing
Technology
T&I
6,111
42,106
16,186
5,644
16,603
18,907
28,189
236
2,316
919
1,994
746
1,030
4,401
3.9%
5.5%
4.1%
35.3%
4.5%
5.5%
15.6%
Skill Application by Service
Area
Service Area
Percent
Reporting
None
Percent
Reporting
Little
Percent
Reporting
Some
Percent
Reporting
Most
Agriculture
15%
17%
44%
24%
Business
14%
23%
44%
20%
FCS
21%
20%
34%
25%
Health
24%
18%
16%
42%
Marketing
7%
11%
41%
41%
Technology
T&I
30%
23%
26%
18%
33%
25%
11%
34%
Impact of Credentials on Earnings
Service Area
Percent of
Yes Earning
More than $9
Percent of No
Earning More
than $9
Difference in
Percentage
Impact
Agriculture
56%
45%
+11%
+20%
Business
38%
33%
+5%
+13%
FCS
25%
26%
-1%
-4%
Health
37%
36%
+1%
+3%
Marketing
55%
36%
+19%
+35%
Technology
T&I
52%
57%
44%
47%
+8%
+10%
+19%
+18%
Conclusions
●
●
●
Going beyond meeting the basic reporting
requirements is difficult; however, the
information can readily be used for program
improvement
Providing the information back to those
involved in the follow-up process in a useful
format gives them more of an incentive to
meet response rate objectives
Using information from previous years to aid
local school divisions in meeting their
response rate objectives can also be
beneficial
Contact Information
●
●
Eric Lichtenberger – [email protected]
Jim Washington – [email protected]