Transcript Job Vacancy Survey Workshop

Job Vacancy Survey
Workshop
April 22-23, 2003
St. Paul MN
Welcome
MDES Commissioner
Harry Mares
Welcome
• Job vacancy information is required under WIA
and the Perkins Act to plan workforce
development policy and programs.
• A job vacancy survey is an innovative product
that provides this information.
Welcome
• The Minnesota Job Vacancy Survey helps us
understand the workforce needs in the state.
• It helps us to plan and implement an effective
workforce development policy.
• This will help Minnesota remain competitive
and economically vital.
• We want to learn from you!
Introductions
Presenters
MDES:
• Oriane Casale
• Rachel Hillman
BLS:
• Christina Chiu
• Jeff Willingham
• Curt Theis
• Annie Tietema
History of
Minnesota’s Survey
Oriane Casale
Minnesota’s History
• Minnesota was one of 7 pilot states funded by
the DOL ETA to conduct a job vacancy
survey.
• In 1999, we partnered with the Metro
Workforce Investment Board and the City of
Minneapolis Employment and Training
Program.
• Conducted two rounds of a Twin Cities
survey.
History Continued
• Survey design based on the Milwaukee Survey of
Job Openings conducted by the University of
Wisconsin-Milwaukee since 1993.
• Used two rounds to refine survey instrument,
sample design, data collection, and analysis.
• Findings are not comparable to current survey (or to
each other).
• Results were well received by partners, media and
Governor’s office.
• This created demand!
History Continued II
• MDES Commissioner saw a need for this
information and was willing to fund statewide
survey.
• We have since produced five rounds of a
statewide survey.
• Success in Minnesota and other states
including Colorado and Wisconsin has meant
further ETA and BLS interest and support –
National Job Vacancy Statistics Workgroup.
Job Vacancy Statistics
Workgroup
Oriane Casale
Find more on this in About JVS,
Introduction.
Background
• The Workforce Information Council
(WIC) has identified obtaining
accurate job vacancy information from
within the States and also nationwide
as a vital goal.
• BLS & ETA
Workgroup Goals
• Employ statistically rigorous methodology.
• Develop comparable statistics across
States and local areas.
• Reduce startup and overhead costs.
• Promote the use of job vacancy statistics.
Participants
•
•
•
•
•
•
•
•
Colorado
Florida (2)
Maine
Maryland (2)
Minnesota (2)
New Mexico
New York
Wisconsin
• ETA
• BLS (4)
See Job Vacancy Statistics Project Brochure!
Progress
• Survey Instrument
• Sample Allocation and Selection System (BLS)
• Data Capture System (MDES)
• Estimates Production Systems (BLS)
• Job Vacancy Statistics Website
• Workshops
Progress Continued
• Access to latest news at www.JVSinfo.org
or e-mail: [email protected]
• JVS Software package
– Sample Allocation and Selection
– Data Capture System
– Estimates Production
• Ongoing technical support provided to the States
• “How to” Guide – About JVS
Survey Timeline
Rachel Hillman
Find more on this in About JVS,
Survey Timeline and Resources.
Survey Timeline
After the sample is drawn...
• Pre-Survey Resources
• Items in Mailings
• Pre-Mailing Activity
See Survey
Timeline
handout!
• Post-Survey Activity
…and before the results are in.
Pre-Survey Resources
Human Resources
Communications
Purchasing
Follow-up Telephone Call
Data Processing
Mailing and Duplicating
Pre-Survey Activities
• Make Detailed Timeline
• Assignments
Pre-Mailing Activity
• Address Correction Process
-Postcard
• Telephone for Contact Name
• Alternate Response Methods
• Supplies
Items in Mailing
• Postcard
• Cover Letter
• Follow-up Insert
• Survey Instrument —
(We’ll talk about that later.)
See Survey Insert Handouts!
Post-Survey Activity
• Data Entry
• Data Cleansing
Cost
Oriane Casale
Find more on this in About JVS, Cost.
Staff Costs
• Survey Coordinator & Analyst
• Technical Support
• Administrative Staff
• Graphic Artist
See Cost Handouts!
Mail or Telephone
A mail survey incurs
costs for printing and
mailing postcards,
letters and surveys
and for calls for
sample refinement
and survey follow-up.
The alternative is to
conduct the survey by
phone or contract with
someone who can, like
the Colorado LMI
Office.
Methods of Funding
•
•
•
•
•
•
One-Stop LMI funds (MN, NJ, FL, WA)
Reed Act (NJ)
WIC Boards (MN)
County/City (MN)
Chamber of Commerce (KS)
State/Private (CO)
Paper Survey
Rachel Hillman
Find more on this in About JVS,
Data Collection: Survey instrument.
Paper Survey
Survey Instrument:
• One page
• Easily Faxable
• Field-tested (Minnesota) and cognitively tested
(BLS)
See Survey Instrument handouts!
Paper Survey Continued
Survey Asks:
Job title — Required
Number of vacancies — Required
Full-time or part-time — Required
Temporary or permanent
Paper Survey II
Survey Asks:
Number of Days Open — 4 categories
Education — 6 categories
Experience — 3 categories
Certificate/License
Wage — Hourly Wage or Monthly, Annual Salary
Benefits — Up to 5 unique benefits
Cognitive Testing of
Survey Instrument
Oriane Casale
Find more on this in About JVS, Data
Collection: Survey instrument.
Cognitive Testing
Purpose:
• Is format easy to follow?
• Do respondents understand survey questions
and do they fill it out as intended?
• Can respondents access information requested
and if so, how easily?
Cognitive Testing Continued
• 23 completed interviews
– 18 in-person
– 5 over the phone
• Respondents were asked to:
– Comment in general on the form
– Express their impressions while filling out the form
– Explain their reasoning for the answers given on the
form
Results of Cognitive Testing
• Overall respondents understood the form,
provided the information we wanted and
had easy access to the information.
• Made changes to instructions and several
columns of information based on results
• Changed education/experience categories
• Wording of seasonal/temporary question
• Changed compensation question
Survey Instrument Modifications
States who have modified the JVS Workgroup
survey instrument:
•
•
•
•
Kansas
Louisiana
Minnesota
Washington
Their surveys are available through JVS
Helpdesk – [email protected]
Minnesota’s Web
Survey
Rachel Hillman
Find more on this in About JVS, Data
Collection: Alternative response methods.
Web Survey
Components of Website:
• Fill out the Survey on-line
• E-mail file containing job vacancy data
• Instructions
• Questions
• Uses and Benefits
• Data
Web Survey Response Rate
3 percent of the firms who responded
filled out the survey on the Internet
Minnesota — Alternative
Response Methods
Rachel Hillman
Find more on this in About JVS, Data
Collection: Alternative response methods.
MN – Alternative Response Methods
•
We offer the following response options:
Mail
Fax
Phone
Internet
Website
E-mail
Alternative Response Methods–
Response Rates
• Website – 9 percent
• Phone – 8 percent
• Fax – 9 percent
• Other – 5 percent
Report Writing &
Analysis
Workforce Supply & Demand Measures
Job Vacancy Rate
Rachel Hillman
Find more on this in About JVS, Workforce
Supply and Demand Indicators.
Job Vacancy Rate
Job Vacancies
Total Employment
Job Vacancy Rate Continued
• The job vacancy rate is the number of
jobs open for hire as a percent of total
employment.
• Job vacancy rate of 2.5 percent— or
fewer than three jobs for every 100 filled
jobs in Minnesota
Job Vacancy Rate Continued II
• Job vacancy rates by
major occupational
group (2-digit SOC)
• Job vacancy rates by
occupation (6-digit SOC)
Estimates
Delivery System
(EDS)
• Job vacancy rates by
firm size
ES-202
• Job vacancy rates by
industry
Estimates Delivery System
(EDS)
Annie Tietema
Find more on this in About JVS, Workforce
Supply and Demand Indicators: Occupational
employment estimates.
Estimates Delivery System 2000
As developed by the North Carolina Employment
Security Commission, in conjunction with the OES
Policy Council Technical Committee
Overview
• EDS 2000 is a system for producing
occupational wage and employment estimates
and publications from the results of the OES
survey
• Employment and wage statistics produced by
the BLS National Office can be imported into
EDS for inclusion in publications
• EDS allows states the flexibility to define and
generate estimates for any sub-state geography
based on county-level building blocks
(townships in New England states)
Overview Continued
• EDS exports estimates in a variety of formats,
including one format suitable for Internet
publications and two database table formats
• EDS will follow the OES bi-annual data
collection, which began with the November 2002
panel.
• While EDS has been SIC-based, the new
version – due out in fall – will operate on the
NAICS industry coding convention.
State Micro-data files
BLS-produced National estimates files
BLS-produced State & MSA estimates files
Where do I get access to EDS?
Contact your state’s OES (Occupational
Employment Statistics) division for information
about your state’s usage of Estimate Delivery
System 2000.
Hiring Demand Index
Rachel Hillman
Find more on this in About JVS, Workforce
Supply and Demand Indicators.
Workforce Supply and Demand
Revisited
Job vacancy rates do not provide
conclusive proof that a workforce
shortage exists
What about Turnover?
Turnover – the rate at which workers cycle in
and out of jobs – distorts interpretation of job
vacancy rates
Hiring Demand Index
• Compare job vacancy rates while
controlling for turnover
• Assumptions:
1. Fields with greatest vacancy rates often have
proportionate hiring activity -- positions get filled!
2. Number of job vacancies meaningful when compared
with number of positions filled in an average month
Hiring Demand Index Continued
Job Vacancy Rate
Turnover Rate
For the Occupational
Group
Job Vacancy Rate
Turnover Rate
For All Occupations
Figure 1.1: Hiring Demand Index for Occupational Groups with 1,000 or More Job
Vacancies in Minnesota
Healthcare Practitioners and Technical
Community and Social Services
Personal Care and Service
Healthcare Support
Occupational Group
Management
Transportation and Material Moving
ALL OCCUPATIONS
Sales and Related
Education, Training and Library
Installation, Maintenance and Repair
Office and Administration Support
Business and Financial Operations
Production
Construction and Extraction
Food Preparation and Serving Related
Hiring Demand Index
Building, Grounds Cleaning and Maintenance
0
0.5
1
1.5
2
Hiring Dem and Index (Average=1.00)
2.5
3
MN — Help Wanted!
•healthcare practitioners and technical
occupations (including registered nurses and licensed
practical nurses)
•community and social service occupations
(including social workers and counselors)
• personal care and service occupations (including
childcare workers, hairstylists, and personal and
home care aides)
• healthcare support occupations (including nursing
aides, orderlies and attendants)
Hiring Demand Index III
• Similar to U.S. BLS’s fill rate from 1991
pilot study
• Index allows identification of occupation
groups where worker shortages are more
serious
Turnover Rates
• CPS Job Tenure Supplement microdata
– Nationwide sample with data elements for
occupation and tenure (n= 50,000)
• Turnover rates were calculated…
Turnover = P + P2 + P3
where P is the share of employment with <1
year tenure
Turnover Rates Continued
• Assumption: Jobs can turn over multiple
times, so turnover will be greater than
number of new hires that persist
Alternate measures of turnover?
• Assume that distribution of tenure in a field
will remain consistent over time.
• Question: How many workers at every level of
tenure must leave to maintain the
distribution?
• A matrix algebra calculation of turnover… but
the resulting estimates aren’t very different
Other Ideas for Analysis
• Comparison to Unemployment
– Job vacancies vs. number of unemployed
– Job vacancy rate vs. insured unemployment
rate
• Others?
4Q 2000
4Q 2001
Job Vacancies
Unemployed
Workers
4Q 2002
Beveridge Curve
• Supply: Insured unemployment is a proxy
indicator of workforce availability
– Many unemployed = lots of competition for jobs
• Demand: Job vacancies as a share of all
jobs
– Few job vacancies = lots of competition for jobs
Beveridge Curve in the Twin Cities,
Fourth Quarter 2002
7.0%
Job Vacancy Rate
6.0%
5.0%
4.0%
3.0%
2.0%
1.0%
0.0%
0.0%
1.0%
2.0%
3.0%
4.0%
5.0%
Insured Unemployment Rate
6.0%
7.0%
Marketing
Statewide Marketing
Substate Regional Marketing
Statewide Marketing
Rachel Hillman
Find more on this in About JVS, Marketing.
What Information Do Job Vacancy
Statistics Provide?
• Identify occupations in demand
• Determine education and experience
needed for job vacancies by region
• Describe job vacancies in terms of:
– Wages
– Benefits
– Part-time/full-time
– Temporary/seasonal or permanent
Steps for Marketing Your JVS
Five steps to building your JVS customer base:
1. Identify the potential audiences in your state
and around the nation.
2. Create user-friendly paper and web
publications oriented toward these audiences
and work to make sure they receive them.
Who Uses Job Vacancy Survey Data
Policy
Makers
Media
Job Seekers
Job
Counselors
Job
Vacancy
Survey
Data
Industry
And
Others!
Human
Resource
Professionals
Workforce
Investment
Boards
Education and
Training
Planners
Possible Publications
• Executive Summary/Report
– Regional
• Brochure
– Regional
– Industry
• Web Publication
• Others?
Statewide Users
• Policy Makers
– General findings
• Educational and Training Planners
– Program possibilities
– Training needs
• Human Resource Professionals
– General Findings
• Job Seekers
– General Findings on ISEEK/Career publications
Statewide Users Continued
• Job Counselors
– General findings
– Specialized requests
• Workforce Investment Boards
– General findings
– Specialized requests
• Industry
– General findings
– Specialized requests
• Media
– General Findings
Other Users?
Steps for Marketing Your JVS Continued
3. Incorporate findings from the job vacancy
survey into the range of information
disseminated through your LMI Office
(publications, articles, interviews, presentations,
training and website).
4. Press release new data and findings to the
media and provide interviews to help the media
interpret and write about this new information.
5. Collect and use customer feedback to
continuously improve publications and web
presence.
How MN Use Job Vacancy Survey Data
Executive
Summary
Technical
Report
Presentations
Specialized
Requests
Job
Vacancy
Survey
Data
Media
Requests
Trends
Articles
Regional
Reports
Review
Supplements
and Regional
Spotlights
JVS Release
• Prepare Website Executive Summary/Report
• Give the radio/newspaper media a little preview
• Press Release
• Prepare paper Executive Summary/Report
• Prepare regional Brochures
Reader Questionnaire
• A variety of users
• Information meet needs & presented in a
clear and understandable manner
Partially — Yes
• A lot of very specific suggestions
Good — Bad
Please see pages 27-28 of 4Q 2002 Minnesota Job
Vacancy Survey for a copy of a Reader Questionnaire!
MN — Regional Marketing
Rachel Hillman
Find more on this in About JVS, Marketing:
Regional brochures/reports.
MN — Regional Users
Regional Analysts:
• WIB Boards
• Job Service Employer Committees (JSEC)
• City/County Boards
Research & Statistics Office:
• Dislocated Workers
• State healthcare groups
• Educational Administrators/Planners
Not Users?
• Chamber of Commerce
• And Others?
Data Capture System
Curt Theis
Rachel Hillman
Find more on this in About JVS, Data
Capture System.
System Requirements
•
Windows 98 operating system or higher
(Windows 2000)
•
Pentium 133 MHz or higher processor
•
8 MB of RAM in addition to operating
system requirements
•
170 MB of available hard disk space
•
CD drive
Installation
• Insert CD
• Start “setup.exe” program
• Follow steps
Let’s give it a try!
Switchboard
Four sections of Switchboard
1.
Data Entry Forms
–
All data entry and SOC coding are done here.
2. Error Reports
–
A series or queries indicating where possible errors occur in
data entry.
3. Tracking Reports
–
Daily accounts of surveys received by various grouping
characteristics.
4. Management
–
Add or delete users, create mailing and contact lists, and link
to or import data tables may be done here.
Data Entry Forms
Four main Data Entry Forms:
1.
Survey Tracking
–
Daily record of surveys received and the method of response.
2. Part A
–
Part A of the survey form (contact information) is entered here.
3. Part B
–
Part B of the survey form (job vacancies) are entered here.
4. SOC Code
–
Job titles are SOC coded in this form
Survey Tracking Form
1. Enter Survey ID
–
Firm mailing address will be displayed at the bottom of the
form. Double check survey for consistency.
2. Enter User Name
3. Response Status
–
Method of response
4. Date Received
5. Round Received
–
What mailing was this survey?
Don’t forget to correct
addresses.
Part A Form
1. Go to Survey ID
–
Firm mailing address and strata will be displayed on the right
hand side. Only surveys that have already been tracked may
be entered.
2. Respondent, title, phone, employment
–
All information on Part A of the survey form is entered here.
3. No Vacancies
4. Comments
Part B Form
1.
Survey ID
–
–
2.
The “Press to Enter Survey ID” button will automatically enter the
last survey ID entered on the Part A form. Firm mailing address and
strata will be displayed on the right hand side.
This field is required.
Full- or Part-time
–
–
3.
All information on Part A of the survey form is entered here.
This field is required.
Job Title
–
Job title is entered as it appears on the survey form.
4.
Permanent or Seasonal
5.
Number of Job Vacancies
–
This field is required.
Part B Form Continued
6.
Length
–
7.
The length of time the job vacancies has been open is entered in days.
Education
–
Education requirement selected from a drop down menu.
8. Experience
–
9.
Experience requirement selected from a drop down menu.
Wage/Wage Period
10. Benefits
–
User defined benefits may be entered using the “Update Benefit Label”
button.
SOC Code Form
1. Coded SOC Title
2. Reported Job Title
–
The job title comes from the Part B form.
3. Number Open/Wage/Education/Experience
–
This information comes from the Part B form.
SOC Code Form Continued
Also included:
• Firm mailing address and strata information
• SOC code, title and definition
• Measure of education for that SOC code
• National and State wages for that SOC code
SOC Coding
Drop down SOC Code List of 30,000 plus job titles
and SOC codes comes from:
• Dictionary of Occupational Titles (DOT)
• Official SOC titles
• Other sources
• Census titles — up to 10,000 more titles coming!
Error Reports
• Benefit Error 1
– At least one benefit and “no benefits” is checked.
• Benefit Error 2
– At least one benefit and “no response” is
checked.
• No Vacancies, Yet Part B
– “No vacancies” box is checked, but there are
jobs recorded in the Part B form for that survey
ID number.
• Part-time with Salary
– The job vacancies is recorded as part-time and
has a salary or large wage.
Error Reports Continued
• Received, Yet No Part A
– The survey has been tracked, but no Part A exists.
– This is the most common error.
• Uncoded Openings
– The job vacancy has not been SOC coded.
• Unusable Part B
– One of the three required fields (full- or part-time, job title,
number of openings) is missing.
• Vacancies, Yet No Part B
– “No vacancies” has not been checked and no Part B records
exist for this survey ID number.
• Zero Openings
– A zero has been entered into the “number of openings” field.
Tracking Reports
A report of how many surveys have been
received by:
•
•
•
•
•
•
•
Area
Date
Size
Round
Industry
Ownership
Vacancy Response
Management
• Add/Delete Users
• Generate Contact List
• Generate Mailing List
• Setup Data Tables
• Link Data Tables
• Import Optional Table
• Generate Results Mailing List
JVS Sample Allocation and
Selection
Christina Chiu
Bureau of Labor Statistics
Statistical Methods Staff
Find more on this in About JVS, Sampling
Methodology.
Sampling
•
•
•
•
•
Sampling Frame
Scope
Stratification
Neyman Allocation
Allocation
– Population variance
– Sampling variance
– Relative standard error
• Software
– Validation of popsamp.txt
Sampling Frame
• Your ES-202 File
•
•
•
•
one quarter of data
one record per establishment
21F EQUI format (see 3.1 of user’s guide)
record length of 725 or 801
Limiting Your Scope
• Definition of Scope
– The target population or universe that
you want to make an inference or draw
conclusions.
– If you do not limit your scope, your
universe is your ES-202 file.
– If you do limit your scope, your universe
is a subset of your ES-202 file.
Limiting Your Scope
• Purpose of Scope
– To include or exclude establishments in the
sample based on certain criteria
• Example: You may want your sample to only
include private establishments, but you want to
exclude temp agencies. Your scope would include
private establishments. Temp agencies would be
out of scope.
Limiting Your Scope
•
Limiting your scope options
1. What if I do not limit my scope?
All establishments on the ES-202 file will be
considered.
2. You may define your scope by OES Scope
or CES Scope - see 1.1.6 of user’s guide
In addition to defining your scope by OES or
CES, you may further define your scope by your
own constraints.
3. You may define your scope by your own
constraints.
Limiting Your Scope
• Scope constraints
•
•
•
•
Employment size of establishment
Industry
Ownership
Geography
Limiting Your Scope
• Scope constraints by Employment
Lower
Bound
Upper
Bound
Scope
Not
Not
Selected Selected
All establishments regardless of
employment size can be included in
the sample.
5
Only establishments with at least 5
employees can be included in the
sample.
Not
Selected
Not
250
Selected
Only establishments with 250 or less
employees can be included in the
sample.
5
Only establishments with at least 5
employees and 250 or less employees
can be included in the sample.
250
Limiting Your Scope
• Scope Constraints by Industry
•
•
•
•
•
•
•
•
•
2-digit SIC
3-digit SIC
4-digit SIC
2-digit NAICS
3-digit NAICS
4-digit NAICS
5-digit NAICS
6-digit NAICS
Combined sector NAICS
Limiting Your Scope
• Scope Constraints by Ownership
– Private sector
– Federal government
– State government
– Local and municipal
Limiting Your Scope
• Scope Constraints by Geography
– County
– Town
– User-defined area
• you must supply your own area file
• see 1.1.3 of user’s guide
Defining Your Strata
• Definition of Strata
– The universe or population is divided into
nonoverlapping groups or strata.
• Purpose of Strata
– To reduce the error that comes from sampling
a population
• Can produce more reliable estimates with
homogeneity
Defining Your Strata
• Strata constraints
–
–
–
–
Employment
Industry
Ownership
Geography
• Stratifying Do’s
– Do stratify by employment size and industry.
– Do stratify by area if you are using an area file.
• Stratifying Don’ts
– Don’t stratify by industry alone.
– Don’t stratify by employment size alone.
– Don’t overstratify - consequence is empty cells
• Example: Avoid stratifying by many size classes
• Example: Avoid stratifying by very specific industries
Defining Your Strata
• Strata constraints by employment
•
•
•
•
0-19, 20-249, 250+
0-4, 5-49, 50, 249, 250+
0-9, 10-49, 50-249, 250,999, 1000-5999, 5000+
0-4, 5-9, 10-19, 20-49, 50-99, 100-249, 250-499, 500999, 1000+
• 0-9, 10-19, 20-49, 50-99, 100-249, 250-499, 500-999,
1000+
• 0-9, 10-19, 20-49, 50-99, 100-249, 250+
• 0-4, 5-9, 10-19, 20-49, 50-99, 100-249, 250+
• If you want to sample establishments with at least one
employee, limit your scope to have a lower bound of
employment of 1.
Defining Your Strata
• Strata constraints by employment
Example: If you plan on sampling 2000
establishments, and your strata is only constrained
by 4-size employment, then your 2000 units will be
distributed among 4 strata.
Strata 1
Strata 2
Strata 3
Strata 4
Size
1
2
3
4
Defining Your Strata
• Strata constraints by
industry
Example: If you plan on
sampling 2000
establishments, and your
strata is only constrained
by employment (4-size)
and 24 industries (2-digit
NAICS sectors), your 2000
units will be distributed
among 96 strata.
Size
Industry
Strata 1
1
Ag
Strata 2
2
Ag
Strata 3
3
Ag
Strata 4
4
Ag
Strata 5
1
Mining
Strata 6
2
Mining
Strata 7
3
Mining
Strata 8
4
Mining
Strata 9
1
Construction
and so on…up to Strata 96
Defining Your Strata
• Strata constraints by ownership
• Strata constraints by geography
– Recommended if you are using an area file
Allocation
• How are units allocated across strata?
• Neyman allocation
– A type of stratified random sampling
– Puts more units in strata that have large Nh or where Sh is more
variable.
nh  n 
N h Sh
N
h
Sh
h
nh is the sample size per stratum.
n is the total sample size.
Nh is the population within a stratum.
Sh is the population standard deviation per stratum.
Allocation
Neyman allocation example
Say you want to take a sample across 5 strata, and you want the total sample
size to be n=60.
Stratum
Nh
Sh
A
44
3.02
B
168
3.77
C
56
5.29
D
16
7.26
E
27
4.58
nh
Wh
N = 311
How do I distribute the sample of n=60 across the five strata?
Allocation
Neyman allocation example
n = 60
nh  n 
Nh Sh
N
h
Sh
h
where
N
h
S h  ( 44  3.02)  (168  3.77)  (56  5.29)  (16  7.26)  ( 27  4.58)  1302.3
Stratum A
nh  n 
Stratum B
Nh Sh
 Nh Sh
nh  n 
Nh Sh
 Nh Sh
h
n h  60 
Stratum C
nh  n 
Nh Sh
N
h
( 44)(3.02)
1302.3
n h  6.12  6
n h  60 
h
Sh
h
(168)(3.77)
1302.3
n h  29.18  29
n h  60 
(56)(5.29)
1302.3
n h  13.65  14
Allocation
Neyman allocation example
n = 60
nh  n 
NhSh
N
h
Sh
h
where
N
h
S h  ( 44  3.02)  (168  3.77)  (56  5.29)  (16  7.26)  ( 27  4.58)  1302.3
Stratum D
nh  n 
Stratum E
Nh Sh
N
h
Sh
nh  n 
Nh Sh
N
h
n h  60 
(16)(7.26)
1302.3
n h  5.35  5
h
Sh
h
n h  60 
( 27)(4.58)
1302.3
n h  5.70  6
Allocation
Neyman allocation example
Results
Stratum
Nh
Sh
nh
Wh= Nh/nh
A
44
3.02
6
7.333
B
168
3.77
29
5.793
C
56
5.29
14
4.000
D
16
7.26
5
3.200
E
27
4.58
6
4.500
Population Variance or Sh
• Definition
– The extent to which the population units or Nh differ
from each other.
• We do not have the true population variance or
Sh, so we estimate Sh by using the estimated
population variance or sh.
• In most cases, we do not have the estimated
population variance of job vacancies because
we do not have any estimates of job vacancies.
Population Variance
• What does the software use for the
estimated population variance of job
vacancies?
– User defined population variance file
– The software assumes that the estimate of job
vacancies is 3% of the employment. The
software calculates the estimated variance of
job vacancies based on this assumption.
Population Variance
• Minimizing sampling error options
– By variance in employment
• When you select this option, you are using variance in
employment as a proxy for the estimated population variance
of job vacancies.
– By mean in employment
• When you select this option, you are using the mean in
employment as a proxy for the estimated population
variance of job vacancies.
• The variance in employment and mean in employment
options provide very similar results.
– By user provided file
• Select this option if you provided your own population
variance file. See 1.1.3 of user’s guide.
Sampling Variance
• Sampling variance
– Definition: The extent to which the population
units or Nh differ from each other.
– Different from population variance
– Minimum of 2 units per strata required to
calculate sampling variance
• Recommend a minimum of 3 units per strata to
account for non-response.
Sampling Variance
• Sampling Variance formula
– Sampling error is equal to the square root of the
sampling variance
– A component of the relative standard error
se(Yˆst )  V (Yˆst ) 
N
h
( N h  nh )
S
2
h
nh
Nh-nh is the finite population correction factor. When
Nh=nh, the estimated sampling variance is zero because
all units in the population are included in the sample.
Relative Standard Error
• What is relative standard error (rse)?
– RSE is your overall sampling error divided by your
estimate of job vacancies.
– The lower the sampling error, the more reliable your
estimate.
– Example: Let’s say that 5,000 job vacancies have
been estimated, and the relative standard error is 5%.
We can say, with 95% confidence, that the “true”
number of job vacancies lie in the interval (4500,
5500).
C.I. = estimate ± 2(sampling error)
= 5000 ± 2(5000*.05)
Relative Standard Error
• Relative standard error formula
rse 
samplinger ror
jobvacancyestimate
rse 
V (Yˆst )
(.03)employment
Allocation
• Two methods of sample selection
– Sample size
• This option is more common since you usually know how
much you can afford to sample.
• Software will not increase sample size due to response rate.
– Example: If you can afford to collect 3000 units and you expect
your response rate to be 50%, then select a sample size of
6000.
– Target relative standard error
• If this option is chosen, a target relative standard error of
15% or less is recommended.
• Can result in high sample size
Allocation
• Target relative standard error by strata
•
•
•
•
By employment size strata
By ownership strata
By industry strata
By geography strata
– This is different from setting an overall target
relative standard error.
– This option is not recommended because we
are setting the desired precision for each
strata.
Allocation
• Certainty units
– By Employment
• If you select this option, all establishments in your
largest size class will be sampled.
– Example: If you select this option and you selected 4-size
classes, then all establishments with 250 or more
employees will be sampled.
– By Ownership
– Example: If you select this option and choose “state,”
then all state government establishments will be
sampled.
Allocation
• Response Rate
– Enter your expected response rate.
– The software assumes the same response rate
across all strata.
– If you selected the sample size option, response rate
does not greatly affect your allocation.
• You must calculate your sample size outside of the system.
• Example: If you can afford to collect 3000 units and you
expect your response rate to be 50%, then select a sample
size of 6000.
– If you selected the target rse option, a low response
rate will generate a larger sample than would a high
response rate.
Software
• Required software:
– JVS Sample Allocation and Selection (JVSSASS)
software created by BLS
– SAS Version 6 or higher
• Required input files
– ES-202 EQUI file
• Optional input files
– User-provided area file
– User-provided population variance file
Software
• Output Files
– popsamp.txt
• verify sample size and scope
• Data Capture System input
– popdata.txt
• Data Capture System input
– jvasel.log
• check for errors
– tblMacroVar.txt
• your “picks”
– jvasel.lst
Validation of popsamp.txt
•
•
•
•
•
Open Excel. View files of all type.
Popsamp.txt is in the job vacancy folder.
Use comma delimiter
View and manipulate values up to three decimal places.
Verify that the overall sample size, or sum of Column K,
is the sample size that you specified in the software.
• Verify that the certainty units you specified have been
selected.
• Verify that you all your sample units are in the scope
you defined.
Validation of popsamp.txt
Column A
own
“0” means that you did not stratify by
ownership
Column B industry
Industry identifier
Column C size
Size identifier
Column D geo
“0” means that you did not stratify by
geography
Column E strata
Strata identifier
Column F
Population per strata
popct=Nh
Column G semp
Sum of employment per strata
Column H memp
Mean of employment
Column I
vemp
Variance in employment
Column J
sd_emp
Standard deviation in employment
Column K sample=nh
Sample size per strata
Column L
Sampling weight per strata
weight=Nh/nh
Job Vacancy Survey:
Estimates Production System
Jeffrey T. Willingham
U.S. Bureau of Labor Statistics
Find more on this in About JVS, Job Vacancy
Survey Estimates Production System.
181
Contents
•
•
•
•
•
•
•
•
•
System Requirements
Graphical User Interface
Data
Screening of Data
Adjustment of Data
Estimation
Confidentiality
Future Versions
Demonstration
System Requirements
•
•
PC with Microsoft Windows™ or NT™
SAS™ v.8
Graphical User Interface
Data
Variable
FIPS
Schnum
CheckDIg
OriWgt
CurWgt
Status
Source
Ori_Emp
Rpt_Emp
County
Town
NAICS
ESOwner
Vac_N_Y
AllocStrata
Area_Type
Spec_Area
Estab
Type Length
character 2
character 5
character 1
numeric 9
numeric 9
character 2
character 1
numeric 6
numeric 6
character 3
character 3
character 6
character 1
character 1
character 12
character 2
character 6
Detail
Decimal Places Variable
Type Length
NA
FIPS
character 2
NA
Schnum
character 5
NA
CheckDig character 1
3
Occ_Code character 7
3
Vacancies numeric 6
NA
PartFull
character 1
NA
Perm_Seas character 1
none
How_Long character 1
none
Edu_Lvl
character 1
NA
Exp_Lvl
character 1
NA
Min_Wage numeric 8
NA
Max_Wage numeric 8
NA
Benefit1
character 1
NA
Benefit2
character 1
NA
Benefit3
character 1
NA
Benefit4
character 1
NA
Benefit5
character 1
Benefit6
character 1
Decimal Places
NA
NA
NA
NA
none
NA
NA
NA
NA
NA
2
2
NA
NA
NA
NA
NA
NA
Screening of Data
Basic Data Edits
•
•
•
•
•
Input: ‘detail’ and ‘estab’ data sets.
Variables: All.
Identifies any discrepancies between the reported data and their expected
characteristics.
If discrepancies are present, then they will be flagged and shown via an
outputted report.
The user will be required to modify any discrepancies using the Data
Capture System.
Screening of Data
Outlier Detection
•
•
•
•
•
Input: ‘detail’ and ‘estab’ data sets.
Variables: Wgt_Diff, Emp_Diff, Min_Wage, Max_Wage, and Vacancies.
Wgt_Diff = |OriWgt - CurWgt|
Emp_Diff = |Ori_Emp - Rpt_Emp|
Utilizes the Quartile Method.
Adjustment of Data
Unit Non-Response
•
•
•






Input: ‘estab’ data set.
Variables: Status.
Utilizes the Non-Response Adjustment Factor:
Viable units are defined as those whose Status code ranges from ‘20-29’
and ‘60-99’ [‘20-29’ = refusals, ‘60-79’ = non-response and data unusable].
Useable units are defined as those whose Status code ranges from ‘80-99’
[‘80-99’ = usable partial response and usable complete response].
V_Emp_i = the weighted employment of Viable records in Size i.
U_Emp_i = the weighted employment of Useable records in Size i.
V_Emp_0 = the weighted, aggregate employment of Viable records.
U_Emp_0 = the weighted, aggregate employment of Useable records.
NRAF
uncollapse d

V _ Emp _ i
U _ Emp _ i
NRAF
collapsed

V _ Emp _ 0
U _ Emp _ 0
Adjustment of Data
Item Non-Response
•
•
Input: ‘detail’ data set.
Variables: How_Long, Perm_TS, Edu_Lvl, Exp_Lvl, Min_Wage, Max_Wage,
Benefit1-Benefit6, and Cert_License.
• Utilizes Nearest-Neighbor Hot-Deck Imputation:
 A distance function, specific to the variables, is defined to locate the nearest
possible neighbor.
 Once a donor is found, it is used to impute for the missing item.
 For this imputation process, successively larger cells are defined until a
donor is located, i.e., imputation begins at the very specific and continues to
the broad.
Estimation
Estimation Levels
geography
A (all)
C (county)
S (specified area)
job type
A (all)
L (by Full/Part Time)
industry
size
A (all)
A (all)
1 (1-digit NAICS)
4 (4-size)
2 (2-digit NAICS)
S (specified size)
3 (3-digit NAICS)
4 (4-digit NAICS)
5 (5-digit NAICS)
6 (NAICS Sector)
7 (NAICS SS)
C (CES NAICS Sector)
D (CES NAICS SS)
C (OES NAICS Sector)
D (OES NAICS SS)
S (specified industry)
experience
education
A (all)
A (all)
L (by experience levels) L (by education levels)
ownership
A (all)
E (ES-202 Ownership)
occupation
A (all)
M (MOG)
D (Detailed SOC)
S (specified occupation)
Estimation
Estimates
•
•
•
•
•
Total Employment
Total Vacancies
Total Vacancy Rate
Population Units
Reporting Units
•
•
•
•
•
Occupational Employment ¹
Estimated Occupational
Vacancies
Occupational Vacancy Rate ¹
Turnover Adjusted Demand ¹
Number of Sampled
Establishments Reporting the
Occupation ²
¹ These statistics will only be available if other data sources are used to
supply estimates of occupational employment and turnover.
² That possess the characteristic at the estimating level -- not weighted.
Estimation
Estimates (continued)
•
•
Job Type
Proportion of Vacancies that are
Permanent
Proportion of Vacancies that are
Temporary/Seasonal
•
•
•
•
Time Open
Proportion of Vacancies that are
Open Less Than 30 Days
Proportion of Vacancies that are
Between 30 Days and 60 Days
Proportion of Vacancies Open
More Than 60 Days
Proportion of Vacancies Always
Open
Estimation
Estimates (continued)
•
•
•
•
•
•
Education
Proportion of Vacancies Requiring
No Diploma
Proportion of Vacancies Requiring
HS/GED
Proportion of Vacancies Requiring
Vocational Education
Proportion of Vacancies Requiring
a Two-year Degree
Proportion of Vacancies Requiring
a Bachelor’s Degree
Proportion of Vacancies Requiring
an Advanced Degree
•
•
•
•
Experience
Proportion of Vacancies Requiring
No Experience
Proportion of Vacancies Requiring
Work Experience
Proportion of Vacancies Requiring
Related Experience
Proportion of Vacancies Requiring
Occupational Experience
Estimation
Estimates (continued)
•
•
•
•
•
•
Benefits
Proportion of Vacancies with
benefit 1 (health insurance)
Proportion of Vacancies with
benefit 2 (paid sick leave)
Proportion of Vacancies with
benefit 3 (paid vacation)
Proportion of Vacancies with
benefit 4 (retirement savings plan
or pension)
Proportion of Vacancies with
benefit 5 (no benefits offered)
Proportion of Vacancies with
benefit 6 (no response)
•
•
•
Hourly Wage
Mean Wage for Vacancies
Minimum Wage for Vacancies
Maximum Wage for Vacancies
Estimation
Sample Formulas
Total Employment
D,estab
 Total_Emp
D,estab
 (CurWgt

*NRAF i*Rpt_Emp
i
i
)
i D,estab
Total Vacancies
 Total_Vaca ncies
D,estab
D,estab
 (CurWgt

*NRAF i*Vacancies
i
i
)
i D,estab
Vacancy Rate
Estimated
D,estab
 Vacancy_Ra te D,estab 
Occupation
al Vacancies
D,detail
Total_Vaca ncies
Total_Emp
D,estab
D,estab
 Occ_Vac_Es t D,detail 
 (CurWgt
i D,detail
Occupation
al Vacancy Rate
D,detail
 Occ_Vac_Ra
te D,detail 
Where Occ_Emp_Es t D,detail comes from a user input file.
Occ_Vac_Es t D,detail
Occ_Emp_Es t D,detail
i
*NRAF i*Vacancies
i
)
Confidentiality
• Similar to the OES Confidentiality guidelines:
 Flag with ‘0’ if there are less than 3 respondents.
 Flag with ‘0’ if the unweighted employment of the largest respondent is
greater than 50% of the weighted employment.
 Flag with ‘0’ if the unweighted employment of the two largest respondents is
greater than 75% of the weighted employment.
 Flag with ‘1’ otherwise.
Future Versions
• Atypical Reporter Review and Adjustment
 This will implement weight adjustments to account for consolidated and
disaggregated reports.
 A consolidated report occurs when more than one unit is sampled and the
employer provides a combined report for all of them.
 A disaggregated report occurs when one unit is sampled but the employer
provides multiple reports that constitute this or a larger unit.
• Variance Estimation
 Variance estimation provides a measure of the degree to which an estimate
varies about the mean of all possible estimates.