Transcript COMPETENCY MODELING - Personnel Testing Council of

Broad-Based Office
Skills Testing
Los Angeles County Office of Education
PERSONNEL COMMISSION
The Problem

Many clerical/secretarial classifications



Each class has a unique examination




Differ on specific job content and levels of responsibility
Many obstacles to class consolidation - logistical, political,
fiscal
Variably content and quality
Not consistently updated
2-3 month recruitment & examination cycle time
No overall competency architecture across the classes


Core competencies and competency progression
Pass point paradoxes (lower pass points for higher level
jobs based on selection ratios)
Purpose and Objectives





Improve testing quality, reliability, and
efficiency
Improve coherence of clerical examination
plans
Reduce redundant testing
Reduce time to place candidates on eligibility
lists
Ensure eligibility lists are always available
Strategy





Core test battery for as many classifications as
possible
Annual calendar of assessment opportunities
(e.g., quarterly)
Computer-based testing
Score banking and certification to different
exam plans
Different weights and cut-off scores for different
classifications based upon relevancy and level
needed.
Strategy:
Similar to SAT & ACT for applying to college
1.
2.
3.
4.
Sign up and take
the test
Apply to the
colleges of choice
Send scores
Colleges set their
own pass-points
and weights for final
selection decisions
1.
2.
3.
4.
Sign up and take
the test
Apply to the jobs of
choice
Scores populated
Each job has its
own pass-points
and weights for final
selection decisions
Score Data Flow
Apply to Take
Clerical Battery
(10 scores)
Exam Plan For
Battery Score
Bank
Apply to Job
Title
Electronic
Certification of Scores
to each Exam Plan3
Exam Plan For
Job Title A 1
Exam Plan For
Job Title B 1
(weights and pass
points for each
score)
(weights and pass
points for each
score)
Eligibilty List A
Eligibilty List B
Electronic Score
Transfer 2
Take Battery –
CBT and
Scanton
Development of the Test Battery






Reviewed job analyses from Agency-Wide Classification
Study - 2004-2007
Re-stated duties into “generalized work activity” (GWA)
statements with no reference to specific job content.
Map GWAs to each classification
Identify relevant assessment tools for GWAs
Via GWAs, map assessment tools to Classifications
Derive relevancy and weight of assessment tools to
classifications.
Example GWAs









Greet, interact, and provide information to others in person and on the
telephone
Verify, input, and correct data and information
Receive, sort, amd categorize information and materials
Understand, interpret, explain, and follow directions, rules, and
regulations
Operate a personal computer and other common electronic office
equipment
Format, edit, keyboard and type documents.
Gather compile, collate, and assemble information and materials
Maintain and coordinate calendars and schedule meetings and
events
Receive, track, and record monetary/numerical information and data
Matrix of GWA with Classifications
Typist Clerk
Secretary
Accounting
Clerk
Receptionist
Greet, interact, and
provide information to
others in person and
on the telephone
1
2
1
2
Verify, input, and
correct data and
information
2
1
2
0
Receive, sort, amd
categorize information
and materials
2
2
2
1
Understand, interpret,
explain, and follow
directions, rules, and
regulations
2
1
2
1
Class
GWA
Competencies Assessed - Tests
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Following Instructions (MC)
English Usage & Grammar (MC)
Microsoft Outlook (MC)
Microsoft Excel (Performance, OPAC)
Microsoft Word Skills (Performance, OPAC)
Telephone - Customer Service (SJ, OPAC)
Sequencing and ordering speed and accuracy (timed)
Computational speed and accuracy (timed)
Checking and comparing speed and accuracy (timed)
Data Entry speed and accuracy (OPAC)
Matrix of Tests with GWAs
Test
GWA
Greet, interact, and
provide information to
others in person and
on the telephone
Verify, input, and
correct data and
information
Receive, sort, and
categorize information
and materials
Understand, interpret,
explain, and follow
directions, rules, and
regulations
Telephone
Customer
Service
Data Entry
speed and
accuracy
Sequencing
and ordering
speed and
accuracy
Following
Written
Instructions
x
x
x
x
x
Matrix of Tests with Classifications
Typist
Clerk
Secretary
Telephone
Customer
Service
1
2
1
2
Data Entry
speed and
accuracy
2
1
2
0
Sequencing and
ordering speed
and accuracy
2
2
2
1
Following
Written
Instructions
2
1
2
1
Class
Test
Accounting Receptionist
Clerk
Result


Linkage of tests-to-tasks-to-classifications to establish
job relatedness
Derived weighting scheme for each classification



Relevant tests
Relevant weights
Job specific competencies identified in the job analyses
are assessed in specialized examination parts



Min Quals
Interview
Performance
Applicable Classifications
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
Clerk
Intermediate Clerk
Senior Clerk
School Clerk
Senior School Clerk
Data Control Clerk
Senior Data Control Clerk
Typist Clerk
Intermediate Typist Clerk
Temporary Office Worker
Senior Typist Clerk
Department Assistant, Dance
Department Assistant, Music
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
Department Assistant, Theater
Department Assistant, Visual Arts
Receptionist
Reader
Information Resources Specialist
Media Dispatching Clerk
Secretary
Division Secretary
Legal Secretary
School Administrative Secretary
Senior Division Secretary
Executive Legal Secretary
Executive Assistant
Affected Positions
Classification
# LACOE Positions
% of Clerical Positions
Senior School Clerk
110
27%
Secretary
78
19%
Division Secretary
46
11%
Senior Typist Clerk
35
9%
School Administrative
Secretary
30
7%
Intermediate Typist Clerk
28
7%
Senior Division Secretary
24
6%
TOTAL
351
87%
Administrative Practices

Score Life:


Test Administration:


Once every 6 months
Retake Scope:


Every 3 months
Test Retake:


2 years
Whole Battery – not individual tests for logistical reasons
Weighted or Qualifying-Only:


Qualifying Pass/Fail-only for Promotional Exams
Weighted for Open recruitment
Administrative Practices

Effective Scores:


Only the candidate’s most recent score, not the highest
score, can be certified to a list.
For candidates whose scores enable them to be placed
on an eligibility list (pass the exam), the certified scores
stands for the duration of eligibility on the respective list.
• Scores already certified are not updated by retakes
• One’s score (and relative ranking) on a list can not be improved
until eligibility expires (one year).

Multiple (10) Pass Points



Low individual test pass points
Reduced Adverse Impact
3 apparent factors:
• Computer skills, speed and accuracy, verbal
Recruitment

Continuous offering of the BBC



Continuous recruitment for all affected classifications
Establishes a pre-tested pool of potential candidates for
the affected classes
For most classifications, limited or no public posting for
job specific recruitments


Notifications sent to pre-tested pool, based on test cut-off
score
Invitations to apply to take the specialized exam parts for
the classification
No need to go to the general population for
each recruitment
Recruit
for Job A
General
Recruit To Take
BBC
Population
PRE-TESTED
BBC POOL
Recruit
for Job B
Recruit
for Job C
Status

Officially launched on February 9th, 2009

Advertised on our PC website & Weekly Job
Opportunities emails.
Two exam administrations in March & April
phased in the BBC in the context of specific
classification recruitments.
 June and October 2009 BBC administrations as
part of annual calendar

Process Improvements Since Start





Candidates are automatically notified via email when
particular job classifications are open for recruitment.
Candidates receive their score report along with an
Examination Information Sheet to address frequently
asked questions.
Scores can (soon) be viewed on-line with ID and
Password
Candidates who miss the Scantron exam portion can reregister to take the next exam administration.
Candidates who are unable to attend the first Computer
Exam can attend the Computer Make-up exam typically
scheduled the following week.
Benefits Received and Expected



Test quality improvement through
 Data monitoring and psychometric analysis
 Empirical validation research
 Development of alternate forms
Administration efficiencies through
 Managed efficiencies in test administration and score
managements
 Minimizing retesting of the same candidates for different
classifications
 Improved communications with candidates
Recruitment enhancement through
 Maintaining a pre-tested pool of applicants
 Having something to offer at job fares
 Reduced eligibility list production cycle time
Next Project: BBA
Broad-Based Administrative Classifications
 Administrative, Analyst, and Professional Series
 Focus on college grads
 Same general model
 Launch in Q1, 2010

Thank you!
Questions?
Comments?