Transcript Slide 1

WWW.FPSI.COM
Maintaining Physical Standards
Using Physical Ability Tests: Are
Your Incumbents Fit for the Job?
August 26, 2011
Dan Biddle, PhD
Stacy L. Bell, MS
Alisa Arnoff, Esq.
Fire & Police Selection, Inc. (FPSI) © 2011
About the Speakers
• Dan Biddle, PhD—President/CEO of Fire & Police
Selection, Inc. (FPSI) and Biddle Consulting Group
• Stacy L. Bell, MS—Executive Vice President of Fire & Police
Selection, Inc.
• Alisa Arnoff, Esq.—Attorney with Scalambrino & Arnoff,
LLP
Fire & Police Selection, Inc. (FPSI) © 2011
Presentation Overview
• Review of critical firefighter physical abilities
• Comparison of fitness testing vs. work-sample testing
• Legal pitfalls associated with arbitrary fitness tests and the
risks associated with employing inappropriate standards at
the incumbent level
• Appropriate techniques to use for setting PAT cutoff scores
for new hires and for incumbents
• Importance of “maintenance/wellness” testing of
incumbents
Fire & Police Selection, Inc. (FPSI) © 2011
Critical Firefighter Physical Abilities
• 1994-1996 Firefighter Physical Ability Validation Study
◦ 331 Fire Suppression Personnel from 41 fire departments identified
the critical physical duties performed by fire suppression personnel:
– Hose drags (dry and charged)
– Ladder raises, removal, and carries
– Walking and operating on ladders
– Searching for fire extension
– Removing conscious/unconscious victims from fire scene
– Ventilation techniques
– Climbing stairs while carrying tools/equipment
– Hoisting operations
Fire & Police Selection, Inc. (FPSI) © 2011
Comparing Fitness Tests to Work-Sample
Tests
•
Examples of Fitness Tests
◦
◦
◦
◦
◦
◦
◦
•
1.5 mile run
Push-ups/Pull-ups
Sit-ups
Aerobic Capacity – Sub Maximal Stress Test
Muscular Endurance & Strength (Arms & Legs)
Flexibility Measurements (Shoulder, Trunk, & Legs)
Body Fat Composition and BMI
Samples of Work Sample Tests
◦
◦
◦
◦
Hose drags: http://fpsi.com/fire/video/fsi1.mpg http://fpsi.com/fire/video/fsi2.mpg
Walking/working on ladders: http://fpsi.com/fire/video/fsi4.mpg
Removal of unconscious/conscious victim: http://fpsi.com/fire/video/fsi7.mpg
Climbing stairs with tools/equipment: http://fpsi.com/fire/video/fsi11.mpg
Fire & Police Selection, Inc. (FPSI) © 2011
What’s the Big Difference Between a Fitness
Test and a Work Sample Test?
• Fitness Tests measure a person’s fitness (in general) and then make the
inference that “if they have fitness level X they should be able to perform
the job of Y”
• Some fitness tests (e.g., VO2 Max/Cardio fitness) require factoring age
and gender into the equation
◦ This can be tricky with various employment discrimination laws and
federal enforcement agencies
• Work Sample tests directly measure whether applicants/incumbents can
perform the job task.
◦ They answer the question: “Irrespective of this person’s age, gender,
race, or disability, can they do the job?”
Fire & Police Selection, Inc. (FPSI) © 2011
NFPA 1583: Standard on Health-Related
Fitness Programs for Firefighters
• In August of 2000, the NFPA published the current standard on healthrelated fitness programs for firefighters. The standard requires:
◦ Fire departments shall require structured participation of the healthrelated fitness program
◦ All members shall participate annually (at least) in a fitness
assessment which measures:
–
–
–
–
–
Aerobic capacity
Body composition
Muscular strength
Muscular endurance
Flexibility
Fire & Police Selection, Inc. (FPSI) © 2011
The Reality of Firefighter Fitness Programs
YES
Population of Community
NO
TOTAL
Number
Depts.
Percent
Number
Depts.
Percent
Number
Depts.
Percent
1,000,000 or more
8
61.5
5
38.5
13
100
500,000 to 999,999
27
71.1
11
28.9
38
100
250,000 to 499,999
38
59.4
26
40.6
64
100
100,000 to 249,999
115
53.5
100
46.5
215
100
50,000 to 99,999
243
49.9
244
50.1
487
100
25,000 to 49,999
501
47.6
552
52.4
1,053
100
10,000 to 24,999
962
33.8
1,881
66.2
2,843
100
5,000 to 9,999
873
24.1
2,756
75.9
3,629
100
2,500 to 4,999
798
17.5
3,774
82.5
4,572
100
Under 2,500
1,639
12.2
11,801
87
13,440
100
TOTAL
5,205
19.8
21,149
80.2
26,354
100
Source: United States Fire Administration, Survey of the Needs of the U.S. Fire Service – 2002
Note: Based on 8,267 departments reporting. Numbers may not add to totals due to rounding.
Fire & Police Selection, Inc. (FPSI) © 2011
The Legal Landscape—In Plain English
• A Fire Department cannot discriminate on the basis of race,
gender, age, religion, national origin, disability, etc.
• Intent is not a required element of a discrimination claim.
• A testing program applied equally to all may be
discriminatory if it screens out “too” many members of a
protected class (has an adverse or disparate impact).
Fire & Police Selection, Inc. (FPSI) © 2011
Disparate or Adverse Impact
• An employer uses a test that is facially neutral, but has an
unjustified adverse impact on members of a protected class
by screening them out.
◦ Every one is given the same test, but members of one or more
protected class do not fare as well as others.
• How many is “too” many?
◦ The 4/5ths or 80% Rule
– Pass rate of the minority group ≥ 80% of the pass rate of the
most successful group.
– Example
• Whites perform best, with a 60% pass rate.
• Possible disparate impact exists if African Americans passed
at rate less than 48% (60% x .8).
Fire & Police Selection, Inc. (FPSI) © 2011
10
Sources of Law
• Federal
◦ Title VII of the Civil Rights Act
◦ The Age Discrimination in Employment Act
◦ The Americans with Disabilities Act
◦ 42 USC §1981, §1983
◦ U.S. Constitution, Equal Protection clause
– 14th amendment, §1
• State and Local
Fire & Police Selection, Inc. (FPSI) © 2011
11
Title VII of the Civil Rights Act
• Prohibits discrimination on the basis of gender in all aspects
of the employment relationship.
– Uniform Guidelines on Employee Selection provide
that physical testing of incumbents must be based on
“professional accepted methods.”
• Must show test is job-related for the position in
question and consistent with business necessity.
Fire & Police Selection, Inc. (FPSI) © 2011
Gender Norming
◦ Having a different set of qualifications given the
immutable physiological differences between the genders
in an attempt to ensure that qualified members of both
sexes are selected, promoted or retained.
– Strength
– Aerobic capacity
Fire & Police Selection, Inc. (FPSI) © 2011
The Age Discrimination in Employment Act
• The ADEA of 1967 prohibits employers from discriminating
against an individual because s/he is age forty years or older.
◦ The ADEA was amended in 1996 to include a public safety
exemption:
– Fire Departments which demonstrate that the age
requirement is a bona fide occupational qualification
(BFOQ) can legally impose maximum age requirements.
Fire & Police Selection, Inc. (FPSI) © 2011
Reasonable Factors Other than Age
• Proposed RFOA regulations consider
◦ To what extent did the employer take steps to assess the
adverse impact on older workers?
◦ How severe is the harm, in terms of the numbers affected
and extent to which preventative or corrective steps are
taken to minimize impact?
◦ Are there other options available?
◦ Why did the employer selected the test it did?
Fire & Police Selection, Inc. (FPSI) © 2011
Age Norming
◦ Having a different set of qualifications given the physical
changes which occur as one ages to control for
concomitant decreases in muscular strength, endurance,
and aerobic capacity attributable to the aging process.
Fire & Police Selection, Inc. (FPSI) © 2011
The Americans with Disabilities Act
• Cannot discriminate against a disabled individual who can do
a job’s essential functions with or without a reasonable
accommodation.
• Cannot discriminate against someone
◦ Who has a record of disability; or
◦ Whom the employer perceives as disabled
Fire & Police Selection, Inc. (FPSI) © 2011
Constitutional Issues
• “No State shall…deny to any person within its jurisdiction
the equal protection of the laws.” (U.S. Const., Art. XIV)
• Gender-based classifications are subject to heightened
scrutiny and will overcome the constitutional challenge if
substantially related to an important governmental interest.
• Age-based classifications are subject to a less rigorous test, as
age, unlike gender, is not a suspect classification; an employer
must only show that the classification reasonably furthers a
legitimate state objective or interest.
Fire & Police Selection, Inc. (FPSI) © 2011
The Typical Testing Case
• Establish adverse impact
◦ Parties agree, or
◦ Plaintiffs present evidence
• FD shows job-related and consistent with business necessity
◦ Both sides offer expert testimony
• Plaintiffs show less restrictive alternatives available
◦ Both sides offer expert testimony
Fire & Police Selection, Inc. (FPSI) © 2011
A Typical Testing Case
• Female firefighters prove disparate impact concerning use of
physical exams
◦ Tests mostly for anaerobic traits (men excel) and not for
aerobic traits (women excel)
• FD justifies use of test
◦ Validated, as each portion of exam designed to test a
representative firefighting task
• Females unable to show a less restrictive alternative
Fire & Police Selection, Inc. (FPSI) © 2011
Physical Ability Testing Standards for
New Hires
• The Uniform Guidelines on Employee Selection Procedures
(1978) state:
◦ “Where cutoff scores are used, they should normally be set so as to be
reasonable and consistent with normal expectations of acceptable
proficiency within the work force. Where applicants are ranked on the basis
of properly validated selection procedures and those applicants scoring below
a higher cutoff score than appropriate in light of such expectations have little
or no chance of being selected for employment, the higher cutoff score may
be appropriate, but the degree of adverse impact should be considered.”
(Section 5H)
Fire & Police Selection, Inc. (FPSI) © 2011
Physical Ability Testing Standards for
New Hires
• FPSI’s standard approach is to utilize the “Modified Angoff Method.”
◦ The Modified Angoff method is the application of the modified method that
received acceptance before the United States Supreme Court in U.S. v. South
Carolina (1971).
◦ The Angoff method produces an average estimate of minimum
competency using opinions from several SMEs.
◦ The modification followed in U.S. v. South Carolina lowered the Angoff
average estimate by one, two, or three standard errors of measurement
(SEMs) after consideration of several statistical and human factors: the size
of the SEM, risk of error (risk of excluding a truly qualified candidate whose
low score does not show the real level of knowledge compared to the risk of
including an unqualified candidate whose low score does show an
unacceptable level of knowledge), internal consistency of the Angoff panel
(e.g., taken individually, the subject matter experts vary in their individual
estimates of minimum competency), supply and demand for at-issue jobs, and
the sex and race/ethnic composition of the at-issue jobs in the work force.
Fire & Police Selection, Inc. (FPSI) © 2011
Physical Ability Testing Standards for
New Hires
• A sample of incumbent fire suppression personnel
“norm” the physical ability test.
• Incumbents identify what a minimally qualified time
should be for the physical ability test (i.e., “normal
expectations of acceptable proficiency”).
• Average the mean opinion time and add one SEM to set
the final cutoff score.
Fire & Police Selection, Inc. (FPSI) © 2011
Physical Ability Testing Standards for
Incumbents (Firefighters and Fire Captains)
• Incumbent opinions are not used to set the cutoff score for
maintenance standards due to bias:
◦ FFs tend to stretch the time limit when they know it will be used to
monitor their job performance (i.e., they tend to overestimate the time)
• Incumbent maintenance standard cutoffs are based upon the
mean of the incumbent time in the “norming” process:
◦ Outliers omitted using a 1.645 SD
◦ Mean plus 1 Standard Error of the Mean + 1.645 Standard Errors of
Difference
• The outcome is the “normal expectations of acceptable
proficiency in the workforce”
Fire & Police Selection, Inc. (FPSI) © 2011
Sample Maintenance PAT Results
IncumbentHistogram
PAT Times
12
Mean = 423.15
10
SEM = 23.69
SED = 33.50
Frequency
8
14.6% “Remediate”
6
4
2
Mean = 423.1463
Std. Dev. = 71.41518
N = 41
0
720
700
680
660
640
620
600
580
560
540
520
500
480
460
440
420
400
380
360
340
320
300
PAT_Score
Fire & Police Selection, Inc. (FPSI) © 2011
Remediate Status of Incumbents
• A total of 6 officers clearly fell outside of the minimum proficiency levels
relating to physical ability expectations (i.e., 95% confidence interval that
these scores are reliably different from the average) and should be required
to improve their abilities through possible dietary changes, weight-loss
programs, and/or physical fitness programs.
• These incumbents fall into a “remediate” category and are asked to retake
the test after 10-16 week training program. The 10-16 week training
program should consist of both cardio-vascular and strength training in
the specific, fire suppression-related work behaviors that are measured by
the test.
• Departments can choose whether they want the training program to be
self directed or conducted by a department-designated exercise specialist.
Fire & Police Selection, Inc. (FPSI) © 2011
Remediate Status of Incumbents
• The 10-16 week remediate process may result in any of the
four possible consequences:
◦ Conditioning program—The incumbent is placed on a program that include
dietary modification and physical training.
◦ Leave of absence—The department may elect to place the incumbent on a
leave of absence until which time the incumbent is able to pass the test.
◦ Disability leave—The department may elect to place the incumbent on
disability leave until which time the incumbent is able to pass the test.
◦ Retirement with pension—The department may elect to terminate
employment with the incumbent following continued attempts to improve test
performance without success.
Fire & Police Selection, Inc. (FPSI) © 2011
The Effect of Age on Test Performance
•
Data from a study consisting of 256 incumbent fire suppression personnel
(mean age = 34.83 years) resulted in a correlation of .397*** when the age of the
incumbent was correlated to test time.
•
Data from a study consisting of 710 firefighter applicants (mean age = 29.47
years) resulted in a correlation of .149*** when the age of the firefighter
applicant was correlated to test time.
•
While age was statistically correlated with physical ability test performance in
both the incumbent and the applicant study, there are a number of other factors
likely contributing to the correlation:
– The motivation level of the applicants vs. the incumbents
– The physical fitness level of the applicants vs. the incumbents
Fire & Police Selection, Inc. (FPSI) © 2011
Importance of “Maintenance/Wellness”
Testing of Incumbents
• Ensuring Safety of the Crew
◦ Nearly 50% of all injuries to civilian firefighters in 2002 were a result
of sprains, strains, and muscular pain—whereby overexertion is
considered the primary causative factor. (NFPA)
◦ Nearly 50% of firefighter fatalities are heart attacks and about half of
those who died had heart-related issues. (NFPA)
◦ “Overweight, out-of-shape fire fighters are an accident waiting to
happen.” (NFPA, 2000)
• Ensuring Safety of the Public
• Reduce Worker Compensation Claims
◦ The estimated total annual cost of firefighter injuries is between $2.8 - $7.8
billion. (NIST, 2005)
Fire & Police Selection, Inc. (FPSI) © 2011
Questions???
• Contact us
◦ Dan Biddle, PhD., CEO/President Fire & Police Selection, Inc.
[email protected]
Phone: 916.294.4250 x. 113
◦ Stacy L. Bell, M.S., Executive Vice President Fire & Police Selection,
Inc.
[email protected]
Phone: 916.294.4242 x. 245
◦ Alisa Arnoff, Esq.—Attorney with Scalambrino & Arnoff, LLP
[email protected]
Phone: 312.629.0546
Fire & Police Selection, Inc. (FPSI) © 2011