Transcript Document 7237677

www.mubeena.net
The Situational Interview
www.mubeena.net
www.mubeena.net
Definition
Job-related situational questions
Situations are specific on-the-job activities
Applicants put in hypothetical situations
Questions can refer to past experience or
future intentions
Differs from structured format by
benchmarking applicant responses
www.mubeena.net
EvolutionTree
Redefinition of Selection Tool
“A valid test cannot be developed until the organization agrees
upon an acceptable definition (measure) of employee behavior.”
Latham & Wexley, 1982
Workforce Prediction
Quality of workforce will
deteriorate
Dyer, 1981
Legal Issues
Selection Process Dilemma
Griggs vs. Duke Power, 1971
www.mubeena.net
Research Literature
Latham, Saari, Pursell, & Campion, 1980
Examined reliability and validity of situational interview by
conducting three studies
Study 1: entry-level position, Study 2: first-line supervisory
position, Study 3: concurrent predictive validity
Results for Study 1: Interview scores were significantly
correlated with every performance criterion including the
overall global rating
Results for Study 2: Interview scores correlated significantly
with three of four criteria – safety, work habits, org
commitment
Results for Study 3: Interview scores significantly correlated
with composite job performance scores, even for women and
black employees
www.mubeena.net
Research Literature
Conclusions of Latham et al. (1980)
Intentions correlate with behavior
Comprehensive job analysis reflect content validity
Interviewee motivation influenced by face validity
Inter-rater reliability high because interviewers
themselves developed scoring key
Emphasis on critical behaviors rather than traits
Past behavior based items could lessen dishonest
responses, it can be verified by previous employer.
However, adverse impact must be kept in check
www.mubeena.net
Research Literature
Pulakos & Schmitt, 1995
Compared validities of experience-based (past performance)
questions and situational (future intentions) questions
Results: Past performance is better predictor of job
performance
Lack of Validity for Situational Items Attributed to:
1.
Responses were evaluated at end of interview. It is necessary
to make the ratings immediately
2.
•
Prior studies indicate that situational interviews involve
lower-level jobs. In this study, the job was complex and
demanding
Major conclusion was that interviews result in lower levels of
adverse impact because they measure cognitive as well as noncognitive performance dimensions
www.mubeena.net
Research Literature
Maurer & Fay, 1988
Two hypotheses: Rater training
Inter-rater agreement
Situational
Inter-rater agreement
Interpretation of results: Situational interviews are more
effective in producing higher inter-rater agreement, even with
little or no training, because of specific rating scales
There was so significant main effect of training
Situational interview is more cost-effective strategy in
comparison with conventional structured interview
www.mubeena.net
Critical Incident Technique
Was first developed and used by John Flanagan and his
students at the University of Pittsburgh in the late 1940s
and early 1950s
Used to identify job behaviors that differentiate
successful performance from unsuccessful performance
Can be useful in job analysis as well as training and
performance appraisal
The development of the graduate situational interview
was based on this technique
www.mubeena.net
Critical Incident Development
Critical incidents were identified by reviewing the job analysis
(CMQ)
Behaviors that could exemplify either good or poor behaviors of
graduate students were chosen
Fourteen job dimensions were identified:
Prioritizing
Stress management
Ability to work with others
Receiving feedback
Self Assessment
Decision making
Group involvement
Participation
Leadership
Ethics
Public speaking
Altruism
Diversity
Retention
www.mubeena.net
Development of Situational
Interview Questions
The critical incidents were used to form the questions to be asked in
the situational interview
The beginning of each question stem entailed a description of a
circumstance involving a critical incident
Example: Critical Incident – Decision Making
Example: You are placed at a company in which your supervisor is
critical and provides no direction to complete a task
The questions ended with a general proposition
Example: “How would you handle this situation?”
Seventeen questions were written using this format
Each critical incident that we identified had a corresponding
question
Two of the critical incidents had more that one corresponding
question, namely stress management and ethics
www.mubeena.net
BARS Development
To facilitate the scoring process a behaviorally anchored
rating scale was developed for each question
The behavioral scales were developed by brainstorming
and determining KSA relevance and importance as rated
by SMEs (job incumbents)
Our anticipation of the responses that we would receive
was used to develop a five point scale
The five point scales included examples of high, average
and low responses
The examples that we agreed represented high, average
and low responses were used as behaviors on the scale
www.mubeena.net
Example
5: Applicant tries to talk to the boss
4: Applicant tries to set own goals and
direction no resolve of conflict.
3: Applicant says nothing.
2: Applicant complains, does not start the
job until the issue is addressed.
1: Applicant leaves the job/talks to boss in
a negative fashion.
www.mubeena.net
Focus Groups
Our questions and behavioral scales were
presented to a focus group
The focus group (subject matter experts)
recommended the following:
- add multitasks to the scales
- reword ambiguous and vague questions
- make sure that each question corresponds to
the critical incident it is describing in a situation
www.mubeena.net
Scoring of the Situational
Interview
Each response on the behavioral scale had a corresponding
score
Scores were rated from one to five
- a 1 or 2 represented a low response
- a 3 represented an average response
- a 4 or 5 represented a high response
The interviewers choose the response that best represents
the interviewees answer
A total score for the interview can be obtained by
summing the ratings for each question
www.mubeena.net
Development of Cutoff Score
The Ebel method was used to determine the cutoff score
- rate each item (1-5)
- determine the percentage of items a minimally
qualified candidate would respond to correctly (a correct
response to each item represented a score of 3 or more)
- multiply this percentage by the number of items
(3 x 17 = 51)
- the cutoff score becomes 51 for the situational
interview
www.mubeena.net
Administration
Participants: 30 I/O graduate students
Two groups of raters: 3 and 2
Selection Tool was administered twice
over a 2 week period.
The Tool was administered in a classroom
setting with other selection tests being
administered at the same time.
www.mubeena.net
Possible Threats To Validity
Random Error:
Noise/Disruption Effects
 Exposure to test questions before
administration
 Fakeablity
 Rater effects(Halo, Contrasts, Leniency
Effects)

www.mubeena.net
Descriptive Data
Mean=64.21
Standard Deviation=5.78
Range=27.67
Skewness=-.845
www.mubeena.net
Score Distribution
7
6
5
4
3
2
Std. Dev = 5.78
1
Mean = 64.2
N = 30.00
0
45.0
50.0
47.5
interview avg
55.0
52.5
60.0
57.5
65.0
62.5
70.0
67.5
72.5
www.mubeena.net
Validity
Correlation with
-Graduate GPA
r=.284, p=.064
-Psychology 249
r=.360, p=.025
-Psychology 283A r=.381, p=.019
-Undergraduate GPA r=.252, p=.089
-GRE
r= -.007, p=.485
www.mubeena.net
Near Future
Standardize scores
Reliability and more validity analysis
Analyze individual questions
www.mubeena.net
Distant Future
How can we modify the interview to
make it a better instrument for selecting
graduate students?
separate information gathering and
evaluation, evaluate after gathering
 modify behavioral anchors
 re-categorize questions so that questions
which co-vary are in the same category
