Transcript Slide 1

PSY 6450, Unit 8
1




Today and Monday: Lecture
Wednesday, no class, T-Day
Monday, 12/02: E8
Wednesday, 12/04
 No lecture
 Return of E8
 Study objectives for ME2

Monday, 12/09: ME2, class meets at 5:00
2

Odd set of material
 Reid et al. (2011) published the only article that
has reviewed (to my satisfaction) staff training
and management programs
 Carbone staff incentive system – only on ppt
 Richman et al. has (a) a very nice simple
measurement system, (b) demonstrates that inservice training is not effective, and (c) selfmonitoring alone is probably not effective over
time
(ba students work in Hss, asked if I could include low-cost interventions; usually target client training, paperwork is very important)
3
 Parsons et al., model for a large scale PM and
systems intervention in residential treatment
facilities
 Green et al., a very cost-effective procedure to
improve the extent to which staff conduct
scheduled training sessions (structural analysis)
 Green et al., increasing the satisfaction of staff by
altering the staff person’s most disliked staff
(4 articles by Denny Reid, Carolyn Green, & Marsha Parsons and colleagues– read everything they have done, both PM and clinical; exquisite, innovative, and
of the highest quality; first challenges to the direct care staff, and then challenges for the organization, but of course there is overlap)
4
Staff management, while similar to OBM interventions in
business and industry, offers some unique challenges
 One is that few professionals in human services are trained
in staff management; rather they are trained to develop
effective training and behavior management programs for
their consumers
 There has been increasing recognition over the past 10
years that staff management skills are essential for
professionals in human services
 Most individuals who obtain graduate degrees to work in
human services, end up as supervisors or managers – they
do not implement the programs with the consumers
themselves, rather they supervise those that do
(most of sm studies have been conducted in group homes or residential facilities for DD; however, the results of that research 5
are clear: there is a need for individuals with expertise in staff mgt in all areas of human services. schools and programs that serve
children with autism, to name a few)


Furthermore, many professionals manage several
different units or programs within human service agencies
and some have started their own human service
organizations, but again they have no or little training in
staff management or organizational systems analysis
Yet, it is quite clear that no matter how well designed a
training or behavior management program is, unless it
is implemented correctly by staff, the consumers will
not benefit
 Functional daily living and job skill training
 Management of disruptive and inappropriate behavior
 Leisure activities, fun things that decrease inappropriate behavior
and may increase “quality of life” for consumers
 Verbal behavior training
6
In business and industry, it’s a given that employees need
to acquire new skills and supervisors/managers need to
know how to supervise their employees
 Most business organizations hire experts in training,
performance management, and organizational systems
analysis, usually in the human resources dept.
 Human service agencies have not done that

(At WMU, over the years, no idea how many of our graduates in human services have told me that
they wished they had taken all of our OBM courses while in graduate school here)
7


Human service professionals, have little or no training in
training, performance management, and organizational
systems analysis, unlike business and industry
And, there aren’t experts in the organization to help them,
unlike in business and industry, who hire experts in
training, performance management, and organizational
systems analysis in “support” units
8


Failure to recognize the importance of staff mgt
Lack of availability of courses in staff mgt/OBM at the
undergraduate and graduate level
 Very few graduate training programs in OBM

OBM courses that are offered typically focus on business and
industry and are marketed that way, thus students pursuing
a career in human services don’t recognize the relevance of
these courses
(WMU has historically been the exception; since 80s; our graduates in academic positions are primarily
the ones who are infusing staff mgt into training of human service professionals, so this is changing)
9

Performance management
 PSY 6450, Psychology of Work

Personnel training
 PSY 6440, Training

Organizational systems analysis
 PSY 6510, Behavioral Systems Analysis
10
Direct care staff are often not well trained when
they enter the organization
 High turnover
 Critical measures are often the behaviors of the
staff or clients, rather than accomplishments

 Implementing training procedures correctly and when
scheduled, correctly responding to client inappropriate
behavior
 Very labor-intensive as a result
(behavior is effervescent unlike accomplishments; need direct observation; )
11

Large number of staff
 Residential facilities require 24/7 staffing; some assisted
living arrangements also require 24/7 staffing
 In day training programs (centers for autistic children) still
need intensive one-on-one training for consumers (i.e.,
verbal behavior training, functional living skills training)
 Adds to the labor intensiveness for supervisors doing
direct observations, performance assessments, and
feedback
12

The accomplishment of the staff really is the
improvement or engagement of the client
 Some studies measure both the behaviors of the staff and the
client, but is this really feasible for an organization to do?
 Can you hold a staff member accountable for the progress of a
particular client when each client is likely to have different
individual needs?
▪ Reason why HSS organizations tend to measure the behaviors of staff –
which is a very reasonable thing: should not hold an employee
accountable for something that is outside his/her control
(May have to alter protocol, frequency of observations/measurement)
13




Pay is typically low
Sometimes staff get “kicked, bitten, and scratched”
They often have little or no professional training before
being hired
Job responsibilities are often not well defined after hire
 Often are excellent at daily care of consumers and scheduled
activities (meal preparation, outings, etc.)
 Not given adequate training or supervision
 Often not given much guidance about what to do when there is “free
time” for consumers
14
What all of this boils down to for staff:
Staff need job training and support
that management is often not trained
to give them.
(don’t blame the staff, but you can’t blame the supervisors either!)
15
Research has consistently shown:

Developmentally disabled individuals who live in residential
facilities or group homes spend ~65% of their time off-task
 That is, not doing any meaningful activities or leisure activities

Direct care staff who work in such facilities spend ~45% of
their time off-task
 That is, not doing any work-related activities
(I ask you to learn these data in SO11A, but I just want to start with these data on this before moving on…)
16

Supervisors in human service settings use the least effective
training and management procedures
 Verbal training methods (lectures and written material) rather than
performance-based training methods (modeling and practice)
 Antecedent interventions (training alone, memos, instructions) rather
than feedback (the most common performance mgt procedure in the
research; money, etc., is too expensive for human service agencies)
 Punishment instead of feedback
▪ In an early study, 90% of supervisors reported they relied on punishment to manage
staff performance problems (even though the available evidence indicates it is not
effective)
(Reid et al. make all of the points I have made about the lack of training for supervisors in their article.
SOs are straightforward and on your own.
17

Why do supervisors use the least effective staff training and
management procedures?
 Lack of training
 Time and effort required by the effective procedures (and in
the case of feedback and effective mgt procedures it is
ongoing time and effort; continue to provide feedback)
(two reasons, always the same )
18



Simple, but effective measurement system
Demonstrates that in-service training is
ineffective
Self-monitoring alone may have immediate
effects but will probably not sustain
performance over time
(redundant – I talked about this when I introduced this unit; study objectives are
straightforward – touch on some of the main points)
19
Rationale of study
To determine whether a self-monitoring procedure,
with minimal supervisory involvement, could
increase staff adherence to scheduled activities and
on-task behavior
 Participants
10 staff members in two houses of an intermediate
care facility (group home) for the developmentally
disabled

20

On-schedule behavior
 Is the staff member in the assigned area for the scheduled activity according
to the posted schedule?
 Does the staff member have all of the materials necessary to conduct the
activity?
 12B Regardless of whether the staff member was actually implementing the
task (that is, the staff member could be off-task in the sense of chatting with
another staff member, drinking coffee, or just interacting “generally” with the
clients; as long as the person was there as scheduled with right materials)

On-task behavior
 Is the staff member engaged in behaviors for any of the three appropriate
activities (group, client/house custodial, or one-on-one training)
 12C Regardless of whether the staff member was implementing the specific
activity that had been scheduled (in other words, even if the staff member
was doing group training when one-on-one training was scheduled; staff
member )
(very nice measures of behavior, simple; apologize for the crowded slide – needs to be on one)
21
Experimental Phases
Measures/
House
Baseline
In-service
Self-Monitor Self-Monitor
+ Feedback
A
50%
50%
80%
94%
B
39%
39%
75%
81%
A
28%
36%
72%
88%
B
28%
28%
77%
80%
On-schedule
On-task
(First, note lack of effectiveness of in-service – SO16 – Not just a self-monitoring procedure – turned daily schedule cards into the
Supv. at the end of the day; implication of fdbk; again; yes, they did get further increases, but the main reason -next slide)
22


The behavior of 5 of the 10 staff members became variable
over time (that’s 50% of the participants)
Supervisory feedback improved both on-schedule and ontask behavior for each of the 5.
 Demonstrates the importance of supervisory feedback and
evaluation

Also suggests that self-monitoring may be effective on a
short-term basis but may not be effective long-term
▪ But, why would we expect self-monitoring to be effective over
the long run?
▪ What consequences are there for self-monitoring or for the selfmonitored performance?
(question: are we doing more harm than good when we publish short term studies that indicate that interventions are
Successful, particularly when the results don’t seem to conform to a solid behavior analysis? Carbone next)
23
Carbone Clinic, center for autism
Valley Cottage, NY, 20 miles north of NYC
10 instructors, 2 classroom supervisors, 40 learners
with autism and other disabilities, age 2-14 years
 Also have 10 OutReach consultants, who conduct
individual assessments and provide consultation to
others
 Conduct a summer institute to train educators in
public and private schools (10 days)



24
Instructors can earn a monthly bonus totaling $300 a
month, $3,600 per year, for exemplary performance
 There are two incentive components, independent

 $150 based on supervisor observations of training skills
 $150 based on accuracy of child’s program/data book
Bonus is publicly announced at the staff meeting that
follows the assessments
 Checks are given to staff at that meeting

(monthly may be an adaptation to HSS because of labor intensiveness; for paperwork – often only based on
clinical training, but paperwork is very important)
25
Names publicly posted in the staff dining room
Performance scores are referred to in the employee’s
annual review
 Performance scores contribute heavily in determining
the size of annual raises and future promotions


components: Observation and feedback, $ incentives, goals/criterion for incentives,
supervisory and public recognition – all of the components of an effective mgt system)
26





Everyone who meets criterion can earn the
incentive (and there is criterion/goal)
Significant amount of money
Public recognition at the next staff meeting
Separate check – the money doesn’t get
“lost” in the person’s regular paycheck
Embedded in the management system – used
to determine pay increases and promotions
27
Following training, unannounced monthly
assessments of performance are conducted
 Supervisor observes instructional sessions using
three to four competency checklists

 Natural environment teaching
 Discrete trial teaching
 Teaching adaptive living skills
 Teaching vocal manding
 Implementation of behavior reduction protocols
(He is willing to give copies of these checklists to individuals who are interested)
28
Supervisors give vocal feedback to instructors after
the observation sessions
 Instructors must score 90% on each of the 3-4
checklists used, with no critical errors to earn the
incentive
 If instructors do not meet the criterion, supervisors
coach instructors and repeat an assessment of
those competencies approximately one week later

29

When staff are stable (no new hires) and staff
are not assigned to new learners (who may
have new competency checklists, thus staff is
still learning the protocol):
 75% to 85% of staff earn the bonus for
teaching/training per month
 25% to 65% of staff earn the bonus for accuracy of
the child’s program/data book
(relatively old data, 2007; training percentage is considerably higher than accuracy of books; exquisite
system and and data: 75-85% of the staff are performing at least 90% of the checklist items correctly
With no critical errors – I wonder how many other agencies can say that about their direct care staff )
30

No experimental design to assess the
effectiveness
 But, replication across new instructors
 $$ paid out has increased over the years as
percentage of instructors who met criterion
increased

No outcome data related to changes in
learner behaviors
 A system to do this is being developed
(Carbone has developed this list; Parsons next)
31
This is the best study I have seen about a large scale
OBM intervention in a human service setting
 The study was conducted in five group homes for the
developmentally disabled
 In the study objectives, I point out some very useful
procedures that could be implemented in any human
service setting although clearly some of the details of
the procedures would have to be modified
 Implemented a total system intervention package

32


There are two experiments
I only have an NFE SO over the first one
because I wanted to focus on the
intervention, but part of the beauty of this
work is having the normative data from the
first when analyzing the results from the
second
33

Experiment 1
 Benchmarking study on treatment and services
 22 living units in six state residential facilities
 18 were certified as intermediate care facilities under Medicaid
(which means services can be reimbursed through Medicaid)

Experiment 2
 Purpose was to develop and implement a comprehensive
management system to improve treatment services in five
group homes
 Group homes were Medicaid certified
 Medicaid had reviewed services and the facilities had been
given a time-limited mandate to improve services or face decertification. Improvement was critical - “critical business issue”
34

On average, what percentage of resident behavior
was off-task?
67%!! (range 0-100%)
When developmentally disabled clients are in group homes, 2/3 of their time is
spent doing things that help them. This suggests that residential facilities are
not fulfilling their active treatment obligations
 On average, what percentage of resident behavior
was active treatment?
19% (range 0-40%)
35
110 Direct care staff
165 Residents
36




Structure (scheduling) and reassignment of
staff
Staff training
Monitoring of staff performance
Supervisory feedback
One of my purposes with the SOs is to point out the systems aspects of the program - they implemented monitoring and feedback systems
for individuals at EACH level of the organization - we often intervene at the direct care staff level, but who provides PM to the group
home supervisors, and to the supervisor of the group home supervisors? We forget to do that, yet are often surprised our interventions don’t last
37


Reassignment and scheduling alone or in
combination are common interventions in
human service settings
What are the benefits?
▪ Task clarification (specification of what they are
supposed to be doing and when)
▪ Decreased conflict with other responsibilities
▪ Individual accountability
▪ Individuals can be identified
▪ Their performance can be measured and evaluated
▪ Their performance can be consequated
(this is important – common that staff are not scheduled, everyone is just supposed to pitch in as needed;
and know when it is needed)
38
•
25A How often did each supervisor or assistant
supervisor observe each staff person?
Once a week
 25B What procedure was used to verify that the
supervisor(s) observed and gave feedback to the
staff member immediately after the observation?
Each staff member initialed the checklist
I am pointing this out because this is basically the same procedure
used by Wilk & Redmon and it permits the assessment of the integrity
of the intervention without observers. Remember this procedure!
(these are, of course, straightforward, but I want to emphasize b)
39



The data on resident behavior collected by researchers (independent of the
preceding measures on staff observations) were summarized and graphed,
and sent to the program director weekly.
The program director sent the graphs along with comments to the area
director, who then sent the appropriate graphs to each group home
supervisor
Note two separate and independent measurement systems
 Were supervisors observing and giving feedback to the direct care staff
 How was the supervisory system affecting resident behavior - was decreasing
resident off-task behavior and increasing active training

Also note that the resident behavior data were collected by:
 8 staff members
 Student interns (number wasn’t specified)

Extremely labor intensive
(also the systems aspect – everyone in the hierarchy was involved – top to bottom)
40
SO28: What very nice contribution does the
normative data provide when analyzing the results
of the study?
 Most studies would have reported the improvement
in resident behavior in comparison to baseline
 During baseline off-task behavior averaged 64%, which
decreased to 41% during the PM intervention
 That looks like a nice decrease (23% decrease) but
residents were still off-task 41% of the time
(cont. on next slide)
41
With the normative data they could also report
 Their baseline average was similar to the average off-
task behavior in the 22 other group homes (18 of which
were Medicaid certified): 64% and 67%, respectively (so
maybe they weren’t doing that badly to begin with!)
 Not only did off-task resident behavior decrease
considerably, but it is now well below the normative
average, so…
(in business & industry, we often call this benchmarking)
42
Not only could the administrators and researchers
show that these group homes had improved
considerably, they could also show that they were
doing considerably better than other state
residential facilities
43



Basically, so you know realistically, what good performance
is given typical staff-to-resident ratios
The residents were profoundly developmentally disabled,
typically nonverbal, and required assistance in self-care
routines
The agency can only hire a certain number of direct care staff
due to budgetary constraints - and usually these type of
organizations are understaffed
 Extremely high staffing ratios: 165 residents, total staff of 127

It is simply unrealistic to assume that it is possible to have
0% off-task resident behavior - so back to the original
question - what is good performance?
44


As the authors note, and I mentioned briefly earlier, while
group home supervisors observed the behaviors of staff and
gave feedback to them weekly immediately after the
observations, neither staff behavior nor supervisor
observation behavior were graphed and fed back to
supervisors or staff
Rather, the feedback that was given was feedback on the %
of off-task resident behavior and % of time residents were
involved in active treatment
 To truly determine a functional relationship between staff and
supervisor behavior and resident behavior, you would have to
measure both (however, I admit I am convinced by the data)
45

The authors make the point, however, that there is a
disadvantage of monitoring staff behavior
 Maintain that staff frequently do not like to be observed and often
react negatively - from mild nervousness to out right hostility
 But, they do not react as negatively when resident behavior is
monitored and reported
 Thus, this may have made it more likely that supervisors would
continue to use the system

It’s an interesting point - but I don’t know how valid it is
 I looked at the reference given, but it was to a book written by Reid et
al. for practitioners, and no data were provided
 It would be an interesting (but difficult) study to conduct
(cont. on next slide)
46
However, Babcock et al. (1992) found that the performance
of nurses was better when they were given formal feedback
on the accomplishment measure for the staff (wearing
gloves when removing soiled sheets), rather than their own
behavior of giving feedback to staff
 So, there are some data to back up Reid et al.
 It’s a interesting issue and question, that is

 Do you give individuals feedback on their behavior or
 Do you give individuals feedback on an accomplishment measure?

Is there a difference in performance and satisfaction of the
workers?
(NFE, but said I might add something: why might feedback on resident behavior rather than on own behavior increase
behavior more and be less aversive?)
47
Questions on the Parsons et al. article?
Discussion?
48




We know we can get short-term
improvements in performance
How do we get maintenance?
There is only one sure way: PM must be
embedded in the management system
Human service settings tend to emphasize
only client service and goals to the exclusion
of PM programs
49
Green, Reid, Perkins, & Gardner (1991)
50




This article presents a very nice objective assessment
of the barriers that can interfere with the
implementation of training programs in human
service settings
Also, it is sensitive to the staff
Staff had been complaining they didn’t have time to
do training
Instead of “dismissing” their complaints, Green et al.
collected data that could help solve the problem
 That is, when were staff available to do training and when
weren’t they because of their competing job
responsibilities
(also, very low cost intervention, and use of a lottery)
51
Study was conducted in a residential facility for
individuals with profound handicaps
 Collected data to determine staff behavior patterns
during the work day – when were they busy and when
did they have “down” time
 Participants were 4 direct-care staff assigned to the
day shift
 20 clients





Profoundly mentally retarded
Non-ambulatory
Serious medical complications
Histories of non-responsiveness to behavior-change
programs
(those of you who want to work in business and industry – how tough is this staff job compared to most jobs?)
52

Five DVs






Basic care
General interaction
Training
Indirect basic care
Nonwork
Observations
 Time-sampling procedure, every 15 minutes, on weekdays
▪ 7:30 – 11:15 am
▪ 1:15 – 3:00 pm
 96 total observations over 26 days
53

Clear patterns for only basic care and
nonwork
 Only three occurrences (involving 1 staff member)
of training activity were noted
54

In the morning
 The number of staff members engaged in direct
basic care was greatest during the earlier time
periods and decreased as the morning progressed
 Conversely, the number of staff engaged in
nonwork increased as the morning progressed
 10:30-11:00 am appeared to be the optimal
morning time to increase structured client
training activities without interfering with basic
care
55

In the afternoon
 Similar pattern, but direct basic care was much
less frequent
 Nonwork was most frequent from 2:00-2:30 pm,
with no observations of basic care during that
period
 2:00-2:30, thus was an optimal time to attempt to
increase client training
56

Scheduled training at
 10:30-11:00 am
 2:00-2:30 pm
 4:00-4:30 pm
▪ added to provide a more comprehensive evaluation staff
mgt program – different staff members were working

Measured direct basic care and training
during those time periods
57


Staff training, first
Staff management, next; four components
 Daily verbal feedback provided to each staff member
contingent on the occurrence and proficiency of client
training activities
 Private weekly feedback provided to each staff
member
 Self-recording; each staff member initialed a chart
posted in the living unit each time he/she conducted a
training activity with an assigned client
(labor intensive!, lottery, next)
58
 Lottery
▪ Held monthly
▪ Staff were eligible if they conducted 80% of their scheduled
training sessions
▪ One person’s name was selected
▪ Prize, could select a special privilege from a list, i.e.,
▪
▪
▪
▪
Free lunch
Extra 30 minutes for lunch or leave 30 minutes early from work
Private parking space
A written commendation letter to be placed in his/her personnel file
▪ Prizes were determined based on recommendations from
staff and supervisors
(note, low cost of prizes – very little out of pocket expense at all )
59

2:00-2:30
 Nonwork decreased from 93% to 8%*
 Training increased from 0% to 91%

10:30-11:00
 Nonwork decreased from 27% to 2%
 Training increased from 7% to 86%

4:00-4:30
 Nonwork decreased from 18% to 5%
 Training increased from 0% to 84%
* Of observations
60
Remember, lottery was held only monthly
Authors did not indicate # of staff who were
eligible in each (which would have been nice to
know)
 8 staff members


 4 from the morning shift and 4 from the afternoon
shift
 If all staff met the eligibility criterion (which probably
did not happen), each staff member would have had a
1 out of 8 chance of winning
 Looks like pretty good odds
61
Green, Reid, Passante, & Canipe (2008)
Final article! Almost done)
62


I like this article because of its emphasis on
increasing the work satisfaction/enjoyment
of supervisors in a human service setting
They have very difficult jobs and this is one of
the few articles (if not the only one in our
field) that has directly addressed that issue
63
Purpose:
Increase work enjoyment/satisfaction of supervisors by
identifying their most disliked task and making it more
attractive
 DVs

 Repeated preference ratings and rankings
▪ During baseline to identify most disliked tasks
▪ After changes to determine if preferences had changed
 Survey after intervention
▪ 7-point scale, did the intervention make their quality of work life better
or worse
▪ Did they want to continue the intervention
▪ In one case a lottery was implemented for staff, so the staff were surveyed with
respect to whether they wanted the lottery to continue
 Objective measures of work quality on the targeted tasks
64



Ms. Tome and Ms. Jones: completion of
monthly progress notes
Ms. Noel: reviewing time sheets
Mr. Davis: conducting staff observations
because it appeared that the staff did not like
having their performance reviewed
65

Completion of monthly progress notes and
review of time sheets (3 of the 4)
 Frequent interruptions!

How was this changed?
 Removed disliked stimuli
▪ Scheduled a specific time in an office away from their work
stations; an office that their staff did not have access to
 Added liked stimuli
▪ Provided them with snacks and bottled water when they
were doing the task
(3 of the four, supervisors, low cost!)
66

Observation of staff’s performance (1 of 4)
 Staff didn’t seem to like it

How was this changed?
 Removed disliked stimuli/added liked stimuli
▪ Added a performance lottery; described as a means for
making observations more pleasant for staff
(4th supervisor, Mr. Davis; removal and adding stimuli the same thing in this case)
67



Performance eligibility for lottery was
determined
Lottery was held monthly
Unusual “lottery” in the sense that each eligible
staff member received a prize (5 total staff)
 As each name was called, the winner selected the prize
he/she wanted from a list of available items

Prizes
 Gift certificates from local stores, re-arrangement of
some work duties, and changes of some aspects of
their work schedules
(this may be why these staff liked the lottery better than those in the previous article; Green et al.; no losers)
68




Preference ratings and rankings increased
substantially for the targeted tasks for all 4
supervisors
All four chose to continue the program
All 5 of the staff who reported to Mr. Davis
said it made their quality of work life
extremely better (SO41)
All five also chose to continue the lottery
(SO45)
(results cont. on next slide)
69

Work quality measures: remained high
 Ms Tome
▪ Baseline: 98% correctly completed progress notes
▪ Intervention: 100% correctly completed progress notes
 Ms Jones
▪ Baseline: 100% correctly completed progress notes
▪ Intervention: 100% correctly completed progress notes
 Ms Noel (ratings by supervisor re time sheets)
▪ Baseline: outstanding
▪ Intervention: outstanding
 Mr. Davis
▪ Baseline: completed 80% of the observation forms
▪ Intervention: completed 100%
70
71