Transcript No Slide Title

What Gets Measured
Gets Undone
Dr. Jim Mirabella
Director of Institutional Research
Professor of Statistics
Florida Community College - Jacksonville (FCCJ)
J. Michael Adams
Corporate Manager, Quality Services
Florida Power & Light/ FPL Group, Inc.
Florida Sterling Conference
Wednesday, May 31, 2000
Disney World, Orlando, Florida
1
What’s getting “undone” ?
• The desired outcome and supporting
processes that the measure(s) intended to
describe
• Why????
– Poor measures drive poor performance!
2
Objectives of the workshop
• Know good measurements from poor
measurements
• Distinguish the implications of good and
poor measures on performance
• Balance an array of measures that favorably
tracks a process’s outcome for planning,
assessing performance, and analysis
• Assess attendees’ measures
• Any other expectation?
3
“Why” Measure at All?
• Measurement is the language of progress
•
•
•
•
and comparison
Provides a sense of where we are AND
where we are going (planning)
Can guide a steady advancement toward
established goals (tracking)
Can identify goal shortfalls, or overachievement (analysis)
Communicates to the work force what is
important to the organization (behavioral)
4
Good measures are born
from SMART goals
•
•
•
•
•
Specific
Measurable
Agreed upon
Realistic
Time-Bound
5
The Challenge of Measuring
• Workers might perceive it as a threat
• Workers might disregard organizational
goals, customers, products and services
• Workers might focus on obtaining favorable
measurements
• Measuring items with no influence on
organizational success = waste of time
– result is a bean-counting approach that focuses
on irrelevant details
• Expected to do something!
6
The Challenges- examples
• “Apples and oranges”
• Impact on comparisons, benchmarking and
performance
• turnover
• consistency
• defects vs. defectives
• commutes (measured in time or miles)
• school achievement
7
“Where” are these
measures
•
•
•
•
Organizational
Process
Department/work unit
Individual
– Exercise
8
Your measures
Jot down your various
performance measures for you, your department, unit and
organization. Indicate the purpose of the measure ( 1=
planning, 2= tracking and analysis; 3: behavioral changes)
• Organization
• Department/ Unit
• Process
• Individual
9
“What” Do We Measure?
•
•
•
•
•
Quality
Delivery
Cycle Time
Waste
Cost
•
•
•
•
•
Defects
Satisfaction
Complaints
Financials
Price
10
Measurement Examples
 Operations-related measures
- reliability
- timeliness of delivery
- order processing time
- errors / defects
- product lead time
- inventory turnover
- cost of quality
- employee
11
Measurement Examples
 Customer-related measures
- customer satisfaction
- customer complaints
- customer retention
 Financial measures
- market share
- sales per employee
- return on assets
- return on sales
12
Baldrige/ Sterling: Results Category
Private, Education, Health Care
7.1:
7.2:
7.3:
7.4:
7.5:
Customer Focus Results; Student
Performance, Patient and other
Customer Focus
Financial and Market; Student and
Stakeholder Focused Results,
Human Resource Result; Budgetary and
Financial Results, Staff and Work
System
Supplier and Partner Results; Faculty
and Staff Results
Organizational Effectiveness
13
Determining and Reviewing
Measurements for Balance
•Customer
Baldrige
Results 7.0
•Financial
•Human Resources
process
dept
•Supplier/ Partner
•Organizational
supplier
14
Desired Outcomes- Airlines
overall organization balance scorecard
• “Assumed Quality”- arrive safely
• Loyal Customers
– Market differentiation, key value attributes:
• on-time arrival
• baggage handling
• customer complaints
• Profitability and Market share
– Satisfied stakeholders (shareholders, partners)
• Employee growth and retention
15
Airline Measurement System
• Individual measures for each flight process
• Group measures for overall airline
• If problems occur with a flight, who do you
blame -- the flight crew or the airline?
• What measures do you affiliate with the
flight crew?
• What measures do you affiliate with the
airline?
• What is important to you for a satisfactory
flying experience?
16
Review your measures
• Are the organization measures reflective of
all Baldrige results items?
• Do the measures reward favorable
behaviors?
• Is there alignment with the contributing
departments and suppliers?
• Are the overall results managed as an
outcome of a process?
17
What’s the opportunity?
• Many organizational measurement systems
are too short, too rigid, or used like a strict
teacher’s ruler ... to whack rather than to
motivate
• Need to replace these outdated
measurement systems with more dynamic
measurement system that motivates
continuous improvement in customer
satisfaction, flexibility, and productivity …
SIMULTANEOUSLY
18
Mis-measurement Systems
• Unless specifically tuned to flight plan,
measurement systems may:
– yield irrelevant / misleading information
– provoke behavior not conducive to
strategy
• Traditional measures ignore requirements &
perspectives of customers (internal/external)
• Bottom-line measures (profitability) too late
for mid-course correction / remedial action
19
Mis-measurement Systems
• Many measurement systems overlook key
non-financial performance indicators
• Measures often used for punishment rather
than to promote learning
• Many measurement systems are inflexible
and limited in what they can do
20
What Should a
Measurement System Do?
• Measures must link operations to strategic
goals
– departments should know how they contribute
separately and together toward strategic mission
• System has to integrate financial / nonfinancial info in a way usable by managers
– managers need right info at right time
• Measurement system’s real value lies in its
ability to focus all business activities on
customer requirements
21
What Should a
Measurement System Do?
• Measure what is important to the customers
• Motivate operations to continually improve
against customer expectations
• Identify and eliminate waste -- of both time
and resources
• Help to accelerate organizational learning and
build a consensus for change when customer
expectations shift
22
Measurement Mismanagement
In an effort to increase market share, a computer service
bureau strategizes to improve the timeliness of its
voicemail service. The goal is to provide new customer
with service w/in 24 hours. To help speed the process, a
program is developed to accept verbal phone orders.
Most new customers were online w/in a day as
promised, causing the company to celebrate their
success. A later audit revealed 70% error rate in order
entry, with 30% of customers disputing their bill and
eventually canceling service..
23
Measurement Mismanagement
A Hi-Tech company sets objective to become highly
profitable by being a product leader. To measure the
performance of its marketing and R&D functions, the
number of new products developed is the most watched
barometer. An internal review reveals that in a sample
of 20 new product introductions, 80% were delivered
over a month late, and significant waste piles up in
production. Management cannot understand why the
accountant’s ink is red – after all, their yardstick tells
them they are developing new products at a record rate.
24
Beware of ……..
Seemingly Simple Measures
• How satisfied are your customers?
• What is your employee turnover?
• What is your client retention rate?
• Is the value of our service worth the
price?
25
Beware of ……..
Treating customer perceptions as
objective measures
• Customer satisfaction  product quality
• Customer satisfaction is a complex
phenomenon
• A well-handled complaint results in higher
customer satisfaction than does no
complaint
• More reasons for complaint  more
dissatisfaction
26
Beware of ……..
Non-specific measurements
• Results are actionable
• Hard to improve what you cannot assess
Failing to measure adequately
• Think BALANCED SCORECARD
• Don’t give employees an outlet for gaming
success
• Identify all areas important to customers
27
Beware of ……..
Using results incorrectly
• Don’t tie results to employee pay unless
employees can directly influence results
• Don’t base employee pay on results that
cannot be measured
28
Recap of Objectives and
Expectations
• Know good measurements from poor
measurements
• Distinguish the implications of good and
poor measures on performance
• Balance an array of measures that favorably
tracks a process’s outcome for planning,
assessing performance, and analysis
• Assess attendees measures
29
Thank you!!!!!!
Jim Mirabella
Mike Adams
30