Transcript e-Global conference - Library Assessment

Our Most Important Challenge
Raising the Assessment Bar:
A Challenge to our Community
Rick Luce, Emory University
Library Assessment Conference
Seattle – August 4, 2008
Environmental Scan
1. University mission evolving:
 Education reform and the Spellings commission  focus on
outcomes and accountability
 Globalization and competition
2. New research methods in a networked world:
 Rise of eScience / eResearch = new ways to work, new
needs and expectations
 Data science & data scientists require new organizational
environments
3. Social drivers:
 Technology enabled social tools to connect & collaborate
My View after 2 years at Emory
Aggressive Strategic Plan completed in first 4 months:



Strategic direction: (1) digital innovations, (2) special
collections, (3) delivery of 21C access, resources, and services
$100M requested in new funds over 5 years
 Concept approval for ~$36M for 3 new capital projects
Implement annual Business Plan – reviewed quarterly
None of which utilize current ARL statistics
We Need a Systems Approach


The human body is a system, our subsystems work together
to keep us healthy
Research libraries are systems, requiring a management
system to keep the subsystems working together to be healthy
Anyone who learns to see the organization as a system can
never again feel satisfied with “improvement” initiatives which
simply change staffing and the org chart but do not tackle the
system itself
Where Assessment Fits
Assessment – a method of planning for improvement

Catalyst for organizational change (not a quick fix)

Gain staff understanding for need for improvement &
commitment to shared improvement goals
Ideally underpinned by a performance measurement
matrix balancing:
 Quality = customer defined goodness – internal &
external
 Time = speed, how fast is the response, agility
 Cost = resources spent on people, processes, or
organizational shifting or rework
Performance Measures: our Vital Signs



Statements without performance measures are
wishful thinking -- without data, we don’t know
We all have volume or transaction data - provides no process
insight
Move focus on product metrics to process metrics





Process performance (statistics, run charts, variation)
Getting to the ‘right’ metrics
What is the value equation?
How do we compare & differentiate ourselves?
Focus on : Customer, process, sponsorship metrics
Hedgehog View: Constancy of Purpose
What are we
best at?
What drives
our economic
(or value)
engine?
Adapted from: Jim Collins. Good to Great. 2001
What are we
passionate
about?
ID Key Success Factors
Characteristics of successful organizations:*
1. Do something others cannot do
2. Do something well that others do poorly, or
3. Do something others have great difficulty doing
well

~10-15% (max.) of research libraries content /
services are unique
What % of the budget resources support
that?
*Prahalad and Hamel. The Core Competencies of the Corporation.
HBR, May/June 1990.
Management Principles of Successful Organizations

Strong customer focus

Effective leadership

Continuous improvement and learning

Management by fact

Fast response

Long-range view of the future

Results orientation

Cooperation, teamwork, partnering
A Value Proposition Example (LANL)

Scientific Productivity & Competitiveness
Enhancing research productivity and competitiveness
 ROI = $4.50 : 1.00

Quality and Business Focused





10 year average: 96% satisfaction rate; 25% delighted
Reduced cost per transaction by a factor of 16
Quality New Mexico ‘Roadrunner’ awards: 1997 & 2000
Federal Library of the Year 1999
“World’s Best Science Library” 2005 Blue Ribbon Review
Follow the Leaders
Malcolm Baldrige National Quality Award: recognizes
organizations practicing the most effective
management methods

High performance is sustainable due to good
management practices

Analysis of 600 winners over 10 years: growth = > 2.5
times as fast as peers, more than 2X more profitable

Examines approach, deployment and results
Baldrige Criteria for Performance Excellence
1. Leadership
2. Strategic Planning
3. Student, Stakeholder, and Market Focus
4. Measurement, Analysis, and Knowledge
Management
5. Workforce Focus
6. Process Management
7. Results
Requires a system both in approach and deployment
Baldrige Applications: Lessons Learned


Accelerated learning using Baldrige framework
“System” is tough to integrate all at once
“Be patient, have discipline” Deming

Importance of supplier partnerships

Difficulty of language translation

Benchmarking data - time series data for competitors –
couldn’t be obtained from libraries
Benchmarking

Process for gaining and applying knowledge to
improve business process performance from a study
of current practices.


A means of using data to identify magnitudes and reasons
for variances in performance.
Intent: comparative process data, best practices


Analyze the operation, know the competition & industry
leaders
Incorporate the best of the best - become the new
benchmark
Missing from Research Library Portfolios

Customer satisfaction index: (delight & loyalty) and perceived
value

Product / service quality (defined by the customer)

Process and operational performance - cycle time, productivity



Employee satisfaction – learning, morale, training, alignment of
strategy direction and rewards
Measuring supplier performance - quality, process variables,
price competitiveness, overall ease of doing business
Financial: cost/value matrix, return on investment, cost
avoidance
Customer Satisfaction Metrics
Maturing our satisfaction assessment
Level 1 – Satisfaction surveys: ‘happiness meters’
Level 2 - What’s important  analysis of customer
importance & satisfaction levels
Level 3 - How do we rate against best in industry
See it from the customers eyes
Measures that matter
Align library vital signs with the organization’s drivers

Quality of product and processes

Innovation

Research leadership

Brand identity

Growing market share

Reducing new product development time

Ability to attract and retain employees

Credibility
Avoiding Pitfalls

Measures that don’t focus on strategy

No accountability

Too many initiatives

Forgetting larger organizational drivers

Lack of discipline

Insulating researchers and managers from scholarly
communication issues
No action without a plan, no plan without data
Business Scorecard - Desired Business Results
1. Customer focus:

Satisfaction, loyalty, value-added
2. Product quality:
 E.g., accessibility, usability, accuracy, completeness
3. Operational process performance:

Productivity, competitiveness, cycle time
4. High performance workforce:

Organization performance assessment, formal process
changes, employee satisfaction
5. Prestigious reputation -- output results
 Strategic performance results, benchmark results,
external assessment scores
A Tale of 2 Libraries: Budget Allocations
$M
Payoff for Successful Quality Implementation
Any road will do if the destination is unknown
The journey to truly superior performance is neither
for the faint of heart nor for the impatient.
The development of genuine expertise requires
struggle, sacrifice, and honest, often painful selfassessment.
HBR: Anders Ericsson (FSU), Michael Prietula (Emory), Edward Cokely (Max Planck)
If you wish to do something for the community,
build a road. If you wish to do something better for the
community, build a bridge – Chinese Proverb
Together let’s build the bridge to a new
level of assessment practice,
supporting continuous improvement
and focusing on outcomes and impact.