Transcript bnchmrk - Economic Development - E-D

Benchmarking in
Economic Development
Programs
Eric L. Rinehart
Economic Development Solutions, Inc
Presentation Objectives

Context
Performance Tracking
 Competitive Benchmarking
 Report Cards/Scorekeeping

Conclusion
 Appendices

Presentation Output

To inform and educate ....
Presentation Outcome

To finish my presentation

Before you finish listening
What is Benchmarking?
Current
Efforts
Relative
World Class
Past Efforts
Absolute
Standards
Future Goal
How Benchmarking is Practiced

Performance Tracking/Monitoring: measuring
the furtherance of organizational objectives

Best Program/Practice Benchmarking:
learning best practices of leading performers

Report Cards/Score-Keeping: compilation of
indicators tracking community movement to the
future
Why Benchmark?
- Pressure for Accountability

Public Expectation

improvement in lives
 return on investment
efficiency of approach
 quality of services
 quantity of services


Organizations
best use of effort
 service to customers

Manager's Operations


Oversight Authorities
GPRA: process
 SEA: elements

Private Sector vs.Public Sector

Needs of Stakeholders:
Private: Similarly minded owners/operators
 Public: Distinct stakeholder perspectives

(good job, right things, actual benefit)

Nature of Results:
Private: Direct measurable results in dollars
 Public: Indirect results; jobs and taxes due to priva

investment, sales, innovation, efficiency
Performance Tracking Techniques
- driven by plans and production
Planning Steps
Production Function
context
Global 'Objective'
mission/vision
Outcome/Benefit
goals/scope
Output/Results
methodology
Process/Activity
resources
Inputs/Resources
Techniques
Summative Eval
Formative Eval
Client Benefit
Public Benefit
Srvc. Delivered
Plan vs Perform.
Quality Mgmt.
Best Practices
Acctng Standard
Performance Tracking: Planning

1993 Gov't Performance & Results Act
Strategic Planning
 Performance Plan
 Program Performance Indicators Rpt


Service Efforts and Accomplishments
Specify Measurable Objectives
 Detail the Strategies Employed
 Publish Activity Indicators
 Publish Outcome Indicators

Performance Tracking: Functions

If Inputs:

And Process: Methods and Procedures

Then Outputs:
Direct Results

Then Impact:
Expected Outcome

Then Goals:
Activities and Resources
Global Objectives & Strategies
Performance Tracking: Techniques

Inputs

Level of Resources
 Accounting Standards
Level of Services
 Plan vs Performance


Process
Competitive Benchmark
 Quality Management

Outputs


Impacts
Customer Benefit
 Public Benefit
 Cost - Benefit

Strategic Plan
- Agency Goals (& Objectives)
Leadership: Increase the capabilities of individuals involved
 Infrastructure: Physical and social infrastructure improvements
 Workforce: Develop a skilled and adaptive workforce
 Capital: Provide access to capital for business growth and expansion
 Technology: Provide access to modern business practices
 Market Development: Linkages between customers/ markets
 Industry: Retention, expansion and attraction

Performance Plan and Report



Dept. Outcomes/Outputs,
for each agency objective
Target Measures,
for each outcome area
Plan vs. Actual Report,
a baseline amount or percentage and
quarterly reports
Tourism Promotion Examples

Strategies
Inputs
 Process

Domestic and International Advertising
Support Local Marketing Organizations
Develop Destination Attractions/Sites
Advertising Media Purchases
Tourist/Traveler Inquiry Fulfillment
Visitor Volume at Welcome Centers
Outputs Percent of callers traveling in Illinois

Outcomes Leisure Traveler Satisfaction

Leisure Travel Expenditures
Jobs Created/Retained
Industry Recruitment Examples

Strategies Recruitment/Visit Team
Advertising and Promotion
State/local Location Incentives

Inputs

Process
Number of Contacts (Presentations) Made
Number of Leads (Suspects) Identified

Outputs
Number of Businesses Attracted
Value of Plant and Equipment Invested

Accuracy, Currency of Data
Response Time to Inquiries
Outcomes
Jobs Created/Retained
Taxes Generated
Competitive Benchmarking Process
(Best Practices)

Key Results:
select critical areas of strategic importance
to the organization
Indicators: select reliable indicators of performance
 Comparison: compare one's performance to others (or

to standards) using these indicators

Learning:
determine where the leaders are and what
can be learned by them

Changes:
make changes in operations based on
what you have learned
Competitive Benchmarking
Private Sector Examples
World Class
Practices

Mgmt, Engineering, Operation
customer responsiveness
 continuous improvement
 quality management
 cycle time reduction
 cross functional workforce
 self directed work teams


Performance
Measures
Productivity, Quality, Efficiency
first pass yields
 scrap & rework costs
 customer rejects
 cycle time
 customer order lead time
 on-time deliveries

Benchmarking within
Economic Development

Best Practices
World Class Practice: Management, Engineering
Operations Practices
 Performance Measures: Productivity, Quality, &
Efficiency of Performance


Best Programs
Formative Analysis: Compare 'architecture' to aid in
the design of a program in its early stages.
 Summative Evaluation: Compare program results to
judge overall worth in relation to alternatives .

Benchmarking "Principles" of
Economic Development

Context:

- Global Economy
- Industry Cluster Focus


- Public Facilities Capacity
- Community Betterment
- Business Vitality
- Marketplace Capability
Vision:
- High Quality Job Creation
- Stable Tax Base

- Community Excellence
Adapted from "Emerging Principles
in Economic Development: A
Benchmarking Tool", June '95.
Organization:
- Regional Collaboration
- Partnerships
Goals:
- Business Competitiveness
Scope:

Process:
- Comprehensive Strategy
- Strategic, Mission Driven
- Customer Orientation
- Measurement and Evaluation
- Learning
Report Cards
- Stated Purposes


the compilation and tracking of 'indicators' to
track community progress towards a desired
future.
the use of marks or 'indicators' to compare
and contrast states or regions.
Report Cards & Score Keeping
- Examples

Compendium of Economic Performance
Peoria Scope
University of Ill Flash Index

Area Report Cards
Business Climate Surveys
CfED Report Card of the States

State Progress Reports
Oregon Benchmarks
Minnesota Milestones
Report Cards
- Not Ready for Prime Time

Relevance: (purpose)

Reliability:

Validity: (marks themselves) The use of existing indicators
Measures context only, and not
follow-up functions, so they won't help you do your job.
(areas for marks) Poor selection of the
phenomena to be studied, they are rarely a cogent reflection of
society and the economy which is more complex.
and measures of historical phenomena cause concern that
outside or unknown events precipitated the effect noted.

Materiality: (the comparisons) The trend indicators typically
used are descriptive and distributive, rather than diagnostic, and
therefore are not necessarily informative or enlightening.
Report Cards
- Alternative Methods Are Needed

Phenomena: State final outcomes expected and therefore
to be considered for measurement.

Rationale:
Describe the cause and effect model to lay
out a logical process to get to the outcomes.
Key Results: Select critical areas of strategic importance.
 Indicators:
Select reliable indicators of performance.
 Comparison: Compare one's performance against relative

or absolute measures using the indicators.

Analysis:
Suggest methods for improvement.
CONCLUSIONS
Embrace 'Benchmarking'
 Positive Attitudes
 Key Concept: Relationships

Benchmarking Efforts:
Economic Development Application

Best Program Benchmarking: Doing the right
things: use formative & summative evaluations.

Performance Monitoring: Doing things right:
document procedures & measure their completion.

Report Cards: Right the things done: compile
realistic indicators of progress towards the future.
Organizational Environment for
Measurement
Personal Leadership
 Openness and Trust


Agreement on Basic Assumptions
Commitment to Evaluation
 Resources for Measurement
 Applications of Results

The Key Concept to Remember
- Relationships
Planning Steps
market analysis
mission/vision
Production Function
Context
Outcome/Benefit
goals/scope
Output/Results
methodology
Process/Activity
resources
Inputs
Techniques
Summative Eval
Formative Eval
Client Benefit
Public Benefit
Srvc. Delivered
Plan vs Perform.
Quality Mgmt.
Best Practices
Acctng Standard
Planning Definitions
Strategic Planning
 Implementation Planning
 Performance Tracking

Strategic Planning Definitions

Strategic Planning. A systematic process by which an
organization identifies and analyzes situations or conditions,
formulates conclusions and statements to address the situation,
devises alternative strategies, & applies resources to change
conditions.
 Vision: A value statement in which a preferred state exists or a
negative state does not exist.
 Mission: Is that which an agency has a calling to do.
 Approach or Scope: General focus or thrust, the broad limits,
within which planning and programming take place.
 Goal: The end to which a design tends; its aim.
 Strategies: Statements of different feasible courses of action
to resolve a situation.
Implementation Plng Definitions

Implementation Planning. The process of determination of a role
and definition of objectives; a selection of the programs required to
fulfill the role; and the development of time-phased plans for the
systematic achievement of those objectives (Harleigh B. Treker,
New Understandings of Administration).
 Program Activity: A project area as defined by the agency.
 Objectives: An objective is a specific, measurable event with
standards of achievement that are to be realized. A job or task
requiring both time and resources.
 Tactics: A job or task requiring both time and resources.
Tactics (or activities or services) state 'how', usually in
chronological order, a strategy is to be accomplished.
Performance Tracking Definitions





Performance Objective: Target level of performance expressed
as a tangible measurable objective.
Output Measure: A defined level of performance of program
efforts, specified in quantifiable, measurable terms.
Performance Indicator: Particular value or characteristic used to
measure outputs or outcomes.
Baseline of Performance: The initial or starting level of activity or
effort, expressed in qualitative or quantitative form.
Plan vs. Performance Measures: A periodic accounting of
appropriate program related measurements so as to compare
actual program results with established program targets.
Evaluation Techniques
The Organizing Framework
 Input Evaluation Techniques
 Process Evaluation Techniques- Numerical
 Process Evaluation Techniques- Qualitative
 Output Evaluation Techniques
 Impact Evaluation Techniques
 Global Objectives Evaluation Techniques

The Production Function
- The Organizing Framework




Inputs: The activities and resources put into an effort (if inputs
are adequate, then processes can be performed).
Process: The methods and procedures used to operate the
program, (if processes are managed efficiently and effectively,
then outputs will be produced).
Outputs: Direct results, often called deliverables or outputs,
themselves are not valuable in their own right (if outputs are
produced then the purpose will be achieved, assuming the correct
outputs have been identified).
Purpose: The expected impact, or outcome, what positive
change we expect to occur (if the purpose is achieved, then this
will achieve the goal).
Input Evaluation Techniques:

INPUT EVALUATION – Inputs are the activities and resources
put into an effort (if inputs are available, then some work process
can be undertaken). Input measures, such as staffing, work
hours dedicated and spending, are routine elements of most
budgets.
 Level of Resources: Number of staff, amount of resources,
number of projects, number of partners.
 Accounting Standards: Application of generally accepted
accounting principles in the management and control of
money.
 Employee Involvement: Application and development of the
workforce as a critical agency asset.
Process Evaluation Techniques
- Numerical:

PROCESS EVALUATION -- Process measurements consider the
methods and procedures used to operate the program, (correctly
managed process will produce outputs expected). This type of
evaluation helps determine how well the program is operating and
identifies opportunities for program improvement.
 Performance Goals: Both indicators and a base line to
provide immediate information on success or failure. Output
indicators define the amount of service units provided,
efficiency indicators are used to compute the costs, either in
terms of dollars or work hours, per unit of output
 Plan versus Performance Measures: Periodic tabulation
and comparison of indicators to determine if the program is
delivering services at the level expected.
Process Evaluation Techniques
- Qualitative:
Project Monitoring Tool: A standard operating procedures
manual, what business call a business plan, listing legal
requirements; administrative (and organizational)
qualifications; the customers targeted, the reporting forms to
be used; list of services to be performed and so on.
 Customer Satisfaction: Survey of how customers perceive
the services and their levels of satisfaction in terms of content,
format and timeliness.
 Competitive Benchmark: Identification of the best practices
of each operations (to improve standard operating procedures
and achieve economies of scale or scope).
 Internal Quality Management: Continuous improvement or
standardization of the service or product being delivered.

Output Evaluation Techniques:

OUTPUT EVALUATION -- Outputs or direct results, sometimes
called deliverables, are valuable as an intermediate measure
(assuming we have identified all the outputs necessary to achieve
the purpose)
 Project Deliverables: Identify direct products of an operation,
i.e., nature of activities; categories of services;
 Plan versus Performance: A routine quarterly or annual
report on whether planned results are achieved.
 Customer Demographics, to identify the types of customer
being served and whether the project is serving the customers
it should. Demographic profile information such as company
size, type of industry, etc. is needed.
Impact Evaluation Techniques:

IMPACT EVALUATION -- (Purpose) The expected impact,
outcome, or results showing what positive change are expected
to occur (if the purpose is achieved, then this will contribute to
the achievement of the global objectives.)
 Customer Benefit: Consider the nature of change or
progress within the firms or individuals assisted.
 Public Benefit (Impact): How has the program changed
lives such as jobs created and retained or new taxes
generated. What side effects have occurred?
 Cost Benefit: Finally, using REMI or other regional
econometric modeling technique, calculate the costs of the
program effort in relation to the benefit
Global Objectives Evaluation
Techniques:


Formative Analysis: Evaluation which deals with program
procedure or quality issues, to determine how to design a
program's operations. It is used to fine-tune and improve
program strategy and enhance delivery. It requires ongoing
input regarding program operation from experts, colleagues
and customers.
Summative Evaluation: Looks at whether the program
actually made a difference, summing up the overall program
usefulness. This type of evaluation requires descriptive
questions (what is happening); explanatory questions (why is
it happening that way); and evaluative questions (how well is
it going).