Transcript Document

Market Transformation Programs:
What they are. How to evaluate them.
Kansas Corporation
Commission
Mitchell Rosenberg, Vice President
Topeka, Kansas
March 26, 2008
Experience you can trust.
Overview
 Market Transformation: Definitions and Reality
 Importance of MT Concepts in Energy Efficiency
Program Design and Evaluation
 Key Challenges in MT Evaluation
 Steps in Meeting MT Evaluation Challenges:
examples from actual programs & studies
 Lessons Learned
2
Definition of Market Transformation
 Market Effect: “a change in the structure or
functioning of a market or the behavior of
participants in a market that results from one or
more energy efficiency program efforts.”
– Typically, these efforts are designed to increase
the adoption of energy-efficient products, services,
or practices, and are causally related to market
interventions.
 Market Transformation: Market effects that
persist once supporting programs are terminated
3
MT References the Product Life Cycle
80%
Market Share
70%
60%
50%
40%
30%
20%
10%
0%
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20
Year
Introduction
Early Acceptance
Take Off
Maturity
SUPPLY CHAIN
Smaller competitors or new
entrants producing
No standardization
Very high prices
Performance problems
Distribution mostly in nonstandard channels
Limited product lines
Little marketing support
1 – 2 larger competitors
enter
Product standards develop
Prices high relative to
standard
Improved performance
Limited distribution
Limited model line build out
Some marketing support
Most large competitors offer
product
Product standards adopted
voluntarily
Prices are higher but in line
with standard models
Product lines built out
Good distribution
Manu. Marketing support
All major competitors in
market
Mandatory standards
Prices approach those of
standard
Producers compete on
price and features
Distribution via all channels
Retailer marketing support
More customers aware
Continued skepticism of
product claims
Small expansion of market
beyond early adopters
Strong demand in
advanced segments
Some demand in all
segments
Strong demand in all
segments
CUSTOMERS
Low level of awareness
Skepticism of product
claims
Only early adopters buying
4
Policy & Programs to Accelerate MT
80%
Market Share
70%
60%
50%
40%
30%
20%
10%
0%
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20
Year
Introduction
SUPPLY CHAIN ORIENTED
Government lab R&D
Sponsored corporate R&D
Technology road mapping
Mediate technology
standard setting
Development of
performance metrics and
testing protocols
DEMAND ORIENTED
Purchase of prototypes or
early models
Develop and publicize case
studies of applications
Early Acceptance
Take Off
Maturity
Vendor technical and sales
training
Co-advertising
Vendor merchandising
support
Development & promotion
of voluntary product
efficiency standards
Product testing
Vendor technical and sales
training
Co-advertising
Vendor merchandising
support
Upstream product subsidies
Initiate consideration of
higher product standards
Develop common service
specifications
Mandatory codes and
standards
Promulgate higher
voluntary standards
Bulk purchase
Customer education
Rebate programs
General EE public relations
Customer education
Rebate programs
General EE public relations
Continued customer
education
Rebate programs for higher
efficiency units only
5
EE Programs do help transform markets:
The case of electronic ballasts
90
300
70
60
200
50
150
40
30
100
20
50
10
R&D: LBL &
CA utilities
Government
Procurement
DSM Programs
@ $50m/yr
1997
1996
1995
1994
1993
1992
1991
1990
1989
1988
1987
1986
1985
1984
1983
0
1982
0
1981
Annual Ballast Shipments, Millions of Units
250
DSM Programs
@ $150m/year
Programs Underway
6
Annual Value of Construction, 1992 Dollars, $ Bil.
80
Electronic
Magnetic
Non-residential
Construction
Value
The case of resource-efficient clothes
washers
40%
35%
Market Share
30%
25%
20%
15%
10%
5%
0%
1996
Local Programs in Effect
ENERGY STAR & Other
Federal Events
1998
1999
2000
2001
2002
2003
2004
2005
2006
1998
2000
2002
2004
2006
12
50
70
100+
90
st
1 ENERGY
STAR
specification
(1997)
DOE
announces new
min. standard
National
promotions
initiated
Federal min
standard
increased
(2007) Fed min
standard and
ENERGY STAR
increased
Manufacturers producing
ENERGY STAR models
8
14
17
21
24
Number of ENERGY STAR
models
18
35
84
125
212*
* Includes only those that meet the revised 2007 specification.
7
Why MT Concepts are Important in
Program Design & Evaluation
 Program designs must be matched to current
stage of development to be cost effective
 Harnessing supply side motivations enhances
cost effectiveness
 High GHG reduction goals
– Need continuous pipeline of energy-efficient
products, services, design practices
– Need to extend adoption of established products to
all relevant market segments
 In some cases, market research provides most
fruitful approach to NTG evaluation
8
Key Challenges in MT Evaluation
 Effects occur over a long time frame
– Events of interest span decades
– Program planning & evaluation cycles typically 3
years or less
 Multiple program sponsors and policy makers
are involved
– Often coordinate activities among jurisdictions
– How tease out effects of one state’s programs?
 Data are difficult to obtain
– Manufacturer shipment data often proprietary
– Distributors, contractors, retailers, designers tend
to be small, scattered, disorganized
9
Steps in Addressing the Challenges of
MT Evaluation
 Market Assessment/Program Logic Model
 Program History Development: Market Presence
Indicators
 Selection of analysis objectives and strategy (ies)
 Appraisal of results/Adjustments for next rounds
 Close parallels to steps in program design and
revision
10
Program Logic Model: Home
Performance
11
Market Assessment/Program Theory
 Key Objectives
– Location on the product life cycle
– Identification of key market actors & segments, motivations,
barriers
– Assess data availability for program management and
evaluation
– Get program planners, implementers, regulators on same
page re: short & long term goals and expectations
 Timing: Should begin in program design phase
 Resources: Avoid over-investment
– Secondary data & reports useful
– Selected local primary research often very useful
12
Program History/Market Presence
 Important to build reference for market effects
and attribution analysis
 Should include national as well as local events &
trends over time
 Data to capture
– Program activities: marketing, training, incentives
– Participation: customer and vendor, numbers and
characteristics of participants and measures,
timing, region
– Efficiency levels supported v. baseline practice
13
Example: EVT clothes washer
programs
2004
2002
2000
1998
NATIONAL PROGRAMS AND MARKETS
12
Local Programs in Effect
ENERGY STAR & Other Federal
Events
st
1 ENERGY STAR
specification
DOE announces
new min. standard
100+
70
50
Federal min
standard increased
National
promotions
initiated
Manufacturers producing
ENERGY STAR models
8
14
17
21
Number of ENERGY STAR
models
18
35
84
125
6.2%
9.3%
16.0%
27.2%
24.9%
27.3%
35.7%
49.6%
Number of Partic. Retailers
40
58
70
75
Number of Incentives Paid
1,950
2,476
2,699
4,129
7
938
1,401
2,131
ENERGY STAR Market Share
LOCAL PROGRAMS AND MARKETS
ENERGY STAR Market Share
Number of ENERGY STAR units
sold without rebates
14
Analysis Strategies
 Market Effects Indicators
– Market share: sales, prevalence of practices, fleet
efficiency indicators (e.g. mean EERs, MPG)
– Price trends
– Changes in codes and standards
 Attribution Analysis
– Cross sectional
– Time Series
– Self-reporting
– Historical
 Best to attempt multiple indicators and
attribution analysis approaches
15
Net Effects of Vermont CW programs:
Cross Sectional & Historical Approach
Model Estimated Mkt. Share
Year
Actual
Market Share
No
Program
No Program
+ No Cum. Effect
% of E Star Sales
Attributed to Program
Program
Only
Program +
Cum Effect Variable
2001
22.6%
12.9%
n/a
34%
n/a
2003
52.6%
46.8%
43.4%
7%
26%
2004
48.9%
46.1%
43.4%
6%
12%
 Results of cross-sectional modeling
– Dependent variable: State level Energy Star CW market share
– Independent variables:
 Program indicators
 Customer Demographics & Energy Prices
 Change indicator
16
VT clothes washer programs: What’s
happening to local program influence
60.00%
50.00%
40.00%
No Program
US Average
Program
VT
30.00%
20.00%
10.00%
0.00%
 Methods
1999 2000
2001 2002
2003 2004
– ‘No-program Area’ market shares rising faster than market areas
 Market conditions
– Impending federal minimum standard changes
– Profitable product for manufacturers
 Fair representation of local program effects?
17
CA commercial lighting programs:
presence in the market
Sponsor Electronic Ballasts Rebated
US EB Shipments
US Utility Ballasts Rebated
As % of Program
As % of all
Area EB Sales
US EB Shipments
As % of US
mm Units
mm Units
mm Units
1992
13.3
0.9
90%
7%
9.4
71%
1993
24.5
1.2
90%
5%
13.2
54%
1994
24.6
2.0
75%
8%
12.2
50%
1995
32.9
1.9
67%
6%
11.1
34%
1996
30.3
1.3
57%
4%
10.5
35%
Total
125.6
7.3
68%
6%
56.4
45%
18
EB Shipments
General Challenges from C&I
Programs




Little available market share data
Complex ‘measures’
Greater customer market segmentation
Multiple levels of supply chain with direct
influence on project-level equipment selection
and design
 Multiple decision makers and criteria in one
customer
 Generally greater reliance on ‘weight of
evidence’ and self-reported approaches than in
residential.
19
Other Potential Approaches to
Attribution or Baseline Development
 Diffusion modeling
 Delphi or other expert judging
 Conjoint analysis of preferences for efficient
substitutes for current products
20
Lessons Learned
 Generate as detailed a story as possible
– Know the history of the program and its relatives
– Know the history of the market and technology
– Know what other programs are doing
– Develop the program logic with local stakeholders
 Know the available data resources
– Sampling, contents, collection methods
– Know what others have done
 Design data collection to the attribution strategy
or strategies
21
Lessons Learned: Maximizing data
opportunities
 Quality of Indicators
– Accuracy, face validity, bias (lack of same)
 Replicability (ability to support historical or time
series analysis)
– Sample frame: captures full population, updated
regularly, documented compilation
– Sampling: keep it kosher and document it
 Comparability (supports cross-sectional analysis)
– Same definitions as data collected elsewhere
– Capitalize on channels to reach national markets
22
The Bigger Picture
 Play well with others
– Markets addressed by local programs are national
and international
– Many program operators are heavily involved
already – get to know them
– Initiatives under way to procure sales & shipment
data
 Independent program influence?
– Many local programs already coordinate
operations, or are developed in explicit reference
to each other (e.g.) codes & standards
– Why try to tease apart effects?
23