Transcript SEMINAR ON STATUS OF MEASURE LIFETIME WORK RTF EUL Kickoff Meeting, August 2011 Lisa A.

SEMINAR ON STATUS OF
MEASURE LIFETIME WORK
RTF EUL Kickoff Meeting, August 2011
Lisa A. Skumatz, Ph.D.
Skumatz Economic Research Associates
[email protected], 303/494-1178
PERSISTENCE / RETENTION
SERA
RETENTION / PERSISTENCE
 Persistence, measure life, EULs, 50% median,
in place and operable
 Protocols, best practices summaries on
samples, data collection, analysis, modeling,
comparisons, documentation
 Some variations in considering adaptations for
behavioral programs
SERA
REMAINING USEFUL
LIFETIME (RUL)







Conceptual issue – early replacement
Intervene at replacement  standard vs. efficient
Intervene early  original vs. efficient for “early” part
 standard vs. efficient for later period
Question is, length of time from end of life?
Few studies; 1/3 ad hoc
Welch & Rogers 2010;


Survey, Weibull, curves for residential equipment
System dynamics / stocks cohorts
 Strong application for behavioral
 adoption curves / timing / lifetimes? Measurement
issue
4
SERA
TECHNICAL DEGRADATION
 Technical degradation (TDFs)
 Addressed in CA-EM&V protocols
 Differences in decay? Very few primary studies
 2 effects - Technical degradation & behavioral /
operational based on quality of use & upkeep – need
studies on combination
 Behavioral very parallel; no studies
5
SERA
LIFETIMES / EULs STUDIES
 CPUC
 Protocols, results, measures with gaps
 Updated lifetimes in report
 All sectors, many measures  incorporated into
DEER (database for energy efficient resources,
energy.ca.gov/deer), protocols
 Reviewed program, savings, measures,
sampling, field work, data validation, analysis,
justifiability
 Strengths, weaknesses, “score”
 Hundreds of millions in shareholder claims
 SCE / DEER
 CPUC / CIEE – nationwide review
 Individual studies
SERA
SCORING CRITERION ON CPUC
RETENTION PROJECT-SUMMARY
Criterion
Description
CPUC
Protocol
Did they meet the CPUC protocol of using the top 10 measures or measures that account
for 50% of the program savings? A score of 3 means that they just met the criteria, but the
discussion of the measures selected and the program savings may have been weak; 5 was
the highest score possible.
Describe the sampling strategy whether it was stratified, random, etc. Describe the sampling
and stratification basis and process. (i.e., customers, rebates, installations, etc.) Is there
adequate discussion of the sampling process? Discuss problems with the strategy / basis if
appropriate. A score of 3 indicated that the sampling strategy and basis was sufficient.
Sampling
Strategy
Fieldwork
and
Validation
What type of fieldwork was used to collect the data?
How well is the data collection process explained? Describe the field work and validation of
the data. This score reflected the adequacy of the field work done in collecting data and the
thoroughness with which the data was validated.
Methodology Is the description of the methodology adequate? Discuss the methodology implemented in
the study. Were alternative models estimated or considered? Was the final selection of
models well justified? Was data attrition sufficiently explained? Does the study adequately
discuss the results of its model?
SERA
EUL STUDY – BEST PRACTICES
ISSUES
 Small sample size
 Population list problems; needs vary by lifetime
of measure
 Omitted other models
 Potential variations by measure type; easy to
add; improves potential fit; not a priori
 Ambiguous failure dates
 Frequent surveys; “bound”, labeling, phone
survey for some
 Poor documentation in report
 Methodology, weighting, failures, conclusions
 Results not put in context
 Over time; other studies
SERA
EUL VALUES USED IN US RESIDENTIAL
Lighting
HVAC
Shell & Other
Lighting – CFL Bulbs: 6-8
years, with some recent work
starting to incorporate
variations based on
assumptions about hours per
day that the bulb operates
HVAC replacement – 15-25
years
Duct sealing and air sealing –
each 15-20 years
HVAC and water heating in
Energy Star – 15-25 years
Insulation 20-25 years
Room A/C – 11-15 years
Windows – 20-35 years
Programmable thermostat –
10-12 years
Weatherization – combined
measures – 20-25 years
Hardwired fixtures – 15-20
years for interior and exterior
fixtures
Lamps (table or touchier) – 510 years for most studies[1],
depending on type
Duct insulation – 20 years
Whole house fans – 25 years
Attic ventilation fans with
thermostat controls – 19 years
Pipe wrap – 10-20 years
Tank temperature turn down –
4-7 years
Occupancy sensors – 10-15
years
9
Note – some differences in values between CA, NW, NE values –
Especially usage / weather based…
SERA
EUL VALUES USED IN US COMMERCIAL
Lighting
HVAC
Lighting – CFL Bulbs: 3.4-6 years with some
recent work starting to incorporate variations
based on assumptions about hours per day that
the bulb operates in business locations
Packaged AC / HP – 12-15 years
Fluorescent fixture – 11-16 years
Hardwired CFL – 10-15 years
HID (interior & exterior) 13-15 years
Chillers – 19-23 years
Economizers – 7-15 years
Energy Management Systems (EMS) – 10-15
years
Motors – 13-20 years
Occupancy sensors – 8-15 years
Daylighting dimming – 9-10 years
10
SERA
ISSUES IN EULs
 Process values lacking (small sample size)
 Some dependent on operating assumptions
 Some end-uses missing / gaps





CFLs, lighting – updated with operating hours
Measures with limited (unreliable or zero) studies – some
with much attributable savings
 Models of air compress, A/C, cook, hvac, refrig/freezer,
process, shell, (motors/pumps, dryers, lighting),
ASD/VSD in some sectors
Missing for plug loads
Building shell – at least verify
Priority depends on future savings, rarity, variations;
waiting hurts EUL data
 Trend toward simplified tables, BUT research shows
strong variations in turnover by business type
11
SERA
EULs FOR BEHAVIORAL
PROGRAMS
 Missing for behavioral / educational programs
 2 studies
 CBSM
 Best practices with nuances - Partial retention;
frequency of data collection; large surveys and
random assignment
 Retention of “upstream” complicated
12
SERA
MEASURE LIFETIME
ESTIMATES / RETENTION
Behavioral / Outreach Programs
Examples
Only 1-2 examples (Harrigan), short term; a few
recent additions (Peters)
Gaps / Issues
Missing for behavioral, education/ training,
commissioning, others
Important –parallel concept to technical degradation
Need “partial” retention concept
Lack of exploration; Months? Years?
Lack of existing research on behavioral retention
Remaining useful lifetime / baseline issues
Best Practices
/Rec’m for
Behavioral /
Outreach
Presumably similar approach, but consider data
acquisition differences and data uncertainties
SERA
VARIATIONS IN
PERSISTENCE
EULs
General results
Early work in NW – gravitation to CA values. CA requires ex post
statistical verification, but for subset of measures – led to
refinements. Some measures with inadequate / missing –
especially behavioral
Variations by
Program type
Almost all EUL results are by measure, not by program design or
incentive provided. Not clear if they should vary by program type.
Variations for
behavioral vs.
measure-based
programs
Missing for behavioral programs including education / training,
commissioning training, and similar programs.
14
SERA
ISSUES / PROBLEMS






Best practices
Results / gaps
TDF
RUL
Behavioral
Key component of program savings



Potential bias away from new, creative
Risk
Complexities for behavioral
 Little primary research / dormant / agreement
15
SERA
CONCLUSIONS – MEASURE
LIFETIMES


Measure lifetimes are a key element in the calculation
of energy savings from energy efficiency programs
Measure lifetimes (and methods) are fairly consistent
for many measure-based programs in residential and
commercial sectors




16
Issue of simplified EUL tables / caution
Shortage of primary research on technical degradation
(TDF); shortage of research on RULs
Virtual absence of studies addressing retention or
persistence of energy savings from behavioral and
education programs
Identifying the measure lifetimes of behavioral and
education programs is complicated as more media
messages on behavior and education “bleed” across
territories
SERA
RECOMMENDATIONS – MEASURE
LIFETIMES

Conduct measure lifetime studies on:





Process equipment, some shell measures, cooking, refrigeration,
and air compressors
Conduct technical degradation studies that account for
mechanical and behavioral performance-related changes
Conduct studies on retention or persistence of energy
savings from behavioral and education programs
Require new behavioral programs to conduct retention
assessments every year or two
Apply different evaluation methods to a variety of behavioral
programs
17
SERA
NEXT STEPS / RESEARCH
NEEDED
Behavioral / Outreach Programs
Retention
Studies of retention of behavioral
 Consider data acquisition differences and data
uncertainties (date?, data collection frequency)
Best practices with nuances – partial retention,
retention of “upstream” complicated
RUL research
Comparison of results for key factors affecting
retention before values can be adopted / transferred as
with measures currently (quality? Audience?, other?)
SERA
CONTEXT DISCUSSION
 Investment dollars at risk
 Debate over precision


Granger – evaluation to avoid making wrong decisions
Multiple applications… varying precision needs?
 Program decisions to be advised include:


Public dollars responsibly spent
Apportionment of dollars between strategies
 Precision based on value / cost of possibility of wrong
decision…

Yes/No vs. precise level of shareholder dollars…
SERA
KEY USES OF EVALUATION
RESULTS / PRECISION NEEDS
By Use / Application
Increasing Rigor
(& cost)
Assessing progress
Deemed
By Considerations
Uncertainty small, low cost, small
value implications, e.g.small resid
programs
Information on C/E
DSM planning input (tradeoff)
Paying participant
Paying utility incentive
Alternative supply
20
High certainty needed, large
Detailed M&V,
site verification, +impacts, large cost
SERA
Persistence
21
SERA
QUESTIONS?
Project Manager contact:
Lisa A. Skumatz, Ph.D.
Skumatz Economic Research Associates (SERA)
762 Eldorado Drive, Superior, CO 80027
Phone: 303/494-1178
Email: [email protected]
SERA