Project Work Plan - RTF Guidelines for Custom Measures and Program Impact Evaluation

January 17, 2012 Regional Technical Forum Presented by: Michael Baker, SBW Richard Ridge, Ridge and Assoc.

Development Process

  Identify critical elements to be addressed in each new appendix  Impact evaluation  Interviews with 13 evaluation managers   Subcommittee discussion and analysis Custom protocols    Review recent BPA work Review documents provided by subcommittee and others Subcommittee discussion and analysis Develop appendices for Guidelines 12/7/2010 2

Interview on Impact Evaluation Critical Elements

    Interview will be completed with 13, representing BPA, ETO, regulators, public utilities and private utilities Subcommittee also asked to answer questions in written form All responses (from interview or written) will be summarized and posted on subcommittee web page Other RTF members invited to complete written response by 1/20/12 3 12/7/2010

Evaluation Manager Interviews

  Identify existing evaluation or reporting protocols Discuss how RTF Guidelines should address the following topics (or others identified by respondents) Reliability Peer Review Research Design Transferability Comparability Use of Program Access to Studies Transparency Administrator Data Frequency Site-Specific M&V Methods Sample Design Other Issues 4 12/7/2010

Identify Custom Protocol Critical Elements

  Review recent BPA work 

Final Report - Research Supporting an Update of BPA’s Measurement and Verification Protocols

        

Regression for M&V Reference Guide Sampling for M&V Reference Guide Existing Building Commissioning an M&V Protocol Application Guide Measurement and Verification Protocol Selection Guide End-Use Metering Absent Baseline Measurement Energy Use Indexing Protocol Energy Modeling Protocol Equipment or End-Use Metering Protocol Engineering Calculations with Verification Protocol

Review other protocols identified by subcommittee or participants in Evaluation Manager interviews 5 12/7/2010

Schedule

        Subcommittee met to review work plan 12/8/11 Work plan review by RTF, 1/17/12 Evaluation manager interviews 1-10 thru 1-19 Subcommittee meetings to select critical elements in meetings during week of February 6th Draft Guidelines appendices prepared and distributed prior to March 13 th RTF meeting Discussion during March RTF meeting Redraft Discussion during April RTF meeting leading to adoption of plan for further work 12/7/2010 6

Use of Data Collected by Program Administrators

  Current Guidelines for Impact Evaluations  UES measures – Delivery Verification  Standard Protocol measures – Faithful application  Custom Protocol - Silent Pre/Post Designs     Basis for UES and Standard Protocols Are they needed (and when) in order for program evaluations to have comparable reliability Difficult for 3 program rd party program evaluations unless data is collected by the How far can we go in specifying measurement and sampling procedures and impose these on program administrators 7 12/7/2010