2013-14 INDUSTRIAL, AGRICULTURAL AND LARGE COMMERCIAL (IALC) ROADMAP

CUSTOM IMPACT EVALUATION

WEBINAR TO PRESENT RESEARCH PLAN

Presentation

July 28, 2014

Kay Hardy, Kris Bradley

RESEARCH PLAN PRESENTATION AGENDA – JULY 28, 2014 » Custom Program Impact Evaluation Research Plan posted to EnergyDataWeb:

• The plan may be downloaded from http://www.energydataweb.com/cpuc/. Select the search tab and then select Evaluation Firm: Itron and Portfolio Cycle 2013-2014. Click the search button and you will see the plan file for downloading.

» Introduction

Kay Hardy (10 min.)

» 2013-14 IALC Research Plan

Kris Bradley (50 min.)

» Group discussion and wrap up

(25 min.)

2013-14 IALC CUSTOM RESEARCH PLAN

» Research Plan was posted for review July 15, 2014» The Research Plan:

• Overview of IALC Work Orders• Composition of 2013 Claims• Evaluation Components and Approaches• Sampling Plan• Timeline and Budget

» Two Work Orders are in place to cover the major IALC Evaluation Activities• Custom measures• NRNC whole-building measures

» Data requests for IOU documentation are proceeding

2013 CLAIMS SUMMARY

Note: SDG&E portfolio gas savings claims are less than the IALC savings claims because the portfolio savings include a substantial component of negative savings from lighting interactive effects that do not generally impact IALC claims.

Energy Savings Claims by IOU

IOUElectric Energy

(GWh)Electric Demand

(MW)

Gas Energy (Million Therms)

IALC Savings ClaimPGE 235 37 21SCE 214 34 1SCG 0 0 13SDGE 50 6 0.8IALC Total 499 77 36

Portfolio Savings ClaimPGE 1,498 243 28SCE 783 140 -3SCG 8 6 26SDGE 322 47 0.4Portfolio Total 2,611 435 51

IALC Percentage of ClaimPGE 16% 15% 75%SCE 27% 24% -SCG 0% 0% 52%SDGE 15% 12% 205%IALC Total 19% 18% 70%

CURRENT IALC WORK ORDERS

» The Custom Impact Evaluation — featuring appropriately rigorous M&V and NTG evaluation — will address gross and net impact measurement after isolating a heterogeneous subset of non-deemed claims in the IALC population.• Focus of the research plan posted and the discussion today

» The NRNC Impact Evaluation will feature M&V and NTG evaluation similar to the Custom Impact Evaluation, but applied to whole-building projects. • Research plan not yet posted

» Custom Lighting Impact Evaluation activities are being re-mapped to the downstream lighting impact evaluation

CLAIMS BREAKDOWN

Category IOUIOU Extract

RecordskW kWh Therms

% kW of IALC

% kWh of IALC

% Therms of IALC

IALC Total All 10,716 76,860 499,167,481 35,867,550 100% 100% 100%

PGE 3,198 37,024 235,077,123 21,275,987 48% 47% 59%SCE 6,469 34,177 214,313,081 560,991 44% 43% 2%SCG 178 0 0 13,209,270 0% 0% 37%SDGE 871 5,659 49,777,277 821,302 7% 10% 2%

PGE 676 0 0 0 0% 0% 0%SCE 37 0 0 0 0% 0% 0%SCG - 0 0 0 0% 0% 0%SDGE 2 0 0 0 0% 0% 0%

PGE - 0 0 0 0% 0% 0%SCE 4,986 2,682 11,645,838 0 3% 2% 0%SCG - 0 0 0 0% 0% 0%SDGE - 0 0 0 0% 0% 0%

PGE 284 10,469 31,649,787 584,886 14% 6% 2%SCE 82 6,493 20,246,798 130,335 8% 4% 0%SCG - 0 0 0 0% 0% 0%SDGE 432 2,749 9,287,030 30,107 4% 2% 0%

PGE 2,238 26,555 203,427,336 20,691,100 35% 41% 58%SCE 1,364 25,002 182,420,445 430,656 33% 37% 1%SCG 178 0 0 13,209,270 0% 0% 37%SDGE 437 2,910 40,490,247 791,195 4% 8% 2%

Custom Total All 4,217 54,467 426,338,028 35,122,221 71% 85% 98%

Zero Claims

Pump Test

NRNC Whole Building

Custom

IOU Total

ADJUSTMENTS TO ORIGINAL EM&V PLAN

Study Name Study Type 2013-14 Budget

Industrial/ Agricultural/ Large Commercial Impact $4,000,000

Custom Project Lower Rigor Impact $500,000

Nonresidential New Construction Impact $750,000

Selected Third Party Programs Impact $1,000,000

NRNC Occupancy Profile Market $100,000

Commercial and Industrial Maintenance Practices, Patterns and Persistence Market $200,000

TOTAL

$6,550,000

Road Map Proposed WO Name Evaluation Type Draft Budget

IALC Custom Impact Evaluation Study Impact $3,892,222

IALC NRNC Impact and Market Evaluation Study Impact and Market $1,404,374

IALC Lighting Measure Impact Evaluation Study Impact $1,253,404

TOTAL $6,550,000

Original EM&V Plan Studies and Budgets

Proposed Work Orders and Budgets

SAMPLING PLAN SUMMARYImpact Evaluation Component PY2013 PY2014 TOTAL

Gross Impact Points Pre-2013* (GRR+NTG) 114 114

Gross Impact Points 2013* and later (76 GRR, 39 NTG) 76 76

Gross Impact Points (GRR+NTG) 125 125

Incremental NTG-Only Points - 50 50

TOTAL 190 175 365

* Refers to the agreement date.

CUSTOM POP. AND GROSS IMPACT SAMPLE

All Projects Claimed in 2013 Proposed Impact Sample All Projects

Pre-2013 Projects 2013+ Projects Pre-2013 Projects 2013+ Projects

IOU

Strata

Boundary M

MB

tu

N Sam

pling U

nits

Stratum

Weight

MM

Btu

N Sam

pling U

nits

Stratum

Weight

MM

Btu

N large

projects

N sm

all projects

N large

projects

N sm

all projects

N total

projects

Percent of

Population

PGE

1 108,541 6 762,850 1 117,000 6 - 1 - 7 100%

2 30,728 17 912,070 1 40,391 6 - 1 - 7 39%

3 13,006 40 771,435 7 134,592 - 8 - 5 13 28%

4 4,696 114 849,826 12 78,917 - 8 - 5 13 10%

5 0 723 736,803 218 180,346 - 7 - 8 15 2%

All - 900 4,032,985 239 551,246 12 23 2 18 55 5%

SCE

1 35,771 6 376,800 1 43,249 6 - 1 - 7 100%

2 11,776 18 375,469 2 44,672 6 - 2 - 8 40%

3 5,545 47 342,902 10 82,397 - 8 - 5 13 23%

4 2,059 106 358,162 20 66,923 - 7 - 5 12 10%

5 0 484 309,897 240 113,875 - 7 - 7 14 2%

All - 661 1,763,231 273 351,116 12 22 3 17 54 6%

SDGE

1 26,458 2 105,848 0 0 2 - - - 2 100%

2 13,847 6 110,364 0 0 3 - - - 3 50%

3 5,682 10 80,022 5 36,863 - 6 - 3 9 60%

4 2,209 25 80,424 9 35,190 - 6 - 7 13 38%

5 0 136 70,519 83 42,196 - 6 - 10 16 7%

All - 179 447,178 97 114,249 5 18 0 20 43 16%

SCG

1 176,435 2 369,132 0 0 2 - - - 2 100%

2 111,279 2 264,750 0 0 2 - - - 2 100%

3 26,000 7 215,353 1 48,223 - 6 - 1 7 88%

4 10,972 12 196,843 5 80,645 - 6 - 5 11 65%

5 0 94 215,801 36 54,890 - 6 - 10 16 12%

All - 117 1,261,879 42 183,758 4 18 0 16 38 24%

All IOUs, 2013 1,857 7,505,272 651 1,200,369 33 81 5 71 190 8%

CUSTOM POPULATION AND NTG SAMPLE

All Projects Claimed in 2013 Proposed NTG Sample

Pre-2013 Projects 2013+ Projects N Completes

IOU Strata Boundary MMBtu

N Sampling

Units

Stratum Weight MMBtu

N Sampling

Units

Stratum Weight MMBtu

Pre-2013 (complete overlap

with GRR)

2013+ (some

overlap with

GRR)

Total

PGE

1 108,541 6 762,850 1 117,000 6 1 7 2 30,728 17 912,070 1 40,391 6 1 7 3 13,006 40 771,435 7 134,592 8 3 11 4 4,696 114 849,826 12 78,917 8 3 11 5 0 723 736,803 218 180,346 7 3 10

All - 900 4,032,985 239 551,246 35 11 46

SCE

1 35,771 6 376,800 1 43,249 6 1 7 2 11,776 18 375,469 2 44,672 6 2 8 3 5,545 47 342,902 10 82,397 8 3 11 4 2,059 106 358,162 20 66,923 7 3 10

5 0 484 309,897 240 113,875 7 3 10

All - 661 1,763,231 273 351,116 34 12 46

SDGE

1 26,458 2 105,848 0 0 2 - 2 2 13,847 6 110,364 0 0 3 - 3 3 5,682 10 80,022 5 36,863 6 3 9 4 2,209 25 80,424 9 35,190 6 3 9 5 0 136 70,519 83 42,196 6 3 9

All - 179 447,178 97 114,249 23 9 32

SCG

1 176,435 2 369,132 0 0 2 - 2 2 111,279 2 264,750 0 0 2 - 2 3 26,000 7 215,353 1 48,223 6 1 7 4 10,972 12 196,843 5 80,645 6 3 9 5 0 94 215,801 36 54,890 6 3 9

All - 117 1,261,879 42 183,758 22 7 29

All IOUs, 2013

1,857 7,505,272 651 1,200,369 114 39 153

TIMELINEActivity Start Date End Date

Evaluation Reporting and Feedback Memoranda First report covering 2013 Feb 2015 Mar 2015 Final Report covering 2013 and 2014 Jan 2016 Mar 2016 Evaluation Project Management and Coordination

Regular and ad-hoc coordination with CPUC June 2014 Ongoing Administrative reporting June 2014 Ongoing

Evaluation Plan

Coordination for plan development with CPUC; review of tracking data for programs and measures

May 2014 Jun 2014

Draft / final evaluation plan submitted to CPUC contract manager Jun 2014 Jul 2014 IOU review and comments on evaluation plan Jul 2014 Jul 2014 Public posting and review of evaluation plan Jul 2014 Incorporate revisions into final plan for public release and final CPUC

approval Jul 2014 Jul 2014

Activities Supporting 2013 Claims Year Impact Report

Collect field data and conduct document reviews, conduct phone surveys for NTG

Jul 2014 Jan 2015

Analyze field data and document findings, analyze NTG phone survey results Jul 2014 Jan 2015 Prepare and submit first annual draft impact report to CPUC contract manager Feb 2015 Mar 2015 Public posting and review of 2013 Claims Year impact report Mar 2015 Incorporate revisions into 2013 Claims Year impact report for public release

and final CPUC approval Mar 2015

Early Feedback on Program Procedures and Free Ridership

Analyze program changes since 2010-12 evaluations, assess compliance with program rules / procedures, free ridership and program influence (presented in evaluation reports)

Jul 2014 Mar 2016

Assess compliance with program rules / procedures, free ridership and program influence (via memorandum)

Jul 2014 Jan 2016

Activities Supporting Final Report

Collect field data and conduct document reviews, conduct phone surveys for NTG

Apr 2015 Jan 2016

Analyze field data and document findings, analyze NTG phone survey results Dec 2015 Feb 2016 Prepare and submit second annual draft impact report to CPUC contract

manager Feb 2016 March 2016

Public posting and review of second annual impact report Apr 2016 Incorporate revisions into second annual impact report for public release and

final CPUC approval May 2016

GROSS IMPACT EVALUATION» This activity will include evaluation of nonresidential, custom-type projects after

they have been installed by the end user and have had savings claimed by the IOU programs.

» This ex-post evaluation activity will focus on estimation of gross realization rates for statistically representative samples of custom projects. » The gross impact evaluation activities will utilize project-specific measurement

and verification (M&V) and application review samples to estimate gross realization rates.

» Greater M&V rigor is planned for large projects in sampling strata 1 and 2. Analysis will most often involve additional on-site metering with extensive engineering analysis to calculate the ex-post impact of selected projects.

» Smaller projects will receive a less rigorous approach in order to most economically analyze the sample points. While on-site verification and some data collection will occur in each of these cases, additional site measurements will be limited to spot measurements and collection/verification of available data (i.e., EMS/SCADA/ex-ante metering).

» 190 PY2013 points and 125 PY2014 points.

PROJECT PRACTICES ASSESSMENTS» Project Practices Assessments (PPAs) are enhanced claims reviews

that will be incorporated into the 13/14 IALC Custom Impact evaluation. » The purpose of the PPA process is to build upon the results of the Low

Rigor Assessment (LRA) process that was part of the 10/12 evaluation in a manner consistent with claims review requirements that are part of ESPI.

» Nested within all gross impact points.

NET TO GROSS EVALUATION

» The proposed NTG evaluation will consist of an interview-based evaluation for a representative sample of selected projects, yielding NTG ratios, and the use of ratio estimation to aggregate to program-level savings estimates.

» Similar to the gross impact approach, a mix of rigorous professional interviews and basic rigor CATI surveys will be conducted.

» 153 PY2013 points and 175 PY2014 points.

NTG OVERALL APPROACH

» Core approach is self-report, as in ‘10-’12. Other approaches are generally not feasible for these types of customers.

» For all sampled projects, the participating customer will be interviewed.» For all participants that are account-managed, the account reps will

also be interviewed to help assess their roles/influence and to help identify the correct respondent.

» If the participant scores the vendor importance highly, a trade ally interview will also be conducted to discern their role/influence.

» Additional market assessments will be conducted for some equipment markets suspected of becoming standard practice, as in ‘06-’08.

» There is a separate work order to address Spillover. Spillover questions will be included in the IALC phone surveys. However the IALC and other Nonresidential program spillover data will be analyzed under this separate Spillover work order.

PROGRESS REPORTING

» Monthly progress reports will be sent to CPUC.» A tracking database will be used to track progress on M&V assignments and NTG

assignments. Built on the experiences of WO033, a revised platform is planned that will incorporate both M&V and NTG activity in one database.

» Periodic meetings, in addition to the above and at a frequency as directed by CPUC, will be held to discuss accomplishments, plans and pending activities / decisions.

Deliverable/Product Revisions By

July 2014 Quarterly Updates

Report By March 2015

Report By May 2016

Research Plan Review and Incorporation of Comments

X

Early feedback Ongoing when

available

Assess free ridership and program influence

X X

Compile report containing consolidated impact results, findings, and recommendations

X X

DISCUSSION