Energizing Indiana 2014 Report

STATE OF INDIANA

INDIANA UTILITY REGULATORY COMMISSION

IN THE MATTER OF THE COMMISSIONS INVESTIGATION, PURSUANT TO IC 8-1-2-58, INTO THE EFFECTIVENESS OF DEMAND SIDE MANAGEMENT (DSM) PROGRAMS CURRENTLY UTILIZED IN THE STATE OF INDIANA, INCLUDING AN EXAMINATION OF ISSUES THAT COULD IMPROVE THE EFFECTIVENESS OF DEMAND SIDE MANAGEMENT PROGRAMS IN THE STATE, INCLUDING CONSIDERATION OF THE ESTABLISHMENT OF AN INDEPENDENT DSM ADMINISTRATOR MODEL ON A STATE-WIDE BASIS RESPONDENTS: ALL JURISDICTIONAL ELECTRIC AND GAS UTILITIES IN THE STATE OF INDIANA

) ) ) ) ) ) ) ) ) ) ) ) ) ) )))

CAUSE NO. 42693-S1

DEMAND SIDE MANAGEMENT COORDINATION COMMITTEES RESPONSE TO JUNE 4, 2015 DOCKET ENTRY

On June 4, 2015 the Indiana Utility Regulatory Commission issued a Docket

Entry in this Cause requiring the Demand Side Management Coordination Committee

(DSMCC) to file (1) TecMarket Works final report concerning the evaluation,

measurement, and verification (EM&V) for Program Year 3 and the entire program

cycle (2012-2014), including cost effectiveness of the programs over the three-year

klaytonNew Stamp

period (the "Final Report"), and (2) TecMarket Works' summary or public version of

the Final Report.

In compliance with the June 4, 2015 Docket Entry issued in this Cause, the

DSMCC hereby submits the attached Final Report, as well as the public report.

Attorney for NORTHERN INDIANA PUBLIC SERVICE COMPANY

-2-

-3-

Certificate of Service

The undersigned hereby certifies that the foregoing was served this 9th day of

June, 2015, via email transmission addressed to:

Jeffrey M. Reed Randall C. Helmen Karol Krohn Office of Utility Consumer Counselor 115 W. Washington Street, Suite 1500 South Indianapolis, Indiana 46204 [email protected] [email protected] [email protected] [email protected]

Charles W. Ritz III Don F. Morton Angela L. Gidley PARR RICHEY OBREMSKEY FRANDSEN & PATTERSON LLP 225 W. Main Street P.O. Box 668 Lebanon, Indiana 46052 [email protected] [email protected] [email protected]

Kay Pashos Kelly Earls Ice Miller LLP One American Square, Suite 2900 Indianapolis, Indiana 46282-0200 [email protected] [email protected]

Teresa Morton Nyhart BARNES & THORNBURG LLP 11 S. Meridian Street Indianapolis, Indiana 46204 [email protected]

Robert E. Heidorn P. Jason Stephenson Michelle Quinn Southern Indiana Gas & Electric Company One Vectren Square 211 N.W. Riverside Drive Evansville, Indiana 47708 [email protected] [email protected] [email protected]

Jennifer W. Terry Timothy L. Stewart Joseph P. Rompala LEWIS & KAPPES, P.C. 1700 One American Square Indianapolis, Indiana 46282 [email protected] [email protected] [email protected]

-4-

Randolph G. Holt Wabash Valley Power Association, Inc. 722 North High School Road P. O. Box 24700 Indianapolis, Indiana 46224 [email protected]

Kelly Karn Melanie D. Price DUKE ENERGY BUSINESS SERVICES, INC. 1000 East Main Street Plainfield, Indiana 46168 [email protected] [email protected]

Ann M. OHara John M. Davis Church Church Hittle & Antrim 938 Conner Street P.O. Box 10 Noblesville, Indiana 46061 [email protected] [email protected] [email protected]

Michael B. Cracraft Steven W. Krohne Hackman, Hulett & Cracraft, LLP 111 Monument Circle, Suite 3500 Indianapolis, Indiana 46204 [email protected] [email protected]

Mike Mooney Hoosier Energy Rural Electric Cooperative,Inc. 7398 State Road 37 Bloomington, Indiana 47402 [email protected]

Peter J. Mattheis Shaun C. Mohler Brickfield, Burchette, Ritts & Stone, P.C. 1025 Thomas Jefferson Street, N.W. Eighth Floor West Tower Washington, DC 20007 [email protected] [email protected]

Deanna Dean-Webster Beau Badeaux DEAN-WEBSTER WRIGHT, LLP 50 S. Meridian Street, Suite 500 Indianapolis, Indiana 46204 [email protected] [email protected]

Christopher M. Goffinet Jeffrey W. Hagedorn HUBER & GOFFINET 727 Main Street Tell City, Indiana 47586 [email protected] [email protected]

Jerome E. Polk POLK & ASSOCIATES, LLC 101 West Ohio Street, Suite 2000 Indianapolis, Indiana 46204 [email protected]

Michael E. Allen LaTona S. Prentice CITIZENS GAS & COKE UTILITY 2020 N. Meridian Street Indianapolis, Indiana 46202 [email protected] [email protected]

Anne E. Becker Lewis & Kappes, P.C. One American Square, Suite 2500 Indianapolis, Indiana 46282 [email protected]

David L. Haselman, Jr. Gregory K. Lawrence McDermott Will & Emery LLP 227 West Main Street Chicago, IL 60606-5096 [email protected] [email protected]

Jennifer A. Washburn Citizens Action Coalition of Indiana, Inc. 603 East Washington Street, Suite 502 Indianapolis, IN 46204 [email protected]

Shaw R. Friedman FRIEDMAN & ASSOCIATES, P.C. 705 Lincolnway LaPorte, Indiana 46350 sfriedman.associa [email protected]

Kerwin Olson Citizens Action Coalition 603 E. Washington Street, Suite 502 Indianapolis, Indiana 46204 [email protected]

Douglas Ahl Energy Center of Wisconsin 455 Science Drive, Suite 200 Madison, Wisconsin 53711 [email protected]

Greg Wagoner Wabash Valley Power Association, Inc. 722 N. High School Road Indianapolis, Indiana 46214 [email protected]

Ken Baker Wal-Mart Stores, Inc. 2001 SE 1Qth Street Bentonville, AR 72716-0550 [email protected]

-5-

Prepared for

The Indiana Demand Side Management

Coordination Committee

address

Submitted by

The Indiana Statewide Core Program Evaluation Team:

TecMarket Works, Cadmus, Opinion Dynamics Corporation, Integral Analytics, and

Building Metrics

2014 Energizing Indiana Evaluation Report

May 1, 2015

An Evaluation of the Core Final Year Energy

Efficiency Programs

address

Core Programs PY3 Report Page i

Table of Contents Executive Summary ....................................................................................................................................... 4

Program Descriptions ............................................................................................................................. 6

Budget and Expenditures ....................................................................................................................... 6

Goals and Ex-Ante Savings Summary ..................................................................................................... 7

Audited Savings Summary ...................................................................................................................... 8

Verified Savings Summary ...................................................................................................................... 9

Ex-Post and Net Savings Summary ....................................................................................................... 11

Lifetime Savings Summary.................................................................................................................... 15

Energizing Indiana Programs High-Level Insight and Findings ............................................................. 15

EM&V Methodology Overview ................................................................................................................... 21

Impact Analysis Approach .................................................................................................................... 21

Uncertainty ........................................................................................................................................... 25

Process Evaluation Approach ............................................................................................................... 26

Home Energy Assessment ........................................................................................................................... 28

Program Design and Implementation .................................................................................................. 28

EM&V Methodology ............................................................................................................................. 29

Program Performance .......................................................................................................................... 30

Impact Analysis ..................................................................................................................................... 30

Summary of Impact Adjustments ......................................................................................................... 40

Process Evaluation ................................................................................................................................ 42

Insights and Recommendations ........................................................................................................... 46

Low Income Weatherization ....................................................................................................................... 48


EM&V Methodology ............................................................................................................................. 49


Impact Evaluation ................................................................................................................................. 50




Residential Lighting ..................................................................................................................................... 66


Core Programs PY3 Report Page ii

EM&V Methodology ............................................................................................................................. 66

Program Sales ....................................................................................................................................... 66

Ex-Ante Energy Savings ........................................................................................................................ 67





Energy Efficient Schools .............................................................................................................................. 82


Evaluation, Measurement, and Verification (EM&V) Methodology .................................................... 84


Ex-Ante Energy Savings ........................................................................................................................ 88


Summary of Impact Adjustments ....................................................................................................... 103

Process Evaluation .............................................................................................................................. 105

Insights and Recommendations ......................................................................................................... 119

Commercial and Industrial ........................................................................................................................ 121

Program Design and Implementation ................................................................................................ 121

EM&V Methodology ........................................................................................................................... 121

Program Performance ........................................................................................................................ 122

Ex-Ante Savings .................................................................................................................................. 122

Impact Analysis ................................................................................................................................... 123

Summary of Impact Adjustments ....................................................................................................... 131

Process Evaluation .............................................................................................................................. 132

Insights and Recommendations ......................................................................................................... 145

Cost-Effectiveness ..................................................................................................................................... 147

Introduction ........................................................................................................................................ 147

Overview of DSMore .......................................................................................................................... 150

Effective Useful Life ............................................................................................................................ 155

Spillover and Freeriders ...................................................................................................................... 155

Utility Inputs ....................................................................................................................................... 156

Results ................................................................................................................................................ 157

Core Programs PY3 Report Page iii

Job Creation .............................................................................................................................................. 161

Appendix A. Lighting Savings Assumptions ............................................................................................... 166

Residential Lighting Program Ex-Post Savings Engineering Assumptions .......................................... 166

Appendix B. Cost-Effectiveness by Utility ................................................................................................. 170

Executive Summary

Core Programs PY3 Report Page 4

Executive Summary

This report covers the Core programs contained within the Energizing Indiana statewide portfolio

implemented for Duke Energy, Indianapolis Power & Light (IPL), Indiana Michigan Power Company

(I&M), Northern Indiana Public Service Company (NIPSCO), and Vectren.

The TecMarket Works Energizing Indiana Program Evaluation Team (Evaluation Team) completed this

report. The Evaluation Team includes TecMarket Works (the Evaluation Administrator), Cadmus,

Opinion Dynamics, Integral Analytics, and Building Metrics.

This report assesses the third program year (PY3) ex-ante, audited, verified, ex-post gross, and net

energy savings achieved by the Energizing Indiana statewide Core programs. Table 1 defines the types of

energy savings presented in this report.

Table 1. Savings Type Definitions

Savings Type Definitions

Ex-Ante Energy savings from the program tracking system, as reported by the third-party administrator (TPA).

Audited Ex-ante values after deemed calculations and measure counts have been confirmed by the evaluation

administrator.

Verified Savings estimated following confirmation of the installation and use of a sample of projects.

Ex-Post Gross Evaluated savings resulting from the installation and use of all program-incented or provided technologies.

Net Savings Evaluated savings directly attributable to the program.

Energy savings presented in this report derive from three types of savings conditions:

Savings from measures installed and used during the third program year.

Savings from measures distributed during the first or second year, but not installed until the

third year.

Continued use and savings from measures installed during the first and second program years.

In addition, the report includes process evaluation findings.

Energizing Indiana consisted of the following five energy-efficiency programs:

Residential Home Energy Assessment (HEA) program.

Residential Low-Income Weatherization (LIW) program (also known as the Income-Qualified

Weatherization program).

Energy Efficient Schools Program (EES): Education; and Schools Audit and Direct Install

(SADI) programs.

Residential Lighting (Lighting) program.

Commercial and Industrial (C&I) Prescriptive Rebates program.

Executive Summary


The programs were delivered by a third-party administrator (TPA), GoodCents, which was hired through

a competitive bid process in 2011. The programs implemented were found to be cost-effective with an

acquisition cost of $0.038 per kWh across the three year program cycle.

This evaluation effort seeks to achieve the industrys highest reliability standards, conforming to

definitions and requirements established by the Indiana Evaluation Framework (the Framework). The

Framework requires reliable studies, with a 90% confidence level and a precision level within 10% over

the standard three-year program cycle, at both the utility and program levels. As this includes five

programs, sponsored by five utility companies, the evaluation presents 25 individual, program-impact

assessments. The report then rolls up utility-specific, energy-impact assessment results to provide

program-level energy impacts, achieving greater than 90% confidence levels and 10% precision levels

for each program and for the portfolio-level results.

This reports language and terminology conforms to the Demand Side Management (DSM) Impact Steps

outlined in the Framework.

While some programs reached goals, across the portfolio the programs achieved savings levels lower

than set goals. Figure 1 shows the progress for portfolio-level savings. These estimates include carryover

savings from bulbs distributed in PY1 and PY2 that were not installed until PY3 for all programs. The

carryover savings account for the increased savings level from Audited to Verified.

Figure 1. PY3 Portfolio-Level Savings

In general, the energy-efficiency program portfolios for each of the utilities as well as the aggregation to

the State of Indiana continue to be cost-effective for the combined PY1 through PY3 under the

Participant Cost, the Utility Cost, and the Total Resource Cost tests. Most of the programs individually

were also found to be cost-effective. In last years report, it was noted that there had been a decline in

Executive Summary


the level of cost-effectiveness for the School Building Assessments program; this program is now cost-

effective for all utilities. The Low Income Weatherization Program continues to not be cost effective.

Program Descriptions The portfolio consisted of five offerings, delivered by GoodCents. PY3 represented the final year of a

three-year program cycle for Energizing Indiana. The programs addressed homes, schools, and

commercial facilities (see Table 2).

Table 2. Energizing Indiana Programs

Budget and Expenditures Overall, the TPA spent 84% of the total PY3 (2014) implementation budget for all programs evaluated in

this report (see Table 3 and

Table 4).

Programs Brief Program Description

Home Energy

Assessment

(HEA)

Walk-through energy audit that: analyzed participant energy use; recommended efficiency measures or

upgrades; and facilitated direct installation of energy-saving measures, including energy efficient

showerheads, CFL bulbs, hot water pipe wrap, and sink aerators.

Low Income

Weatherization

(LIW)

Walk-through home energy assessment that included all HEA elements, plus full diagnostic testing

(blower-door) for the home. Auditors recommended weatherization measures or upgrades that

facilitated direct installation of low-cost, energy-saving measures, including energy-efficient

showerheads, CFL bulbs, sink aerators, pipe wrap, water heater tank wrap and air sealing. In addition,

eligible homes received attic insulation.

Residential

Lighting

Upstream discounts on a variety of lighting products (CFLs, LEDs, and lighting fixtures). The program

worked with retailers and manufacturers across the state to offer reduced prices at the point-of-sale.

Energy Efficient

Schools (EES)

Two components:

Education Program, worked with fifth-grade students, teaching them about energy efficiency and

how they could make an impact at school and home. Participating schools received classroom

curriculum and Energizing Indiana take-home efficiency kits.

SADI, worked with schools to assess all energy systems to determine if they operated efficiently.

Assessment results guided schools to install appropriate upgrades and rebates available through the

C&I program. The schools also received a bundle of direct-install measures at no cost.

Commercial and

Industrial (C&I)

Prescriptive rebates to facilities, based on the installation of energy efficiency equipment and system

improvements. Additional tracks targeted to small businesses included the Opt-in Direct Install program

and targeted relationships with trade allies. Rebates addressed lighting, variable frequency drives (VFDs),

Heating, Ventilation, and Air Conditioning (HVAC), and efficient ENERGY STAR commercial kitchen

appliances. In addition, the program began offering direct-mail CFLs kits, starting in Fall 2012.

Executive Summary


Table 3. PY3 Budget and Expenditures by Program and Statewide

Program PY3 Budget PY3 Expenditures % of Budget Goals

HEA 16,122,422 15,410,785 96%

LIW 6,810,490 6,709,524 99%

Lighting 5,109,080 4,982,396 98%

EES 5,005,270 4,859,674 97%

C&I 11,049,876 5,128,301 46%

Portfolio Total 44,097,138 37,090,680 84%

Table 4. PY3 Budget and Expenditures by Utility and Statewide

Utility PY3 Budget Goals PY3 Expenditures % of Budget Goals

Duke 18,645,593 14,055,315 75%

IM 3,613,424 3,367,744 93%

IPL 10,147,175 9,087,366 90%

NIPSCO 7,763,976 6,701,934 86%

Vectren 3,926,970 3,878,321 99%

Portfolio Total 44,097,138 37,090,680 84%

Goals and Ex-Ante Savings Summary Figure 2 demonstrates the distribution of electric savings among programs planned to be achieved by

the portfolio in PY3.

Figure 2. PY3 Percentage, by Program, Toward kWh Goal

Error! Reference source not found. summarizes the TPAs ex-ante savings, compared to savings goals

for 2014. As delivered, the Lighting and Schools programs exceeded their goals, while HEA and LIW

marginally fell short of the program goals. The C&I program achieved 46% of the goal; the TPA reports

this was because the programs terminated at different dates throughout the year, causing uncertainty

about available rebates and participation opportunities.

9%2%

15%

4%71%

Home Energy Assessment

Low Income Weatherization

Lighting

Energy Efficent Schools

Commercial and Industrial

Executive Summary


Table 5. 2014 Statewide kWh Ex-Ante Savings by Program for PY3*

Error! Not a valid bookmark self-reference. demonstrates the distribution of savings among programs

reported (ex-ante) in 2014.

Figure 3. PY3 Percent (by Program) Toward Ex-ante kWh Savings

Audited Savings Summary Table 6 presents the audited energy and demand savings for each program in PY3.

Table 6. PY3 Statewide Audited Savings by Program

Program kWh Ex-Ante kWh Audited kW Ex-Ante kW Audited

HEA 50,232,989 50,098,068 6,664 6,614

LIW 11,011,111 10,953,115 1,138 1,124

Lighting 98,722,324 102,247,800 11,763 12,148

EES 25,149,719 25,149,719 230.70 230.53

C&I 211,401,651 189,709,000 36,034 39,186

Portfolio Total* 396,517,794 378,157,702 55,830 59,303

* Values may not add perfectly to total values due to rounding of decimals.

13%3%

25%

6%

53%



Lighting

Energy Efficent Schools


Program kWh Goal kWh Ex-Ante % of Goals kW Goal kW Ex-Ante % of Goal

HEA 56,019,020 50,232,989 90% 6,932 6,664 96%

LIW 11,966,981 11,011,111 92% 1,085 1,138 105%

Lighting 98,707,000 98,722,324 100% 15,697 11,763 75%

EES: Education 21,774,000 21,775,231 100% n/a n/a n/a

EES: SADI 2,001,216 3,374,488 169% n/a 230.7 n/a

C&I 463,244,374 211,401,651 46% 68,693 36,034 52%

Portfolio Total** 653,712,591 396,517,794 61% 92,407 55,830 60%

*Source: From GoodCents Portal, year-end savings statement, pulled January 26, 2015

** Values may not add perfectly to total values due to rounding of decimals.

Executive Summary


Figure 4 demonstrates the distribution of audited savings among the programs. The Commercial and

Industrial and Lighting programs accounted for more than three-quarters of the savings.

Figure 4. PY3 Percentage (by Program) Toward kWh Audited Savings

Verified Savings Summary The savings estimates excludes savings from measures not yet installed. Figure 5, below, shows the

verified savings per program (for PY3 distributed measures only).

Figure 5. PY3 Percent (by Program) Toward kWh Verified Savings

Table 7 compares ex-ante savings to verified savings by program, and Table 10 compares ex-ante savings

to verified savings by utility per program. The reports program-specific sections include further details.

Table 7. PY3 Statewide Ex-Ante and Verified Savings by Program for PY3

* Values may not add perfectly to total values due to rounding of decimals.

13%3%

27%

7%

50%



Lighting

Energy Efficient Schools


14%

3%

25%

7%

52%



Lighting



Program kWh Ex-Ante kWh Verified Realization

Rate

% of

Goal

kW Ex-

Ante

2014 kW

Verified

Realization

Rate

% of

Goal

HEA 50,232,989 48,500,431 97% 86% 6,664 6,229 93% 90%

LIW 11,011,111 10,660,250 97% 89% 1,138 1,087 95% 100%

Lighting 98,722,324 88,824,765 90% 90% 11,763 10,553 90% 67%

EES 25,149,719 25,211,586 100% 106% 230.70 226.31 98% n/a

C&I 211,401,651 185,326,275 88% 40% 36,034 38,156 106% 56%

Portfolio Total* 396,517,794 358,523,307 90% 55% 55,830 56,251 101% 61%

Executive Summary


Some measures, such as CFLs, contain carryover savings. Carryover savings refers to the installation of a

measure after the year it was distributed. For example, if a CFL was purchased or given to a participant

in 2013, but not installed until 2014, the savings from that measure would not be applied until the actual

installation date. This is based on in-service rate (ISR) research, also discussed in subsequent chapters,

that indicates CFLs in storage will go into use within three years from the date of receiving or purchasing

them. Verified carryover savings occur for all programs except for C&I, for which the extra lighting saving

are introduced in the ex-post stage for reasons explained below. Table 8 presents the carryover savings

from PY1 and PY2 occurring in PY3 for HEA, LIW, Lighting and EES.

Table 8. PY1 and PY2 Verified Carryover Savings by Program (kWh)

Table 9 and Table 10 provide the program-level and utility level ex-ante and verified savings. These

tables include savings achieved from measures distributed during PY1 and PY2, but not installed until

PY3 (for all programs but C&I). Within program chapters and provided in Table 10, utilities can see

additional verified savings applied based on the savings accrued from measures that continue to be

installed, even if they were not distributed and directly counted in the current years ex-ante savings.

Table 9. Portfolio Ex-Ante and Verified Savings by Program from PY1-PY3 Achieved in PY3*

* Values include CFL kWh savings carryover from PY1 and PY2 distribution, installed in PY3 for HEA, LIW, Lighting and Schools.


Program PY1 Carryover PY2 Carryover Total Carryover in PY3

HEA 832,334 2,696,924 3,529,258

LIW 114,843 248,664 363,507

Lighting 11,419,159 17,822,962 29,242,120

EES 4,060,354 3,713,825 7,774,179

C&I n/a n/a n/a

Portfolio Total 16,426,690 24,482,374 40,909,064

Program kWh Ex-Ante kWh Verified Realization

Rate

% of

Goal

kW Ex-

Ante

kW

Verified

Realization

Rate % of Goal

HEA 50,232,989 52,029,689 104% 93% 6,664 6,844 103% 99%

LIW 11,011,111 11,023,757 100% 92% 1,138 1,130 99% 104%

Lighting 98,722,324 118,066,886 120% 120% 11,763 14,483 123% 92%

EES 25,149,719 32,985,765 131% 139% 231 227 98% n/a

C&I 211,401,651 185,326,275 88% 40% 36,034 38,156 106% 56%

Portfolio

Total** 396,517,794 399,432,371 101% 61% 55,830 60,840 109% 66%

Executive Summary


Table 10. Portfolio Ex-Ante and Verified Savings by Program by Utility from PY1-PY3 Achieved in PY3*

* Values include CFL kWh savings carryover from PY1 and PY2 distribution, installed in PY3 for HEA, LIW, Lighting and School.


Ex-Post and Net Savings Summary The Evaluation Team determined ex-post gross evaluated savings through: engineering analysis; building

simulation modeling; billing analysis; or metering analysis.

Program kWh

Ex-Ante kWh Verified

Realization

Rate

% of

Goal

kW

Ex-Ante

kW

Verified

Realization

Rate

% of

Goal

Duke

HEA 17,714,195 18,019,534 102% 95% 2,272 2,298 101% 99%

LIW 3,451,650 3,566,090 103% 97% 324.98 335 103% 102%

Lighting 42,744,285 48,455,761 113% 113% 5,086 5,935 117% 88%

EES 13,593,070 17,993,841 132% 137% 50.3 49 98% n/a

C&I 87,659,164 76,846,828 88% 31% 12,406 13,136 106% 35%

Total Duke** 165,162,364 164,882,055 100% 50% 20,138 21,754 108% 47%

I&M

HEA 2,181,517 2,401,606 110% 107% 239 266 111% 113%

LIW 1,529,696 1,517,142 99% 99% 139.27 137 98% 100%

Lighting 15,747,122 19,702,788 125% 126% 1,874 2,423 129% 97%

EES 2,376,678 3,027,880 127% 141% 33.5 33 98% n/a

C&I 23,783,183 20,849,641 88% 60% 3,684 3,901 106% 76%

Total I&M** 45,618,196 47,499,057 104% 84% 5,970 6,759 113% 84%

IPL

HEA 15,377,976 16,056,511 104% 96% 2,083 2,141 103% 100%

LIW 2,315,573 2,293,183 99% 96% 254.12 249 98% 114%

Lighting 15,928,050 19,836,782 125% 124% 1,895 2,443 129% 96%

EES 4,886,278 6,004,672 123% 129% 55.9 55 98% n/a

C&I 46,771,718 41,002,652 88% 52% 6,905 7,312 106% 60%

Total IPL** 85,279,595 85,193,800 100% 72% 11,194 12,200 109% 74%

NIPSCO

HEA 9,472,965 9,801,056 103% 84% 1,254 1,320 105% 97%

LIW 2,084,444 2,042,904 98% 81% 213.69 207 97% 90%

Lighting 14,799,870 18,550,566 125% 125% 1,773 2,276 128% 96%

EES 2,783,872 4,060,188 146% 181% 68.3 67 98% n/a

C&I 41,269,639 36,179,227 88% 43% 10,954 11,599 106% 94%

Total NIPSCO** 70,410,790 70,633,941 100% 61% 14,263 15,469 108% 95%

Vectren

HEA 5,486,336 5,750,982 105% 89% 815 819 100% 94%

LIW 1,629,748 1,604,438 98% 86% 205.94 202 98% 120%

Lighting 9,502,997 11,520,988 121% 121% 1,136 1,407 124% 93%

EES 1,509,821 1,899,184 126% 120% 22.7 23 99% n/a

C&I 11,917,947 10,447,926 88% 72% 2,084 2,207 106% 88%

Total Vectren** 30,046,849 31,223,518 104% 92% 4,264 4,658 109% 110%

Executive Summary


Ex-post savings adjustments could include changes to the following:

Baseline assumptions.

Weather.

Occupancy levels.

Hours-of-use (HOU).

Decreased or increased production levels.

1. Other assumptions resulting from the impact analysis research.

Net savings reflect ex-post savings, with the net-to-gross (NTG) ratio applied. These NTG ratios account

for freeridership and short-term spillover effects, which ultimately contribute to the programs cost-

effectiveness calculations. Table 11 shows the statewide ex-post and net savings by program

attributable to PY3 program actions only (no carryover included). Figure 6 and Figure 7 show the

distribution of ex-post and net savings by program.

Figure 6. Percent (by Program) Toward kWh Ex-Post Savings

Table 11. Statewide Ex-Post and Net Savings by Program

*EES program broken out by subprogram to show program-specific level NTG ratios ** Values may not add perfectly to total values due to rounding of decimals.

13%3%

24%

6%

55%



Lighting



Program kWh Ex-Post kWh NTG

Ratio kWh Net kW Ex-Post kW NTG Ratio kW Net

HEA 43,390,531 83% 35,865,838 5,408 83% 4,493

LIW 9,015,003 100% 9,015,003 1,146 100% 1,146

Lighting 82,994,936 49% 40,667,519 11,385 49% 5,578

EES: Education* 18,842,953 100% 18,932,112 2,714 106% 2,864

EES: SADI* 1,155,874 247% 2,857,865 102 100% 102

C&I 190,457,751 81% 154,989,113 33,424 77% 25,663

Portfolio Total** 345,857,050 n/a 262,327,450 54,179 n/a 39,846

Executive Summary


Figure 7. PY3 Percent toward kWh Net Savings

The Evaluation Team includes the carryover savings through for all impact metrics from the time that

they are introduced into the impact savings stream.

Ex-post carryover savings for all programs, shown in Table 12, maintain the NTG values from the year

they were acquired by the participant.

Table 12. PY1 and PY2 Carryover Savings by Program

Table 13. Statewide Ex-Post and Net Savings by Program from PY1-PY3 Achieved in PY3*

Program kWh Ex-Post kWh NTG

Ratio kWh Net kW Ex-Post kW NTG Ratio kW Net

HEA 46,262,705 83% 38,217,984 5,771 83% 4,791

LIW 9,315,612 100% 9,315,612 1,186 100% 1,186

Lighting 106,518,541 50% 52,900,581 14,450 50% 7,164

EES 26,477,252 103% 27,308,135 3,630 101% 3,660

C&I 214,268,670 84% 180,466,797 39,705 82% 32,385

Portfolio Total** 402,842,780 n/a 308,209,109 64,742 n/a 49,186

* Values include CFL kWh savings carryover from PY1 and PY2 distribution, installed in PY3 for all programs.


Table 14. Statewide Ex-Post and Net Savings by Program by Utility from PY1-PY3 Achieved in PY3*

Program kWh Ex-Post kWh NTG kWh Net kW Ex-Post kW NTG kW Net

Duke

HEA 15,913,643 83% 13,263,973 1,991 84% 1,663

14%

3%

16%

8%59%



Lighting



Program PY1 Carryover PY2 Carryover Total Carryover in PY3

Ex-Post kWh NTG Ex-Post kWh NTG Ex-Post kWh NTG

HEA 696,807 77% 2,175,368 84% 2,872,174 82%

LIW 95,603 100% 205,006 100% 300,609 100%

Lighting 8,831,204 57% 14,692,400 49% 23,523,604 52%

EES 2,878,465 94% 3,599,961 78% 6,478,425 85%

C&I 23,810,919 107% n/a n/a 23,810,919 107 %

Portfolio Total 36,312,998 n/a 20,672,735 n/a 56,985,731 n/a

Executive Summary


Program kWh Ex-Post kWh NTG kWh Net kW Ex-Post kW NTG kW Net

LIW 2,858,557 100% 2,858,557 348 100% 348

Lighting 43,734,871 50% 21,705,924 5,933 50% 2,940

EES 15,063,666 99% 14,970,101 2,078 101% 2,101

C&I 88,829,977 84% 74,812,534 14,107 82% 11,617

Total Duke** 166,400,714 127,611,088 24,456 18,669

I&M

HEA 1,985,025 83% 1,653,851 266 83% 221

LIW 1,243,398 100% 1,243,398 138 100% 138

Lighting 17,524,744 50% 8,720,465 2,386 50% 1,185

EES 2,304,795 106% 2,452,504 311 101% 312

C&I 25,749,233 86% 22,061,488 4,558 84% 3,844

Total I&M** 48,807,195 36,131,706 7,658 5,700

IPL

HEA 14,060,746 83% 11,691,647 1,749 83% 1,459

LIW 2,078,128 100% 2,078,128 242 100% 242

Lighting 17,831,760 50% 8,867,738 2,415 50% 1,199

EES 4,818,732 107% 5,160,184 663 102% 673

C&I 45,519,135 83% 37,908,532 7,297 80% 5,872

Total IPL** 84,308,502 65,706,228 12,366 9,445

NIPSCO

HEA 8,806,890 80% 7,050,199 1,073 81% 871

LIW 1,705,645 100% 1,705,645 194 100% 194

Lighting 16,727,994 50% 8,305,648 2,254 50% 1,118

EES 2,844,280 107% 3,041,716 365 98% 356

C&I 41,841,659 84% 35,243,751 11,390 80% 9,117

Total NIPSCO** 71,926,469 55,346,958 15,277 11,656

Vectren

HEA 5,496,401 83% 4,558,314 692 83% 577

LIW 1,429,885 100% 1,429,885 265 100% 265

Lighting 10,699,172 50% 5,300,807 1,462 49% 723

EES 1,445,778 116% 1,683,630 214 101% 217

C&I 12,328,665 85% 10,440,492 2,353 82% 1,934

Total Vectren** 31,399,901 23,413,128 4,986 3,716

* Values include bulb drop and CFL kWh savings carryover from PY1 and PY2 distribution, installed in PY3. ** Values may not add perfectly to total values due to rounding of decimals.

Figure 8 presents savings achieved by each program for each category of savings assessed in this study.

Executive Summary


Figure 8. Savings by Program in Each Savings Category

Lifetime Savings Summary Once a measure is installed the savings continue to accrue throughout the lifetime of that measure.

Degradation, code changes, and dual baselines are considered to ensure the lifetime savings calculations

are correct. Across the three year program cycle, the Core programs saved approximately 11 million

megawatt hours.

Table 15. PY1-PY3 Ex-Post Gross Lifetime Savings

Program PY1 Lifetime

Savings (MWh)

PY2 Lifetime

Savings (MWh)

PY3 Lifetime

Savings (MWh)

Total Lifetime

Savings (MWh)

HEA 94,901 602,845 272,309 970,054

LIW 56,952 114,283 71,666 242,901

Lighting 248,615 592,619 479,167 1,320,401

EES 457,056 283,820 162,231 903,106

C&I 1,263,147 3,648,755 2,649,463 7,561,365

Portfolio Total 2,120,671 5,242,322 3,634,835 10,997,828

Energizing Indiana Programs High-Level Insight and Findings The Evaluation Team considers these finding significant, but they do not represent all insights and

findings. Each program evaluation section of this report expands on these findings and includes minor

additional insights from the analysis efforts.

Besides presenting the enclosed results, the Evaluation Team created separate, program-specific binders

for each utility that contain detailed information on variable assumptions and sources.


The HEA Program offered walk-through audits and direct-installations of energy-efficiency measures.

The HEA program delivery model instructed auditors to install as many measures as possible in homes

-

100,000,000

200,000,000

300,000,000

400,000,000

500,000,000

600,000,000

700,000,000

Goal Ex-Ante Audited Verified Ex-Post Net

kWh

Sav

ings

Home Energy Assessment Low Income Weatherization Lighting

Energy Efficient Schools Commercial and Industrial

Executive Summary


(without leaving measures behind) and recording all installed measures in the program tracking

database. Figure 9 presents an overview of its savings achievements.

Figure 9. HEA Overview of Savings

Key evaluation findings include the following:

In 2014, the program met 80% its participation goals, and 90% and 96% of the kWh and kW

savings goals (ex-ante compared to goals), while using 98% of its budget.

In PY3, the TPA made a number of enhancements to the database, allowing it to more

accurately capture the auditors activities.

Customers remained satisfied with the program, rating it an 8.9 mean satisfaction score on a

10-point scale.


The LIW program offers walk-through audits and direct installations of energy-efficiency measures. The

program includes air sealing and, in some cases, attic insulation. This program only targets low-income

populations. To qualify, the participant must have a reported income of up to 200% of the Federal

Poverty Level. Figure 10 presents an overview of the savings achievements.

Executive Summary


Figure 10. LIW Overview of Savings


In 2014, the program met 85% its participation goals, and 92% and 105% of the kWh and kW

savings (ex-ante compared to goals), while using 96% of its budget.

In PY3, the TPA made a number of enhancements to the database, allowing it to more

accurately capture the auditors activities.

Customers remained satisfied with the program, rating it an 8.6 mean satisfaction score on a

10-point scale.

Residential Lighting

The Residential Lighting program met its ex-ante savings in PY3. Agreed-upon gross savings calculations

generally were applied accurately and consistently. The program continued to offer discounts on a

variety of lighting products. Due to a 30% goal reduction and, ultimately, a program ramp-down, the

retailer footprint decreased from 16 in PY2 to 14 in PY3, and the total number of participating

storefronts dropped from 865 in PY2 to 310 in PY3. Standard CFLs continued to represent the bulk of

bulb sales through the program. Figure 11 presents an overview of program savings achievements.

Executive Summary


Figure 11. Lighting Overview of Savings


In PY3, 92% of all bulbs sold through the program were standard CFLs. Specialty CFLs

represented 7% of total bulb sales, while program LED sales were less than 1%. Three- and four-

bulb packs were most commonly sold for standard CFLs, with two-bulb packs most common for

specialty CFLs, LEDs were sold exclusively in single packs.

Participating retailers consisted of a range of store types, with Walmart leading the number of

participating store fronts (as well as sales).

Program tracking data were clean, and, in most cases, savings were calculated using proper,

agreed-upon assumptions.

The lighting markets future remains uncertain. Utilities that continue to run programs will want

to monitor and assess the market periodically for the most appropriate product mix and savings

assumptions (such as baseline wattages and installation rates).


The EES program offers energy-efficiency kits for students and energy assessments for school buildings,

coupled with installation of low-cost, energy-saving measures. In 2014, the program achieved 106% of

its energy-savings goals and 100% of its participation goals, while using 98% of its budget. Figure 12

presents an overview of the savings achievements.

Executive Summary


Figure 12. Schools Overview of Savings


As in PY2, several School Audit and Direct Install program measures experienced significantly

negative ex-post adjustments. These occurred for the following reasons:

The reported location of measure installations, such as occupancy sensors, did not maximize

potential savings; and

A lack of data existed regarding the circumstances in which measures were installed.

The Education Program continued to meet distributed kit goals and produce high verified

savings for the third year in a row. Savings from CFLs distributed in PY1 and PY2 but not installed

until PY3 also helped bolster verified savings.


The C&I program fell short of meeting its savings goal for PY3. In 2014, the program achieved 46% of its

energy-savings goals and 52% of its demand savings goals, while using 46% of its budget. Figure 13

presents an overview of the programs savings achievements.

Executive Summary


Figure 13. C&I Overview of Savings


The TPA reported program end dates across the five utilities affected program participation.

Ex-post savings were higher than audited, based on the Evaluation Teams use of full-load hours

in the Indiana technical reference manual (TRM).

Ninety-five percent of lighting and non-lighting participants reported satisfaction with the

program overall.

Confidence and Precision

The evaluation plans were structured to attempt to achieve the 90/10 levels by program. As the

programs were delivered by the same implementer in a similar fashion across utilities, the approach

made sense.

Table 16 shows the confidence and precision achieved for each program. Overall, the portfolio verified

savings achieved 9.9% precision at 90% confidence and each program achieved precision well below

10%.

Table 16. Confidence and Precision

Program Precision

HEA 3.6%

LIW 2.8%

Lighting 5.5%

EES 3.7%

C&I 5.7%

Portfolio 9.9%

EM&V Methodology Overview



Overall, this evaluation sought to quantify each programs energy and demand savings (via the impact

evaluation) and to understand and help improve the Energizing Indiana Programs performance (via the

process evaluation).

Process Evaluation: The process evaluation included documenting program efforts and improving design

and delivery, involving the following key efforts: a review of program materials; in-depth interviews with

third-party administrator program and implementer staff; participant surveys; and

participant/nonparticipant contractor in-depth interviews. The process evaluations sought to answer the

following overall questions:

Does the program, as designed and implemented, meet its goals?

Can improvements be made in the program design and implementation processes, including

marketing and database tracking efforts?

What specific customer/contractor insights could improve the program and increase

satisfaction levels?

Impact Evaluation: The impact evaluation sought to accurately quantify energy savings and to provide

information leading to more accurate energy savings, including demand reduction and estimates in

future program years. Data were gathered from various sources, including the following: the tracking

database, management interviews, independent operational observation, and participant surveys. Data

analysis included conducting an audit of the tracking system, analysis of participant survey data,

conducting statistical and engineering analysis of ex-ante savings, and other efforts. The section below

describes the evaluation approach and its application.

Impact Analysis Approach This section describes the typical steps taken in conducting impact evaluations of Indiana-based

demand-side management (DSM), using steps consistent with the approaches described in the Indiana

Evaluation Framework (the Framework).1 These steps guided the Evaluation Teams evaluation

approaches.

In addition to outlining the evaluation approaches, this section provides summary descriptions of

actions taken to determine savings estimates for the overall Energizing Indiana portfolio and for

program- and utility-specific energy and demand savings. Where therm savings could be accounted for,

each program detailed those savings in the program specific section. Since therm savings are not a part

of the contracted savings goals, they were not rolled up to the portfolio level. Figure 14 illustrates steps

used in the impact evaluation approach, and Table 17 elaborates on the actions taken within each of

1 Indiana Evaluation Framework. TecMarket Works. September 25, 2013.



these steps. The methods employed varied by program to accurately represent the program designs and

target populations.

Figure 14. DSM Impact Evaluation Steps

Table 17. EM&V Impact Analysis Steps

Savings Categories Definition, Sources, and Activities

Goals Savings targets set by the DSMCC for each program year, per program.

Ex-Ante Savings

Reported savings values provided by the third-party administrator (TPA), as shown in its portal.

Savings resulted from the TPAs Energizing Indiana activities, using savings values and assumptions

provided in its scope of work.

Audited Savings

Once the following activities had been conducted through , audited savings were produced:

Reviewing the program tracking databases.

Reviewing hardcopy program applications from a sample to verify consistency with data

recorded in the program tracking databases.

Checking ex-ante savings estimates and calculations to ensure the implementer/utility

applied the agreed-upon values appropriately/correctly.

Adjusting program tracking data as necessary to correct errors or identified omissions.

Recalculating program savings, based on the adjusted, audited number of measures and

on any errors found in the program tracking data.

Verified Savings Verified savings could be computed after confirming measures had been installed and remain

operational. This step typically employed a stratified random sample of installations, selected for



Savings Categories Definition, Sources, and Activities

detailed analysis. Typical methods for collecting necessary data included telephone surveys and/or

site visits. This step could result in adjustments in total savings to address issues such as the

following:

Measures rebated but never installed, or not installed in the correct program year;

Measures not meeting program qualifications;

Measures installed but later removed; or

Measures improperly installed.

This step did not alter the per-unit, energy-saving values.

Ex-post Gross Evaluated

Savings

Ex-post gross evaluated savings could be determined using the following: engineering analysis,

building simulation modeling, billing analysis, metering analysis, or other accepted impact

evaluation methods. Adjustments to the verified savings could include: changes to the baseline

assumption; adjustments for weather; adjustments for occupancy levels; adjustments for

decreased or increased production levels; and other adjustments following from the impact

analysis approach.

Net Savings

Determined by adjusting the ex-post evaluated gross savings estimates to account for a variety of

circumstances related to program attribution, including savings-weighted freerider and spillover

effects. Total Net Savings involved extrapolation of sample-based ex-post net savings to the

population of Energizing Indiana participants. This incorporated adjustments for freeridership,

participant spillover, and market effects (applied at the end of the current program cycle):

Net-to-Gross Ratio = (1- freerider adjustment + participant spillover adjustment + market effects

adjustment)

Freerider Assessment

Free riders are participants who would have undertaken the installation of some or all of the measures

incented through the programs. For the percentage of participants saying they would have made a more

energy-efficient choice without the program, savings would be adjusted by the difference in energy use

of the program-provided technology and the efficiency of the technology they would have installed

without the program.

The evaluation effort selected the baseline approach at the same time it defined the freeridership

approach due to linkages between these two conditions. The individual program chapters report each

approach used in the Energizing Indiana program evaluations.

Spillover Assessment

Spillover savings must be added to program-induced net savings when a participant replicates actions

that save energy in their facilities, and they attribute their non-program-incented or provided actions to

the program. Spillover savings in the Energizing Indiana PY1 evaluation report only included short-term

participant spillover savings, representing but a portion of spillover savings typically achieved

by programs.

That is, more spillover savings typically occur than those identified in the evaluation efforts. Spillover

savings include actions taken between program participation and identified by the evaluations survey

with participants.



Longer-term spillover savings, after completion of an evaluation effort, as a portion of the participant

population repeat actions caused by the program. These savings may be captured in the Market Effects

Study report which will be published July 1, 2015. Nonparticipants also take spillover actions if directly

or indirectly influenced by a program, but, typically, remain unaware of that influence. Savings from

groups of unaware nonparticipants, taking actions due to a programs influence but not perceiving that

influence, more traditionally are termed market effects savings. This report only includes participant

spillover.

To assess spillover, surveys asked participants if they had taken additional actions to save energy in their

facilities or homes.2 If they reported taking additional actions, follow-ups asked them to describe that

action in sufficient detail to allow estimation of savings achieved. An attribution question then indicated

whether the program influenced or caused the action.

If respondents reported actions influenced by the program, surveys asked them to rate that influence

level on a 1 to 10 scale, allowing the percent of the influence to be attributed, based on the score

provided. Using a 1 to 10 scale, cause could be attributed using a direct linear conversion of their score

to a percentage (2=20% / 5=50% / 8=80%).

EISA Baseline Adjustment Approach

In 2007 the United States passed the Energy Independence and Security Act (EISA), which restricts sales

of standard incandescent light bulbs (standard bulbs), but allows bulbs in the supply chain to be sold via

retail sales. Upon exhausting the supply of standard bulbs, new incandescent bulbs cannot be

manufactured, distributed, or sold in the United States.

If standard bulbs no longer prove available, savings from standard bulbs cannot be used to estimate

program savings. Rather, the energy-saving baseline must reflect the market and products available for

sale. As such, the Evaluation Team developed and implemented a baseline adjustment approach that

accounted for retailer-reported product availability and adjusted baseline wattages accordingly.

This baseline adjustment approach built upon a longitudinal mystery shopper study in Indiana. As part of

the study, the Evaluation Team called a representative sample of retail stores to inquire if a store

offered 100- and 75-watt bulbs for sale, how many they carried, and other questions regarding future

availability.

The first wave, conducted in January 2013, was used for the PY2 CFL impact calculations. At the time of

the survey, EISA only affected 100-watt incandescents, therefore, research focused solely on assessing

the availability of these bulbs. The research indicated that availability of 100-watt incandescent light

bulbs eroded, yet a considerable number of stores stocked and offered those light bulbs.

2 For the HEA program, responses to audit recommendations were not included in spillover; rather, they were

included as direct program impacts. For the LIW program, captured spillover was identified as Audit

Recommendations.



The second wave was completed in January 2014 and informed baseline adjustments for PY3 impact

calculations. At the time of the study, EISA impacted 75-watt incandescents, and, as such, the study

explored availability of both 100- and 75-watt incandescents. This study indicated that, while the

availability of 100- and 75-watt incandescent bulbs further eroded due to EISAs effects, a sizable

number of Indiana stores continued to carry those bulbs.

Table 18 presents the best baseline adjustments made in PY2 (2013) and PY3 (2014) as well as projected

adjustments for future years. Stand-alone memos, submitted to the DSMCC upon the studies

completion, summarize detailed survey results and baseline adjustment recommendations.

Table 18. Lighting Baseline Wattage with EISA

100-watt 75-watt 60-watt 40-watt

Year Phase Baseline

(Watts) Year Phase

Baseline

(Watts) Year Phase

Baseline

(Watts) Year Phase

Baseline

(Watts)

2012 0% 100 2012 0% 75 2012 0% 60 2012 0% 40

2013 55% 84.6 2013 0% 75 2013 0% 60 2013 0% 40

2014 60% 83.2 2014 60% 61.8 2014 0% 60 2014 0% 40

2015 70% 80.4 2015 80% 57.4 2015 55% 50.65 2015 60% 33.4

2016 80% 77.6 2016 100% 53 2016 60% 49.8 2016 80% 31.2

2017 90% 74.8 2017 100% 53 2017 70% 48.1 2017 100% 29

2018 100% 72 2018 100% 53 2018 80% 46.4 2018 100% 29

2019 100% 72 2019 100% 53 2019 90% 44.7 2019 100% 29

2020 100% 72 2020 100% 53 2020 100% 43.0 2020 100% 29

Uncertainty Program evaluations seek to reliably determine energy and demand savings, along with some

reasonable accuracy level. Whenever selecting a population sample to represent a populationwhether

of appliances, individuals, households, premises, or organizationssome amount of sampling error

occurs. Sampling error arises as only a portion of actual values can be measured (e.g., metering energy

consumption over one week, metering 5% of the affected equipment). Different samples lead to

different estimates of energy and demand savings.

If randomly drawing a sample from the population, the sampling error should be random and provide an

unbiased estimate of true savings. The Evaluation Team used precision to characterize sampling error,

defined as the degree that additional measures would produce the same or similar results. Convenient

measures, such as confidence intervals and statistical significance tests, provided quantitative estimates

of the uncertainty introduced through sampling.

As each program, across each utility, will achieve a level of 90% confidence with a 10% precision rate

across the three-year program cycle, each sample was designed to limit uncertainty and to maintain a

high level of accuracy within the evaluation. Survey findings in their program-specific sections include

the number (n=) of participants sampled throughout PY3.



Process Evaluation Approach Although the process evaluation efforts differed somewhat for each program, to an extent, these

studies followed a similar theme and approach. The process evaluation consisted of program-specific

efforts, designed to address each programs researchable issues.

Sampling Design

The evaluation included on-site surveys and field EM&V work for a sample of participating sites. The

Evaluation Team developed a sampling plan for each project, outlining: the sampling design; the

population; and the sample sizes needed to meet the evaluations overall precision requirements. This

reports program sections provide the sampling detail for each program. Samples generally derived from

the participant tracking data, per the sampling plan.

The sampling design depended on: the analysis data quality objectives; development of the sample

frame; and potential uses of the data. These designs included the following:

Simple random sampling;

Stratified random sampling;

Two-stage or cluster sampling;

Nested sampling (time-of-use meters used within a smaller sample of interval meters); and

Systematic sampling.

Table 19 presents the program-specific evaluation efforts conducted.

Table 19. EM&V Activities by Program

Programs Process Evaluation

Approach Impact Evaluation Approach Market Effects Evaluation Approach

Residential Programs

Residential

Lighting

Lead: Opinion

Dynamics

TPA interview

Materials review

Database analysis

Database audit

Engineering analysis and

verification

Ex-post gross savings analysis

TPA interview

Materials and database review

Secondary research

General population telephone

survey

General population on-site visits

Engineering analysis

HEA

Lead: Opinion

Dynamics

TPA interview

Materials review

Database analysis

Participant survey

Database audit


verification


Estimate NTG

LIW

Lead: Opinion

Dynamics

TPA interview

Materials review

Database analysis

Participant survey

Database audit


verification


EES

Lead: Cadmus

TPA interviews

Student surveys

Parent surveys

Teacher surveys

TPA interviews

Student surveys

Parent surveys

Facility staff interview



Programs Process Evaluation

Approach Impact Evaluation Approach Market Effects Evaluation Approach

Facility staff interview Deemed savings review

Nonresidential Program

Commercial and

Industrial

Lead: Cadmus

TPA interviews

Participating customer

(lighting and non-lighting)

surveys

Participating trade ally

surveys

TPA interviews

Participating customer (lighting and

non-lighting) surveys

Engineering file review

TPA interview

Review of C&I Core program

performance and database for PY1

through PY3

Secondary research of market

trends for Lighting and HVAC

General population survey with C&I

customers via phone

Survey with participating trade

allies via internet

Engineering analysis




Program Design and Implementation The HEA program offered a walk-through audit and direct installation of energy-efficiency measures for

single-family homes. Energy auditors conducted a walk-through assessment of participating customers

homes to evaluate the homes energy performance. While on-site, auditors installed energy-efficiency

measures, such as CFLs and energy-efficient faucet aerators. In addition, customers received a report

about their homes energy use, suggesting further actions to reduce customers energy consumption.

The program ended on or before December 31, 2014.

The HEA program delivery model instructed auditors to install as many measures as possible in homes

(without leaving measures behind) and recording all installed measures in the program tracking

database. Auditors could install up to 20 CFLs in a participants home. While available in previous years,

the program discontinued offering water heater tank wrap and Smart Strip measures due to the

measures minor contribution to program savings.

In PY3, the program reached 39,992 households. With the expectation that the program would end in

December 2014, the overall level of audits by month decreased, compared to previous program years;

visits averaged about 3,335 per month. Figure 15 shows the number of audits completed in each month

throughout the program lifecycle.

Figure 15. HEA Audits Completed by Month in PY1, PY2, and PY3

Source: Energizing Indiana PY3 program tracking database.

The TPA employed a number of tactics to draw customers into the program, including Internet based

outreach, mailings, community partners, and neighborhood canvassing. The TPA canvassed

neighborhoods and encouraged residents to participate in the HEA program. In PY3 (unlike in previous

402

987

1,5641,546

1,351

1,3281,345

2,123

1,955

4,725

7,795

5,153

6,885

4,883

5,153

6,4926,347

5,6445,889

6,311

5,393

6,716

6,1276,216

4,070

3,910

4,259

4,016

3,364

3,0023,129

2,765

3,246

2,9893,003

2,239

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

9,000

Co

un

t o

f H

EA A

ud

it V

isit

s

PY2 PY3PY1 n=30,274PY2 n=70,056PY3 n=39,992

PY1



program years, when canvassing efforts followed a shotgun approach), the TPA focused on geographic

areas where field advisors were active. This reduced travel time and allowed canvassers to focus more

deeply on strategic areas. The TPA also integrated mailing efforts with canvassing efforts, rather than

focusing on areas with low saturation, as in previous years. The intent was to provide two to three

touches before canvassers arrived. This strategy was employed in all participating IOU territories,

except for I&M, where canvassing did not take place during PY3.

In the past, the TPA enlisted marketing assistance from a number of community partners, and nonprofit

organizations, with community partners awarded $25 dollars for each recruited participant. The TPA

ended this incentive on September 30, 2014, to ensure any pending audits would be completed before

the end of the program.

To help assure the success in the canvassing measures, the TPA launched two monthly ally-training

efforts in PY3, offered every two weeks on a rotating basis: Employee Technical and Vehicle (ETC), and

QC Metric Training. The TPA ETC courses focused on topics related to basic program implementation

and customer service: professionalism, attire, analyzing the customer report, general field topics,

company policies, and vehicle use policies. QC Metric Training covered 89 different aspects of a

successful home energy audit. In each class, field supervisors reviewed 10 of these aspects and answer-

related questions.

EM&V Methodology The evaluation of the HEA program incorporated impact and process elements. Table 20 provides an

overview of tasks used for evaluating this program.

Table 20. Program Evaluation Overall Tasks

Action Details

Implementer Interviews Interviewed three implementation staff.

Program Database Review/

Verification Reviewed the participant data tracking database.

Participant Interviews Conducted telephone survey with 202 program participants to establish

installation rates, ascertain NTG, and collect process information.

Impact Analysis

Audited the measure installations reported in the program database.

Verified the reported measure installations using participant interviews.

Performed engineering calculations of installed measures.

Estimated NTG (freeridership and spillover), based on self-reported data from

participant interviews.

Changes were minimal in PY3, and the program implemented the design changes that occurred in PY2,

including ex-ante values established by measure (rather than by home) and direct installations of all

measures (rather than CFL distribution).

Sampling Design

The Evaluation Team developed a sampling plan to produce telephone survey findings within the

established bounds of 90% confidence level and 10% precision at the program level. The Team



conducted 202 interviews with program participants in September 2014. Table 21 provides the sample

frame, response and cooperation rates.

Table 21. Participant Survey Sample Design*

Action Population Completions Cooperation Rate Response Rate**

Telephone Participant Interviews 40,022 202 40% 17%

* Response rate gives the percentage of participants who completed an interview out of all of the participants who were called, even if we did not actually speak with them (for example, getting an answering machine). Cooperation rate gives the percentage of participants who completed an interview out of all of the participants with whom we actually spoke. ** In general, a rate response above 10% is considered acceptable.

Program Performance Table 22 shows how ex-ante savings compared to goals established in program planning. Ex-ante savings

reported by the program did not reflect adjustments resulting from the evaluation. The TPA reported

the program met 90% of the kWh, 96% kW, and 85% of therm goals.

Table 22. Ex-Ante Statewide Results

Utility

kWh kW Therms

Goal Ex-Ante % of

Goal Goal Ex-Ante

% of

Goal Goal Ex-Ante

% of

Goal

Duke 18,967,000 17,714,195 93% 2,323 2,272 98% 767,255 528,697 69%

I&M 2,182,000 2,181,517 100% 236 239 102% 92,327 60,154 65%

IPL 16,670,039 15,377,976 92% 2,136 2,083 98% 669,823 528,237 79%

NIPSCO 11,726,990 9,472,965 81% 1,365 1,254 92% 477,921 612,978 128%

Vectren 6,472,991 5,486,336 85% 873 815 93% 257,704 189,022 73%

Statewide 56,019020 50,232,989 90% 6,932 6,664 96% 2,265,030 1,919,088 85%

In terms of homes served, the program achieved 80% of its participation goals, as shown in Table 23.

Table 23. Number of Homes Served by the HEA Program

Utility Number of Homes

Goal Ex-Ante % Achieved

Duke 17,563 13,539 77%

I&M 2,106 1,632 77%

IPL 15,117 11,809 78%

NIPSCO 9,568 8,417 88%

Vectren 5,321 4,588 86%

Statewide 49,675 39,985 80%

Impact Analysis This impact evaluation included a database review of volume of measures installed in each participating

home. The review sought to capture volume and ex-ante savings, and to estimate audited savings. The



Evaluation Team also conducted an engineering analysis of measures installed to establish ex-post

savings.

Audited Savings

The impact evaluation process began with the review of the program database and summed the

measure volume installed in homes. In addition, the Evaluation Team captured volumes of canvassing

measures distributed through customer recruitment. Table 24 shows measures installed in participant

homes (before canvassing). CFLs were the prevalent measures installed through the program.

Table 24. Audited Measure Quantities

Measures in Database Duke I&M IPL NIPSCO Vectren Statewide

CFLs Bulb 210,194 26,169 185,891 139,192 69,489 630,935

Pipe Wrap Home 3,230 566 2,934 327 598 7,655

Energy Efficient Showerhead Showerhead 5,445 696 4,250 409 1,010 11,810

Faucet Aerators Aerator 10,946 1,534 8,436 941 2,149 24,006

Audit Recommendations Home 13,541 1,634 11,809 8,418 4,590 39,992

Table 25 shows the count of CFLs distributed through canvassing, as they appeared in the program

database.

Table 25. Audited Canvassing Measure Quantities

Measures Duke I&M IPL NIPSCO Vectren Statewide

CFLs 9,108 0 10,661 3,626 5,385 28,780

To arrive at audited savings, the Evaluation Team multiplied the agreed-upon ex-ante deemed savings

by the volume of measures installedboth the canvassing efforts and the measures installed in

participating homes (i.e., the number of CFLs * ex-ante deemed savings per CFL)to arrive at the

programs total audited energy, demand, and gas savings, as shown in Table 26.

Table 26. Audited Energy Savings from Direct Install and Canvassing Measures Combined

Utility Audited kWh Audited kW Savings Audited Therms

Duke 17,654,071 2,249 735,102

I&M 2,173,091 237 96,527

IPL 15,328,224 2,066 618,825

NIPSCO 9,466,026 1,251 313,423

Vectren 5,476,656 811 212,225

Statewide 50,098,068 6,614 1,976,102



The per unit ex-ante kWh savings (by measure) is shown in Table 27.

Table 27. Ex-Ante Per Unit kWh Savings, by Measure and by Utility

Measure Utility

Duke I&M IPL NIPSCO Vectren

CFL 50 50 50 50 50

Pipe Wrap 83.2 87.1 74.7 89 83.9

Showerhead 441 441 441 441 441

Aerator 68.1 68.1 68.1 68.1 68.1

Recommendations 247.2 247.2 247.2 247.2 247.2

Canvassing CFLs 41.9 0 41.9 41.9 41.9

Smart Strips 102.8 102.8 102.8 102.8 102.8

Carryover Canvassing CFLs 41.9 41.9 41.9 41.9 41.9

Verified Savings

To establish verified savings, the Evaluation Team established an installation rate by comparing the ex-

ante measure volume against the volume customers reported as installed in the participant surveys (the

installation rate). The survey data also enabled calculation of a persistence rate by establishing

measures that remained installed after the audits. The Evaluation Team multiplied this installation rate

and the persistence rate by the audited savings to establish verified savings. The installation rate

reflected statewide values (rather utility-specific) due to survey sample sizes established to reach the

agreed-upon confidence and precision at the statewide level over the three-year program cycle.

For CFLs distributed through the canvassing activities, the Evaluation Team did not perform primary

research with program participants and, as such, applied the installation and persistence rate as

calculated through the residential lighting program. Table 28 shows installation rates and persistence

rates.

Table 28. Statewide Verification Adjustments

Measure Statewide Installation Rate Statewide Persistence

CFLs 99.93% 97.32%

Pipe Wrap 100% 100%

Energy Efficient Showerhead 94.7% 100%

Faucet Aerators 88.4% 100%

Audit Recommendations 100% 100%

Canvassing CFLs 79% 99%

For the most part, installation rates reflected customers confirmation that the majority of measures

reported were installed during the audit. Statewide persistence rates indicated most measures

remained in place, except for a small volume of CFLs. For example, 88.4% of Faucet Aerators were

confirmed as having been installed by customers. Among the faucet aerators that were confirmed,

customer report that 100% were still installed.

Table 29 shows the resulting verified energy, demand, and gas savings after applying installation and

persistence rates to audited savings.



Table 29. Audited and Verified Savings from Audits and Canvassing Activities, without Carryover

Utility Audited kWh Verified kWh Audited kW Verified kW Audited

Therms

Verified

Therms

Duke 17,654,071 17,067,361 2,249 2,121 735,102 731,068

I&M 2,173,091 2,108,623 237 230 96,527 95,945

IPL 15,328,224 14,808,692 2,066 1,932 618,825 616,297

NIPSCO 9,466,026 9,224,529 1,251 1,195 313,423 316,368

Vectren 5,476,656 5,291,226 811 750 212,225 212,248

Statewide 50,098,068 48,500,431 6,614 6,229 1,976,102 1,971,926

As per the DSMCC request, the Evaluation Team established realization rates based on verified savings.

Table 30 shows verified savings and utility-specific realization rates. The precision around the HEA

realization rates for the PY1-PY3 period is 3.6% at 90% confidence.

Table 30. Verified Realization Rates from Audit and Canvassing Activities, without Carryover

Utility Ex-Ante

kWh

Verified

kWh

Reali-

zation

Rate

Ex-Ante

kW

Verified

kW

Reali-

zation

Rate

Ex-Ante

Therms

Verified

Therms

Reali-

zation

Rate

Duke 17,714,195 17,067,361 96% 2,272 2,121 93% 528,697 731,068 138%

I&M 2,181,517 2,108,623 97% 239 230 96% 60,154 95,945 159%

IPL 15,377,976 14,808,692 96% 2,083 1,932 93% 528,237 616,297 117%

NIPSCO 9,472,965 9,224,529 97% 1,254 1,195 95% 612,978 316,368 52%

Vectren 5,486,336 5,291,226 96% 815 750 92% 189,022 212,248 112%

Statewide 50,232,989 48,500,431 97% 6,664 6,229 93% 1,919,088 1,971,926 103%

For PY3, the program claimed additional savings from CFLs dropped off at customers homes during PY1

and PY2 (the program design changed to only allow direct installation of program measures). A

percentage of these CFLs were placed in storage during the year distributed, with up to 99% of stored

bulbs expected to be installed within a few years. This caused savings to occur several years after initial

purchases. Hence, the program should receive additional credit for bulbs left behind with customers for

future installation. The Evaluation Team defined these as Carryover Savings.

To establish PY1 and PY2 Carryover savings, the Evaluation Team followed the Uniform Methods

Project (UMP)3 recommended method.

3 The UMP is a framework and set of protocols established by the U.S. Department of Energy for determining

energy savings from energy-efficiency measures and programs. More details are available online:

https://www1.eere.energy.gov/office_eere/de_ump_protocols.html.



The UMP explains most bulbs placed into storage (99%) become installed within two years and

recommends calculating the installation rate for two years after the bulb purchase, as follows:

2 = 3 =99% 1

2

Where:

ISR = In-service rate

Y1 = Year 1, the year the bulb was purchased

Y2, Y3 = Years 2 and 3, the two years following the bulbs purchase date

99% = Percentage of program bulbs installed within three years, including the program year

Per the UMP method, the Evaluation Team applied a 55% installation rate for PY2 left-behind CFLs

installed in PY3, and applied a 45% installation rate for PY1 left-behind CFLs in PY3.

Table 31 summarizes additional carryover savings from bulbs distributed in PY1 and PY2, but installed in

PY3.

In addition, the table includes canvassing measures excluded from the PY2 analysis. In PY2, the

Evaluation Team had not received completed measure counts for canvassing measures, thus the PY2

analysis excluded some Smart Strip measures from the PY2 analysis (representing less than one-tenth of

1% of the HEA programs savings). As such, savings associated with this portion of the Smart Strips

installations were included in the PY3 analysis. While this increased demand and energy savings, it

resulted in negative gas savings.



Table 31. Verified PY1 and PY2 Carryover Savings from CFLs Left Behind in Homes (Added to PY3)

Utility Program

Year Measure

Quantity Left in

Homes

Installation

Rate

Persistence

Rate

Verified

Quantity

Verified Carryover

kWh

Verified

Carryover kW

Verified Carryover

Therms

Duke PY1 PY1 CFLs 9,212 45% 97.3% 4,034 201,717 24 -3,913

Duke PY2 PY2 CFLs 25,190 55% 97.3% 13,483 674,166 81 -13,079

Duke PY2 PY2 Smart Strips 15 100% 100% 15 1,542 0 0

Duke PY2 PY2 Canvassing CFLs 3,604 50% 99% 1,784 74,749 71 0

I&M PY1 PY1 CFLs 3,718 45% 97.3% 1,628 81,414 10 -1,579

I&M PY2 PY2 CFLs 7,029 55% 97.3% 3,762 188,119 23 -3,650

I&M PY2 PY2 Smart Strips 228 100% 100% 228 23,438 3 0

I&M PY2 PY2 Canvassing CFLs 1 50% 99% 0 12 0.01 0

IPL PY1 PY1 CFLs 16,391 45% 97.3% 7,178 358,917 43 -6,963

IPL PY2 PY2 CFLs 28,135 55% 97.3% 15,060 752,984 90 -14,608

IPL PY2 PY2 Smart Strips 633 100% 100% 633 65,072 8 0

IPL PY2 PY2 Canvassing CFLs 3,416 50% 99% 1,691 70,845 68 0

NIPSCO PY1 PY1 CFLs 5,731 45% 97.3% 2,510 125,493 15 -2,435

NIPSCO PY2 PY2 CFLs 14,306 55% 97.3% 7,657 382,875 46 -7,428

NIPSCO PY2 PY2 Smart Strips 12 100% 100% 12 1,234 0 0

NIPSCO PY2 PY2 Canvassing CFLs 3,227 50% 99% 1,597 66,925 64 0

Vectren PY1 PY1 CFLs 2,959 45% 97.3% 1,296 64,794 8 -1,257

Vectren PY2 PY2 CFLs 11,435 55% 97.3% 6,121 306,038 37 -5,937

Vectren PY2 PY2 Smart Strips 697 100% 100% 697 71,652 8 0

Vectren PY2 PY2 Canvassing CFLs 833 50% 99% 412 17,273 16 0

Statewide - - 136,771 - 69,800 3,529,258 615 -60,848

Note: the canvassing CFLs carryover savings include canvassing CFLs from the LIW program.



Table 32, Table 33, and Table 34 show total verified energy, demand, and gas savings, respectively,

including savings from PY3 program activities as well as carryover savings from PY1 and PY2.

Table 32: PY3 Verified kWh Savings, with Carryover

Utility Verified kWh from

Audits PY1 Carryover kWh PY2 Carryover kWh

PY3 Total Verified

kWh

Duke 17,067,361 201,717 750,457 18,019,534

I&M 2,108,623 81,414 211,570 2,401,606

IPL 14,808,692 358,917 888,902 16,056,511

NIPSCO 9,224,529 125,493 451,034 9,801,056

Vectren 5,291,226 64,794 394,962 5,750,982

Statewide 48,500,431 832,334 2,696,924 52,029,689

Table 33. PY3 Verified kW Savings, with Carryover

Utility Verified kW from

Audits PY1 Carryover kW PY2 Carryover kW PY3 Total Verified kW

Duke 2,121 24 152 2,298

I&M 230 10 25 266

IPL 1,932 43 166 2,141

NIPSCO 1,195 15 110 1,320

Vectren 750 7.8 62 819

Statewide 6,229 100 515 6,844

Table 34. Verified PY3 Therm Savings, with Carryover

Utility Verified Therms from

Audits

PY1 Carryover

Therms

PY2 Carryover

Therms

PY3 Total Verified

Therms

Duke 731,068 -3,913 -13,079 714,076

I&M 95,945 -1,579 -3,650 90,716

IPL 616,297 -6,963 -14,608 594,726

NIPSCO 316,368 -2,435 -7,428 306,506

Vectren 212,248

Documents

Energizing Indiana 2014 Report