Embed Size (px)
Citation preview
Rocky Mountain Power
Wyoming
See ya later, refrigerator®:
Program Evaluation Report
2011-2012 October 23, 2013
Rocky Mountain Power
Prepared by:
Jason Christensen
Emily Miller
Josh Keeling
Kate Bushman
Steve Cofer
Hossein Haeri, Ph.D.
Cadmus
i
Table of Contents Glossary of Terms......................................................................................................................................... iii
Executive Summary ....................................................................................................................................... 1
Summary of Key Findings ....................................................................................................................... 1
Key Impact Findings ......................................................................................................................... 1
Key Process Findings ........................................................................................................................ 3
Cost-Effectiveness Results ............................................................................................................... 3
Summary and Recommendations .......................................................................................................... 5
Program Description and Overview .............................................................................................................. 6
Program Participation ............................................................................................................................ 6
Impact Evaluation ....................................................................................................................................... 10
Methodology ........................................................................................................................................ 10
Sampling Approach ........................................................................................................................ 10
Impact Estimation Method ............................................................................................................ 11
Kit Savings Algorithm and Assumptions ........................................................................................ 12
Evaluated Gross Savings ....................................................................................................................... 14
Gross Annual Unit Energy Consumption ........................................................................................ 14
Refrigerator Regression Model ...................................................................................................... 15
Freezer Regression Model ............................................................................................................. 15
Extrapolation .................................................................................................................................. 16
Kit Savings ...................................................................................................................................... 16
UEC Summary ................................................................................................................................ 17
In-Service Rates .................................................................................................................................... 17
Appliance Part-Use Factor ............................................................................................................. 17
CFL Installation Rate ...................................................................................................................... 19
Tracking Database Review and Verification .................................................................................. 19
Net-to-Gross ......................................................................................................................................... 20
Freeridership .................................................................................................................................. 21
Secondary Market Impacts ............................................................................................................ 22
Integration of Freeridership and Secondary Market Impacts ....................................................... 23
Induced Replacement .................................................................................................................... 24
ii
Spillover ......................................................................................................................................... 25
Final Net-to-Gross .......................................................................................................................... 28
Summary of Impact Findings ................................................................................................................ 28
Process Evaluation ...................................................................................................................................... 30
Methodology ........................................................................................................................................ 30
Program Implementation and Delivery ................................................................................................ 31
Program History and Program Management ................................................................................ 31
Program Staffing and Training ....................................................................................................... 32
Delivery Structure and Processes .................................................................................................. 32
Forms and Incentives ..................................................................................................................... 33
Marketing ............................................................................................................................................. 34
Approach ........................................................................................................................................ 34
Targeting ........................................................................................................................................ 35
Comparison with Nonparticipants ................................................................................................. 35
Customer Response .............................................................................................................................. 37
Satisfaction ..................................................................................................................................... 37
Barriers ........................................................................................................................................... 39
Quality Assurance/Quality Control ....................................................................................................... 39
Benchmarking ....................................................................................................................................... 40
Cost-Effectiveness ....................................................................................................................................... 42
Appendix A: Participant Demographics ...................................................................................................... 46
Appendix B: Precision Calculations ............................................................................................................. 48
Appendix C: Participant Survey Instrument ................................................................................................ 50
Appendix D: Nonparticipant Survey Instrument ......................................................................................... 70
Appendix E: Logic Model ............................................................................................................................. 77
Appendix F: Refrigerator NTG Combined Decision Tree............................................................................. 80
Appendix G: Freezer NTG Combined Decision Tree ................................................................................... 81
Appendix H: CFL Engineering Calculations and Assumptions ..................................................................... 82
Hours of Use ......................................................................................................................................... 82
Waste Heat Factor ................................................................................................................................ 85
iii
Glossary of Terms
Analysis of Covariance (ANCOVA)
An ANCOVA model is an ANOVA model with a continuous variable added.
Analysis of Variance (ANOVA)
An ANOVA model explains the variation in the independent variable, based on a series of characteristics
(expressed as binary variables with values of zero or one, indicating the absence or presence of the
characteristics).
Coefficient of Determination (R2)
The R2 indicates the proportion of variance in a dependent variable explained by a regression equation,
and takes values between zero and one. An R2 of zero indicates that the independent variables have no
explanatory power. An R2 of one indicates that 100% of the variability in the dependent variable is
explained by changes in the independent variables.
Evaluated Gross Savings
Evaluated gross savings represent the total savings resulting from a program, before adjusting for
freeridership or spillover. They are most often calculated for a given measure, ‘i,’ as:
Evaluated Net Savings
Evaluated net savings are the total savings resulting from a program, net of what would have occurred in
the program’s absence. These savings can be attributed to the program, and are calculated as:
Freeridership
Freeridership in energy-efficiency programs is participants who would have adopted the energy-efficient
measure in the program’s absence. This is often expressed as the freeridership rate, or the proportion of
evaluated gross savings that can be classified as freeridership.
Gross Unit Energy Savings
For the SYLR program, gross unit energy savings are the evaluated in situ unit energy consumption for
the recycled unit, adjusted for part-use.
In-Service Rate (ISR)
The ISR (also called the installation rate) is the proportion of incented measures actually installed.
iv
Net-to-Gross (NTG) Ratio
The NTG ratio is a ratio of net savings to gross savings. Analytically, NTG is defined as:
Net Realization Rate
The net realization rate is a comparison of evaluated net savings to reported gross savings.
P-Value
A p-value indicates the probability that a statistical finding might be due to chance. A p-value less than
0.10 indicates that, with 90% confidence, the finding is statistically significant.
Part-Use Factor
The part-use factor is the portion of the year that equipment is operating. That is, if a given measure has
a part-use factor of 0.5, it was operational for six months out of the year, on average.
Spillover
Spillover is the adoption of an energy-efficiency measure that was induced by the program’s presence,
but was not directly funded by the program. As with freeridership, the spillover rate is expressed as a
proportion of evaluated gross savings.
T-Test
The t-test is a general statistical test of difference. In regression analysis, a t-test is applied to determine
whether the estimated coefficient differs significantly from zero. A t-test with a p-value less than 0.10
indicates that there is a 90% probability that the estimated coefficient is different from zero.
1
Executive Summary
Rocky Mountain Power contracted with Cadmus to conduct an impact and process evaluation of its See
ya later, refrigerator® (SYLR) Program for the 2011 and 2012 program years. To evaluate program gross
and net energy savings for the impact evaluation, Cadmus used secondary meter data analysis, surveys
of program participants and nonparticipants, and a review of the program tracking data. To evaluate the
effectiveness of program processes, Cadmus conducted in-depth interviews with program staff involved
in different aspects of the program.
The evaluation data consisted of:
Telephone surveys with 238 participating Wyoming customers;
Telephone surveys with 53 nonparticipating Wyoming customers;
Review of Wyoming program materials; and
In-depth interviews with program management and program administrator staff.
Summary of Key Findings
Key Impact Findings
The key impact evaluation findings are:
In 2011, the SYLR Program recycled 1,036 refrigerators and freezers and distributed 801 energy-
savings kits; in 2012, participation increased to 1,111 total units and 878 energy-savings kits.
The part-use factor (portion of the year that the equipment is in operation) for both
refrigerators and freezers fell within expected ranges at 0.90 and 0.86, respectively.
After adjusting for part-use, the gross per-unit savings were 1,130 kWh for refrigerators and 944
kWh for freezers.
The net per-unit savings were 443 kWh for refrigerators and 477 kWh for freezers. These values
are lower than the evaluated per-unit savings for 2009-2010,1 primarily due to changes in the
evaluation methodology.2
The overall NTG decreased from 78% in the 2009-2010 evaluation to 42%. Again, this was
primarily driven by changes in the evaluation methodology, as well as the fact that fewer 2011-
2012 participants said they would keep the unit in the absence of the program.
1 The evaluated per-unit net savings in the 2009-2010 evaluation were 657 for refrigerators and 517 kWh for
freezers, with NTGs of 57% for both refrigerators and freezers.
2 See the Impact Estimation Method section for more detail.
2
Gross savings for energy-savings kits (57 kWh) were 21% lower than the reported per-unit
savings of 72 kWh.
Participants reported installing 80% of the compact fluorescent lamps (CFLs) provided in the
energy-savings kit offered through the program. In addition, 51% of surveyed participants
recalled receiving the kit containing CFLs.
Table 1 summarizes program participation, gross savings (reported and evaluated), and evaluated net
savings for 2011 and 2012.3 Table 2 and Table 3 show savings for each program year.
Table 1. 2011 and 2012 Program Savings by Measure
Measure Evaluated
Participation
Reported
Gross
Savings
(kWh)
Evaluated
Gross
Savings
(kWh)
Gross
Realization
Rate
Evaluated
Net
Savings
(kWh)
Net
Realization
Rate
Refrigerator
Recycling 1,746 2,013,240 1,973,701 98% 772,850 38%
Freezer
Recycling 401 505,530 378,616 75% 191,349 38%
Energy-
Savings Kit 1,007 143,561 57,430 40% 57,430 40%
Total 3,154 2,662,331 2,409,747 91% 1,021,630 38%
Table 2. 2011 Program Savings by Measure
Measure Evaluated
Participation
Reported
Gross
Savings
(kWh)
Evaluated
Gross
Savings
(kWh)
Gross
Realization
Rate
Evaluated
Net
Savings
(kWh)
Net
Realization
Rate
Refrigerator
Recycling 831 953,670 939,373 99% 367,834 30%
Freezer
Recycling 205 329,130 193,557 59% 97,822 30%
Energy-
Savings Kit 480 77,193 27,400 35% 27,400 35%
Total 1,516 1,359,993 1,160,331 85% 493,057 36%
3 Throughout this report, the table totals may not sum due to rounding. Precision estimates, for means and totals
(such as savings) are expressed in relative terms, while those for proportions and ratios (such as NTG) are expressed in absolute terms.
3
Table 3. 2012 Program Savings by Measure
Measure Evaluated
Participation
Reported
Gross
Savings
(kWh)
Evaluated
Gross
Savings
(kWh)
Gross
Realization
Rate
Evaluated
Net
Savings
(kWh)
Net
Realization
Rate
Refrigerator
Recycling 915 1,059,570 1,034,328 98% 405,016 38%
Freezer
Recycling 196 176,400 185,059 105% 93,527 53%
Energy-
Savings Kit 527 66,368 30,030 45% 30,030 45%
Total 1,638 1,302,338 1,249,417 96% 528,573 41%
Key Process Findings
The following are the key process evaluation findings:
Collaboration between Rocky Mountain Power and the program administrator proved effective,
due to a longstanding working relationship. Program staff reported effective communication
and smooth implementation.
Participant satisfaction was high in the 2011 and 2012 program years: 99% of the surveyed
participants reported being very or somewhat satisfied with the program. An overwhelming
majority of participants expressed satisfaction with the program sign-up process and incentive
levels. The survey did not reveal notable complaints.
Participants learned of the program through various channels, the two most common being bill
inserts and television advertising. The online sign-up process and the telephone sign-up process
both proved simple and easy to understand, and participants reported high satisfaction levels
with both channels.
Cost-Effectiveness Results
As shown in Table 4, the program was cost-effective across the evaluation period according to four of
the five primary cost-effectiveness tests: PacifiCorp’s Total Resource Cost (PTRC) test, Total Resource
Cost (TRC) test, Utility Cost Test (UCT), and Participant Cost Test (PCT).
The program was cost-effective with relatively high benefit/cost ratios of 1.72 and 1.56 from the PTRC
and TRC test perspectives. The SYLR program was not cost-effective based on the Ratepayer Impact
Measure (RIM) test.4
4 The RIM test is included as a measure of the program’s impact on all ratepayers, and it is common for
demand-side management (DSM) programs to have a RIM ratio of less than one.
4
Table 4. 2011 and 2012 Evaluated Program Cost-Effectiveness Summary
Cost-Effectiveness Test Levelized
$/kWh Costs Benefits Net Benefits
Benefit/Cost
Ratio
PTRC 0.0516 $332,456 $571,040 $238,584 1.72
TRC 0.0516 $332,456 $519,127 $186,671 1.56
UCT 0.0516 $332,456 $519,127 $186,671 1.56
RIM
$907,341 $519,127 -$388,214 0.57
PCT
$0 $1,429,879 $1,429,879 N/A
Table 5 and Table 6 show the program’s cost-effectiveness for the 2011 and 2012 program years,
respectively.
Table 5. 2011 Evaluated Program Cost-Effectiveness Summary
Cost-Effectiveness Test Levelized
$/kWh Costs Benefits Net Benefits
Benefit/Cost
Ratio
PTRC 0.0463 $163,613 $312,013 $148,400 1.91
TRC 0.0463 $163,613 $283,649 $120,036 1.73
UCT 0.0463 $163,613 $283,649 $120,036 1.73
RIM
$460,588 $283,649 -$176,940 0.62
PCT
$0 $729,994 $729,994 N/A
Table 6. 2012 Evaluated Program Cost-Effectiveness Summary
Cost-Effectiveness Test Levelized
$/kWh Costs Benefits Net Benefits
Benefit/Cost
Ratio
PTRC 0.0582 $180,949 $277,599 $96,650 1.53
TRC 0.0582 $180,949 $252,363 $71,414 1.39
UCT 0.0582 $180,949 $252,363 $71,414 1.39
RIM
$478,785 $252,363 -$226,422 0.53
PCT
$0 $750,067 $750,067 N/A
5
Summary and Recommendations Although participation was slightly lower than expected in both 2011 and 2012, the SYLR Program ran
smoothly with no major implementation issues, and experienced high customer satisfaction. The
program cost-effectively achieved net savings of 1,021,630 kWh over the two-year period.
Based on evaluation results, Cadmus offers the following recommendations:
Rocky Mountain Power should consider adjusting its expected per-unit savings to reflect the
evaluated per-unit gross savings values of 1,130 kWh for refrigerators, 944 kWh for freezers,
and 57 kWh for kits, as found in this evaluation.
The program administrator and Rocky Mountain Power should continue with improved tracking
of energy-savings kit delivery, including recording whether or not a kit was delivered. This
change was already implemented beginning in 2013.
Rocky Mountain Power should consider revising the measures provided in the kits, as the Energy
Independence and Security Act of 2007 (EISA) standards in 2014 will lead to decreased savings
for 60-watt replacement CFLs. If kit measures are revised, Rocky Mountain Power should update
expected per-unit savings for kits accordingly.
For future cost-effectiveness calculations, Cadmus recommends that Rocky Mountain Power
update measure lives to align them with the values adopted in the most recent Regional
Technical Forum (RTF) measure workbooks:.
Refrigerators: 7 years5
Freezers: 5 years
CFLs (kits): 6 years6
5 Refrigerators and freezers found in:
http://rtf.nwcouncil.org//measures/res/ResFridgeFreezeDecommissioning_v2_5.xlsm 6 CFLs found in: http://rtf.nwcouncil.org/measures/res/ResCFLLighting_v2_2.xlsm
6
Program Description and Overview
The Wyoming See ya later, refrigerator (SYLR) residential refrigerator and freezer recycling program
serves as part of Rocky Mountain Power’s ongoing demand-side management (DSM) resource
acquisition strategy.7 Rocky Mountain Power’s overarching objective with the program is to decrease
electricity usage (kWh) by removing and recycling inefficient secondary refrigerators and freezers and
older primary refrigerators. The program encourages those shopping for replacement units to consider
ENERGY STAR®-labeled models, and refers them to the Home Energy Savings (HES) Program, where they
may be eligible for incentives for other energy-efficiency measures and services. In addition to reducing
energy consumption and lowering participants’ electricity bill, participating appliances are recycled in an
environmentally sound manner.8
Beginning operation in 2009, the SYLR Program provides residential customers with a $30 incentive for
each recycled appliance. Beginning in October 2011, Rocky Mountain Power increased the incentive to
$40 per appliance. Participants receive an incentive for up to two refrigerators or freezers. Renters who
own their appliances may participate, and apartment complex owners or managers who provide tenants
with appliances are eligible. Participants also receive an energy-savings kit, which includes: two 13-watt
CFLs, a refrigerator/freezer thermometer card, energy-savings educational materials, and information
on other Rocky Mountain Power residential efficiency programs.
Qualifying refrigerators and freezers must be in working condition when picked up and be at least
10 cubic feet in size. Rocky Mountain Power contracted with JACO Environmental, Inc. (the program
administrator) to implement the program in Wyoming. The program administrator disables and removes
the appliances, and recycles at least 95% of the materials, including the refrigerant.
Program Participation Participation in the SYLR program tends to be seasonal, with the highest participation during summer.
As shown in Figure 1, the SYLR Program saw a steady increase in participation from spring through
summer in 2012. There was also a notable increase in November and December of 2011 after the
incentive increased.
7 See ya later, refrigerator® has been registered to PacifiCorp through the U.S. Patent and Trademark Office
since April 6, 2010, under registration number 3770705.
8 Environmentally-sound disposal of this equipment includes proper disposal of oils, PCBs, mercury, and CFC-11
foam; and recycling of CFC-12, HFC-134a, plastic, glass, steel, and aluminum.
7
Figure 1. Program Participation by Month and Year
Figure 2 shows the four-year trends in program units’ age and size. During this period, the average
freezer age and size did not change noticeably. The average refrigerator age and size declined slightly,
with some variation over time.
Figure 2. Average Unit Age and Size by Year
The refrigerator configurations of program units also changed, increasing in side-by-side models from
2009 to 2012 (Figure 3). There was also a decrease in the proportion of single-door units.
8
Figure 3. Refrigerator Configuration by Year
As shown in Figure 4, freezer configurations did not exhibit an appreciable trend from 2009 to 2012.
Figure 4. Freezer Configuration by Year
9
These trends are consistent with Cadmus’ observations of other recycling programs. As recycling
programs mature, the composition of recycled appliances tends to change. In their infancy, these
programs recycle more secondary appliances (particularly those in use for only a portion of the year).
Such units tend to be older, smaller, and located in unconditioned spaces, such as garages or
basements. They also tend to be less efficient, and are more likely to be single-door units.
10
Impact Evaluation
Methodology This section describes the methodology Cadmus used to evaluate the program’s impact. The report
presents two types of evaluated savings: evaluated gross savings and evaluated net savings. To
determine these values, Cadmus applied four steps (shown in Table 7) to the reported gross program
savings. The evaluation defined reported gross savings as electricity savings (kWh) that Rocky Mountain
Power included in its 2011 and 2012 annual program reports.
Table 7. Impact Estimation Steps
Saving Estimate Step Action
Evaluated Gross Savings
1 Verify accuracy of data in program database
2 Perform statistical/engineering analysis to evaluate per-unit savings
3 Adjust evaluated gross savings with installation rate/part-use factor
Evaluated Net Savings 4 Apply net-to-gross adjustments
Step one (verifying the accuracy of data in the program database) included reviewing the program
tracking database to ensure that participation and reported savings matched the 2011 and 2012 annual
reports.
Step two (performing a statistical/engineering analysis to evaluate per-unit savings) involved estimating
refrigerator, freezer, and CFL savings.
Step three (adjusting the evaluated gross savings with the installation rate/part-use factor) determined
the mean proportion of the year in which recycled appliances were used, as well as the number of CFLs
program participants installed. Using a telephone survey, Cadmus collected information to estimate an
installation rate and part-use factor, which Cadmus then used to calculate evaluated gross savings.
Step four (applying net-to-gross [NTG] adjustments) determined the net savings. Through participant
and nonparticipant telephone surveys, Cadmus estimated freeridership, secondary market effects (i.e.,
the program’s impact on the availability of used appliances), induced replacement, and spillover.9
Sampling Approach
Cadmus developed survey samples of randomly selected program participants and nonparticipants,
seeking precision of ±10% at the 90% confidence level at the measure level. The evaluation determined
sample sizes assuming a 0.5 coefficient of variation (CV). Cadmus applied a finite population correction
to determine the necessary sample size. Table 8 shows the planned and achieved sample sizes by target
group.
9 This report’s Net-to-Gross section provides a detailed description of how Cadmus estimated these parameters.
11
Table 8. Sample Sizes by Target Group
Target Group Population Target Sample Size Achieved Sample Size
Participants 2,147 444 238
Nonparticipants N/A 70 53
*Due to the small population, Cadmus was unable to attain the target number of completed surveys. However, all
efforts were made to attain the target without placing undue burden on customers: up to five attempts were
made to reach each participant, and (as described below) surveys were conducted in three rounds to capture
feedback close to the time of participation.
Cadmus randomly selected 238 survey participants from the population of 2,147 unique participants.
Participant surveys were conducted in three rounds:
1. Quarters 1 and 2 of 2011;
2. Quarters 3 and 4 of 2011; and
3. Quarters 1 through 4 of 2012.
This approach allowed Cadmus to provide Rocky Mountain Power with interim results from the 2011
surveys, as well as ensuring that customers were contacted close to the time when they participated in
the program. All results presented in this report are weighted to reflect the distribution of participants
across the evaluation period.
Cadmus selected the 53 nonparticipants through screening questions that were asked from a random
sample of Rocky Mountain Power customers residing in Wyoming. For this evaluation, nonparticipants
are defined as customers who had recently disposed of a refrigerator or freezer outside of the Rocky
Mountain Power program.
Impact Estimation Method
Cadmus’ impact evaluation methodology for the 2011-2012 program years was informed by recent
guidelines developed by the U.S. Department of Energy (DOE). In 2011, DOE launched the Uniform
Methods Project (UMP) with the goal to “strengthen the credibility of energy savings determinations by
improving EM&V, increasing the consistency and transparency of how energy savings are determined.”10
The UMP identified seven common residential and commercial DSM measures and enlisted a set of
subject matter experts to draft evaluation protocols for each measure or subject area. Refrigerator
recycling was one of the seven identified measures. The DOE recruited Cadmus to manage the UMP
process, as well as be the lead author, for the refrigerator recycling protocol.
Through a collaborative process that included reviews by a technical advisory group and a steering
committee, as well as a public review and response period, the UMP resulted in a set of protocols that
capture the collective consensus of the evaluation community. Each protocol establishes broadly
10
U.S. Department of Energy. “About the Uniform Methods Project.” Last modified January 21, 2013. Accessed
June 4, 2013. http://www1.eere.energy.gov/office_eere/de_ump_about.html.
12
accepted best practices for evaluating the key measures in the category, including identifying and
explaining key parameters, data sources, and gross- and net-related algorithms.
This evaluation follows the methodology outlined in the refrigerator recycling protocol, which largely
mirrors the method Cadmus used for evaluating the 2009-2010 SYLR program, with the following
changes recommended by UMP:
1. Prospective Part-Use. UMP recommends that part-use be assessed based on how the recycled
appliance would likely have been used if not recycled, rather than how it was previously used.
For example, if a primary refrigerator would have become a secondary refrigerator independent
of the program, its part-use should reflect the average usage of secondary refrigerators and not
primary refrigerators.
2. Induced Replacement. UMP recommends that appliance replacement is unavoidable and
naturally occurring. As a result, program savings should not be estimated as the difference in
energy consumption between the recycled appliance and the appliance that replaces it. The
exception to this rule is when recycling programs induce a replacement that otherwise would
not have occurred.
3. Secondary Market Impacts. UMP considers the program’s impact on the used appliance market.
The secondary market impact adjustment accounts for changes in the availability of used
appliances due to the program. Therefore, in this impact evaluation, Cadmus accounted for the
program’s impact on the used appliance market.
4. Regression Model Specification. UMP stipulates a model specification for estimating each
appliance’s annual energy consumption when it is not feasible to use utility-specific in situ
metering and modeling. The UMP model reflects the availability of more winter metering data
and the need to create a more universal and weather symmetrical model (i.e., one that accounts
for the effects of heating and cooling degree days [HDDs, CDDs]).
More information about UMP is available on the DOE’s Website.11
Kit Savings Algorithm and Assumptions
With each appliance pick up scheduled, participants received an energy-savings kit, which contained:
Two 13-watt CFLs;
One refrigerator thermometer; and
Energy-savings educational materials and other program references.
11
U.S. Department of Energy. “Uniform Methods Project for Determining Energy Efficiency Program Savings.”
Last modified April 9, 2013. Accessed June 14, 2013. http://www1.eere.energy.gov/office_eere/de_ump.html.
13
The following algorithm estimated CFL savings:
( )
Where:
ΔWatts = Wattage of baseline bulb minus wattage of kit CFL
ISR = In-service rate, or the percentage of units installed
HOU = Hours-of-use (per day)
WHF = Waste heat factor, adjustment to account for lighting impacts on HVAC
consumption
365 = Constant for days per year
1,000 = Constant for conversion of watts to kilowatts
The ISR captured CFLs installed, removed, and replaced by other energy-efficient light bulbs. Specifically:
( )
Cadmus estimated the baseline wattage by comparing the lumen output of kit CFLs to their
incandescent equivalents. Each 13-watt CFL within the kit has 900 lumens of output. The baseline
incandescent equivalent wattages are determined based on the lumens output of the bulb.
A 60-watt incandescent bulb has a range of 750-1,049 lumens,12 which is within the range to be
equivalent to the 13-watt kit CFL. Cadmus chose to use 60 watts as the baseline because it is the
incandescent bulb of equivalent lighting output.
Cadmus calculated average HOU using ANCOVA13 model coefficients, estimated from a combined
multistate, multiyear database of light logger data, compiled by recent Cadmus CFL HOU studies. This
model expressed average HOU as a function of room type, existing CFL saturations, and the presence of
children in a home. Appendix H provides a more detailed exploration of the impact methodology used to
estimate CFL HOU.
12
Congress signed EISA into law on December 19, 2007. The new law contains provisions for phasing in more
efficient incandescent lamps based on rated lumens. The 60-watt incandescent bulb lumens ranges were
based on rated lumen ranges as defined by EISA.
13 ANCOVA, or analysis of covariance, refers to a type of statistical modeling.
14
The values for all explanatory variables were based on response data from the participant survey, except
for CFL saturation. Data on CFL saturations were taken from the Northwest Energy Efficiency Alliance
2011 residential building stock assessment.14 Figure 5 shows distributions of bulbs by room types.
Figure 5. Location of Bulbs Installed
Evaluated Gross Savings
Gross Annual Unit Energy Consumption
Cadmus used the UMP-specified regression model to estimate the unit energy consumption (UEC) for
refrigerators, and followed the UMP recommendation to use a similar model to estimate freezer UEC. .
The coefficient of each independent variable indicates the influence of that variable on daily
consumption, holding all other variables constant.
A positive coefficient indicates an upward influence on consumption
A negative coefficient indicates a downward effect on consumption
14
Northwest Energy Efficiency Alliance. 2011 Residential Building Stock Assessment: Single-Family Characteristics
and Energy Use. September 18, 2012. http://neea.org/docs/reports/residential-building-stock-assessment-
single-family-characteristics-and-energy-use.pdf?sfvrsn=8
15
The value of the coefficient indicates the marginal impact of a one-point increase in the independent
variable on UEC. (For instance, a 1-cubic foot increase in refrigerator size results in a 0.059 kWh increase
in daily consumption.)
In the case of dummy variables, the value of the coefficient represents the difference in consumption if
the given condition is true. For example, in Cadmus’ refrigerator model, the coefficient for the variable
indicating whether a refrigerator was a primary unit is 0.560; this means that, all else being equal, a
primary refrigerator consumes 0.560 kWh more per day than a secondary unit.
Refrigerator Regression Model
Table 9 shows the UMP model specification Cadmus used to estimate annual energy consumption of
refrigerators in 2011 and 2012, along with the model’s estimated coefficients.
Table 9. Refrigerator UEC Regression Model Estimates (Dependent Variable = Average Daily kWh, R-square = 0.30)
Independent Variables Coefficient p-Value
Intercept 0.805 0.166
Age (years) 0.021 0.152
Dummy: Manufactured Pre-1990 1.036 <.0001
Size (cubic feet) 0.059 0.044
Dummy: Single Door -1.751 <.0001
Dummy: Side-by-Side 1.120 <.0001
Dummy: Primary 0.560 0.008
Interaction: Unconditioned Space x HDDs -0.040 0.001
Interaction: Unconditioned Space x CDDs 0.026 0.188
Freezer Regression Model
Table 10 details the final model specifications Cadmus used to estimate the energy consumption of
participating freezers, along with the results.
Table 10. Freezer UEC Regression Model Estimates (Dependent Variable = Average Daily kWh, R-square = 0.38)
Independent Variables Coefficient p-Value
Intercept -0.955 0.237
Age (years) 0.045 0.001
Dummy: Manufactured Pre-1990 0.543 0.108
Size (cubic feet) 0.120 0.002
Dummy: Chest Freezer 0.298 0.292
Interaction: Unconditioned Space x HDDs -0.031 <.0001
Interaction: Unconditioned Space x CDDs 0.082 0.028
16
Extrapolation
After estimating the final regression models, Cadmus analyzed the corresponding characteristics (the
independent variables) for participating appliances (as captured in the program administrator’s program
database). Table 11 summarizes the program averages or proportions for each independent variable.
Table 11. 2011-2012 Participant Mean Explanatory Variables
Appliance Independent Variables Participant Population Mean Value
Refrigerator
Age (years) 25.74
Dummy: Manufactured Pre-1990 0.62
Size (cubic feet) 17.26
Dummy: Single Door 0.06
Dummy: Side-by-Side 0.23
Dummy: Primary 0.68
Interaction: Unconditioned Space x HDDs* 2.73
Interaction: Unconditioned Space x CDDs* 0.09
Freezer
Age (years) 34.23
Dummy: Manufactured Pre-1990 0.86
Size (cubic feet) 17.31
Dummy: Chest Freezer 0.30
Interaction: Unconditioned Space x HDDs* 8.04
Interaction: Unconditioned Space x CDDs* 0.28
* CDDs and HDDs are the weighted average from Typical Meteorological Year (TMY3) data for weather stations
that Cadmus mapped to participating appliance ZIP codes. TMY3 uses median daily values for a variety of weather
data collected from 1991–2005.
To estimate the average annual UEC, Cadmus applied the model coefficients to the independent
variables. For example, using values from Table 10 and Table 11, the estimated annual UEC for freezers
can be calculated as:
( [ ]
[ ] [ ]
[ ] [ ]
[ ])
Kit Savings
Table 12 shows final inputs and gross savings estimated for CFLs distributed in the energy-savings kits.
Table 12. Unadjusted CFL Savings (Not Including Adjustment for In-Service Rate)
Incandescent Watts
CFL Watts
HOU Waste Heat
Factor Gross Annual kWh
(per bulb) Gross Annual kWh
(per kit) 60 13 2.3 0.90 36 72
17
UEC Summary
Table 13 reports the evaluated average annual UEC for refrigerators and freezers recycled through the
SYLR Program during 2011 and 2012. The section following the table describes adjustments Cadmus
made to these estimates to determine the gross per-unit savings estimates for participant refrigerators
and freezers.
Table 13. Estimates of Per-Unit Annual Energy Consumption
Appliance Ex Post Annual UEC (kWh/year) Relative Precision( 90% confidence)
Refrigerators 1,256 7.3%
Freezers 1,098 14.4%
Energy-Savings Kits 71 4.2%
In-Service Rates
Appliance Part-Use Factor
Participants used some of the refrigerators and freezers recycled through the program for part of the
year. Cadmus calculated a weighted average part-use factor, representing the three participant usage
categories (operated year-round, operated for a portion of the preceding year, or unplugged and not
operated), as defined by the appliance’s operational status during the year before it was recycled. For
example, Cadmus gave participants who did not use their appliance at all the year before it was recycled
a part-use factor of zero, as no immediate savings were generated by their appliance’s retirement.
Using information gathered through the participant surveys, Cadmus took the following steps to
determine part-use, as outlined in UMP.
1. Cadmus determined whether recycled refrigerators were primary or secondary units. Cadmus
considered all stand-alone freezers as secondary units.
2. Cadmus asked those participants who indicated they had recycled a secondary refrigerator or
freezer if the appliance had been operated year-round, operated for a portion of the preceding
year, or unplugged and not operated. Cadmus assumed that all primary units were operated
year-round.
3. Cadmus asked participants who indicated that they only operated their secondary refrigerator
or freezer for only a portion of the preceding year to estimate the total number of months the
appliance was plugged in. This allowed for calculating the portion of the year for which the
appliance was in use. Cadmus determined that the average secondary refrigerator and freezer
operating part-time had part-use factors of 0.19 and 0.58, respectively.
These three steps resulted in information about how refrigerators and freezers were operated prior to
recycling (Table 14).
18
Table 14. Historical Part-Use Factors by Category
Usage Type
and Part-Use
Category
Refrigerators Freezers
Percent of
Recycled
Units
Part-Use
Factor
Per-Unit
Energy Savings
(kWh/year)
Percent of
Recycled
Units
Part-Use
Factor
Per-Unit
Energy Savings
(kWh/year)
Secondary Units
Only n=23
Not in Use 9% 0.00 -
Used Part Time 13% 0.19 244
Used Full Time 78% 1.00 1,256
Weighted
Average 100% 0.81 1,015
All Units
(Primary and
Secondary)
n=65 n=49
Not in Use 5% 0.00 - 12% 0.00 -
Used Part Time 5% 0.19 244 4% 0.58 640
Used Full Time 90% 1.00 1,256 84% 1.00 1,098
Weighted
Average 100% 0.92 1,151 100% 0.86 945
Next, Cadmus asked surveyed participants how they would likely have operated their appliances had
they not recycled them through SYLR. For example, if surveyed participants indicated they would have
kept a primary refrigerator independent of SYLR, Cadmus asked whether they would have continued to
use the appliance as their primary refrigerator or if they would have relocated and used it as a
secondary refrigerator.
Cadmus then combined the part-use factors listed in Table 14 with participants’ self-reported action had
the program not been available. This resulted in the distribution of likely future usage scenarios and
corresponding part-use estimates. The weighted average of these future scenarios, shown in Table 15,
produced SYLR’s 2011-2012 part-use factor for primary and secondary refrigerators (0.90) and freezers
(0.86).15
15
Since the future usage of discarded refrigerators is unknown, Cadmus applied the weighted average part-use
value of all refrigerators that would have been discarded independent of the program (0.92). This approach
acknowledges that the next owner of discarded appliances might use them as primary or secondary units.
19
Table 15. Part-Use Factors by Appliance Type
Use Prior to
Recycling
Likely Use Independent
of Recycling
Refrigerator Freezer
Part-Use
Factor
Percent of
Participants
Part-Use
Factor
Percent of
Participants
Primary
Kept (as primary unit) 1.00 0%
Kept (as secondary unit) 0.81 7%
Discarded 0.92 57%
Secondary Kept 0.81 10% 0.86 26%
Discarded 0.92 25% 0.86 74%
Overall 0.90 100% 0.86 100%
CFL Installation Rate
On average, survey data indicated participants initially installed 1.59 of the two bulbs received, resulting
in an 80% installation rate. This is slightly down from the 83% found in the 2009-2010 evaluation. Figure
6 shows the proportion of participants installing zero, one, or two bulbs.
Figure 6. Number of Bulbs Installed
Tracking Database Review and Verification
The program administrator tracked and provided Cadmus with three types of program data:
1. Data on recycled appliances (stored in a “Units” database);
2. Information about appliance pick ups (stored in an “Orders” database); and
3. Data about customers (stored in a “Customers” database).
These integrated databases allowed the program administrator to record information collected via the
call center or Website, along with on-site data collected during pick ups and post-pick-up data recorded
20
during recycling. The program administrator’s client Web portal provided the Rocky Mountain Power
program manager with real-time access to collected data and other program results.
Cadmus reviewed the program administrator’s databases and compared participation recorded in the
databases with the participation reported in Rocky Mountain Power’s annual reports. Reported
quantities matched the database, as shown in Table 16 below.
Verification of Kit Recipients
The program administrator’s database did not include records on the energy-saving kits reported.
Therefore, Cadmus used its participant survey to verify the receipt of kits. There was a substantial
discrepancy between the number of reported and verified energy-savings kits. Approximately 49% of
survey respondents did not recall receiving the kit when their appliance was picked up.
Cadmus identified this low recall issue in the preliminary analysis of the 2011 participant survey. To
ensure the low recall was not due to how the survey was worded, Cadmus added a screening question
to the participant survey to ensure that the person who was present when the appliance was picked up
was the same person who took the survey. This ensured that a respondent who did not recall receiving a
kit was actually present when the kit was provided. Cadmus also interviewed the program administrator
facility manager to assess any possible process changes that might have explained the discrepancies.
This interview yielded no explanation for the fact that less than 100% of participant reported receiving a
kit, although the manager did mention that some customers decline the kits at pickup. 16 Therefore,
Cadmus applied an adjustment to the number of verified kits based on the survey.
Table 16 outlines the reported and verified measure quantities.
Table 16. 2011 and 2012 Reported and Verified Measure Quantities
Measure 2011 2012 Total Difference in Totals
Identified Verified Identified Verified Identified Verified Nominal Proportion
Refrigerators 831 831 915 915 1,746 1,746 - 0%
Freezers 205 205 196 196 401 401 - 0%
Energy-Savings Kits
953 480 1,037 527 1,990 1,007 983 49%
Net-to-Gross Cadmus used the following formula to estimate net savings for recycled refrigerators:
16
The Process Evaluation discusses additional tracking of kit receipt, which was implemented beginning in 2013.
21
Where:
Gross Savings = The evaluated in situ UEC for the recycled unit, adjusted for
part-use;
Freeridership and
Secondary Market Impacts = Program savings that would have occurred in the program’s
absence;
Induced Replacement = Average additional energy consumed by replacement units
purchased due to the program;
Spillover = Non-programmatic savings induced by the program.
Applying the UMP protocol introduced an additional parameter related to net savings—secondary
market impacts—and requires a decision-tree approach to calculating and presenting net program
savings. Cadmus did not follow this approach for the 2009-2010 impact evaluation.
The decision tree—populated by the responses of surveyed participants—presents savings under all
possible scenarios of participants’ actions with the discarded equipment. Cadmus used a weighted
average of savings under these scenarios to calculate the net savings attributable to the program.
Throughout this chapter, specific portions of the decision tree are included to highlight specific aspects
of the net savings analysis. Cadmus also provided the entire decision trees in Appendix F (refrigerators)
and Appendix G (freezers).
Freeridership
The first step of the Cadmus freeridership analysis was to ask participants if they had considered
discarding the participating appliance prior to learning about the program. If the participant indicated no
previous consideration to dispose of the appliance, Cadmus categorized them as a non-freerider and
excluded them from the subsequent freeridership analysis.
Next, Cadmus asked all the remaining participants (i.e., those who had considered discarding their
existing appliance before learning about SYLR) a series of questions to determine the distribution of
participating units likely to have been kept versus discarded absent the program. There are three
possible scenarios independent of program intervention:
1. Unit is discarded and transferred to someone else.
2. Unit is discarded and destroyed.
3. Unit is kept in the home.
22
To determine the percentage of participants in each of the three scenarios, Cadmus asked surveyed
participants about the likely fate of their recycled appliance had it not been decommissioned through
the SYLR Program. Cadmus categorized their responses into the following options:
Kept the appliance.
Sold the appliance to a private party (either an acquaintance or through a posted
advertisement).
Sold or gave the appliance to a used appliance dealer.
Gave the appliance to a private party, such as a friend or neighbor.
Gave the appliance to a charity organization.
Left the appliance on the curb with a “Free” sign.
Had the appliance removed by the dealer from whom the new or replacement refrigerator was
obtained.
Hauled the appliance to a landfill or recycling center.
Had the appliance picked up by local waste management company.
Once Cadmus determined the final assessments of participants’ actions independent of SYLR, Cadmus
calculated the percent of refrigerators and freezers that would have been kept or discarded (the results
are shown in Table 17).
Table 17. Final Distribution of Kept and Discarded Appliance
Stated Action Absent
Program
Indicative of
Freeridership
Refrigerators
(n=161)
Freezers
(n=175)
Kept No 16% 27%
Discarded Varies by Discard Method 84% 73%
Secondary Market Impacts
If a participant would have directly or indirectly (through a market actor) transferred the program-
recycled unit to another Rocky Mountain Power customer absent the program, Cadmus determined
what that would-be acquirer might have done since that unit was unavailable due to the program. Some
would-be acquirers would find another unit, while others would not. This possibility reflects the
awareness that some acquirers were in the market for a refrigerator and would acquire another unit,
while others were not (and would have taken the unit opportunistically). It is difficult to quantify this
absent program-specific information regarding the change in the total number of refrigerators and
freezers (both overall and for used appliances) that were in use before and after implementing the
program. Without this information, UMP recommends that the evaluators assume that half of the
would-be acquirers found an alternate unit. Without information to the contrary, Cadmus applied the
UMP recommendation to this evaluation.
23
Next, Cadmus determined whether the alternate unit was likely to be another used appliance (similar to
those recycled through the program) or a new standard-efficiency unit (presuming fewer used
appliances are available due to program activity).17
For the reasons discussed above, it is difficult to estimate this distribution definitively. The UMP again
recommends a midpoint approach when primary research is unavailable: evaluators should assume that
half of the would-be acquirers would find a similar used appliance, and half would acquire a new,
standard-efficiency unit.
Cadmus used the ENERGY STAR Website to determine the energy consumption of these new, standard-
efficiency appliances. Specifically, Cadmus averaged the reported energy consumption of new, standard-
efficiency appliances of comparable sizes and similar configurations as the program units.
Figure 7 details Cadmus’ methodology for assessing the program’s impact on the secondary refrigerator
market and for applying the recommended midpoint assumptions when primary data were unavailable
(a freezer-specific diagram is provided in Appendix G). As evident in the figure, accounting for market
effects results in three savings scenarios: full per-unit gross savings, no savings, and partial savings (i.e.,
the difference between the energy consumption of the program unit and the new, standard-efficiency
appliance that was acquired instead).
Figure 7. Secondary Market Impact for Refrigerators
Integration of Freeridership and Secondary Market Impacts
After estimating the parameters of the freeridership and secondary market impacts, Cadmus used the
UMP decision tree to calculate the average per-unit program savings net of their combined effect.
Figure 8 shows how Cadmus integrated these values into an estimate of savings net of freeridership and
secondary market impacts. Again, Cadmus applied secondary market impacts as a result of UMP: this
was not accounted for in previous SYLR evaluations.
17
It is also possible that the would-be acquirer would select a new ENERGY STAR® unit. However, most
customers who are in the market for a used appliance would upgrade to the next lowest price point (a
standard-efficiency unit).
24
To ensure that survey participants provided the most reliable responses possible, and to mitigate
socially desirable response bias to the greatest extent possible, Cadmus averaged the participant and
nonparticipant transfer and disposed ratios.
Figure 8. Estimation of Net Savings for Refrigerators
Induced Replacement
UMP states that evaluators must account for the energy consumption of replacement units only when
the program induced that replacement (i.e., when the participant would not have purchased the
replacement refrigerator in the absence of the recycling program). In the case of non-induced
replacements, the energy consumption of the replacement appliance is not germane to the savings
analysis, since that appliance would have been purchased or acquired regardless of the program. The
acquisition of another appliance in conjunction with participation in SYLR does not necessarily indicate
induced replacement. Again, this is consistent with the methods outlined in the UMP.
Cadmus used the results of the participant surveys to determine which of the replacement refrigerators
and freezers were acquired by SYLR participants due to the program. The survey results indicated that
SYLR reduced the total number of used appliances operating within Rocky Mountain Power’s Wyoming
service territory, and raised the average efficiency of the active appliance stock. Across both appliance
types, 30% of participants did not replace their recycled appliance.
Next, Cadmus used participant survey results to estimate the proportion of replacements that were
induced by the customer’s participation in SYLR. Specifically, Cadmus had asked each participant
indicating they replaced the participating appliance: “Would you have purchased the new
refrigerator/freezer without the incentive you received for recycling the old one?” Since an incentive of
$30 to $40 is unlikely to be sufficient motivation for most participants to purchase an otherwise
unplanned-for replacement unit (which can cost $500 to $2,000), Cadmus asked a follow-up question of
participants who responded “No.” This question, intended to confirm the participant’s assertion that the
program alone caused them to replace their appliance, was: “Just to confirm: you would not have
25
replaced your old refrigerator/freezer without the Rocky Mountain Power incentive for recycling, is that
correct?”
To further increase the reliability of these self-reported actions, the induced replacement analysis also
considered: 1) whether the refrigerator was a primary unit, and 2) the participant’s stated intentions in
the absence of the program. For example, if a participant would have discarded their primary
refrigerator independent of the program, it is not possible that the replacement was induced (since it is
extremely unlikely the participant would live without a primary refrigerator). However, for all other
usage types and stated intention combinations, induced replacement was a viable response.
Figure 9. Estimation of Induced Replacement
As expected, only a portion of the total replacements was induced: the program induced 2% and 5% of
all refrigerator and freezer participants, respectively, to acquire a replacement unit.
Table 18. 2011-2012 Induced Replacement Rates
Appliance Induced Replacement Rates
Refrigerator 2%
Freezer 5%
Spillover
Spillover refers to additional savings generated by program participants due to their program
participation, but not captured by program records. Spillover occurs when participants choose to
purchase additional energy-efficient measures or adopt energy-efficient practices due to a program, but
do not participate in another program to receive an incentive for those additional measures. These
customers’ savings are not automatically counted toward the utility’s programmatic savings.
Cadmus estimated spillover from program participants’ adopting additional measures as a result of their
participation. A small effect revealed by a survey may translate into a large effect for the population,
because survey results are applied to the population of eligible participants.
Energy-efficiency programs’ spillover impacts get added to program results. In contrast, freeriders’
impacts reduce a programs’ net savings.
26
For the SYLR program, Cadmus measured spillover by asking a sample of participants who purchased
and received an incentive for a particular measure if, due to the program, they installed another
efficient measure or undertook other energy-efficiency activities. Respondents were asked to rate the
relative influence of the SYLR program and incentive on their decisions to pursue additional savings.
Spillover questions were used to determine whether program participants installed any other energy-
savings measures since participating in the program. Savings participants received from additional
measures were considered spillover savings if the program significantly influenced their decisions to
purchase those additional measures, and if they did not receive additional incentives for those
measures.
SYLR program participants were specifically asked whether they installed the following measures, which
were associated with quantifiable spillover:
High-efficiency dishwashers
High-efficiency washing machines
High-efficiency refrigerators
High-efficiency water heaters
If the participant installed one or more of these measures, they were asked additional questions about
which year they purchased the measure, and whether they received an incentive for the measure. If
applicable, participants were asked how influential the SYLR program was on their purchasing decisions
(participants could answer not at all, not very, somewhat, or very influential). Participants expressed
mixed responses regarding the program’s influence on these actions, with 10% finding the program very
influential (Figure 10).
27
Figure 10. Program Influence on Installing Additional Measures (with 90% Confidence Intervals)
Fourteen percent of participants claimed to have installed energy-efficient measures or changed their
behaviors after participating in the SYLR Program. However, only three such purchases represent
quantifiable savings: energy-efficient refrigerators, clothes washers, and dishwashers. Other measures,
such as weatherization, water heaters, and HVAC, are difficult to quantify accurately based on survey
data, and thus were not included in the spillover analysis. In addition, CFLs, which are commonly
mentioned, were not counted because of a high likelihood of double-counting savings claimed by the
HES Program through upstream CFL sales. Cadmus calculated participant spillover by estimating savings
attributable to additional measures installed, and whether respondents stated that Rocky Mountain
Power was very influential in their decisions. Measures were counted if they were eligible for program
incentives but incentives were not requested.
For calculating spillover savings, Cadmus used a top-down approach. The analysis began using a subset
containing only survey respondents who indicated they installed additional energy-savings measures
after participating in the SYLR program, but without receiving any incentives. From this subset, Cadmus
removed participants who indicated the program was not very influential on their decisions to purchase
additional measures.
For the remaining participants with legitimate spillover savings, Cadmus estimated energy savings from
additional measures installed. The savings values, calculated by Cadmus, were matched to additional
measures installed by survey participants.
Table 19 summarizes participant survey spillover responses for the SYLR program. Appliance per-unit
savings were derived from Cadmus’ evaluation of 2011 and 2012 HES gross savings values. Cadmus
28
assumed CFL savings equaled those calculated for energy-savings kits. Total spillover savings
represented 0.3% of refrigerator savings. There were no spillover savings attributable to freezer
participants.
The spillover results for the SYLR Program are shown in Table 19.
Table 19. Program Spillover in 2011 and 2012
Measure
Total
Spillover
Savings
(kWh)
Participant
Population
Savings (kWh)
Spillover
Percent
Freezers 0 91,585 0%
Refrigerators 391 136,788 0.3%
Final Net-to-Gross
As summarized in Table 20, Cadmus determined the final net savings as gross savings less freeridership,
secondary market impacts, and induced replacement kWh, plus the spillover savings.
Table 20. 2011 and 2012 NTG Ratios
Scenario
Gross Per-
Unit
Savings
Freeridership and
Secondary Market
Impacts (kWh)
Induced
Replacement
kWh
Induced
Additional
Savings (Spillover)
Net
kWh NTG
Refrigerator 1,130 683 8 3 443 39%
Freezer 944 444 23 0 477 51%
After adjusting for freeridership and spillover, the net per-unit savings were 443 kWh for refrigerators
and 477 kWh for freezers. These values are somewhat lower than the evaluated per-unit savings for
2009-2010, primarily due to changes in evaluation methodology.
Summary of Impact Findings Table 21 summarizes the results of Cadmus’ impact analysis for 2011 and 2012. Table 22 summarizes
the results for just 2011, and Table 23 summarizes the results for just 2012.
29
Table 21. 2011 and 2012 Program Savings by Measure
Measure Evaluated
Participation
Reported
Gross
Savings
(kWh)
Evaluated
Gross
Savings
(kWh)
Gross
Realization
Rate
Evaluated
Net
Savings
(kWh)
Precision at
90%
Confidence
Net
Realization
Rate
Refrigerator
Recycling 1,746 2,013,240 1,973,701 98% 772,850 42% 38%
Freezer
Recycling 401 505,530 378,616 75% 191,349 37% 38%
Energy-
Savings Kit 1,007 143,561 57,430 40% 57,430 9% 40%
Total 3,154 2,662,331 2,409,747 91% 1,021,630 33% 38%
Table 22. 2011 Program Savings by Measure
Measure Evaluated
Participation
Reported
Gross
Savings
(kWh)
Evaluated
Gross
Savings
(kWh)
Gross
Realization
Rate
Evaluated
Net
Savings
(kWh)
Precision at
90%
Confidence
Net
Realization
Rate
Refrigerator
Recycling 831 953,670 939,373 99% 367,834 42% 39%
Freezer
Recycling 205 329,130 193,557 59% 97,822 37% 30%
Energy-
Savings Kit 480 77,193 27,400 35% 27,400 9% 35%
Total 1,516 1,359,993 1,160,331 85% 493,057 33% 36%
Table 23. 2012 Program Savings by Measure
Measure Evaluated
Participation
Reported
Gross
Savings
(kWh)
Evaluated
Gross
Savings
(kWh)
Gross
Realization
Rate
Evaluated
Net
Savings
(kWh)
Precision at
90%
Confidence
Net
Realization
Rate
Refrigerator
Recycling 915 1,059,570 1,034,328 98% 405,016 42% 38%
Freezer
Recycling 196 176,400 185,059 105% 93,527 37% 53%
Energy-
Savings Kit 527 66,368 30,030 45% 30,030 9% 45%
Total 1,638 1,302,338 1,249,417 96% 528,573 33% 41%
30
Process Evaluation
This section presents detailed staff interview findings and participant and nonparticipant survey results.
The areas of focus include:
Effectiveness of the delivery structure and implementation strategy;
Marketing approaches;
Customer satisfaction; and
Internal and external communications.
Methodology Cadmus conducted the following process evaluation research:
Document review, including:
Past evaluations,
Logic models, and
The program Website.
Utility staff and administrator interviews.
Participant and nonparticipant surveys.
Cadmus developed stakeholder interview guides to collect information about key topics from program
management staff. JACO Environmental implements the SYLR Program (the program administrator).
Cadmus conducted three stakeholder interviews: one each with the program managers at Rocky
Mountain Power and JACO (both of whom oversee appliance recycling programs in all five of
PacifiCorp’s service territory states), and one with the manager of JACO’s appliance recycling facility in
Salt Lake City, Utah. The following items were discussed during the interviews:
Program history;
Process flow;
Program design versus program implementation;
Changes in implementation and program marketing; and
Strengths and areas for improvement.
Cadmus conducted interviews by phone, and then contacted interviewees via e-mail for follow-up
questions and clarifications.
31
In addition, Cadmus conducted telephone surveys with participant and nonparticipant customers.
Cadmus designed the survey instruments to collect data about the following topics:
Customer information. Demographic information and household characteristics.
Program process. Details to inform the following performance indicators:
What are the participation motivations and barriers?
Are program incentives set correctly?
Is the program process effective?
How satisfied are customers with the program?
What are the program’s strengths and/or areas for improvement?
Program Implementation and Delivery Drawing on stakeholder interviews and participant and nonparticipant survey response data, this section
discusses the SYLR Program implementation and delivery. Cadmus updated logic models to reflect the
2011 and 2012 program processes based on information gathered through program staff interviews (see
Appendix E).
Program History and Program Management
The SYLR Program launched in Wyoming in 2009. According to program staff, participation started out
lower than expected possibly due to a weak national economy at the time the program launched. Since
that time, program participation in Wyoming has remained fairly low.
According to the program administrator, they worked with Rocky Mountain Power to establish 2011–
2012 program goals based on prior program performance and harvest rates.18 The contract between
Rocky Mountain Power and the program administrator included these projected participation levels, but
did not apply any penalty in the case of lower-than-expected participation. Beginning in 2013, Rocky
Mountain Power issued a new RFP and designed the contract so that the program administrator will
incur a financial penalty if the SYLR Program does not meet its participation goals. Additionally, Rocky
Mountain Power aligned the participation goals for 2013 more closely with recent program
performance.
Rocky Mountain Power receives a monthly invoice and report from the program administrator that
includes the number of pick ups, the reason(s) each unit was rejected, and the time lapse for mailing
incentive checks. In 2011, Rocky Mountain Power staff observed reporting inconsistencies, so in 2012
they began comparing monthly reports, invoices, and the dashboard to make sure they are consistent.
This improved monitoring appears to have resolved any inconsistencies, and this evaluation verified that
2011 and 2012 reported unit counts were consistent with the program administrator’s databases.
18
The harvest rate is the number of units recycled through the program in a given year divided by the total
number of residential customer accounts in the service territory.
32
Program Staffing and Training
JACO Environmental implemented the SYLR Program for Rocky Mountain Power in 2011-2012, and has
been the implementer since the program’s inception. The program staff also included a Rocky Mountain
Power program manager, PECI as a marketing contractor, and Appliance Distribution as a subcontractor
to JACO. Appliance Distribution’s role and the addition of PECI are discussed in more detail in the
Delivery Structure and Marketing sections below.
All program stakeholders reported that staffing levels were adequate, and the working relationships
among parties involved in program implementation were effective. Training occurred as needed for new
employees, but no major staff changes were reported for the 2011-2012 program, with the exception of
PECI’s new role as marketing contractor.
Delivery Structure and Processes
Rocky Mountain Power and the program administrator reported that they designed the program
similarly to other appliance recycling programs already operating in other states.
Rocky Mountain Power and the program administrator followed four main program delivery steps:
1. Marketing.
2. Sign-up/scheduling.
3. Appliance pick up.
4. Incentive payment.
They tailored the marketing messages to appeal to owners of older and secondary refrigerators,
although there were no minimum age requirements for appliances to participate.
Rocky Mountain Power’s Wyoming customers who were interested in disposing of an eligible appliance
could obtain information or sign up to participate through Rocky Mountain Power’s website or by calling
the program administrator toll-free. When participants signed up, the program administrator collected
details about how customers learned of the program, verified eligibility, and scheduled pick-up times.
The customer received a window of time for appliance pick up on a specific day, and was required to
have the appliance plugged in and running upon pick up.19 In 2011, the program administrator reduced
the pick-up window from four hours to two hours, in order to provide better customer service and to
match the two hour pick-up window offered by Sears.
The time between scheduling and pick up in 2011-2012 averaged 11 days, which was relatively
consistent with that in 2009-2010. The program administrator noted that pick-up wait times tended to
be shortest in urban areas, while customers in outlying areas experienced longer waits.
19
The program administrator estimated that 2-3% of pick ups were typically ineligible for participation because
the appliance was not working. Similarly, the program administrator reported that roughly 1-2% of units
scheduled for pick up were ineligible for participation due to their size.
33
At the scheduled time, the program administrator picked up and verified that the appliance was in
working condition, and collected data about the appliance age, size, configuration, and features. In late
2011, the pick-up crew began using hand-held computer devices to perform a variety of quality
assurance and quality control (QA/QC) functions and enable the pick-up process. The program
administrator took a photo of each unit and recorded its model number and unit number. Customers
signed the hand-held device when the pick up was complete.
During appliance pick up, the program administrator provided participants with an energy-savings kit
containing: two 13-watt CFLs, a refrigerator thermometer, energy-savings educational materials, and
information about Rocky Mountain Power’s other energy-efficiency program offerings. During 2011 and
2012, the receipt of the kit was not tracked in the program administrator’s database. However,
beginning in 2013, the program administrator added tracking of the kit for each customer. Under the
new tracking procedure, each participant will indicate on the hand-held device whether or not they
received a kit at the time of their pick-up.
The program administrator brought appliances to Appliance Distribution’s facility in Salt Lake City for
decommissioning and recycling. The program administrator then mailed incentive checks to participants.
Forms and Incentives
The SYLR Program requires minimal paperwork from participating customers. The sign-up process can
be completed by phone or online, and does not require the customer to fill out lengthy forms.
Customers who sign up by phone are asked for information, including their address, the location of the
unit, and a few screening questions. Customers who sign up online respond to these questions through
a brief, one-page online form.
Customers appreciated the simplicity of the sign-up process: 97% of surveyed customers reported being
very or somewhat satisfied with the program sign-up process. Customers were also satisfied with the
convenience of the pick-up process: 94% reported that it was very easy or somewhat easy to schedule a
convenient pick-up time.
Participating customers reported high satisfaction levels with the incentive amount (99% reported being
very satisfied or somewhat satisfied). Also, 90% said they would have participated in the program with a
lower incentive, and 82% said they would have participated without an incentive at all.
34
Marketing
Approach
The program marketing components underwent several changes during 2011 and 2012:
In 2011, the program administrator rebranded the kits and incentive checks with the Rocky
Mountain Power logo, instead of the JACO logo. Program staff reported that these changes,
while small, are important because they affect the customers’ perception of the program.
The program administrator updated the marketing materials it distributes with messages
coordinated with the Home Energy Savings Program, which provides customers with an
incentive for purchasing an energy-efficient appliance. According to program staff, it is natural
to promote recycling when advertising incentives for efficient new appliances.
Participants became aware of the program through a variety of ways. TV advertisements and bill inserts
were the most frequently mentioned channels (Figure 11), which is consistent with findings from prior
program evaluations. In a separate question, 22% of participants indicated that TV is the best way for
Rocky Mountain Power to communicate energy-efficiency opportunities, and 72% said that bill inserts
are the best method.
Figure 11. How Participants Learned About the Program
In 2012, the program administrator selected PECI (the program administrator for HES) to replace Runyon
Saltzman & Einhorn as the marketing subcontractor. In prior years, Runyon Saltzman & Einhorn had
developed marketing materials and worked with the program administrator to develop an overall
marketing strategy and approach. The Rocky Mountain Power and program administrator staff reported
that the program administrator had already contracted with PECI to work on collateral for similar
35
appliance recycling programs in Oregon; because Rocky Mountain Power and the program administrator
were pleased with the results of those efforts, they wanted to shift their business to PECI. While
administrator staff reported that the first six months working with PECI were a challenge because the
primary SYLR staff had left, the working relationship appears to have improved over time.
According to program and administrator staff, the program administrator closely examines past pick-up
trends and uses that information to develop marketing plans.
Another change in 2012 was that mass media, which used to be handled by Runyon Saltzman & Einhorn,
is now handled by a marketing contractor to PacifiCorp. Observations about the seasonality of the
program—with higher participation in the spring and fall—led program administrator staff to
recommend that TV ads and bill inserts align with seasonal trends. Program staff reported that TV is an
effective medium for the SYLR Program, which is the only energy-efficiency program with TV
advertisements.
Targeting
Program and administrator staff reported that they do not exclusively target specific customer segments
for the SYLR Program, just any customer who has a second refrigerator or freezer. Administrator staff
noted that PECI sent out a mailing to customers who participated in the Home Energy Savings Program
and received a rebate for a new appliance, as these customers may have an extra unit that could be
recycled. PECI also receives market data from retailers with the ZIP codes where most new units are
purchased.
Compared to customers in the general population, SYLR Program participants were more likely to be
homeowners of a single-family residence. The 2011-2012 demographic results are fairly consistent with
the 2009-2010 survey, with no statistically significant differences. Table 24 shows the average
demographics for participants surveyed.
Table 24. Participant Demographics
Characteristic Participants 2009-2010 Participants 2011-2012
Average Head of Household Age 57.4 59.6
Homeownership 95% 92%
Average Household Size (number of people) 2.6 2.9
Proportion Earning Less Than $50k 41% 29
The vast majority of 2011 and 2012 participants (92%) live in a single-family residence, with less than 8%
living in a multifamily or manufactured home. As the participant contact information was self-reported
(and could have been a landline or cell phone), the survey was less likely to experience bias for
respondents with landlines, as often seen in random-digit-dial surveys.
Comparison with Nonparticipants
When a nonparticipant population differs demographically from the participant population, it could be
due to misplaced or incomplete targeting of marketing efforts. Cadmus tested the similarities between
36
nonparticipants and participants to rule out marketing not reaching some eligible demographic groups.
For example, if a large portion of nonparticipants live in mobile homes but few participants live in
mobile homes, the mobile home market may have been overlooked.
One area in which participants and nonparticipants differed was the number of years they have lived in
their home. As shown in Figure 12, 75% of participants have lived in their home for more than five years,
compared to just 49% of nonparticipants.
Figure 12. Years Lived at Current Residence
Table 25 shows the results of t-tests for differences between participants and nonparticipants20 for a
series of relevant characteristics. Age, household size, and homeownership were significantly different
between participants and nonparticipants, with p-values of 0.00 for both age and household size and
0.01 for home ownership.
Home types did not differ significantly: Cadmus’ chi-square test for independence between the two
groups indicated they do not differ with 90% confidence (p-value=0.99).
20
All t-tests assumed unequal sample sizes and variances.
37
Table 25. T-Tests for Demographic Differences Between Participants and Nonparticipants
Characteristic Participants Nonparticipants Difference p-value
Average Head of Household Age 57.8 42.3 15.6 0.00
Single-Family Homeownership 95% 79% 18% 0.01
Average Household Size (number of people) 2.9 3.3 0.9 0.00
Proportion Earning Less Than $50k 29% 40% 18% 0.88
Customer Response
Satisfaction
Most participants experienced high overall satisfaction rates with the program: approximately 86% of
participants reported being somewhat or very satisfied with the program (see Figure 13). When asked
about program specifics, such as scheduling and incentive amounts, participants expressed similar
satisfaction levels. These high levels of customer satisfaction are common for utility appliance recycling
programs.
Figure 13. Overall Program Satisfaction
Participants’ willingness to recommend the program to others reflects their positive perceptions of the
program. Figure 14 shows that 88% of participants said they were somewhat or very likely to
recommend the program.
38
Figure 14. Likelihood of Recommending Program to Others
Ninety-five percent of customers reported having positive experiences with the program scheduling
process, with 5% expressing scheduling concerns (Figure 15).
Figure 15. Level of Difficulty with Scheduling
Program and administrator staff noted that the SYLR program rarely receives customer complaints. The
pick-up staffs’ use of hand-held computers allows them to communicate quickly with their call center,
enabling all involved parties to communication efficiently and knowledgeably with the customer if there
are any problems, such as with locating their home or picking up the unit.
39
Customers have gone out of their way to compliment program administrator staff and find ways to
encourage more recycling. For example, the SLYR facility manager in Salt Lake City was contacted by a
Salt Lake City resident who thinks highly of the SYLR Program and wants her Girl Scouts troop to
promote it. The troop would ask participating customers to donate their incentive to the troop; this
model has been implemented successfully in other states where the program administrator operates
appliance recycling programs.
Barriers
Overall, participants did not report notable complaints or issues during the surveys, and based on the
SYLR process evaluation, Cadmus noted no significant barriers. The program functions smoothly, likely
due to its longevity in the Wyoming market and the program administrator’s experience.
Quality Assurance/Quality Control The SYLR Program has multiple QA/QC checkpoints to facilitate quality service delivery and accurate
data tracking. The biggest change during 2011-2012 was the pick-up and warehouse crews using hand-
held devices. When a pick-up crew arrives at a customer’s home, they verify that the unit is in working
condition and fits the size criteria. If the unit passes those two tests and therefore meets the program
criteria, the crew enters the model number, unit number, size, and age into the hand-held device, and
takes a picture of the unit from a specific angle. If the unit does not meet the program criteria, they still
take a picture and record why the unit was not accepted. The pick-up crew also indicates if they caused
any damage during their visit. The information they uploaded to the hand-held device reaches the
program administrator database within five minutes and is available to all authorized program users.
When the unit arrives at the warehouse, the warehouse staff scans the unit and the appliance picture
taken by the pick-up staff appears. This serves as a verification that the correct unit arrived at the
warehouse to be processed for recycling.
The program administrator and Rocky Mountain Power track the time it takes for a customer to receive
their incentive check, and they track the time between when a customer sets an appointment and their
unit is picked up. On a quarterly basis, the program administrator verifies that the incentive processing
time is below the requirement (30 days); they reported that this requirement is consistently met.
According to Cadmus’ assessment of the program administrator’s tracking data, the average time
between the pickup date and the check date was 21 days. However approximately 4% of records
showed a wait longer than 30 days, and the maximum wait was 61 days. The program administrator also
examines why units were rejected and tracks the volume of rejections to make sure there is no increase.
In addition to the QA/QC performed by Rocky Mountain Power and the program administrator, an
independent contractor performs follow-up inspections at a random sample of participant homes. These
inspections ensure that pick-up procedures were followed and any issues are reported to Rocky
Mountain Power and the program administrator.
40
Benchmarking Cadmus benchmarked the 2011-2012 SYLR Program against four other similar utility appliance recycling
programs. Since the reports for three of those benchmarked utility programs are not publically available,
all four programs are cited in this report anonymously as Utility 1, Utility 2, Utility 3, and Utility 4. Utility
2 and Utility 3 offered their appliance recycling programs in 2011, while Utility 1 and Utility 4 offered
theirs in 2012.
Figure 16 shows a comparison of the three common methods utilities use to inform participants about
their respective appliance recycling programs. A higher proportion of participants learned of the SYLR
Program through TV compared to all the benchmarked counterparts. While the program awareness
resulting from print media was somewhat similar across utilities (except for Utility 1), more SYLR
participants (and Utility 1 participants) learned about the appliance recycling program through bill
inserts than participants in the other three utility programs.
Figure 16. Main Method for Hearing About Program
Figure 17 compares participants who stated being either somewhat or very satisfied with their
respective appliance recycling program, or who rated their program overall as a 7 or higher out of 10.
The SYLR Program had overall customer satisfaction levels slightly lower than the other four programs,
but the difference was not statistically significant.21
21
Using a two sample t-test, Cadmus confirmed that the difference in satisfaction levels between the SYLR
Program and the benchmarked utility programs was insignificant.
41
Figure 17. Overall Program Satisfaction
42
Cost-Effectiveness
In assessing cost-effectiveness, Cadmus analyzed program costs and benefits from five different
perspectives, using Cadmus’ DSM Portfolio Pro22 model. The benefit/cost ratios Cadmus used for these
tests are described in the California Standard Practice Manual for assessing DSM program cost-
effectiveness. The five tests perspectives were:
1. PacifiCorp Total Resource Cost (PTRC) Test: This test examined program benefits and costs from
Rocky Mountain Power and Rock Mountain Power customers’ perspectives, combined. On the
benefit side, it included avoided energy costs, capacity costs, and line losses, plus a 10% adder
to reflect non-quantified benefits. On the cost side, it included costs incurred by both the utility
and participants.
2. Total Resource Cost (TRC) Test: This test also examined program benefits and costs from Rocky
Mountain Power and Rocky Mountain Power customers’ perspectives, combined. On the benefit
side, it included avoided energy costs, capacity costs, and line losses. On the cost side, it
included costs incurred by both the utility and participants.
3. Utility Cost Test (UCT): This test examined program benefits and costs from Rocky Mountain
Power’s perspective only. The benefits included avoided energy, capacity costs, and line losses.
The costs included program administration, implementation, and incentive costs associated with
program funding.
4. Ratepayer Impact Measure (RIM) Test: All ratepayers (participants and nonparticipants) may
experience rate increases designed to recover lost revenues. The benefits included avoided
energy costs, capacity costs, and line losses. This test included all Rocky Mountain Power
program costs and lost revenues.
5. Participant Cost Test (PCT): From this perspective, program benefits included bill reductions and
incentives received. Costs included a measure’s incremental cost (compared to the baseline
measures), plus installation costs incurred by the customer.
Table 26 summarizes the five tests’ components.
22
DSM Portfolio Pro has been independently reviewed by various utilities, their consultants, and a number of
regulatory bodies, including the Iowa Utility Board, the Public Service Commission of New York, the Colorado
Public Utilities Commission, and the Nevada Public Utilities Commission.
43
Table 26. Benefits and Costs Included in Various Tests
Test Benefits Costs
PTRC
Present value of avoided energy and capacity
costs* with 10% adder for non-quantified
benefits
Program administrative and marketing costs
and costs incurred by participants**
TRC Present value of avoided energy and capacity
costs*
Program administrative and marketing costs
and costs incurred by participants**
UCT Present value of avoided energy and capacity
costs*
Program administrative, marketing, and
incentive costs
RIM Present value of avoided energy and capacity
costs*
Program administrative, marketing, and
incentive costs, plus the present value of lost
revenues
PCT Present value of bill savings and incentives
received Incremental measure and installation costs
* Includes avoided line losses.
** Incentive costs are typically excluded from the TRC as transfer payments. However, for appliance
recycling programs such as SYLR, participants do not incur costs. Therefore, the incentive cost is treated
differently from incentives in typical DSM programs. It is not excluded from the TRC, and instead it is
treated as an administrative cost since it is not offsetting any participant costs. This means that for SYLR the
UCT and the TRC costs are equal.
Table 27 provides cost analysis inputs, including the evaluated energy savings for each year, and the
discount rate, line loss, and program costs. Rocky Mountain Power provided all of these values, except
the evaluated energy savings and evaluated participation. Cadmus derived the discount rate and
inflation from Rocky Mountain Power’s 2011 Integrated Resource Plan.
44
Table 27. Selected Cost Analysis Inputs
Input Description 2011 2012 Total
Participation
Refrigerators 831 915 1,746
Freezers 205 196 401
Energy-Savings Kits 480 527 1,007
Measure Lives*
Refrigerators 8.5 6.0 N/A
Freezers 8.5 9.0 N/A
Energy-Savings Kits 8.5 5.0 N/A
Evaluated Gross Savings
(kWh/year)** 1,160,331 1,249,417 2,409,747
Discount Rate 7.17% 7.17% N/A
Line Loss 7.96% 9.51% N/A
Inflation Rate 1.8% 1.8% N/A
Total Program Costs $163,613 $180,949 $344,563
*Measure lives were provided by Rocky Mountain Power in the annual report data. 2011 used an average
measure life for all measures.
**Savings are at meter, while benefits account for line loss.
The SYLR program benefits included energy savings and their associated avoided costs. For the cost-
effectiveness analysis, Cadmus used energy savings derived from this study’s evaluated kWh. Measure
lives used (Table 27) were from annual report data provided by Rocky Mountain Power. Maintaining
consistency with the annual reports allows for more direct comparison of reported and evaluated
results.23 For all of analyses, Cadmus used the avoided costs associated with PacifiCorp’s 2011 IRP 35
Percent Load Factor Eastside Residential Whole Home Decrement.24
Table 28 presents the program cost-effectiveness analysis results, including the evaluated NTG25 for all
program measures for the evaluation period (2011-2012), but not accounting for non-energy benefits
(except those represented by the 10% conservation adder included in the PTRC test). The cost-
23
For future cost-effectiveness analysis, Cadmus suggests updating measure lives to align with the values used in
the most recent RTF measure workbooks.
24 The IRP decrements are detailed in Addendum, Chapter 2 of PacifiCorp’s 2011 Integrated Resource Plan:
http://www.pacificorp.com/content/dam/pacificorp/doc/Energy_Sources/Integrated_Resource_Plan/2011IRP
/2011IRP-Addendum_20110627.pdf 25
Evaluated NTG is 39.2% for refrigerators and 50.5% for freezers.
45
effectiveness analysis results indicate that the program was cost-effective from all perspectives, except
the RIM test.26 A benefit/cost ratio greater than 1.0 is considered cost-effective.
Table 28. Evaluated 2011 and 2012 Program Cost-Effectiveness Summary
Cost-Effectiveness Test Levelized
$/kWh Costs Benefits Net Benefits
Benefit/Cost
Ratio
PTRC 0.0516 $332,456 $571,040 $238,584 1.72
TRC 0.0516 $332,456 $519,127 $186,671 1.56
UCT 0.0516 $332,456 $519,127 $186,671 1.56
RIM
$907,341 $519,127 -$388,214 0.57
PCT
$0 $1,429,879 $1,429,879 N/A
Table 29 and Table 30 show the program’s evaluated cost-effectiveness for the 2011 and 2012 program
years, respectively.
Table 29. Evaluated 2011 Program Cost-Effectiveness Summary
Cost-Effectiveness Test Levelized
$/kWh Costs Benefits Net Benefits
Benefit/Cost
Ratio
PTRC 0.0463 $163,613 $312,013 $148,400 1.91
TRC 0.0463 $163,613 $283,649 $120,036 1.73
UCT 0.0463 $163,613 $283,649 $120,036 1.73
RIM
$460,588 $283,649 -$176,940 0.62
PCT
$0 $729,994 $729,994 N/A
Table 30. Evaluated 2012 Program Cost-Effectiveness Summary
Cost-Effectiveness Test Levelized
$/kWh Costs Benefits Net Benefits
Benefit/Cost
Ratio
PTRC 0.0582 $180,949 $277,599 $96,650 1.53
TRC 0.0582 $180,949 $252,363 $71,414 1.39
UCT 0.0582 $180,949 $252,363 $71,414 1.39
RIM
$478,785 $252,363 -$226,422 0.53
PCT
$0 $750,067 $750,067 N/A
26
The RIM test is included as a measure of the program’s impact on all ratepayers, and it is common for DSM
programs to have a RIM ratio of less than one.
46
Appendix A: Participant Demographics
Table A-1. Home Type Characteristics
Home Characteristics Percent of Respondents Precision at 90% Confidence
Home Type (n=355) - -
Single-family Home 92% 2.8%
Townhome or duplex 2% 1.5%
Mobile home or trailer 6% 2.5%
Own/Rent (n=355) - -
Own 95% 2.3%
Rent 5% 2.3%
Table A-2. Household Characteristics
Household Characteristics Mean Standard Deviation Precision at 90%
Confidence
Participant Age (n=216) 57.8 1.13 3%
Number of Residents (n=229) 2.9 0.28 16%
Figure A-1. Distributions of Household Sizes
47
Figure A-2. Distributions of Participant Ages
48
Appendix B: Precision Calculations
To determine the uncertainty level of savings results, Cadmus considered the effect of sampling error on
all estimates presented in the report. Sampling error refers to uncertainty introduced by the use of
sampled data to infer characteristics of the overall population. These data include survey results, meter
data, and data from secondary sources. Cadmus used sampled data to estimate the parameters of per-
unit savings calculations (such as installation rates) and to estimate the consumption of specific
equipment types (such as in billing analysis).
Sampling error has been reflected in estimated confidence intervals. Unless otherwise noted, Cadmus
estimated intervals at 90% confidence, indicating a 90% confidence that the true population value fell
within the given interval. Cadmus calculated confidence intervals for means, proportions, regression
estimates, and any calculated values using sample estimates as an input. Cadmus calculated all
confidence intervals using the following standard formula for estimating uncertainty for proportions and
means:
√
Where:
1.645 = the z-score for a 90% confidence interval, and
s2 = the sample variance.
In some cases, the uncertainty of estimates came from multiple sources. For example, for summed
estimates, such as those for total program savings, Cadmus calculated the root of the sum of the
squared standard errors to estimate the confidence interval:27
( ) √( ) (
)
In some cases, Cadmus multiplied estimates. For instance, net savings calculations involved combining
gross estimates with an in-service rate and/or NTG ratio estimated from participant surveys. For these
results, Cadmus calculated combined standard errors for the final estimates. In cases where the
relationship was multiplicative, Cadmus used the following formula:28
√ ( ) (
) (
)( )
27
This approach to aggregating errors follows methods outlined in: Schiller, Steven, et al. “National Action Plan
for Energy Efficiency.” Appendix D. 2007. Available online: www.epa.gov/eeactionplan.
28 Cadmus derived this formula from: Goodman, Leo. “The Variance of the Product of K Random Variables.”
Journal of the American Statistical Association (1962).
49
To ensure transparency of the error aggregation process, Cadmus reported precision for both individual
and combined estimates, where relevant.
50
Appendix C: Participant Survey Instrument
Introduction
Hello, my name is [INSERT FIRST NAME] from [INSERT COMPANY NAME]. I'm calling on behalf of
[UTILITY] to ask you some survey questions about the See ya later, refrigerator recycling program.
[IF RESPONDENT DOES NOT EXPRESS RESERVATIONS SKIP TO S1, IF RESPONDENT EXPRESSES
RESERVATIONS AT THIS POINT, PERSUADE:] I’m not selling anything, we are interested in your opinions
to help improve our programs, and to understand how to assist customers in saving money on their
utility bills. Your responses will remain confidential.
Screening Questions
S1. According to our records, someone in your household signed up to recycle an appliance through
the [UTILITY] See ya later, refrigerator Program. Are you that person?
1. Yes
2. No
-98. DON’T KNOW [TERMINATE]
-99. REFUSED [TERMINATE]
S2. [ASK IF S1=2] Is that person available to speak with?
1. Yes [START FROM S1 WITH NEW RESPONDENT]
2. No [TERMINATE, ARRANGE CALLBACK IF POSSIBLE]
S3. Were you present during the appliance pick-up?
1. Yes
2. No
-98. DON’T KNOW [TERMINATE]
-99. REFUSED [TERMINATE]
S4. [ASK IF S3=2] Is that person available to speak with?
1. Yes [START FROM S1 WITH NEW RESPONDENT]
2. No [TERMINATE, ARRANGE CALLBACK IF POSSIBLE]
51
Measure Verification
A1. [ASK IF QUANTITY_REF>0] Program records indicate that you received an incentive for having
[INSERT QUANTITY_REF] refrigerator(s) recycled by the program around [INSERT DATE OF PICK-
UP]. Is this correct?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED [TERMINATE]
A2. [ASK IF A1=2 OR 98] How many refrigerators did you recycle through the [INSERT UTILITY]
program?
1. [RECORD] [IF A2=0, RECODE QUANTITY_REF=0]
-98. REFUSED
-99. DON’T KNOW
A3. [ASK IF QUANTITY_FRZ>0] Program records indicate that you received an incentive for having
[INSERT QUANTITY_FRZ] freezer(s) recycled by the program around [INSERT DATE OF PICKUP].
Is this correct?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED [TERMINATE]
A4. [ASK IF A3=2 OR 98] How many freezers did you recycle through the [INSERT UTILITY] program?
1. [RECORD] [IF A4=0, RECODE QUANTITY_FRZ=0]
-98. REFUSED
-99. DON’T KNOW
52
Awareness
B1. How did you learn about the [INSERT UTILITY] appliance recycling program? [DO NOT READ
LIST. RECORD UP TO THREE RESPONSES]
1. Newspaper/Magazine/Print Media
2. Bill Inserts
3. Rocky Mountain Power/Pacific Power Website
4. Other Website
5. Internet Advertising/Online Ad
6. Family/Friends/Word-of-Mouth
7. Rocky Mountain Power/Pacific Power Representative
8. Radio
9. TV
10. Billboard/Outdoor Ad
11. Retailer/Store
12. Sporting Event
13. Home Shows/Trade Shows
14. Appliance Recycling Contractor
15. Other [RECORD VERBATUM]
-98. DON’T KNOW
-99. REFUSED
53
B2. What are the best ways for [INSERT UTILITY] to inform you about energy-efficiency offerings like
the appliance recycling program? [DO NOT READ. PROMPT IF NECESSARY. RECORD UP TO
THREE RESPONSES]
1. Newspaper/Magazine/Print Media
2. Bill Inserts
3. Rocky Mountain Power/Pacific Power Website
4. Other Website
5. Internet Advertising/Online Ad
6. Family/Friends/Word-of-Mouth
7. Rocky Mountain Power/Pacific Power Representative
8. Radio
9. TV
10. Billboard/Outdoor Ad
11. Retailer/Store
12. Sporting Event
13. Home Shows/Trade Shows
14. Appliance Recycling Contractor
15. E-mail from Rocky Mountain Power/Pacific Power
16. Other [RECORD VERBATUM]
-98. DON’T KNOW
-99. REFUSED
B3. How would you rate your current understanding of energy-efficiency? Would you say you…
[READ LIST. RECORD FIRST RESPONSE]
1. Have no knowledge of energy-efficiency
2. Are somewhat knowledgeable about energy-efficiency
3. Are very knowledgeable about energy-efficiency
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
Appliance Description
H1. [READ IN APPLIANCE CHARACTERISTICS INCLUDED IN PARTICIPANT SAMPLE [TYPE = “REF” OR
“FRZ”] [TYPE DETAIL = CONFIGURATION (TOP FREEZER, CHEST, ETC.)] [ASK IF
QUANTITY_REF>0, AND QUANITITY_FRZ>0] The next few questions focus on just one appliance.
Since you recycled more than one appliance, please answer these questions about the [INSERT
TYPE] [INSERT TYPE DETAIL].
54
H2. During the time just before you decided to get rid of the appliance, was it being used as your
main appliance, or had it been a secondary or spare? [IF RESPONDENT RECYCLED MORE THAN
ONE APPLIANCE, DIRECT THEM TO RESPOND REGARDING THE FIRST APPLIANCE THEY
RECYCLED.] [IF RESPONDENT IS UNSURE: ] A main refrigerator is typically in the kitchen, and a
spare refrigerator is usually in the garage or basement and might not be in use all the time.
1. Main
2. Secondary or Spare
-98. DON’T KNOW
-99. REFUSED
H3. [ASK IF H2=2] How long were you using it as a spare before you recycled it through the
program?
1. [RECORD VALUE IN MONTHS. IF RESPONDENT ANSWERS IN YEARS, MULTIPLY BY 12 AND
RECORD VALUE IN MONTHS]
-98. DON’T KNOW
-99. REFUSED
H4. During the year before you recycled it, was the appliance plugged in and running… [READ LIST]
1. All the time
2. For special occasions only
3. During certain months of the year only
4. Never plugged in or running
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
H5. [ASK IF H4=2 OR H4=3] If you were to add up the total time it was running as a spare in the last
year before you recycled it, how many months would that be? [IF RESPONDENT IS UNSURE:]
Your best estimate is okay.
1. [RECORD MONTHS 1-12]
-98. DON’T KNOW
-99. REFUSED
55
H6. Where was the [INSERT APPLIANCE TYPE] located during most of the year before you recycled
it?
1. Kitchen
2. Garage
3. Porch/Patio
4. Basement
5. Yard/Outside
6. Other [RECORD]
-98. DON’T KNOW
-99. REFUSED
H7. Was the location heated?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
H8. Was the location air-conditioned?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
H9. Would you say the [INSERT APPLIANCE TYPE] you recycled… [READ LIST. RECORD FIRST
RESPONSE]
1. Worked and was in good physical condition
2. Worked but needed minor repairs
3. Worked but had some major problems
4. Didn’t work
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
H10. Did you get a new [INSERT APPLIANCE TYPE] to replace the one you recycled?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
56
H11. [ASK IF H10=2] Do you plan to get a replacement appliance in the near future?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
H12. [ASK IF H10=1] Would you have purchased the new [INSERT APPLIANCE TYPE] without the $30
incentive you received for recycling the old one?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
H13. [IF H10=1 AND H12=2] Just to confirm: you would not have replaced your old [INSERT
APPLIANCE TYPE] without the [INSERT UTILITY] incentive for recycling, is that correct?
1. Correct
2. Incorrect
-98. DON’T KNOW
-99. REFUSED
H14. Is the [INSERT APPLIANCE TYPE] you replaced it with an ENERGY STAR or high-efficiency model?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
H15. Had you already considered getting rid of this [INSERT APPLIANCE TYPE] before hearing about
[INSERT UTILITY]’s appliance recycling program?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
H16. Without the existence of [INSERT UTILITY] appliance recycling program, what would you most
likely have done with your old [INSERT APPLIANCE TYPE]? Would you have… [READ LIST]
1. Gotten rid of it
2. Kept it
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
57
H17. [ASK H16=1] Would you have gotten rid of it within a year of when the program took it, or more
than a year later?
1. Within a year of when the program took it
2. More than a year later
-98. DON’T KNOW
-99. REFUSED
H18. [ASK H16=2] If you had kept it, would you have used it full time, stored it unplugged, or used it
occasionally?
1. Used full time
2. Stored it unplugged
3. Used it occasionally
-98. DON’T KNOW
-99. REFUSED
H19. If you had kept the [REFRIGERATOR, FREEZER], would you have kept it in the same location you
mentioned earlier? That is would it have been located in [READ IN ANSWER FROM H6]?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
Consideration of Alternatives and Freeridership
Now I have a few questions about the different options you might have considered before recycling your
appliance. For each statement, please tell me whether you gave serious consideration to the associated
action.
F1. How would you have disposed of the unit if the program had not been available? [ALLOW ONLY
ONE ANSWER] [PROGRAMMER: LIST SHOULD BE READ IN RANDOM ORDER]
1. Selling it to a private party (friend, family member, craigslist) [SKIP TO NEXT SECTION]
2. Selling it to a used appliance dealer
3. Giving it away for free private party (friend, family, craigslist) [SKIP TO NEXT SECTION]
4. Leave it on curb with free sign[SKIP TO NEXT SECTION]
5. Give it away to charity organization
6. Having it removed by the dealer you got your new or replacement [INSERT APPLIANCE
TYPE] from [SKIP TO NEXT SECTION]
7. Taking it to a dump or recycling center yourself [or friend or family member]
8. Hiring someone to take it to a dump or recycling center [SKIP TO NEXT SECTION]
58
[READ IN UNIT AGE INCLUDED IN PARTICIPANT SAMPLE] [IF F1=2 AND AGE >15 OR F1=5 OR
F1=7, THEN READ F2 ALONG WITH THE CORRESPONDING DETAILS:]
[READ IF F1=2 AND AGE >15] Used appliance dealers typically only buy units that are less than
15 years old and are in very good condition.
[READ IF F1=5] Many charity organizations (such as Goodwill, Salvation Army) do not accept
large appliances.
[READ IF F1=7] Appliances are heavy and require a truck, trailer, or large vehicle to relocate.
Local waste transfer stations charge a fee to dispose of large appliances.
F2. [If F1=2 and AGE >15 OR F1=5 OR F1=7] If you had not been able to [READ IN ANSWER FROM
F1], how would you have disposed of it? [ALLOW ONLY ONE ANSWER BUT DO NOT ALLOW
PREVIOUS ANSWER]
1. Selling it to a private party (friend, family member, craigslist)
2. Selling it to a used appliance dealer
3. Giving it away for free private party (friend, family member, craigslist)
4. Leave it on curb with free sign
5. Give it away to charity organization
6. Having it removed by the dealer you got your new or replacement [INSERT APPLIANCE
TYPE] from
7. Taking it to a dump or recycling center yourself (or friend or family member)
8. Hiring someone to take it to a dump or recycling center
9. Kept it
F3. Would you have participated in the program without the incentive check?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
CFL Installation
E1. Was a free kit containing two CFL light bulbs and energy information given to you at the time of
pick-up?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
59
E2. [IF E1≠1, SKIP TO SP1] How would you rate the information found in this kit? Would you say it
was… [READ LIST]
1. Very helpful
2. Somewhat helpful
3. Not very helpful
4. Not at all helpful
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
E3. [ASK IF E2≠98 OR 99] Why did you assign this rating? [DO NOT READ LIST. RECORD MULTIPLE]
1. Information was too general
2. Already aware of information
3. Information did not apply
4. Used the suggestions provided in information
5. Written well
6. Passed information along to others
7. Other [RECORD VERBATIM]
-98. DON’T KNOW
-99. REFUSED
E4. How many of the CFLs that came in the kit did you install?
1. None
2. One
3. Two
-98. DON’T KNOW
-99. REFUSED
E5a. [ASK IF E4=2 OR 3] What type of bulbs were in the socket before you installed the CFLs? [READ
LIST IF NECESSARY]
1. Incandescent (or “traditional” bulbs)
2. Another CFL
3. LED
4. Florescent
5. Halogen
6. Empty
-98. DON’T KNOW
-99. REFUSED
60
E5b. [ASK IF E4=1 OR 2] Why didn’t you install [IF E4=1, “them?” IF E4=2, “the other CFL?” [DO NOT
READ LIST. RECORD MULTIPLE]
1. Did not fit fixtures
2. Intend to install later
3. Do not like style
4. Do not like quality
5. Defective product
6. Other [RECORD VERBATIM]
-98. DON’T KNOW
-99. REFUSED
E6. [ASK IF E4=2 OR 3] Where did you install the CFL(s)? [DO NOT READ. RECORD UP TO TWO]
1. Bedroom
2. Bedroom (unoccupied)
3. Basement
4. Bathroom
5. Closet
6. Dining Room
7. Foyer
8. Garage
9. Hallway
10. Kitchen
11. Office/Den
12. Living Space
13. Storage
14. Outdoor
15. Utility
16. Other [RECORD VERBATIM]
-98. DON’T KNOW
-99. REFUSED
E7. Did you install the refrigerator thermometer included in your energy-savings kit?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
61
E8. [ASK IF E7=1, ELSE SKIP TO E10] After installing the thermometer, did you change the
temperature setting on your refrigerator?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
E9. [ASK IF E7=1, ELSE SKIP TO E10] Did you increase or decrease the temperature setting in your
refrigerator?
1. Increase
2. Decrease
-98. DON’T KNOW
-99. REFUSED
E10. Do you remember receiving a booklet with information about how to save energy?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
E11. [ASK IF E10=1, ELSE SKIP TO SP1] Have you followed any of the advice mentioned in the
booklet? If so, what advice?
1. Yes, [RECORD VERBATIM]
2. No
-98. DON’T KNOW
-99. REFUSED
Spillover and Market Impact
SP1. Since participating in the See ya later, refrigerator Program, have you participated in any other
incentive programs offered by [INSERT UTILITY]?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
62
SP2. [ASK IF SP1=1] Which programs did you participate in?
1. [RECORD VERBATIM]
-98. DON’T KNOW
-99. REFUSED
SP3. [ASK IF SP1=1] How influential was the recycling program in your decision to participate in other
[INSERT UTILITY] energy-efficiency programs? Would you say it was… [READ LIST]
1. Very influential
2. Somewhat influential
3. Not very influential
4. Not at all influential
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
SP4. [ASK IF SP1=2] Based on your experience in recycling your appliance, how likely are you to
participate in another utility energy-efficiency program? Would you say you are… [READ LIST]
1. Much more likely
2. Somewhat more likely
3. No more or less likely
4. Less likely to participate in another program
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
SP5. Besides recycling your old [APPLIANCE TYPE], have you made other energy-efficiency
improvements or purchases on your own since participating in the appliance recycling program?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
SP6. [ASK IF SP5=1] What did you install or purchase? [DO NOT READ. RECORD MULTIPLE]
1. High-efficiency dishwasher
2. High-efficiency washing machine
4. High-efficiency refrigerator
5. High-efficiency water heater
6. CFLs
7. Other [RECORD VERBATIM]
-98. DON’T KNOW
-99. REFUSED
63
SP6a. [ASK IF SP5=1] Did you receive an incentive for any of those items?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
SP7. [ASK IF SP5=1] How much did your experience with the See ya later, refrigerator Program
influence your decision to install other high-efficiency equipment on your own? Would you say
it was… [READ LIST]
1. Very influential
2. Somewhat influential
3. Not very influential
4. Not at all influential
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
Program Satisfaction
G1. How satisfied are you with the [INSERT UTILITY] See ya later, refrigerator Program overall?
Would you say you are… [READ LIST]
1. Very satisfied
2. Somewhat satisfied
3. Somewhat dissatisfied
4. Very dissatisfied
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
G2. [ASK IF G1= 2, 3, or 4] Why do you give it that rating? [DO NOT READ. RECORD MULTIPLE]
1. Incentive was too small
2. Contractor never called me back
3. Contractor never showed up/showed up late
4. Contractor was unreliable/unprofessional
5. Difficult to get an appointment time that was convenient for me
6. Wanted to use a different (non-program) contractor
7. Other [RECORD VERBATIM]
-98. DON’T KNOW
-99. REFUSED
64
G3. How satisfied are you with the sign-up process for the program? Would you say you are… [READ
LIST]
1. Very satisfied
2. Somewhat satisfied
3. Somewhat dissatisfied
4. Very dissatisfied
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
G4. How easy was it to schedule a convenient pick-up time? Would you say it was… [READ LIST]
1. Very easy
2. Somewhat easy
3. Somewhat difficult
4. Very difficult
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
G5. How satisfied are you with the appliance pick-up portion of the program? Would you say you
are… [READ LIST]
1. Very satisfied
2. Somewhat satisfied
3. Somewhat dissatisfied
4. Very dissatisfied
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
G6. Did the crew that picked up your appliance check to see if it was working before they took it
away?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
65
G7. How satisfied are you with how quickly you received your incentive? Would you say you are…
[READ LIST]
1. Very satisfied
2. Somewhat satisfied
3. Somewhat dissatisfied
4. Very dissatisfied
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
G8. How satisfied are you with the amount of the incentive? Would you say you are… [READ LIST]
1. Very satisfied
2. Somewhat satisfied
3. Somewhat dissatisfied
4. Very dissatisfied
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
G9. Would you have participated in the program if the amount of the incentive had been less?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
G10. How likely are you to recommend the [INSERT UTILITY] See ya later, refrigerator Program to
friends and family members? Would you say you are… [READ LIST]
1. Very likely
2. Somewhat likely
3. Not very likely
4. Not at all likely
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
G11. Is there anything you would suggest to improve the [INSERT UTILITY] See ya later, refrigerator
Program?
1. [RECORD VERBATIM]
-98. DON’T KNOW
-99. REFUSED
66
Demographics
I have just a few more questions about your household. Again, all your answers will be strictly
confidential.
D1. Which of the following best describes your house? [READ LIST]:
1. Single-family home
2. Townhouse or duplex
3. Mobile home or trailer
4. Apartment building with four or more units
5. Other [RECORD]
-98. [DO NOT READ] REFUSED
-99. [DO NOT READ] DON’T KNOW
D1A. About how old is your home?
1. [RECORD # YEARS OLD – IF ACTUAL YEAR IS PROVIDED, TRANSLATE TO YEARS RELATIVE TO
2011]
-98. DON’T KNOW
-99. REFUSED
D2. Do you rent or own your home?
1. Own
2. Rent
3. Other [RECORD]
-98. REFUSED
-99. DON’T KNOW
D3. How long have you lived at that location?
1. Less than one year
2. Two to five years
3. More than five years
-98. REFUSED
-99. DON’T KNOW
D4. Including yourself and any children, how many people currently live in your home?
1. [RECORD]
-98. REFUSED
-99. DON’T KNOW
67
D4a. [ASK IF D4 >1] Are any of the people living in your home under the age of 18?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
D5. Can you please tell me in what year you were born?
1. [RECORD]
-98. REFUSED
-99. DON’T KNOW
D6. In 2010, was your pre-tax household income above or below $50,000?
1. Below $50,000
2. Above $50,000
3. Exactly $50,000
-98. DON’T KNOW [SKIP TO C1]
-99. REFUSED [SKIP TO C1]
D7. [ASK IF D6=1] Which of the following categories best represents your household income in
2010? Please stop me when I read your category:
1. Under $10,000
2. $10,000 to under $20,000
3. $20,000 to under $30,000
4. $30,000 to under $40,000
5. $40,000 to under $50,000
-98. REFUSED
-99. DON’T KNOW
D8. [ASK IF D6=2] Which of the following categories best represents your household income in
2010? Please stop me when I read your category:
1. $50,000 to under $60,000
2. $60,000 to under $75,000
3. $75,000 to under $100,000
4. $100,000 to under $150,000
5. $150,000 to under $200,000
6. $200,000 or more
-98. REFUSED
-99. DON’T KNOW
68
D9. What is the primary heating source for your home? [READ LIST IF NEEDED]
1. Forced air natural gas furnace
2. Forced air propane furnace
3. Air source heat pump
4. Ground source heat pump
5. Electric baseboard heat
6. Gas fired boiler/radiant heat
7. Oil fired boiler/radiant heat
8. Passive solar
9. Pellet stove
10. Wood stove
11. Other [RECORD]
-98. DON’T KNOW
-99. REFUSED
D10. [IF D9=1-11] How old is the primary heating system? [RECORD RESPONSE IN YEARS]
1. [RECORD 1-100]
-98. Don’t Know
-99. Refused
D11. What type of air conditioning system, if any, do you use in your home? [READ LIST IF
NECESSARY, INDICATE ALL THAT APPLY]
1. Central Air Conditioner
2. Room Air Conditioner
3. Evaporative Cooler
4. Air Source Heat Pump
5. Ground Source Heat Pump
6. Whole House Fan
7. No Cooling System Before
8. Other [RECORD]
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
D12. [ASK IF D11≠7] How old is your primary cooling system?
1. [RECORD RESPONSE IN YEARS]
-98. DON’T KNOW
-99. REFUSED
69
Conclusion
C1. How satisfied are you with the service that [INSERT UTILITY] provides overall? Would you say
you are… [READ LIST]
1. Very satisfied
2. Somewhat satisfied
3. Somewhat dissatisfied
4. Very dissatisfied
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
C2. Do you have any additional feedback or comments?
1. Yes [RECORD VERBATUM]
2. No
-98. REFUSED
-99. DON’T KNOW
That concludes the survey. Thank you very much for your time and feedback.
70
Appendix D: Nonparticipant Survey Instrument
Introduction
Hello, my name is [INSERT FIRST NAME] from [INSERT COMPANY NAME]. I'm calling on behalf of
[UTILITY] to ask you some survey questions that will help [UTILITY] improve their energy-efficiency
programs.
[IF RESPONDENT DOES NOT EXPRESS RESERVATIONS, SKIP TO S1. IF RESPONDENT EXPRESSES
RESERVATIONS AT THIS POINT, USE THE FOLLOWING SCRIPT TO PERSUADE]: I’m not selling anything,
we are interested in your opinions to help improve our programs, and to understand how to assist
customers in saving money on their utility bills. Your responses will remain confidential.
S. Screening Questions
S1. Did you discard a refrigerator or freezer in 2011 or 2012? By discard, we mean getting rid of it
either by selling it, giving it away, having someone pick it up, or taking it to the dump or a
recycling center.
1. Yes, refrigerator(s)
2. Yes, freezer(s)
3. Yes, both appliances
4. No [TERMINATE]
-98. DON’T KNOW [TERMINATE]
-99. REFUSED [TERMINATE]
S2. Did the appliance(s) work? [IF RESPONDENT IS UNSURE, SAY:] Even if it didn’t get cold, did the
appliance turn on when it was plugged in?
1. Yes
2. No [TERMINATE]
-98. DON’T KNOW [TERMINATE]
-99. REFUSED [TERMINATE]
S3. Did you have the appliance(s) picked up through [INSERT UTILITY]’s See ya later, refrigerator
Program?
1. Yes [TERMINATE]
2. No
-98. DON’T KNOW [TERMINATE]
-99. REFUSED [TERMINATE]
71
S4. [INSERT UTILITY] offers an incentive to pick up and recycle old working refrigerators and
freezers. If you had participated, a contractor would have picked the appliance up at your home
and provided you with $30 later in the mail. Are you sure your appliance wasn’t picked up by the
utility program?
1. Yes, I’m sure it wasn’t picked up by the program or I received no incentive
2. No, I did get the incentive check [TERMINATE]
-98. I still don’t know for sure [TERMINATE]
-99. REFUSED [TERMINATE]
[TERMINATION SCRIPT: “Those are all the questions we have for you. Thank you very much for your
time.”]
N. Nonparticipant Awareness
N1. Were you aware of the [INSERT UTILITY] appliance recycling program prior to getting rid of your
appliance?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
72
N2. [ASK IF N1=1, ELSE SKIP TO N4] How did you learn about the [INSERT UTILITY] appliance
recycling program? [DO NOT READ LIST. RECORD UP TO THREE RESPONSES]
1. Newspaper/Magazine/Print Media
2. Bill Inserts
3. Rocky Mountain Power/Pacific Power Website
4. Other Website
5. Internet Advertising/Online Ad
6. Family/Friends/Word-of-Mouth
7. Rocky Mountain Power/Pacific Power Representative
8. Radio
9. TV
10. Billboard/Outdoor Ad
11. Retailer/Store
12. Sporting Event
13. Home Shows/Trade Shows
14. Appliance Recycling Contractor
15. Other [RECORD VERBATIM]
16. Postcard
17. Direct Mail
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
N3. What made you decide not to have your appliance picked up through the [INSERT UTILITY]
appliance recycling program? [DO NOT READ. RECORD UP TO THREE RESPONSES.]
1. Unit didn’t qualify
2. Did not know how to sign up
3. Was not able to schedule a convenient pick-up time
4. Too much hassle
5. Other [RECORD VERBATUM]
-98. DON’T KNOW
-99. REFUSED
73
N4. What are the best ways for [INSERT UTILITY] to inform you about energy-efficiency offerings like
the appliance recycling program? [DO NOT READ. PROMPT IF NECESSARY. RECORD UP TO
THREE RESPONSES]
1. Newspaper/Magazine/Print Media
2. Bill Inserts
3. Rocky Mountain Power/Pacific Power Website
4. Other Website
5. Internet Advertising/Online Ad
6. Family/Friends/Word-of-Mouth
7. Rocky Mountain Power/Pacific Power Representative
8. Radio
9. TV
10. Billboard/Outdoor Ad
11. Retailer/Store
12. Sporting Event
13. Home Shows/Trade Shows
14. Appliance Recycling Contractor
15. E-mail from Rocky Mountain Power/Pacific Power
16. Other [RECORD VERBATUM]
17. Postcard
18. Direct Mail
-98. DON’T KNOW
-99. REFUSED
N5. How would you rate your current understanding of energy efficiency? Would you say you…
[READ LIST. RECORD FIRST RESPONSE]
1. Have no knowledge of energy efficiency
2. Are somewhat knowledgeable about energy efficiency
3. Are very knowledgeable about energy efficiency
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
A. Appliance Characteristics
[IF MORE THAN ONE APPLIANCE WAS DISCARDED, SAY:] For the rest of the survey, I’d like you to focus
on only one of the appliances you got rid of. Please answer these questions about the appliance you
discarded most recently.
[IF ONLY ONE APPLIANCE WAS DISCARDED, SAY:] For the rest of the survey, we would like to ask you
about the appliance you discarded.
74
A1. At the time you discarded it, approximately how old was the appliance?
1. [RECORD AGE IN YEARS]
-98. Don’t know
-99. Refused
A2. Before you made the decision to get rid of the appliance, in what room was the appliance
used/located?
1. Kitchen
2. Garage
3. Porch/patio
4. Basement
5. Other [RECORD]
-98. DON’T KNOW
-99. REFUSED
A3. Would you say the appliance …? [READ LIST, RECORD ONLY ONE RESPONSE]
1. Worked and was in good physical condition
2. Worked but needed minor repairs
3. Worked but had some major problems
-98. [DO NOT READ] DON’T KNOW
-99. [DO NOT READ] REFUSED
A4. Did you get a new appliance to replace the one you got rid of?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
A5. [ASK IF A4=1, ELSE SKIP TO A6] Is the appliance you replaced it with an ENERGY STAR or high-
efficiency model?
1. Yes
2. No
-98. DON’T KNOW
-99. REFUSED
75
A6. How did you get rid of your old appliance? [IF NEEDED, PROMPT:] For example, did you sell it or
give it away?
1. Sold it to a private party, either by running an ad or to selling to someone I know
2. Sold it to a used appliance dealer
3. Gave it away to a private party, such as a friend or neighbor
4. Gave it away to a charity organization, such as Goodwill Industries or a church
5. Had it removed by the dealer I got my new or replacement appliance from
6. Hauled it to the dump myself
7. Hauled to a recycling center myself
8. Had someone else pick it up for junking or dumping
9. Kept it
10. Some other way [RECORD VERBATIM]
-98. DON'T KNOW
-99. REFUSED
Demographics
D1. Which of the following best describes your house? [READ LIST]:
1. Single-family home
2. Townhouse or duplex
3. Mobile home or trailer
4. Apartment building with four or more units
5. Other [RECORD]
-98. [DO NOT READ] REFUSED
-99. [DO NOT READ] DON’T KNOW
D2. Do you rent or own your home?
1. Own
2. Rent
3. Other [RECORD]
-98. REFUSED
-99. DON’T KNOW
D3. How long have you lived at that location?
1. Less than one year
2. Two to five years
3. More than five years
-98. REFUSED
-99. DON’T KNOW
76
D4. Including yourself and any children, how many people currently live in your home?
1. [RECORD]
-98. REFUSED
-99. DON’T KNOW
D5. Can you please tell me in what year you were born?
1. [RECORD]
-98. REFUSED
-99. DON’T KNOW
D6. In 2010, was your pre-tax household income above or below $50,000?
1. Below $50,000
2. Above $50,000
3. Exactly $50,000
-98. DON’T KNOW [SKIP TO CLOSING STATEMENT]
-99. REFUSED [SKIP TO CLOSING STATEMENT]
D7. [ASK IF D6=1] Which of the following categories best represents your household income in
2010? Please stop me when I read your category:
1. Under $10,000
2. $10,000 to under $20,000
3. $20,000 to under $30,000
4. $30,000 to under $40,000
5. $40,000 to under $50,000
-98. REFUSED
-99. DON’T KNOW
D8. [ASK IF D6=2] Which of the following categories best represents your household income in
2010? Please stop me when I read your category:
1. $50,000 to under $60,000
2. $60,000 to under $75,000
3. $75,000 to under $100,000
4. $100,000 to under $150,000
5. $150,000 to under $200,000
6. $200,000 or more
-98. REFUSED
-99. DON’T KNOW
CLOSING SCRIPT: Those are all the questions I have. [INSERT UTILITY] appreciates your input. Thank you
for your time.
77
Appendix E: Logic Model
Inputs: Funds, experienced staff, market knowledge, network of partners, appliance recycling infrastructure, program design
Rocky Mountain Power See ya later, refrigerator® Program Logic ModelO
utpu
tsA
ctiv
ities
Long
-Ter
m O
utco
mes
(4
-7 y
ears
)In
term
edia
te O
utco
mes
(2
-3 y
ears
)S
hort-
Term
Out
com
es
(0-1
yea
r)
Marketing & Outreach
Legend
Output or Outcome Activity
Process Flow
Quality ControlMeasurement & Verification
Marketing materials disseminated to
customers with working refrigerators and freezers
and customers who recently purchased a
new unit
Customers enroll in program
EM&V team conducts
evaluation
3 4
Inspections/reviews of program operations performed
8
Effectiveness of program operations confirmed
9
Rocky Mountain Power gains experience designing and marketing
programs to this market segment; informs next program planning cycle
Long term energy savings goals are
achieved
Optimum program
performance maintained
15
Persistent energy and demand
savings
23
19 20
Program savings verified
14
18
Environmental benefits achieved
from sound recycling practices
1
6
Immediate kWh & KW savings
Inefficient appliances permanently removed
from grid, energy saving kits distributed
to program participants
7
12Customer energy
efficiency education; program awareness
5
Customers gain knowledge of benefits of efficient
appliances and costs of operating inefficient
equipment; increased program awareness
10
Increased program penetration;
fewer inefficient appliances on grid
11
17
Incentives provided to
customers for appliance recycling
2
13
16
24
22
21
78
Table D-1. See ya later, refrigerator Program Logic Model Links: Working Hypotheses and Indicators
Link Working Hypotheses Indicators
1 Marketing and outreach lead to targeting
communications to residential customers with
refrigerators and freezers and those who recently
purchased a new unit.
Number of eligible potential participants that express interest;
marketing materials in bill inserts, on company website, in
schools, in newspapers and on radio; presence at seminars,
conferences, home shows, and community events
2 Incentives lead to customers enrolling in the
program.
Number of participants; participant interviews indicate role of
incentives on enrollment activities
3 Measurement and verification lead to the
evaluation team conducting an evaluation.
Completed evaluation informs future program cycles
4 Quality control leads to inspections being
performed.
Number of inspections indicate that quality control occurred
5 The delivery of marketing materials leads to
increased customer awareness regarding energy
efficiency and the program.
Increased customer awareness regarding energy efficiency
identified in surveys
6 Marketing efforts lead to customers enrolling in
program.
Number of participants enrolled in the program who indicate they
were reached by marketing efforts
7 Customer participation results in removing
inefficient appliances from the grid.
Number of appliances recycled due to participation in the
program
8 The evaluation leads to confirming program
effectiveness.
Implementer interviews (qualitative); evaluation identifies best
practices
9 Inspections and reviews leads to confirming
program effectiveness.
Implementer interviews (qualitative); inspections and reviews
should be indicated as improving program effectiveness
10 Education leads to program awareness. Participant interviews (qualitative) should indicate that education
led to program awareness
11 Removing inefficient appliances from the grid
leads to increased program penetration.
Number of appliances recycled compared to overall market
12 Removal of inefficient appliances leads to kWh
and kW savings.
Energy/demand savings generated expressed in kW and kWh
13 kWh and kW savings leads to persistent demand
savings.
Energy/demand savings over time; Participant interviews
regarding measure persistence
14 Confirming effective program operations leads to
verified program savings.
Implementer interviews (qualitative); effective program theory
and demonstrated links indicate savings are attributable to the
program
15 Confirming effective program operations leads to
the maintenance of optimum performance.
Implementer interviews (qualitative); program operations should
be confirmed as effective
16 Increased program awareness leads to fewer
inefficient appliances on the grid.
Interviews regarding awareness and resulting behavior
17 Fewer inefficient appliances on the grid lead to
persistent energy savings.
Market study/ number of appliances recycled; participant
interviews regarding measure persistence
18 Verified program savings leads to persistent
energy and demand savings.
Energy/demand savings over time expressed in kW and kWh
19 Verified program savings leads to Pacific Power
gaining experience with designing and marketing
programs.
Implementer interviews (qualitative); the increased experience
will be investigated
20 Maintaining optimal performance leads to Pacific
Power gaining experience with designing and
marketing programs.
Implementer interviews (qualitative); increased experience will be
investigated
79
Link Working Hypotheses Indicators
21 Fewer inefficient appliances on the grid lead to
environmental benefits.
Energy/demand savings quantified using engineering estimates,
analysis of reduced need to build power plants, environmental
impacts of power plants that were not built quantified using EPA
and other secondary data
22 Fewer inefficient appliances on the grid lead to
achieving long term energy savings.
Energy/demand savings, analysis of reduced need to build power
plants
23 Persistent energy savings lead to achieving long
term energy savings.
Energy/demand savings in kW and kWh using engineering analysis
and assessed over time
24 Pacific Power gaining experience with designing
and marketing programs leads to achievement of
long term energy savings goals.
Implementer interviews (qualitative); interviews will determine if
the experience is positively impacting program processes and
outcomes
80
Appendix F: Refrigerator NTG Combined Decision Tree
81
Appendix G: Freezer NTG Combined Decision Tree
82
Appendix H: CFL Engineering Calculations and Assumptions
Hours of Use Cadmus estimated CFL hours of use (HOU) using a multistate modeling approach, built on light logger
data collected from five states: Missouri, Michigan, Ohio, Maine, and Maryland. Cadmus chose these
data rather than data from the most recent California evaluation primarily because all these states have
relatively new CFL programs compared to California, where residential CFL programs have been in place
for a number of years.
Metering Protocol
Following whole-house lighting audits, Cadmus installed up to five loggers in each participant home.
Metering periods varied by utility, ranging from three months to one year. For homes with five or fewer
CFL fixture groups, Cadmus field staff installed light loggers on every CFL fixture. For homes with more
than five CFL fixture groups, field staff randomly selected which five fixtures to meter. This method
relied on systematic sampling, which involved installing a logger on every nth CFL fixture (the nth number
also based on the number of total possible CFL fixtures).
During the logger removal process, field staff collected additional data for evaluating data quality and
for determining if loggers had failed, had been tampered with, or had been removed. Moreover, prior to
removing each logger, field staff noted whether the logger had been correctly installed and the
orientation of its sensor.
Model Specification
To estimate HOU, Cadmus determined the total “on” time for each individual light logger per day, using
the following guidelines:
If a light logger did not record any light for an entire day, the day’s HOU was set to zero.
If a light logger registered a light turned on at 8:30 p.m. on Monday, and turned off at 1:30 a.m.
on Tuesday morning, 3.5 hours were added to Monday’s HOU and 1.5 hours to Tuesday’s HOU.
Cadmus modeled both weekday and weekend daily HOU as a function of room type, the presence of
children in the home, and CFL saturations in the home. This was done using two analyses of covariance
(ANCOVA) models, one for each day type.
ANCOVA models are regression models, which model a continuous variable as a function of a single,
continuous explanatory variable (in this case, CFL saturation) and a set of binary variables. This way, an
ANCOVA model simply serves as an analysis of variance (ANOVA) model with a continuous explanatory
variable added. Cadmus chose this specification due to its simplicity, making it suitable in a wide variety
of contexts. Though the model lacks the specificity of other methods, it offers estimates not nearly as
sensitive to small differences in explanatory variables, compared to more complex methods. Therefore,
these models can produce consistent estimates of average daily HOU for a given region, using its specific
distribution of bulbs by room and household type, and by the existing CFL saturation.
83
Cadmus specified final models as cross-sectional, ANCOVA regressions for day-type29 (j), and bulb (i), as:
Where:
CFL Saturation = the proportion of CFL bulbs in the home;
Kids = a dummy variable30 equal to one, if the household has children
under 18 living in the home, and zero otherwise;
Kitchen = a dummy variable equal to one, if the bulb is in the kitchen, and
zero otherwise;
Basement = a dummy variable equal to one, if the bulb is in the basement, and
zero otherwise;
Outdoor = a dummy variable equal to one, if the bulb is outdoors, and zero
otherwise;
Bedroom = a dummy variable equal to one, if the bulb is in the bedroom, and
zero otherwise;
Bathroom = a dummy variable equal to one, if the bulb is in the bathroom. and
zero otherwise; and
Other = a dummy variable equals to one, if the bulb is in a low-use room
(such as a utility room, laundry room, or closet), and zero otherwise.
Cadmus tested the potential influences of other demographic and regional variables in model
specifications, such as: latitude, income, education, and home characteristics. However, these variables
were not included as their estimated coefficients did not differ significantly from zero or produced signs
inconsistent with expectations.
Final Estimates and Extrapolation
As shown in Table 31, not all the two models’ estimated coefficients differed significantly from zero for
both day types, most likely due to differences in schedules between days. Nevertheless, Cadmus
included the same independent variables in each model for better cross comparability.
29
The two day-types for this analysis were weekend and weekday. Cadmus defined weekends as Saturday and
Sunday as well as the following federal recognized holidays: Christmas, Thanksgiving, Labor Day, Memorial
Day, New Year’s Day, Fourth of July, Presidents’ Day, and Veterans’ Day.
30 Dummy variables are binary, taking only values of either zero or one. Coefficients for these variables can be
interpreted as the difference in mean values between the two mutually exclusive groups.
84
Table 31. HOU Model ANCOVA Estimates
Coefficient Weekday Weekend
Parameter Estimate p-value* Parameter Estimate p-value
Intercept 2.38 <.0001 2.17 <.0001
CFL Saturation -0.82 0.08 -0.34 0.57
Kids 0.94 0.01 1.28 0.10
Kitchen 1.26 0.00 0.75 0.17
Basement 0.33 0.78 -1.29 0.00
Outdoor 1.27 0.12 -0.02 0.98
Bedroom -1.06 0.00 -1.57 0.00
Bathroom -0.85 0.06 0.92 0.62
Other -1.36 0.01 -1.91 <.0001
* P-values indicate the degree of confidence to which analysis asserts the given coefficient equals zero. In other
words, it is the probability that the effect of a given variable on HOU is random. Therefore, a lower p-value
indicates a higher degree of confidence in the estimated effect.
Cadmus used these model parameters to predict average daily use for SYLR by taking the sum of the
product of each coefficient shown in Table 31, and its corresponding average independent variable.
Table 32 shows independent variables used for SYLR. Except for CFL saturation, Cadmus estimated
independent variables using 2009–2010 participant survey data. Due to a lack of detailed CFL saturation
data for Rocky Mountain Power’s Wyoming service area, Cadmus used secondary data to estimate CFL
HOUs by room.
Table 32. Weekday HOU Estimation Input Values
Variable Value
CFL Saturation 29.9%
Kids 37%
Kitchen 18%
Basement 4%
Outdoor 6%
Bedroom 16%
Bathroom 15%
Other 13%
Using these values, the following equation calculated a 2.32 average weekday HOU:
( [ ]
[ ] [ ] [ ] )
85
Using the same method, Cadmus calculated the weekend HOU using parameter estimates from the
weekend model. The weighted average of these two values then provided the average annual HOU:
Table 33. HOU by Day Type
Day HOU
Weekday 2.32
Weekend 2.26
Overall 2.30
Precision calculations for model estimates accounted for sampling errors in model estimates and sample
inputs, which largely arose from participant surveys. Precision of individual HOU estimates can be
impacted by the precision of logger data model estimates and the accuracy of model inputs used for
extrapolation. Cadmus estimated the final relative precision for the CFL HOU in the SYLR program at
±4.1 percent with 90 percent confidence.
Waste Heat Factor The waste heat factor (WHF) is an adjustment representing interactive effects of lighting measures on
heating and cooling equipment operation. For this evaluation, Cadmus applied the WHF adjustment to
lighting savings estimates that was recommended in the 2009-2010 evaluation report. The WHF applied
in this evaluation was calculated for the potential study Cadmus conducted for Rocky Mountain Power
Wyoming in 2012 using the following methodology.
Cadmus calculated SYLR’s WHF using ASHRAE data on heating and cooling degree days (HDD and CDD,
respectively) in Rocky Mountain Power’s service territory. In addition, Cadmus used the 2006 Energy
Decisions Survey data31 to determine the saturation of heating and cooling equipment types in
Wyoming.
To determine the portion of the year heating or cooling equipment operates, and, therefore, when
lighting would affect heating or cooling energy consumption, Cadmus used the Northwest Power and
Conservation Council’s workbook to estimate the interaction for ENERGY STAR lighting savings in the 6th
Regional Power Plan.32
This calculator estimates heating and cooling interactions based on building simulation models for a
variety of HVAC equipment and cities around the region. Cadmus estimated savings for CITY, as
representative for the Wyoming territory, by using a weighted average of HDD and CDD from cities
across the region to most closely match the Wyoming territory.
31
http://www.pacificorp.com/content/dam/pacificorp/doc/Energy_Sources/Demand_Side_Management/
DSM_VolumeI_2011_Study.pdf.
32 http://www.nwcouncil.org/energy/powerplan/6/supplycurves/res/EStarLighting_NewFY09v1_0.xls.
86
This calculator determined heating and cooling interactions for zonal heating and heat pumps. To
estimate interactions for electric forced air furnaces, a heating system efficiency of 75 percent (to
account for duct losses) was included. The cooling interaction from heat pumps was used for all electric
cooling systems, as shown in the following table.
Table 34. HVAC Interactions
HVAC System Space Heating Interaction Space Cooling Interaction
Zonal 61% N/A
Heat Pump 47% -5.3%
These interactions were then weighted by the market share of electric heating and cooling systems.
The heating interaction was calculated as follows, where the summation is over the three electric
heating types:
∑( )
In addition:
The combined -9.5 percent adjustment could be applied to electricity savings for all interior lighting
measures to account for a net increase in electric heating and cooling load due to more efficient lighting.
Weighting for the interior/exterior distribution estimated through participant surveys, Cadmus found
the final WHF to be 90.5 percent (as shown below).
( )
( )
Applying this adjustment to the evaluated gross per-unit savings estimates for energy-savings kits yields
a WHF-adjusted gross per-unit savings estimate of 71.5 kWh. For this SYLR program period, the WHF
was determined to be 90.5%.
