Evaluation of the Massachusetts New Homes with ENERGY … · 2020-07-15 · The evaluation work conducted for the 2008 Program includes: • PRISM billing analyses of homes built

22 Haskell Street, Cambridge, MA 02140 Phone: (617) 497-7544 Fax: (617) 497-7543

www.nexusmarketresearch.com

Evaluation of the Massachusetts New Homes with ENERGY STAR® Program

2008 Findings and Analysis

Final Report

July 2, 2009

Submitted to:

Joint Management Committee

Submitted by:

Nexus Market Research Dorothy Conant

MA New Homes with ENERGY STAR—2008 Overall Report

Contents EXECUTIVE SUMMARY ......................................................................................................................................... I

PROGRAM OVERVIEW ................................................................................................................................................. I CONCLUSIONS ............................................................................................................................................................ II RECOMMENDATIONS ................................................................................................................................................. III

1 PROGRAM STATUS AND ACCOMPLISHMENTS ..................................................................................... 1 2 BILLING ANALYSES ....................................................................................................................................... 3

2.1 INITIAL BILLING ANALYSIS .............................................................................................................................. 3 2.2 UPDATED BILLING ANALYSIS .......................................................................................................................... 3

2.2.1 Occupant Behavior ................................................................................................................................ 4 2.2.2 Weather Data ......................................................................................................................................... 4

3 NON-ENERGY IMPACTS ................................................................................................................................ 6 3.1 HOMEOWNER SURVEY FINDINGS ..................................................................................................................... 6 3.2 BUILDER INTERVIEW FINDINGS ........................................................................................................................ 7

4 CFL PROCESS REVIEW ............................................................................................................................... 10 4.1 SATISFACTION WITH THE 2008 CFL PROCESS ................................................................................................ 10 4.2 CFL SELECTION, ORDERING, DELIVERY AND INSTALLATION ........................................................................ 11

4.2.1 Who Selected CFLs ............................................................................................................................. 11 4.2.2 Ordering and Installing CFLs .............................................................................................................. 11 4.2.3 CFLs Installed in Empty Sockets or Replaced Incandescent Bulbs ..................................................... 12 4.2.4 Number of CFLs Installed ................................................................................................................... 12 4.2.5 CFLs Failing ........................................................................................................................................ 12 4.2.6 Verification of Installed CFLs ............................................................................................................. 12

4.3 CHANGES IN TYPES OF CFLS INSTALLED—2007 TO 2008 ............................................................................. 12 5 BUILDER INTERVIEWS FOR ANNUAL PROGRAM ASSESSMENT ................................................... 14

5.1 MARKETING ENERGY STAR HOMES ........................................................................................................... 14 5.2 CHOOSING A HERS RATER ............................................................................................................................ 15 5.3 TRAINING AND TECHNICAL SUPPORT ............................................................................................................. 15

6 CONCLUSIONS AND RECOMMENDATIONS .......................................................................................... 16 6.1 CONCLUSIONS ................................................................................................................................................ 16 6.2 RECOMMENDATIONS ...................................................................................................................................... 17

APPENDIX A BILLING ANALYSES

APPENDIX B ASSESSMENT OF NON-ENERGY IMPACTS

APPENDIX C REVIEW OF 2008 CFL INSTALLATION PROCESS

APPENDIX D BUILDER INTERVIEW REPORT

Nexus Market Research

MA New Homes with ENERGY STAR—2008 Overall Report


Tables TABLE 3-1: MEAN NEI VALUES FROM THE HOMEOWNER SURVEY ....................................................... 7

Figures FIGURE 1-1: ENERGY STAR HOMES—PERCENTAGE OF ALL NEW HOMES COMPLETED

IN MASSACHUSETTS ................................................................................................................................ 2 FIGURE 1-2: ENERGY STAR, EMU AND CODE PLUS HOMES—PERCENTAGE OF ALL NEW

HOMES COMPLETED IN MASSACHUSETTS ...................................................................................... 2 FIGURE 2-1: HEATING ADJUSTMENT FACTOR – 1973 TO 2000 WEATHER DATA,

OUTLIERS EXCLUDED, FIXED-RANGE TAU ...................................................................................... 5 FIGURE 3-1: BUILDERS AND HOMEOWNERS WHO SAY ENERGY STAR HOMES PROVIDE

POSITIVE NEIS ........................................................................................................................................... 8 FIGURE 3-2: BUILDER AND HOMEOWNER VALUATION OF NEIS ............................................................ 9 FIGURE 4-1: WHO INSTALLED CFLS ................................................................................................................ 11 FIGURE 4-2: TYPE OF CFLS INSTALLED—2007 AND 2008 .......................................................................... 13

MA New Homes with ENERGY STAR—2008 Overall Report Page i

Executive Summary This document summarizes the evaluation work completed on the Massachusetts New Homes with ENERGY STAR® Program run by the Joint Management Committee (JMC) in Massachusetts for 2008. Individual evaluation reports on which the findings and analysis contained in this document are based are included as appendices. The evaluation work conducted for the 2008 Program includes:

• PRISM billing analyses of homes built through the Program in 2006 and 2007 providing adjustment factors for both the User-defined Reference Home (UDRH) and the as-built ENERGY STAR-qualified home that may be applied to Beacon software-based estimates of energy usage in order to calculate energy savings from the Program. (Billing Analyses, Appendix A)

• Estimation of the value to homeowners of seven non-energy impacts (NEIs) of ENERGY STAR homes using a survey of 70 homeowners who have purchased ENERGY STAR-qualified homes over the past two years, and in-depth interviews with 30 builders who participated in the Program in 2008. (NEI Analysis, Appendix B)

• Interviews with 30 ENERGY STAR builders who participated in the free Compact Fluorescent Lamp (CFL) component of the Program in 2008 assessing the builders’ experience selecting, ordering and installing CFLs on their own rather than having HERS raters do installations as well as the impact of the 2008 protocol on the number and type of CFLs installed. (CFL Process, Appendix C)

• Interviews with the 30 ENERGY STAR builders who participated in the free CFL component of the Program assessing their satisfaction with the Program, marketing ENERGY STAR homes, choosing a HERS rater and Program training offerings with comparisons to findings from 2007 interviews. (Builder Interviews, Appendix D)

Program Overview The Massachusetts New Homes with ENERGY STAR Program qualified more than 980 housing units in 2008, bringing the total number of housing units ENERGY STAR qualified since Program inception to more than 14,300. The total number of housing units ENERGY STAR qualified in 2008 is 23% lower than in 2007, the number of total building permits issued in Massachusetts in 2008 is 36% lower and the estimated number of housing units completed in Massachusetts in 2008 is 33% lower. With the number of statewide completed housing units falling more sharply than the number of homes qualified, the penetration of ENERGY STAR housing units climbed. The estimated percentage of new housing units completed in Massachusetts that are ENERGY STAR qualified increased from 9% in 2007 to 10% in 2008, the penetration of single family ENERGY STAR homes climbed from 7% to 10%, and the penetration of multi-family ENERGY STAR units fell from 14% to 11%.


MA New Homes with ENERGY STAR—2008 Overall Report Page ii

Conclusions Evaluations of the 2008 Program covered relatively diverse areas—adjustment factors from billing analysis, valuation of NEIs, and review of the CFL installation process. These evaluations do, nonetheless, point to several overall conclusions and recommendations.

The recession and housing market decline of 2008 has, not surprisingly, had significant effects on the Program. The number of ENERGY STAR homes qualified dropped, even as the penetration of ENERGY STAR housing increased, reflecting a much sharper percentage drop in the estimated number of housing units completed statewide than in the number of ENERGY STAR-qualified housing units built. However, 2008 also saw increases in the builders’ and, likely, the homeowners’ perceptions of the value of having the ENERGY STAR label on a home. More specifically,

• The builders’ perception of the value of having the ENERGY STAR label on a home has increased with almost all (93%) of interviewed builders saying that building ENERGY STAR homes and being able to market them as ENERGY STAR qualified is very (70% of builders) or somewhat (23% of builders)valuable in today’s housing market. This is a marked improvement from 2007 when 78% of builders called for the Program to conduct more consumer marketing because of low homebuyer interest in ENERGY STAR homes.

• While homeowner awareness of ENERGY STAR homes was not directly measured in 2008, over one-half (57%) of interviewed builders say awareness of and/or interest in buying ENERGY STAR homes has increased, and the percentage of builders saying almost all homebuyers are aware of ENERGY STAR homes doubled from 5% in 2007 to 10%.

• Homeowners who know they have bought ENERGY STAR homes place high values on non-energy impacts (NEIs) with close to nine out of ten believing their homes offer more thermal comfort than other new homes and valuing this feature, on average, almost as much as yearly energy bill savings. Homeowners also place a high value on energy bill protection—a feature they say they largely figure out on their own.

The Program’s steps toward a market-driven model, having builders chose their own HERS raters and giving them the option to select and install CFLs themselves, also appear to be working well. More specifically,

• Builders like being able to choose their HERS rater. Most builders stayed with the HERS raters they were assigned in 2007; the 14% of interviewed builders who opted to change HERS raters have successfully selected and established good working relationships with different raters.

• A majority (55%) of the 22 interviewed builders who have installed free CFLs under both the old and new processes like selecting, ordering and installing the free CFLs themselves better than having their HERS rater select and install the CFLs, since under the new process the builders are able to install the CFLs on their schedule, typically at the same


MA New Homes with ENERGY STAR—2008 Overall Report Page iii

time the lighting fixtures are installed. Just over one-fourth (27%) prefer having their HERS raters select and install CFLs and 18% are indifferent, saying both processes work equally well.

• Listing marketing support offerings on the application form and asking builders to check the options they are interested in is very likely a factor contributing to the likelihood of builders taking advantage of available marketing support. The percentage of interviewed builders taking advantage of available marketing support options increased from 18% of builders interviewed in 2007 to 50% of builders interviewed in late 2008 and early 2009.

Recommendations The overall conclusions of the 2008 evaluations lead to several recommendations aimed at maintaining the Program’s effectiveness as it continues its transition to a market-based model in a recessionary housing market. Several recommendations deal with marketing the Program.

• Continue marketing the Program to consumers to increase homebuyer awareness of and interest in the importance of energy efficiency and the benefits of buying and living in an ENERGY STAR home.

• Continue to encourage builders to take advantage of marketing support available through the Program. In particular, encourage all participating builders to display ENERGY STAR signs at their projects—signs are relatively inexpensive and builders say the signs bring in customers and make it more likely they will ask about what goes into building an ENERGY STAR home.

• Produce more case studies appropriate for use in marketing to builders and homebuyers. There are now several builders saying they have homebuyers coming to them looking for an ENERGY STAR home and several builders saying buyers are very happy with the low operating cost of their ENERGY STAR homes.

• NEIs should be an integral part of marketing for ENERGY STAR homes. Builders should be especially encouraged to talk to prospective buyers about noise reduction, indoor air quality, and safety (or make sure their salespeople do so). Consider sharing some of the results of the NEI study with builders, in particular to show them the differences between home buyers’ ratings of NEI values compared to how builders think buyers would rate them.

It is also important to maintain flexibility and prepare for future developments.

• The Program’s plan going forward to give builders the option of selecting, ordering and installing the free CFLs themselves or having their HERS rater select and install them is a good step. With the Program now requiring builders to install CFLs in at least 50% of all hard-wired screw-based fixtures, offering both approaches allows experienced ENERGY STAR builders who are comfortable selecting CFLs on their own and prefer having the CFLs on site to install when the light fixtures are installed to do so.


MA New Homes with ENERGY STAR—2008 Overall Report Page iv


Builders who are new to the Program or who are not very knowledgeable about the variety of CFLs available or which ones work best in specific applications will benefit from having their HERS raters choose and install the CFLs. Offering both processes will likely maximize the number of CFLs installed through the Program.

• Start planning for training builders to meet ENERGY STAR 2011 requirements. Educating builders in 2009 about changes likely to be implemented in 2011 and assuring them that in 2010 the Program will provide the training needed to ensure they can meet the new requirements will help maintain builder participation and the increasing penetration rate of ENERGY STAR homes in the market.

MA New Homes with ENERGY STAR—2008 Overall Report Page 1

1 Program Status and Accomplishments The Massachusetts New Homes with ENERGY STAR Program certified more than 980 housing units in 2008, bringing the total number certified since Program inception to more than 14,300.

Residential new construction programs in Massachusetts began in 1991 with The Energy-Crafted Homes (ECH) Program and transitioned to the ENERGY STAR Homes Program in April 1998. In 2007 the Program’s name was changed to Massachusetts New Homes with ENERGY STAR. The JMC’s New Homes with ENERGY STAR Program is part of a national energy efficiency campaign developed by the U.S. EPA. Nationally, 108,727 ENERGY STAR homes were certified in 2008, and over 947,600 ENERGY STAR-qualified homes have been built since the national program was introduced in 1996.1 The Massachusetts Program certified over 980 housing units in 2008, bringing the total number of housing units certified through the Program to more than 14,300; another 400 plus housing units were completed under Code Plus options in 2008.

The Code Plus option available to housing units signed in 2008 enabled housing units that failed to meet all the requirements for ENERGY STAR qualification, but had air infiltration of 6 ACH CFM 50 or lower and duct leakage of 8% or lower to participate in the Program, earn an incentive and have free CFL bulbs installed in appropriate sockets. In 2009, builders can not sign up for the Code Plus option, but builders new to the Program may be given the option to have their first project default to Code Plus.

One general indicator of the Program’s progress is the penetration of ENERGY STAR homes in the marketplace. Figure 1-1 shows the estimated percentage of new homes completed in Massachusetts that are ENERGY STAR qualified climbed from 3% in 1999 to 16% in 2006, then dropped to 9% in 2007 and increased to 10% in 2008. The estimated penetration of single family ENERGY STAR homes, which was at 7% in both 2006 and 2007, climbed to 10% in 2008. The estimated penetration of multi-family ENERGY STAR units dropped sharply in 2007 because multi-family units in high-rise buildings not built under the residential building code became ineligible for ENERGY STAR qualification. In 2008 the estimated penetration of ENERGY STAR multi-family units fell from 14% to 11%. (EPA recently developed modified performance and prescriptive path requirements for attached housing and expanded its definition of qualifying multi-family housing to include units in buildings of five or fewer stories permitted as having a residential use-group even if built under commercial building code.2)

1 Preliminary estimates provided by EPA: final 2008 data are not yet available. 2 http://www.energystar.gov/index.cfm?c=bldrs_lenders_raters.nh_attached_housing


http://www.energystar.gov/index.cfm?c=bldrs_lenders_raters.nh_attached_housing



Figure 1-1: ENERGY STAR Homes—Percentage of All New Homes Completed in Massachusetts

1%3% 4%

6%

9% 9% 10%12%

16%

9%

18%19%

35%

11%

25%

14%

6% 6% 7% 7% 7%

10%10%

0%

5%

10%

15%

20%

25%

30%

35%

40%

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

All ENERGY STAR Completions Multi-family ENERGY STAR Single Family ENERGY STAR

Multi-family

Single Family

Program ENERGY STAR Completions as Percent of Statewide Completions

Total Completions

Including homes completed in 2008 that did not meet all the requirements for ENERGY STAR qualification, but participated in the Program through the Code Plus option, the Program impacted an estimated 15% of homes completed in Massachusetts in 2008: 12% of single family homes and 20% of multi-family housing units. (Figure 1-2 )

Figure 1-2: ENERGY STAR, EMU and Code Plus Homes—Percentage of All New Homes Completed in Massachusetts

1%3% 4%

6%9% 9% 10%

12%

18% 19%

48%

20%

25%

20%

6% 6%

20%

11% 15%

8%7% 7%

12%

-5%

5%

15%

25%

35%

45%

55%

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

All Completions All Multi-family All Single Family

Multi-family

Single Family

Program ENERGY STAR, EMU & Code Plus Completions as Percent of Statewide Completions

Total Completions


2 Billing Analyses The first task of the 2008 evaluation effort was to perform a PRISM billing analysis of homes built through the Program in 2006 and 2007 in order to provide adjustment factors for both the User-defined Reference Home (UDRH) and the as-built ENERGY STAR-qualified home that may be applied to Beacon software-based estimates of energy usage in order to calculate energy savings from the Program. Beacon is the software program currently used by ICF International, the implementation contractor, to estimate energy usage and savings in ENERGY STAR Homes. PRISM estimates weather adjusted heating and/or cooling usage and overall usage for a home through a model incorporating both billing data and actual weather conditions. The PRISM billing analysis issued an initial report recommending an adjustment factor of 0.734 on November 7, 2008. After conducting a series of further data explorations in response to the comments of several reviewers, the team updated the analysis and recommended a final adjustment factor of 0.774, issued on March 19, 2009.

2.1 Initial Billing Analysis The initial billing analysis used models with the closest fit between the normalized weather data and the billing data (i.e., an explained variance R2 of at least 0.70 or better) to develop various adjustment factors using a scatter plot with the x-axis representing Beacon estimated usage and the y-axis representing PRISM estimated usage. Multiple adjustment factors were calculated based on various subsets of the models. The recommended heating adjustment factor of 0.734 (with a 90% confidence interval of 0.693 to 0.775) was developed from the subset of 91 PRISM models for the 87 single-family homes and the four multifamily buildings with an R2 of 0.90 or better (that is, models with very high explained variance) and that were not identified to be outliers based on the percentage difference in their PRISM and Beacon estimated energy usage.

The initial billing analysis also calculated two cooling adjustment factors, but they were based only on 16 single-family homes, and the results indicated such dramatically different estimated usage between PRISM and Beacon models that the factors were not considered reliable. The report on the initial billing analysis is included as Appendix A-1.

2.2 Updated Billing Analysis The adjustment factor provided by the initial billing analysis, 0.734, would reduce the Program’s claimed savings by 26.6%. Both the NMR team and the reviewers expressed concern over the size of this adjustment and, thus, the team explored two main areas—occupant behavior and weather—in an effort to explain why PRISM estimates of actual usage differed from Beacon estimates and perhaps to develop a revised adjustment factor that would not reduce claimed savings by such a large amount.



2.2.1 Occupant Behavior After a review of variable bias impacts, the NMR team advised against further research into the behavioral component for differences between PRISM and Beacon estimates of heating usage. The reason for this is that, if there is little or no relationship between the Beacon heating usage engineering estimate and occupant behavior, then modeling the relationship between the normalized actual heating consumption and the engineering estimate without behavioral data will not bias the correction factor estimate. In other words, adding occupant behavior variables to the billing analysis will not markedly affect the resulting adjustment factor, although it would improve the explained variance in the model.

2.2.2 Weather Data The Beacon and PRISM models use different weather data for estimating heating usage in the same house. Beacon models estimate usage based on normalized hourly weather data, which, until recently were only available for weather stations located in moderate to large cities; Massachusetts homes are matched with stations in Albany, Boston, Hartford, Providence, and Worcester. Furthermore, the Beacon models used in the comparison relied on a weather data normalization series period from 1961 to 1990. In contrast PRISM models base energy usage on normalized daily weather data that have been available for smaller, often more local weather stations for some time; these weather data were used since they are likely to more accurately reflect the conditions in many homes. However, the quality of the data from the smaller stations varied for earlier years, with reliable data available only from 1996 to the present.

The use of different weather data in the original Beacon and PRISM models discussed in Section 2.1 is significant since the average number of annual heating degree days in Massachusetts has declined from a normalized 6,467 in the 1961 to 1990 time series to 6,100 in the 1996 to 2007 series and 6,079 for the 2006 to 2008 time period with billing data available. Several analyses explored weather effects. The NMR team ran the PRISM models again using weather data from 1973 to 2000 for the same stations used by the Beacon model and ICF ran 14 Beacon models using normalized weather data from 1991 to 2005. The NMR team also ran the PRISM models using the 1973 to 2000 weather data but limited the heating reference temperature to 45 to 65 degrees.




The last analysis improved the adjustment factor from 0.734 to 0.774. Figure 2-1 presents the scatter plot from this analysis.

Figure 2-1: Heating Adjustment Factor – 1973 to 2000 Weather Data, Outliers Excluded, Fixed-Range Tau

(PRISM R2 ≥ 0.90)

y = 0.774x

0

200

400

600

800

1000

1200

1400

1600

0 500 1000 1500 2000 2500

PRISM Heating Usage (Therms)

Beacon Heating Usage (Therms)

Sample size = 82, multifamily = 2, single-family attached = 39, single-family detached = 41

Based on the updated analysis, the NMR team recommended that efforts to bring the Beacon and PRISM model estimates closer together, thus increasing the adjustment factor, focus on coordinating the weather series used in the normalization period. The weather data used should be more recent than the 1961 to 1990 period currently used for the Beacon model to take into account the warming of the climate, and likely savings in future years. In the meantime, the recommended adjustment factor has been updated to 0.774. Based on recent information provided by the Sponsors, it is the team’s understanding that the Beacon models will be updated with more recent weather data for future estimation of energy savings for ENERGY STAR-qualified homes. The memorandum on the updated billing analysis is included as Appendix A-2.


3 Non-Energy Impacts The value to homeowners of seven non-energy impacts (NEIs) of ENERGY STAR homes was calculated using a survey of 70 homeowners who have purchased ENERGY STAR-qualified homes over the past two years. The values assigned to the NEIs by the homeowners were then compared to how much 30 builders who participated in the Program in 2008 thought buyers of their homes valued NEIs associated with energy efficiency. The NEIs, which had not been previously studied in connection with the Massachusetts Program, are:

• Thermal comfort in terms of temperatures and draftiness • Noise levels in terms of the amount of outdoor noise the home’s occupants can hear

inside the house • Lighting quality combined with longer lighting life given the use of CFLs and fluorescent

fixtures • Indoor air quality given reduced air infiltration and improved ventilation helping keep out

dust, pollen, humidity, and car exhaust • Safety of the new home’s heating system along with improved ventilation making the

home safer in terms of carbon monoxide levels and gas leaks • Anticipated ease of selling or leasing the home in the future and higher resale/leasing

value • Knowledge of some protection against rising energy prices; that is, as energy prices

increase, energy bill savings will also increase

3.1 Homeowner Survey Findings Homeowners overwhelmingly believe their new homes provide positive NEIs in thermal comfort (86%), a higher resale or rental value (80%), and protection against energy bill increases (93%). Majorities also believe their new homes provide noise reduction (67%) and better lighting features (61%), while fewer believe their homes provide better indoor air quality (51%) or more safety (46%). For all the NEIs studied, homeowners most often say they learned of the existence of an NEI before deciding to buy or build the home and that the presence of NEIs influenced their decision to buy or build an ENERGY STAR home.

Homeowner survey respondents were asked to estimate an annual monetary value for the NEIs they experience in their homes. The survey used a direct scaling method, asking respondents to value NEIs as a percentage of energy savings assuming ENERGY STAR homes save $400 per year in energy costs compared to similar, newly constructed non-ENERGY STAR homes. The survey first asked homeowners if they believed their new home had a particular NEI and whether it was positive or negative. Taking the thermal comfort NEI as an example, respondents were asked if they believed their new home, because it is an ENERGY STAR home, is more comfortable than other new homes, less comfortable, or no different in its comfort level. Those who believed it was more comfortable were asked to place a value per year on this increased



comfort either in dollars or as a percentage of energy savings. Those who believed it was less comfortable were asked how much the decreased comfort took away from the value of living in an ENERGY STAR home, either in dollars or as a percentage of energy savings. NEI values for those who believed their new home was no different in comfort level from non-ENERGY STAR new homes were set to zero. Table 3-1 presents the mean values assigned to NEIs by homeowner survey respondents; values have been set to zero for those who did not believe their home provided a particular NEI and to missing for those who could not express a value either in dollars or as a percentage of energy savings. The means in Table 3-1 include both positive and negative NEI values. Upper and lower bounds of values are calculated at a 90% confidence level; the lower bounds provide a conservative estimate that may be considered for planning purposes.

Table 3-1: Mean NEI Values from the Homeowner Survey (all homeowner survey respondents except for those who cannot assign an NEI value)

Thermal comfort (n=61)

Noise reduction

(n=64)

Lighting life/quality

(n=63)

Indoor air

quality (n=61)

Safety (n=63)

Resale rental value

(n=53)

Energy bill protection

(n=51) Total

Mean NEI Value

Dollars $279 $146 $144 $126 $105 $259 $386 $1445 %Bill Savings 70% 37% 36% 32% 26% 65% 97% 361%

Lower Bound NEI Value*

Dollars $210 $110 $105 $91 $72 $170 $161 $919 % Bill Savings 53% 28% 26% 23% 18% 43% 40% 230%

Upper Bound NEI Value*

Dollars $348 $182 $183 $161 $138 $348 $611 $1971 % Bill Savings 87% 45% 46% 40% 35% 87% 153% 493%

*Calculated at a 90% confidence level.

3.2 Builder Interview Findings Builders are more likely than homeowners to believe ENERGY STAR homes have positive NEIs in the cases of thermal comfort, outdoor noise reduction, better indoor air quality, safety, and a higher resale or rental value, as shown in Figure 3-1. Furthermore, the proportion of builders who believe ENERGY STAR homes are quieter and have better indoor air quality is significantly higher than that of homeowners at the 90% confidence level. However, the proportion of builders who believe the lighting life and quality of ENERGY STAR homes is a positive feature for homeowners is significantly lower, at the 90% confidence level, than the proportion of homeowners holding this view.



Figure 3-1: Builders and Homeowners Who Say ENERGY STAR Homes Provide Positive NEIs

Builders are more likely to discuss thermal comfort than any other NEI with their customers, and homeowners are more likely to remember hearing about this NEI than any other NEI from the builder. While builders also claim they often discuss noise reduction, lighting, indoor air quality, resale or rental value, and energy bill protection with customers, fewer homeowners remember hearing about these NEIs from builders.

Overall, builder estimates of NEI value to homeowners are higher than those provided by homeowners, though, at a global level the results are similar enough to reinforce their validity. For individual NEIs, builder mean values are considerably higher than homeowner means for thermal comfort, noise reduction, indoor air quality, and safety; in the cases of indoor air quality and noise reduction, builder estimates are significantly higher at the 90% confidence level. This is due, in both cases, to the higher proportion of builders than homeowners who believe ENERGY STAR homes are quieter and have better indoor air quality, as shown in Figure 3-2. Estimates for resale or rental value are very close for the two groups and builders give considerably lower estimates for lighting features and energy bill protection. (Figure 3-2)




Figure 3-2: Builder and Homeowner Valuation of NEIs

It should be noted that the lower mean value estimate for lighting life and quality among builders than among homeowners is largely due to the higher proportion of builders who applied a negative value to this NEI. While only four out of 70 homeowners applied negative values to CFLs, four out of 30 builders did so. It would thus appear that builders are more likely to believe that homeowners do not like CFLs than is actually the case. The report on NEI assessment is included as Appendix B.


4 CFL Process Review Thirty builders interviewed about their experiences selecting, ordering and installing the free CFLs available to them generally found the 2008 process workable. Prior to 2008 builders could have their HERS raters install free CFLs in all appropriate sockets when the final inspection was conducted; builders did not pay anything for the CFLs and there was no limit to the number of CFLs that could be installed. In 2008, builders became responsible for selecting, ordering and installing the free CFLs and received an incentive of $2 for each CFL they installed. In 2009 the $2 incentive to builders per CFL installed was discontinued and all participating homes are now required to have CFLs installed in at least 50% of available hard-wired screw-based fixtures. Builders have the option of installing the free CFLs themselves or having their HERS rater install them at the final inspection. As in previous years, the free CFLs are available for all appropriate sockets; there is no limit to the number of free CFLs that may be installed per home.

4.1 Satisfaction with the 2008 CFL Process All interviewed builders say they learned about the changes in how to get the free CFLs from their HERS rater and/or ICF and most (77%) say that they found the instructions for selecting and ordering CFLs easy to understand and follow. A majority (55%) of the 22 interviewed builders who have installed free CFLs under both the old and new processes prefer selecting, ordering and installing the free CFLs themselves, just over one-fourth (27%) prefer having their HERS raters select and install the CFLs and 18% are indifferent, saying both processes work equally well. The feature builders like most about the 2008 process is being able to install the CFLs on their schedule, typically at the same time the lighting fixtures are installed; what builders like most about the pre-2008 process is that it is less work for them and there is no paperwork. Six of the eight interviewed builders who have experience only with the 2008 process commented that they like the 2008 process; the other two builders did not comment.

These findings support the Program’s plan going forward to give builders the option of selecting, ordering and installing the free CFLs themselves or having their HERS rater select and install them. Clearly, different builders have different needs and preferences as well as different levels of knowledge about and experience with CFLs. With the Program now requiring builders to install CFLs in at least 50% of all hard-wired screw-based fixtures, builders who are new to the Program or who are not very knowledgeable about the variety of CFLs available or which ones work best in specific applications will benefit from being able to have their HERS raters choose and install the CFLs. Builders without much experience with CFLs may, on their own, be likely to install the minimum number of CFLs required by the Program, but if HERS raters select and install the CFLs they can encourage the installation of CFLs in more than 50% of appropriate sockets. Offering both processes will likely maximize the number of CFLs installed through the Program.



4.2 CFL Selection, Ordering, Delivery and Installation Builders describe a variety of configurations with respect to who selected, who ordered and who installed the free CFLs. The most common configurations are: the builder selected and ordered the CFLs and the electrician installed them (20%); the builder selected, ordered and installed the CFLs (17%); and the builder selected and ordered the CFLs and the construction crew installed them (17%).

4.2.1 Who Selected CFLs Overall, 70% of interviewed builders say they selected what CFLs to order. Almost two-thirds of interviewed builders (63%) say they, or their electrician, did not need help selecting the CFLs; builders who needed help report they received the help they needed from ICF, their electrician, Energy Federation Inc. (EFI), the lighting supply house or the website3. Several builders commented on why they did not ask their electrician to select the CFLs. Builders say they are more familiar with CFLs and that electricians do not know that much about bulbs because when they (electricians) order fixtures from a supply house, the supply house selects the bulbs.

4.2.2 Ordering and Installing CFLs Almost all interviewed builders (28 of 30) say they did the actual ordering of the CFLs. Most interviewed builders (77%) ordered all the CFLs they needed in one order, 20% identified additional sockets they needed to order CFLs for after they received their first order and 3% (one builder) usually makes two orders per home so that he does not get too many CFLs at one time. Builders say CFLs are delivered within a few days of being ordered; none of the builders interviewed report CFL delivery interfering with their construction schedule.

Figure 4-1 shows one-third of interviewed builders say their electrician installed the CFLs, just over one-fourth (27%) say their construction crew installed the CFLs, one-fifth say they installed the CFLs and one-fifth gave other responses.

Figure 4-1: Who Installed CFLs

3 More than one builder mentioned visiting “the website.” They did not say whether they were referring to the Program’s website or EFI’s website.



Most interviewed builders (77%) say they were able to install all the CFLs they ordered. Builders who were not able to install all the CFLs they ordered report some did not fit, some were defective, or that they had extra CFLs. If only a few CFLs were defective or did not fit, builders typically reported buying replacements locally. If they have extra CFLs, builders say they install them in their next ENERGY STAR home.

4.2.3 CFLs Installed in Empty Sockets or Replaced Incandescent Bulbs One-half of the interviewed builders say they installed the free CFLs in empty sockets, 23% say they replaced incandescent bulbs, 23% say some CFLs were installed in empty sockets and some replaced incandescent bulbs, and one builder (3%) says he installed some CFLs in empty sockets and replaced existing CFLs in other sockets. Builders who replaced incandescent bulbs typically said that they saved the incandescent bulbs they removed or gave them to the homeowner. Going forward, all 15 builders who installed CFLs in empty sockets in 2008 and 9 of the 15 builders who replaced incandescent bulbs in at least some sockets say that they plan on asking their electrician not to install incandescent bulbs in sockets for which they have ordered CFLs.

4.2.4 Number of CFLs Installed Builders who installed CFLs under both the old and new processes report installing the same number of CFLs (13 of 22 builders) or more CFLs (9 of 22 builders) under the 2008 process than they think their HERS raters would have installed under the old process.

4.2.5 CFLs Failing One-half of the 30 interviewed builders report having some CFLs fail. Builders who commented on CFL failures say that that very few fail—only one or two per home. When only a few CFLs fail most builders say that they either use CFLs they have on hand or purchase new ones on their own to replace the ones that failed.

4.2.6 Verification of Installed CFLs Only 10% of builders say that their HERS rater counts the number of CFLs installed; 30% say their HERS rater verifies the CFLs are installed, but do not mention the HERS rater counting the CFLs; 20% say that their HERS rater verifies CFLs are installed, but does not count them; and 30% say that they are not aware of anyone verifying that the CFLs they ordered were installed. Ten percent of builders reported either that the final inspection on their home was done before the CFLs were installed or that their homes had not yet had the final inspection at the time of the interview.

4.3 Changes in Types of CFLs Installed—2007 to 2008 ICF provided data on the number of CFLs, by bulb type, they installed and verified in the 2007 and 2008 Program years. Almost all (91%) of the bulbs installed in 2008 were ordered and




installed by builders. Figure 4-2 shows that spirals remain the most frequently installed type of CFL, but that they accounted for a much smaller percentage of CFLs in 2008 (38% vs. 49% in 2007). Globes also accounted for a smaller percentage of 2008 CFLs (3% vs. 7% in 2007). Capsules accounted for 22% of all CFLs in both 2007 and 2008. Reflectors and floods, candelabra-based bulbs, 3-way bulbs, and dimmable floods and spirals all accounted for larger percentages of total bulbs in 2008 than in 2007, though some of the percentages are very low. The drop in the percentage of spirals installed likely reflects builders becoming more familiar with and comfortable with the variety of CFL choices now available to them. ICF installed an average of 25 CFLs per housing unit installing CFLs in 2007. In 2008, the average number of CFLs installed per home installing CFLs was 44.

Figure 4-2: Type of CFLs Installed—2007 and 2008


5 Builder Interviews for Annual Program Assessment This year’s builder interviews focused on three basic topics: marketing ENERGY STAR-qualified homes, choosing HERS raters and satisfaction with the training and support provided by the Program. Thirty ENERGY STAR builders were interviewed in December 2008 and January 2009. Although the questions in this year’s interviews are not identical to the questions asked in interviews with 40 ENERGY STAR builders conducted in November and December 2007, they are similar enough to make comparisons that suggest several very positive trends.4

• In the 2007 interviews, 78% of builders called for the Program to conduct more consumer marketing because of low homebuyer interest in ENERGY STAR homes. This year, 93% of interviewed builders (28 out of 30 builders) say that building and marketing ENERGY STAR-qualified homes in the current housing market is very valuable (70%; 21 builders) or somewhat valuable (23%; 7 builders).

• In the 2007 interviews, only five percent of builders said almost all homebuyers were aware of ENERGY STAR homes. This year, over one-half (57%) of interviewed builders say awareness of and/or interest in buying ENERGY STAR homes has increased, and the percentage of builders saying almost all homebuyers are aware of ENERGY STAR homes doubled to 10%.

• In the 2007 interviews only seven builders (18%) said they received help marketing their homes from the Program. This year, one-half of the 30 interviewed builders say they are taking advantage of marketing support options offered by the Program to help them market their ENERGY STAR homes.

One of the most encouraging findings from this year’s interviews is four builders reporting homebuyers looking for an ENERGY STAR home and another two builders saying that certification helped sell some of their homes.

5.1 Marketing ENERGY STAR Homes As described above, 93% of interviewed builders say that building ENERGY STAR homes and being able to market them as ENERGY STAR qualified is very or somewhat valuable in today’s depressed and very competitive housing market. Almost all interviewed builders (90%) also say that homebuyers are showing more awareness of and/or interest in energy efficiency. Most important from a Program perspective, a majority of interviewed builders (57%) report homebuyers are showing more awareness of and/or interest in buying an ENERGY STAR-qualified home. One-half of the 30 interviewed builders say they are taking advantage of marketing support options offered by the Program to help them market their ENERGY STAR homes. Another three builders say they plan to take advantage of marketing support on their next

4 Please note that due to the small sample sizes the results of the 2007 and 2008/2009 interviews should not be considered statistically representative of the population of Massachusetts ENERGY STAR builders; rather, the results should be viewed as qualitative in nature.




projects. Eleven of the 15 builders taking advantage of marketing support from the Program say that they think it is helping them sell their ENERGY STAR homes by letting buyers know they build energy-efficient homes and moving people to ask questions about what is involved in building an ENERGY STAR home.

5.2 Choosing a HERS Rater Interviewed builders like being able to choose their HERS raters and most builders who had a HERS rater assigned to them in 2007 continued to work with the same rater in 2008. Only 3 of the 22 interviewed builders who built ENERGY STAR homes in both 2007 and 2008 chose a different rater to work with in 2008. Their reasons for changing raters all address the need to have a good working relationship with their rater. Most importantly, all three builders who changed raters are satisfied with the raters they are now working with.

5.3 Training and Technical Support Roughly three-fourths (73%) of the 30 builders interviewed in late 2008 or early 2009 say they are satisfied with the level and amount of training/support available to them through the Program and/or their HERS rater. This is consistent with what the 40 ENERGY STAR builders interviewed in 2007 said; 78% of builders interviewed in 2007 said they were either extremely satisfied (25%) or satisfied (53%) with the technical support they received.

The 30 builders interviewed in late 2008 or early 2009 are virtually split between whether or not they think the Program needs to offer additional training—15 say no, 14 say yes and one does not know. One production builder who commented on why he does not think additional training is needed says, “We build the same house over and over, so once we have it down the only time we need more training is when the rules change.” Four other builders who commented say that there is help available if you ask for it or study what is available on the Program’s website.

The 14 builders who would like to see additional training offerings provide a variety of suggestions including: refresher courses; annual review meetings with small groups of ten or fewer builders to go over Program changes and what they (the builders) are doing well and how they could improve; seminars on the HERS system, the certification process and Program changes; seminars or training on new technologies and educational marketing; and training on a variety of specific technical topics.


6 Conclusions and Recommendations Evaluations of the 2008 Program covered relatively diverse areas—adjustment factors from billing analysis, valuation of NEIs, and review of the CFL installation process. These evaluations do, nonetheless, point to several overall conclusions and recommendations.

6.1 Conclusions The recession and housing market decline of 2008 has, not surprisingly, had significant effects on the Program. The number of ENERGY STAR homes qualified dropped, even as the penetration of ENERGY STAR housing increased, reflecting a much sharper percentage drop in the estimated number of housing units completed statewide than in the number of ENERGY STAR-qualified housing units built. However, 2008 also saw increases in the builders’ and, likely, the homeowners’ perceptions of the value of having the ENERGY STAR label on a home. More specifically,

• The builders’ perception of the value of having the ENERGY STAR label on a home has increased with almost all (93%) of interviewed builders saying that building ENERGY STAR homes and being able to market them as ENERGY STAR qualified is very (70% of builders) or somewhat (23% of builders)valuable in today’s housing market. This is a marked improvement from 2007 when 78% of builders called for the Program to conduct more consumer marketing because of low homebuyer interest in ENERGY STAR homes.

• While homeowner awareness of ENERGY STAR homes was not directly measured in 2008, over one-half (57%) of interviewed builders say awareness of and/or interest in buying ENERGY STAR homes has increased, and the percentage of builders saying almost all homebuyers are aware of ENERGY STAR homes doubled from 5% in 2007 to 10%.

• Homeowners who know they have bought ENERGY STAR homes place high values on non-energy impacts (NEIs) with close to nine out of ten believing their homes offer more thermal comfort than other new homes and valuing this feature, on average, almost as much as yearly energy bill savings. Homeowners also place a high value on energy bill protection—a feature they say they largely figure out on their own.

The Program’s steps toward a market-driven model, having builders chose their own HERS raters and giving them the option to select and install CFLs themselves, also appear to be working well. More specifically,

• Builders like being able to choose their HERS rater. Most builders stayed with the HERS raters they were assigned in 2007; the 14% of interviewed builders who opted to change HERS raters have successfully selected and established good working relationships with different raters.



• A majority (55%) of the 22 interviewed builders who have installed free CFLs under both the old and new processes like selecting, ordering and installing the free CFLs themselves better than having their HERS rater select and install the CFLs, since under the new process the builders are able to install the CFLs on their schedule, typically at the same time the lighting fixtures are installed. Just over one-fourth (27%) prefer having their HERS raters select and install CFLs and 18% are indifferent, saying both processes work equally well.

• Listing marketing support offerings on the application form and asking builders to check the options they are interested in is very likely a factor contributing to the likelihood of builders taking advantage of available marketing support. The percentage of interviewed builders taking advantage of available marketing support options increased from 18% of builders interviewed in 2007 to 50% of builders interviewed in late 2008 and early 2009.

6.2 Recommendations The overall conclusions of the 2008 evaluations lead to several recommendations aimed at maintaining the Program’s effectiveness as it continues its transition to a market-based model in a recessionary housing market. Several recommendations deal with marketing the Program.



• Produce more case studies appropriate for use in marketing to builders and homebuyers. There are now several builders saying they have homebuyers coming to them looking for an ENERGY STAR home and several builders saying buyers are very happy with the low operating cost of their ENERGY STAR homes.

• NEIs should be an integral part of marketing for ENERGY STAR homes. Builders should be especially encouraged to talk to prospective buyers about noise reduction, indoor air quality, and safety (or make sure their salespeople do so). Consider sharing some of the results of the NEI study with builders, in particular to show them the differences between home buyers’ ratings of NEI values compared to how builders think buyers would rate them.




It is also important to maintain flexibility and prepare for future developments.

• The Program’s plan going forward to give builders the option of selecting, ordering and installing the free CFLs themselves or having their HERS rater select and install them is a good step. With the Program now requiring builders to install CFLs in at least 50% of all hard-wired screw-based fixtures, offering both approaches allows experienced ENERGY STAR builders who are comfortable selecting CFLs on their own and prefer having the CFLs on site to install when the light fixtures are installed to do so. Builders who are new to the Program or who are not very knowledgeable about the variety of CFLs available or which ones work best in specific applications will benefit from having their HERS raters choose and install the CFLs. Offering both processes will likely maximize the number of CFLs installed through the Program.

• Start planning for training builders to meet ENERGY STAR 2011 requirements. Educating builders in 2009 about changes likely to be implemented in 2011 and assuring them that in 2010 the Program will provide the training needed to ensure they can meet the new requirements will help maintain builder participation and the increasing penetration rate of ENERGY STAR homes in the market.

Appendix A

Billing Analyses

Appendix A-1

Initial Billing Analysis



REPORT ON PRISM BILLING ANALYSIS AND ENERGY USAGE ADJUSTMENT FACTORS DEVELOPED IN

SUPPORT OF THE MASSACHUSETTS NEW HOMES WITH ENERGY STAR® PROGRAM

DRAFT November 7, 2008

Submitted to: Joint Management Committee

Submitted by: Nexus Market Research, Inc.

The Cadmus Group Dorothy Conant

Draft Report: Billing Analysis of ENERGY STAR-qualified homes Page i

Table of Contents

1 Executive Summary ................................................................................................................ 1

2 Introduction ............................................................................................................................. 2

2.1 Key Findings: Adjustment Factors ................................................................................... 2

3 Summary of Sample Development and Final Sample Size .................................................... 4

4 Data Cleaning Procedures Prior to PRISM Analysis .............................................................. 6

4.1 Matching billing data to requested addresses ................................................................... 6

4.2 Identifying problematic properties or months of data ...................................................... 6

4.2.1 Months with low usage ............................................................................................. 7

4.2.2 Months with high usage ............................................................................................ 7

4.2.3 Other issues with monthly billing data ..................................................................... 8

5 PRISM Analysis Procedures ................................................................................................... 8

6 Criteria for Inclusion in Adjustment Factor Calculation ........................................................ 9

6.1 Billing-data reasons for exclusion from adjustment factor .............................................. 9

6.2 Beacon File reasons for exclusion from adjustment factor ............................................ 10

6.3 PRISM reasons for exclusion from adjustment factor ................................................... 10

6.4 Handling of multifamily buildings ................................................................................. 11

7 Results: Adjustment Factors ................................................................................................. 11

7.1 Heating Adjustment Factors ........................................................................................... 12

8 Cooling Adjustment Factors ................................................................................................. 18

9 Potential Reasons for Differences from Beacon ................................................................... 19

10 Variation from the Previous Study Approach ....................................................................... 20

11 Conclusion ............................................................................................................................ 22


Draft Report: Billing Analysis of ENERGY STAR-qualified homes Page ii


List of Figures

Figure 7-1: Heating Adjustment Factor – All Buildings Included ............................................... 13

Figure 7-2: Heating Adjustment Factor – Outliers Excluded ....................................................... 14

Figure 7-3: Heating Adjustment Factor – Buildings Strongest PRISM Models .......................... 15

Figure 7-4: Heating Adjustment Factor – Strongest PRISM Models and Outliers Excluded ..... 16

Figure 8-1: Cooling Adjustment Factor – All Buildings Included ............................................... 18

Figure 8-2: Cooling Adjustment Factor –Outlier Excluded ......................................................... 19

Draft Report: Billing Analysis of ENERGY STAR-qualified homes Page 1

1 Executive Summary This report summarizes the procedures, methods, and results of a PRISM billing analysis of heating and cooling in homes built through the Massachusetts New Homes with ENERGY STAR® Program (the program) in 2006 and 2007. It provides adjustment factors for both the User-defined Reference Home (UDRH) and the as-built ENERGY STAR-qualified home that the program sponsors could apply to Beacon software-based estimates of energy usage in order to calculate energy savings from the program. The billing analysis team is lead by Nexus Market Research (NMR). M. Sami Khawaja and Matei Perussi of The Cadmus Group (Cadmus) provided guidance and training to NMR on the PRISM analysis; Dorothy Conant advised the team on various aspects of the project.

The NMR team requested billing data for 464 ENERGY STAR-certified home addresses from the sponsors and received data on electric usage for 524 homes or units in multifamily buildings and gas usage for 320 homes or units in multifamily buildings. More electric records were received than requested because our request sometimes grouped entire multifamily buildings while the data we received covered individual units in those buildings. Fewer gas records were received than requested because some homes did not have active accounts while others did not have natural gas service.

After receiving the billing data, the team searched for potential problems in the datasets, including, but not limited to, the following issues:

• Lack of 12 months of usable data for the individual home or unit • Months in which the home or unit was vacant • Unusual shifts in usage, typically associated with either the completion of the

construction period of a home or its shift from being vacant to occupied • Meter readings that covered more or less than a month of usage • Estimated meter readings as opposed to actual ones

When possible, the team fixed problems in the dataset (e.g., combining two sequential readings to add up to a month, averaging an estimated reading with a sequential actual reading). However, if no solution existed (e.g., lack of 12 months of data, vacancies, usage associated with the construction phase), the problematic months were removed from the data set. Only those homes or units with 12 months of usable data entered the PRISM analysis. The team ran separate PRISM analyses for gas and electric usage, using normalized weather data from 1997 through 2008, the period for which reliable data was available for all weather stations.

Using the models with the closest fit between the normalized weather data and the billing data (i.e., an explained variance R2 of at least 0.70 or better) and that had positive relationships with either seasonal heating or cooling usage, the team developed various adjustment factors using a scatterplot with the x-axis representing Beacon estimated usage and the y-axis representing PRISM estimated usage. A regression trendline was fitted to the data, and the slope of that line serves as the adjustment factor for PRISM and UDRH estimates of energy usage. Multifamily buildings remained in the analysis if every unit in the building had 12 months of usable billing data, a strong R2, and a positive relationship with either heating or cooling. The PRISM



estimated cooling or heating usage was summed across units in the building to provide a comparable value to the Beacon estimated usage, which covered the entire building.

Although multiple adjustments factors were calculated based on various subsets of the models, the team recommends that the sponsors apply a heating adjustment factor of 0.734 (with a 90% confidence interval of 0.693 to 0.775) to the estimates of energy usage in ENERGY STAR and UDRH homes. This factor was developed from the subset of 91 PRISM models for the 87 single-family homes and the four multifamily buildings with an R2 of 0.90 or better (that is models with very high explained variance) and that were not identified to be outliers based on the percentage difference in their PRISM and Beacon estimated energy usage. The recommended factor reduces claimed energy savings by 26.6%. The team also calculated two cooling adjustment factors, but they were based only on 16 single-family homes, and the results indicated such dramatically different estimated usage between PRISM and Beacon that we believe they are not reliable. Therefore, we do not provide a recommended adjustment factor for cooling.

2 Introduction This report summarizes the procedures, methods, and results of a PRISM billing analysis of heating and cooling in homes built through the Massachusetts New Homes with ENERGY STAR® Program (the program) in 2006 and 2007. It provides adjustment factors for both the User-defined Reference Home (UDRH) and the as-built ENERGY STAR-qualified home that the program sponsors could apply to Beacon software-based estimates of energy usage in order to calculate energy savings from the program. Beacon is the software program currently used by ICF International, the implementation contractor, to estimate energy usage and savings in ENERGY STAR Homes. The billing analysis team is lead by Nexus Market Research (NMR). M. Sami Khawaja and Matei Perussi of The Cadmus Group (Cadmus) provided guidance and training to NMR on the PRISM analysis; Dorothy Conant advised the team on various aspects of the project.

2.1 Key Findings: Adjustment Factors Adjustment factors were calculated using two different approaches. The first approach fits a regression trend line to a scatterplot; the slope of that line serves as the estimated adjustment factor; the 90% confidence interval around the estimate is also reported. The second approach produces an adjustment factor which is the ratio of the sum of the PRISM estimated usage across all addresses, to the sum of estimated usage across all addresses using Beacon. In both cases, the closer the adjustment factor is to one, the closer the PRISM estimates of usage align with Beacon estimates. The same adjustment factors are expected to be applied to both the UDRH and as-built homes because there are no billing data for UDRH homes.1

1 The UDRH is a hypothetical baseline home. The features of the UDRH home represent the typical or average features installed in newly built homes in Massachusetts as identified in the 2005 baseline study of new homes in Massachusetts.



As Table 1 shows, all of the adjustment factors calculated for heating suggest that Beacon overestimates heating usage (i.e., adjustment factors are less than one). The team recommends that the sponsors use the adjustment factor of 0.734, associated with the sample for which there is a robust PRISM model fit (R2 ≥ 0.90) and that excludes outlying buildings.2 This adjustment factor also has the highest explained variance of the scatterplot estimates and closely adheres to criteria used in the previous study for inclusion of buildings in the analysis.3 In contrast, the PRISM based analysis fails to provide a reasonable adjustment factor for cooling. PRISM has difficulty identifying relatively low electric cooling use, and the analysis of data for homes with central air conditioning suggests most homes use central air conditioning on an as needed basis rather than using it consistently throughout the summer. Therefore, PRISM calculates unreliable estimates of cooling usage.

Table 1: Heating and Cooling Adjustment Factors for Energy Use

Energy Use

Group of Buildings Used n

Adjustment Factors by Approach Scatterplot Ratio

Estimate SEa 90% Confidence

Intervalb Estimate Low High

Heating

PRISM R2 ≥ 0.70 123 0.742 0.031 0.691 0.793 0.764

PRISM R2 ≥ 0.70, no outliers 117 0.686 0.023 0.648 0.724 0.712

PRISM R2 ≥ 0.90 96 0.783 0.032 0.730 0.836 0.808


Cooling PRISM R2 ≥ 0.70 16 7.881 1.698 5.088 10.674 7.112


a SE = standard error b The z-score for the 90% confidence interval is 1.645. The confidence interval is estimate ± z x SE.

It is also important to note that the adjustment factors developed using both approaches are similar for all heating-related groupings of buildings. However, the two approaches diverge substantially for cooling-related factors, particularly when outliers are removed. Since the adjustment factor estimates based on the scatterplot approach have less sensitivity to large error

2 Outliers were determined by using a box plot (see description on page 12) of the percentage difference between PRISM and BEACON estimates. The values considered outliers ranged from -57% to -97%. 3 Harley, Bruce of Conservation Services Group (2002) Summary Findings for Energy Consumption Analysis of the ENERGY STAR® Homes Program for the Joint Management Committee. Submitted to William Blake of National Grid on June 15, 2002.



differences between the PRISM and Beacon estimates, the team recommends using the estimates derived from that approach over the ratio approach.4

It is not recommended that the sponsors use any of the calculated adjustment factors for cooling. Given that both the previous and current studies encountered similar difficulties using PRISM for estimating cooling usage in Massachusetts, the sponsors may want to seek alternative methods for calculating cooling adjustment factors.

The data presented in Table 2 show the impact of applying the scatterplot adjustment factor from PRISM models with an R2 of 0.90 or better without outliers. Applying the factor reduces estimated usage proportionately for both the UDRH and as-built home, and the overall claimed savings also decrease proportionately. A similar example for cooling is not presented because the recommendation is not to use the estimated cooling adjustment factors.

Table 2: Effect of Adjustment Factors on Energy Savings Estimates Beacon Estimate Example Heat

Usage in Therms PRISM

Multiplier Adjusted Usage

UDRH 1000 0.734 734 As Built Home 800 0.734 587 Savings 200 0.734 147

3 Summary of Sample Development and Final Sample Size As shown in Table 3 on the next page, the NMR team requested billing data for 464 ENERGY STAR-certified home addresses from the sponsors, comparable to the request made in the previous study in 2002.5 The request covered billing data for the addresses listed in participant files provided by the program implementation contractors for homes certified in 2006 and from January through March of 2007 under the new regulations mandating completion of the Thermal Bypass Checklist for all certified homes and duct leakage testing in specific situations. For multifamily buildings, the billing data requests sometimes specified individual units and, in other cases, covered the entire building due to inadequate information on the number of units in the building or how they are designated (e.g., letter, numbers, etc.).

Overall, the sponsors delivered data for 320 gas and 524 electric addresses (see Table 3 for more detail). The greater number of electric records reflects the fact that some homes are heated with fuel oil, propane, or electricity and that gas is more frequently master-metered than is electricity.

4 In the ratio approach, large errors become incorporated into the estimate and can greatly affect the ratio in the same manner that outliers can have a strong influence on an average. While large errors also influence the results of the scatterplot approach, the methodology of fitting a line minimizes error across all data points as much as possible, therefore reducing the impact that individual errors have on the estimate. 5 Harley (2002).



The team received more records in response to our request than did the previous study, particularly natural gas addresses.6

After completing the data cleaning and analyses described in detail in Chapter 4, a total of 123 addresses were used to develop a heating-adjustment factor that the sponsors may apply to the Beacon estimates of energy usage. This includes 116 single-family homes (split nearly evenly between attached [SFA] and detached [SFD] homes) and seven multifamily buildings (all with four or fewer units); the previous study had numerically and proportionately more multifamily buildings in its 2002 sample, although all the multifamily buildings in that study were also small.7 All but one of the buildings used to develop adjustment factors in the current study is heated with natural gas. No records from 2006 were used because ICF has not run the 2006 projects through Beacon and it was determined that too few would pass the cleaning procedures to warrant this additional step on a cost-effectiveness basis. It is important to note that the current study treated each unit in a multifamily building individually in the data cleaning and PRISM analysis until the calculation of the adjustment factors. Only then did we group individual multifamily units together to represent the entire building. The seven multifamily buildings referenced in Table 3 reflect the number remaining in the calculation of the adjustment factor after aggregating individual units into whole buildings.

The team also developed two adjustment factors for electric cooling based on 16 homes (split evenly between SFA and SFD). The previous study’s cooling adjustment factor was based on data from 105 buildings. The greater number of buildings in the previous study may result from their inclusion of multifamily buildings for which they had data only on some units. As discussed in Section 10 below, the current study only included multifamily buildings for which we had data on all units, which excluded all multifamily buildings with five or more units as well as some of the buildings with four or fewer units. The greater number of buildings in the previous study may also possibly reflect that the buildings were slightly older than those in our sample, and, therefore, more likely to have all units occupied.

6 It is possible that part of the difference can be explained by the previous study counting individual buildings and the current study counting individual units. 7 The greater number of multifamily buildings in the previous study may reflect the different way in which the researchers handled multifamily buildings for which they did not have data for all units.



Table 3: Records Requested, Received, and Used in Analysis Previous Study Current Study Address Requests Submitted 439 464a

Addresses Received 319 634b

Gas 82 320 Electricity 300 524c

Buildings Used to Develop Heating Adjustment 45 123 Single family buildings 15 116 Multifamily buildings 30 7

Buildings Used to Develop Cooling Adjustment 105 16 a For some multifamily buildings, the request covered individual units, while, for others, it included the entire building because the lists lacked information on the number of units in the building or complex. b The total includes 210 houses/units for which the team received both gas and electric data. Some records are for individual units in multifamily buildings while others represent the entire master-metered building. c The sponsors provided more electricity addresses than requested due to inexact information regarding the number of units in multifamily and single-family attached developments.

4 Data Cleaning Procedures Prior to PRISM Analysis After receiving the billing data from the sponsors, the NMR team cleaned the data to prepare it for use in PRISM.

4.1 Matching billing data to requested addresses The first task involved matching the addresses received to those listed in the data requests to the sponsors. Most single-family homes—both detached and attached—matched those in our requests, although occasionally the addresses contained unexpected lot numbers, unit designations, or separate accounts for lighting. Although the original lists often named only the entire multifamily building, the sponsors usually provided data for individual units, as anticipated. It was not always clear, however, if the sponsors had sent data for all units in the building, even after reviewing the number of individual files ICF used in the Beacon program to determine a building’s ENERGY STAR status. Ultimately, a building entered the calculation of the adjustment factor only if it was very clear that the entire building (or groups of units) was included on the same meter or if the number of units with usable data matched the number of units included in REM/Rate files.

4.2 Identifying problematic properties or months of data Matei Perussi of Cadmus, an advisor to NMR on this project, worked with the NMR team to identify potential problems or outliers in the billing data, looking at both individual months of data as well as the entire set of billing records for a particular address. When a solution existed the team fixed the problem (e.g., averaged data across estimated and actual readings). If no solution existed, the team excluded the problematic month or address.

Common problems found in the billing data include the following: • Months with very little usage, suggesting a vacancy • Months with very high usage, suggesting either a commercial space or usage during the

building’s construction



• Readings taken more or less frequently than monthly (e.g., 14 days or 60 days) • Duplicate meter readings • Estimated meter readings • Public or other unoccupied spaces • Possible presence (based on the pattern of the billing records) of other equipment that

could affect readings (e.g., swimming pool heaters or pumps)

The exact reason for the problems and the solutions to them varied based on the context of the reading or other information on the building, as discussed in the following sections.

4.2.1 Months with low usage By far the most common problem with the billing data involved usage so low relative to other months that it suggested the house or unit was vacant during a single month or multiple months. For electricity, all months with zero usage and months with low usage relative to other months while taking the season into account were considered to be possible vacancies.8 For gas, the team allowed zero usage in summer months but not in winter months; otherwise, gas vacancies were identified based on relative usage compared to other months and taking the season into account.

Many of these vacancies occurred during the earliest months for which billing data were available for the property—most likely before anyone moved into the newly completed house or unit. A few vacancies, however, occurred in the middle of the period with available billing data. While most of these likely point to temporary vacancies associated with vacations or the seasonal use of homes, detailed examination of the data also uncovered addresses for which earlier usage occurred during the construction period and then the address became vacant prior to occupancy.

The solution for all potential vacancies was to remove them from the PRISM analysis, using standard PRISM coding procedure (that is, denoting usage as -1). Section 6.1 discusses the role of vacancies in including or excluding addresses from the calculation of the adjustment factor.

4.2.2 Months with high usage The research team found few cases of higher than usual usage for particular addresses, relative to other months of data. The few cases pointed to greater usage early in the period with available billing data, probably during a construction phase, followed by lower usage following the completion and occupation of the building. In such situations, these high usage months were excluded from the analysis using the same standard PRISM procedure of denoting usage as -1 in the PRISM files.

Some addresses had higher usage than what one would typically expect for all months of billing data available. While flagged as potential commercial properties, closer examination revealed that most of these addresses were either master-metered buildings or large single-family homes. They remained in the analysis, assuming they met all other criteria for inclusion.

8 The decision of whether or not to classify a month as low-usage followed no exact rules but instead depended on the unique circumstance of each address and the judgment of the researchers.



4.2.3 Other issues with monthly billing data The other issues—and their solutions—were more straightforward than those involving unusually high or low usage. In particular, this study addressed them in the following ways:

• Combined subsequent readings for two adjacent months when neither covered an entire month – for example, if one reading spanned 20 days and the following reading 10 days, the two were readings were combined into one monthly reading of 30 days

• Combined a short reading (less than 30 days) with a longer one (more than 30 days) and averaged the results between them when a short and long reading appeared in consecutive months

• Proportioned readings when they covered more than one month (e.g., 60 days) to estimate usage for individual months

• Kept one reading in cases where the sponsors sent duplicate readings • Summed the average daily consumption when two estimated readings (usually one short

and one long estimate) appeared in consecutive months and averaged the results between the two months

• Flagged all other problem months or potentially problematic properties (e.g., public spaces, homes with pools) for further review when developing the adjustment factor

5 PRISM Analysis Procedures PRISM estimates weather adjusted heating and/or cooling usage and overall usage for a home through a model using both billing data and actual weather conditions. Therefore, in order to run PRISM, the team first had to match buildings to their nearest weather station. This was accomplished using a mapping provided by Cadmus of all Massachusetts zip codes to the nearest weather station. The Beacon software, like the REM/Rate® software used in the previous study, uses hourly temperature data normalized over a 30 year period. The National Climate Data Center (NCDC), the repository for weather station data, keeps records of actual and normalized hourly data only for larger weather stations located in cities. Therefore, in the Beacon analysis, ICF matches buildings to the weather station of the nearest New England city that has hourly data. In contrast, PRISM relies on the average daily temperature, which is available from smaller weather stations. Using these smaller, local stations allows one to pair buildings with stations closer to them, likely providing a more accurate estimate of temperature than stations located in cities further away.9 For this reason, the current study relied on daily temperature data for smaller weather stations downloaded from the NCDC website.10 The team downloaded data for the past 12 years (1997 through 2008) as data prior to 1997 were unreliable and often missing.

The temperature and billing data were prepared following standard PRISM procedures, namely creating separate temperature and billing files for each weather station. Likewise, separate files were created for natural gas and electricity data. The team then ran the data in PRISM for each

9 Note that the use of different weather stations serves as one potential source of difference between PRISM and Beacon estimates of energy usage. 10 National Climate Data Center National Climate Data On-line. Available at http://www7.ncdc.noaa.gov/CDO/cdoselect.cmd?datasetabbv=GSOD&countryabbv=&georegionabbv=



weather station and fuel type, specifying that PRISM normalize the weather based on temperatures reported from June 1, 1997 through May 31, 2008.

PRISM has three different types of models: heating only, cooling only, or a combined heating/ cooling model. Each model estimates the weather Normalized Annual Consumption (NAC). The full equation for calculating NAC and the associated heating and cooling components of usage from the PRISM model based on α and β parameters is as follows:

where:

α = base-level consumption per day δh = 1 (for heating-only and heating/cooling) models δc = 1 (for cooling-only and heating/cooling) models β = slope (specified as either heating or cooling per degree day) H0 (τh) = Normal (12 year average in our case) annual heating degree days at heating reference temperature τh C0 (τc) = Normal (12 year average in our case) annual cooling degree days at cooling reference temperature τc

In heating-only models the cooling portion of the equation is set to zero, and in the cooling-only models the heating portion of the equation is set to zero.

PRISM allows the user to specify that it run one of the three models; alternatively, the user can choose to run an “automated” model and let PRISM pick which of the three models—heating, cooling, or heating/cooling—best fits the data. The team ran automated models for both natural gas and electric data. As expected, PRISM always chose a heating model for the natural gas accounts. However, although the Beacon files or data supplied by the sponsors listed only 17 homes with any electric heat, PRISM chose a mixture of heating and cooling (but never a mixed model) for the electric addresses. The team also forced PRISM to run the electric data with a cooling model.

6 Criteria for Inclusion in Adjustment Factor Calculation After running the PRISM models, the team further reviewed the billing data, Beacon files, and PRISM results to select the buildings that entered our calculations of adjustment factors. As shown in Table 3 (Section 3, page 4), applying the criteria discussed below meant that a total of 123 buildings entered the calculation of the heating adjustment factors (59 SFA, 57 SFD, and seven multifamily buildings) and 16 homes (split evenly between SFA and SFD) entered the calculation of the cooling adjustment factors.

6.1 Billing-data reasons for exclusion from adjustment factor The lack of 12 or more usable months of data for the entire building (i.e., a single-family home or every unit in a multifamily building that was grouped together in one REM/Rate file to determine ENERGY STAR status) serves as the most common billing-data reason for removing



buildings from the calculation of adjustment factors. Most of these records were excluded because the houses and individual units in multifamily buildings had not been occupied for at least 12 months by the time the team requested the billing data. For others, the sponsors provided 12 or more months of billing data, but the removal of individual months due to vacancies or other reasons meant that 12 months of usable billing data were not available.

Vacancies serve as the second most common reason for removal of buildings from the calculation of the adjustment factors. PRISM does not develop reliable models if dramatic breaks exist in the period of billing data entered into the program; furthermore, it needs representation of billing data from most months of the year in order to normalize the billing data reliably. For these reasons, a building was removed from the analysis if two or more months of vacancies occurred in the middle of a period for which billing data were available and the remaining data did not provide a full year cycle for the building.11

Finally, buildings were removed if the usage as determined by the billing data was unusually high and the team could not verify that it represented a master-metered building. This applied to very few buildings, and was most often associated with senior citizen communities or multifamily housing complexes, suggesting that the reading included public spaces such as lobbies, hallways, or recreation facilities.

6.2 Beacon File reasons for exclusion from adjustment factor The team removed records if ICF had not run Beacon estimates of usage. Most buildings excluded for this reason were completed in 2006 before ICF became the implementation contractor for the program. After reviewing the billing data and PRISM results for buildings completed in 2006, the team found that many of them would be excluded from the adjustment factor calculation for other reasons discussed in this section. Therefore, it was not cost effective to have ICF produce Beacon usage estimates for projects completed prior to their taking over as implementation contractor for the few usable buildings it would provide.

6.3 PRISM reasons for exclusion from adjustment factor There were two PRISM-related reasons why buildings may have been excluded from the analysis: 1) if the explained variance (R2) of the PRISM model for the building was less than 0.70, and 2) if the model indicated a negative relationship (i.e., β < 0) with heating or cooling data, depending on the model under consideration. When a model has a low R2 or negative relationship with heating or cooling data, PRISM is essentially reporting that it cannot find seasonal changes in usage. In contrast, R2 of 0.70 or higher and a positive slope with heating or cooling data typically serve as strong predictors of weather normalized heating or cooling usage for the home or unit.

11 For example, assume the team had 16 months of data beginning in November 2006 and running through February 2008, but data from December 2006 to February 2007 were excluded due to vacancy. In this situation, this study would keep the address in if the billing data for December 2007 to February 2008 were good but would exclude the address if December 2007 to February 2008 also indicated a vacancy.



6.4 Handling of multifamily buildings Since individual multifamily units had been occupied for different amounts of time, the PRISM models were run for each unit in multifamily buildings separately. Then the team summed the individual weather adjusted estimates of heating or cooling usage for all the units in the building to arrive at a whole-building estimate of usage for comparison to Beacon, which estimates usage for the entire building. The individual units all still had to adhere to the other criteria listed in this section. Only seven multifamily buildings—all with four or fewer units—met the full set of criteria for inclusion in the calculation of heating adjustment factors, and no multifamily buildings met the criteria for inclusion in the cooling adjustment factors.

7 Results: Adjustment Factors In order to compare the PRISM results to the Beacon estimates of energy usage for heating and cooling, the team converted Beacon heating and cooling usage from MMBTUs to therms for heating and kWh for cooling.12 The cooling estimate included both cooling usage due to ventilation and the use of other cooling equipment (e.g., air conditioners, heat pumps) in the buildings. The billing data for Unitil Gas and Berkshire Gas were converted from ccf to therms.

The team then calculated two types of adjustment factors, both of which replicate the efforts of the previous study in 2002. In the first, a standard scatterplot was developed using Excel, with Beacon estimates of yearly heating or cooling usage as the horizontal ‘x’ axis and PRISM estimates of yearly, weather adjusted heating or cooling usage as the vertical ‘y’ axis. The team fit a linear regression trend line to the scatterplot with the following equation:

where y = PRISM estimated usage, m = adjustment factor, and x = Beacon estimated usage. As in most linear equations, b is the y-intercept, with the y-intercept term set equal to zero in all of the equations, assuming that both methods would identify zero usage for the same buildings. To adjust the Beacon data, one multiplies the Beacon estimated usage by the adjustment factor. The previous study used a very similar method with one exception: the current study examined adjustment factors for PRISM models at both R2 = 0.70 and R2 =0.90, while the previous study only included the latter. We also calculated the 90% confidence factor for these estimates, presented earlier in Table 1 (Section 2.1, page 2).

The second type of adjustment factor is based on the ratio of the sum of PRISM estimated heating or cooling usage to the sum of the Beacon estimated heating or cooling usage across all buildings included in the calculation. This is the method used in the previous study to develop a cooling adjustment because the scatterplot approach failed to provide a reliable, usable estimate (the same is true in the current study as reported below in Section 8).13

12 Million BTUs; the study references MMBTU instead of MBTU to avoid confusion, as some people use MBTU to refer to 1,000 BTUs and others to 1,000,000 BTUs. 13 The adjustment factor developed in the previous study using the scatterplot approach resulted in a negative R2 indicating that the model was a very poor fit to the data.



7.1 Heating Adjustment Factors The team calculated numerous heating adjustment factors based on different groups of buildings. All factors are presented below (ranging 0.684 to 0.808) suggesting that Beacon consistently overestimates usage compared to PRISM. This differs from the previous report where the adjustment factor of 1.01 pointed to a slight overestimate by REM/Rate. The differences in the estimated adjustment factors between the two studies may reflect a number of circumstances discussed in more detail in Section 10, but summarized here as follows:

• Potential differences in the predictions of energy usage between the REM/Rate software (used in the previous study) and the Beacon software (used in the current study)

• Different approaches to handling multifamily buildings for which data were not available for all units that resulted in a much larger proportion and number of multifamily buildings in the previous study compared to the current one, which may, in turn, affect the adjustment factors

• Divergent treatment of outliers: the previous study applied a weighting scheme to them, while the current study typically excluded them

• Exclusion of buildings because of estimated readings: this occurred in the previous study but estimated readings were not a problem in the current study

• Estimation of a separate adjustment factor for the UDRH was possible in the previous study because they had access to the information needed to run REM/Rate predictions of energy usage on a sample of baseline homes; the same was not true for the current study as a recent baseline study was not designed to collect the data needed to run REM/Rate or Beacon on the inspected baseline homes



The first two adjustment factors we calculated and present in this paragraph include all 123 buildings (122 heated with natural gas, one with electricity) with a PRISM R2 of 0.70 or higher. The first adjustment factor is 0.742 using the scatterplot approach (Figure 7-1), and the second is a comparable 0.764 using the ratio approach. In other words, PRISM suggests that heating-related usage in these homes is about 75% of what Beacon estimates it to be.

Figure 7-1: Heating Adjustment Factor – All Buildings Includeda (PRISM R2 ≥ 0.70)

y = 0.742x

0

500

1000

1500

2000

2500

0 500 1000 1500 2000

PRISM Heating Usage (therms)

Beacon Heating Usage (therms)

a n = 123: 7 multifamily, 59 SFA, 57 SFD



The next adjustment factors we calculated exclude six buildings (all SFD homes) found to be outliers using the box plot approach on the percentage difference between the PRISM and Beacon estimates.14 The resulting adjustment factor using the scatterplot approach is reduced to 0.686 (Figure 7-2), and using the ratio approach is reduced to 0.712.

Figure 7-2: Heating Adjustment Factor – Outliers Excludedab (PRISM R2 ≥ 0.70)

y = 0.686x

0

200

400

600

800

1000

1200

1400

1600

0 500 1000 1500 2000



a n = 117: 7 multifamily, 59 SFA, 51 SFD b Outliers based on percentage difference between PRISM and Beacon estimates

14 The box plot is a commonly used approach in statistics of graphically depicting groups of numerical data through their five-number summaries (the smallest observation, lower quartile (Q1), median (Q2), upper quartile (Q3), and largest observation). It also indicates which observations, if any, might be considered outliers based on the five-number summary. See http://www.netmba.com/statistics/plot/box/ for a discussion. In the heating models, the box plot always identified -57% or worse as outliers on the percentage difference between PRISM and Beacon.


http://en.wikipedia.org/wiki/Five-number_summary

http://en.wikipedia.org/wiki/Quartile

http://en.wikipedia.org/wiki/Median

http://en.wikipedia.org/wiki/Quartile

http://en.wikipedia.org/wiki/Outlier

http://www.netmba.com/statistics/plot/box/


We then developed adjustment factors using the identical criteria from the previous study where the PRISM models with an R2 of 0.90 or better are included in the calculation of adjustment factors. The 96 buildings meeting these criteria produce an adjustment factor of 0.783 using the scatterplot approach (Figure 7-3), and 0.808 using the ratio approach.

Figure 7-3: Heating Adjustment Factor – Buildings Strongest PRISM Modelsa (PRISM R2 ≥ 0.90)

y = 0.783x

0

500

1000

1500

2000

0 500 1000 1500



a n = 96: 4 multifamily, 45 SFA, 47 SFD



Similarly, the team also created two adjustment factors for buildings with a PRISM R2 of 0.90 or better that excluded outliers. The scatterplot approach yields an adjustment of 0.734 (Figure 7-4), while the ratio approach gives an estimate of 0.761. The team recommends the sponsors use this scatter plot adjustment factor of 0.7341.

Figure 7-4: Heating Adjustment Factor – Strongest PRISM Models and Outliers Excludeda

(PRISM R2 ≥ 0.90)

y = 0.734x

0

200

400

600

800

1000

1200

1400

1600

0 500 1000 1500



a n = 91: 4 multifamily, 45 SFA, 42 SFD b Outliers based on percentage difference between PRISM and Beacon estimates

Here it is important to note that the scatter plot approach is also known as simple linear regression. Because we have set the y-intercept equal to zero, one is not able to calculate reliable explained variances (R2) for the equations for statistical reasons related to using the uncorrected sums of squares.15 However, two commonly used statistical packages (Stata and SPSS) reported the explained variance of this model to be 0.902, although each package warns that this explained variance is unreliable and should not be interpreted as providing a description of the fit of the model to the data. Therefore, to aid in interpretation of how well the recommended adjustment factor explains the relationship between PRISM and Beacon estimates, we also ran the model allowing the intercept to vary from zero. The explained variance of this model is 0.453, likely a better description of how well the model fits the data.

15 See http://www.minitab.com/support/answers/answer.aspx?ID=1037


http://www.minitab.com/support/answers/answer.aspx?ID=1037


The team also calculated separate adjustment factors for the one-half of the homes that Beacon predicts has the lowest usage (i.e., below 750 therms) and the second-half of the homes that Beacon predicts has the highest usage (i.e., 750 therms or more). Although the fit of the adjustment factors varied by consumption level, the adjustment factors were not substantially different from the recommended adjustment factor of 0.734. Thus, this shows that the adjustment factors can be applied at any consumption level to adjust the Beacon heating usages.



8 Cooling Adjustment Factors The team only calculated four cooling adjustment factors because just 16 buildings met the criteria for inclusion in the development of these factors. In addition, the results based on these 16 buildings show a very large gap between the PRISM estimates of cooling usage and the Beacon estimates. PRISM has difficulty identifying relatively low electric cooling use, and the analysis of data for homes with central air conditioning suggests most homes use central air conditioning on an as needed basis rather than using it consistently throughout the summer.

In particular, a cooling adjustment factor of 7.889 resulted from using the scatterplot method (Figure 8-1), and 7.112 using the ratio approach. In other words, the PRISM results suggest that cooling usage is between seven and eight times higher than Beacon estimates it to be, a difference which we believe to be highly unlikely and, instead, probably reflects how confused the PRISM program is by the relative low cooling usage in New England. It is also worth noting the differences in the scales for the x- and y-axes; while the x-axis runs from zero to 2,000, the y-axis runs from zero to 30,000. Visually, it appears that the differences between the PRISM and Beacon estimates of usage are small, but this is a trick of eye played by the scales. In reality, the estimates differ a great deal.

Figure 8-1: Cooling Adjustment Factor – All Buildings Includeda

(PRISM R2 ≥ 0.70)

y = 7.889x

0

5,000

10,000

15,000

20,000

25,000

30,000

0 500 1000 1500 2000

PRISM Cooling in kWh

Beacon Cooling in kWh (Ventilation and Other)

a n = 16, 8 SFA and 8 SFD



We also calculated the cooling adjustment, removing the single outlier for which PRISM predicts usage of over 25,000 kWh while Beacon predicted usage of about 1,700 kWh. This action reduced the scatterplot adjustment factor to 2.985 and the ratio adjustment factor to 4.781, still leading to results that suggest that PRISM and Beacon are far apart in their estimates of cooling-related usage (Figure 8-2).

Figure 8-2: Cooling Adjustment Factor –Outlier Excludedabc

(PRISM R2 ≥ 0.70)

y = 2.985x

0

500

1,000

1,500

2,000

2,500

3,000

3,500

4,000

4,500

5,000

0 200 400 600 800 1000 1200 1400 1600

PRISM Cooling in kWh

Beacon Cooling in kWh (Ventilation and Other)

a n = 15, 8 SFA, 7 SFD b Outlier identified by extremely high PRISM estimated usage

Furthermore, these estimates differ tremendously from the more reasonable adjustment factor of 0.779 in the previous study, which was developed using the ratio approach due to the poor fit of the scatterplot trend line to the data.16 The team is not at all confident in the reliability and usefulness of the cooling adjustment factors developed in the current study as both rely on what may be considered to be very poor PRISM estimates of cooling usage.

9 Potential Reasons for Differences from Beacon We identified two potential reasons—beyond estimation error in Beacon or PRISM or unexpected customer behavior—to explain why PRISM estimates of usage for individual homes

16 Had the previous study used the adjustment factor developed from the regression approach, it would have been 0.6901. The current team has concerns not so much about the ratio approach but more about using the faulty PRISM data for either the scatterplot or ratio approach. If the PRISM data are poor predictors of cooling usage, they will be poor predictors in either approach.



differ from Beacon estimates. The reasons do not directly explain the size of the adjustment factor, although the error in each individual estimate influences the calculated factor.

First, as mentioned above, while both models use normalized weather data, the NMR team used the nearest weather station with daily data. Beacon requires hourly data, which are available only for larger weather stations that are not always the nearest station to the building under consideration. Therefore, the weather station used in the current analysis could lead to divergent PRISM and Beacon estimates. PRISM likely provides the more accurate estimates because the temperature reading is usually taken closer to the building.

Second, although PRISM calculates base load energy use and heating or cooling use for the fuel under consideration, it cannot distinguish between seasonal usage for heating or cooling and other seasonal use of the fuel (e.g., increased winter natural gas use for water heating; summer electricity usage for pool heaters). Therefore, PRISM would likely count these uses as heating or cooling when, in reality, they reflect some other use of the fuel. Note that this situation applies to buildings for which PRISM overestimates summer cooling or winter heating usage, a situation that in the current study did not frequently occur in the heating data but occurred in the cooling data.

10 Variation from the Previous Study Approach The NMR team largely replicated the methods used in the previous study. However, our study differs from the previous study in a number of ways that could affect the results. Although it is not possible to say whether these factors move the current results closer to or further away from Beacon estimates of usage, they represent important deviations from the methods used in the previous study that may explain why the adjustment factors presented here are greater than those found in that previous study.

First, and perhaps most importantly, the current study compared PRISM results to Beacon predictions of energy usage; the previous study compared PRISM results to REM/Rate predicted usage. The greater adjustment factors found in the current study may be an artifact of the differences between Beacon and REM/Rate usage estimates.



Second, the two studies treated multifamily records differently. The previous study ran PRISM for individual units in multifamily buildings and then summed them into a whole building estimate that matched the REM/Rate files. The current team followed this same basic approach. However, the studies differ on handling buildings for which billing data were available for only some units. While the current study excluded these buildings from the analysis, the previous study took the following approach:

“If PRISM results were not obtained for all the accounts in a multi-family building k, the sum of the PRISM results was multiplied by

k

k

nN

,

where Nk = the number of accounts in multi-family building k and nk = the number of accounts in multi-family building k for which PRISM results were obtained.”17

This approach increased the number of multifamily buildings in the previous analysis—30 of 45 in the previous study versus seven of 123 in the current one, although it is important to note that both studies ultimately used only small multifamily buildings. The previous study also assumed that the units for which billing information was available were representative of the units without such information. The current team was not prepared to make this same assumption because the characteristics (e.g., size, layout, and features) of units occupied immediately after construction—or their residents—may not be comparable to those occupied at a later date. In the end, the previous study had a greater number and proportion of multifamily buildings in its heating analysis (30 of 45) than the current study did (seven of 123), which may have some influence on the results, although which direction is not clear. Together, these two divergent decisions mean that multifamily buildings have a much greater impact on the 2002 results than in the current study, and this may affect the adjustment factors.

Third, the current study also treated outlying months of billing data differently. The previous study applied weights (although the methodology was not explained in detail) to individual outlying billing records before running the final models. The current study excluded outlying months but included the buildings as long as they had 12 months of non-outlying and otherwise good billing data based on criteria described earlier in this report.

Fourth, estimated readings were very rare in the dataset for the current study and no buildings were excluded solely because they had too many estimated readings. Based on our reading of the 2002 report, it appears that some buildings were removed due to estimated readings.

The final issue relates only to the adjustment factor for the UDRH and not ENERGY STAR-qualified homes. The previous study was able to provide a separate adjustment factor for the UDRH because the researchers had REM/Rate files on homes that were inspected in the 2001 baseline study of new residential construction practices conducted for the Massachusetts Board of 17 Geopfrich (2002) Memorandum regarding PRISM Analysis. Submitted to Bruce Harley, Bill Blake, and John Livermore on February 13, 2002. Page 4.




e

Building Regulations and Standards (BBRS).18 The previous study was able to run PRISM models on billing data from the BBRS homes and compare the results to the REM/Rate results. Then thprevious study compared the REM/Rate and UDRH predictions of energy usage for each home in the BBRS group. With these two pieces of information, a UDRH adjustment factor was developed.19 A similar approach could not be used in the current study because all the data needed to run REM/Rate or Beacon were not collected on the homes inspected in the 2005 baseline study. Given budget limitations, a conscious decision was made when designing the 2005 baseline study to collect less detailed information on individual homes in order to increase the number of homes inspected because ENERGY STAR certification requirements, the HERS index scoring approach, and the REM/Rate software would soon be changing.

11 Conclusion The NMR team recommends using the heating adjustment factor of 0.734, associated with buildings with the most robust PRISM estimates, excluding outliers. This factor should be applied to both UDRH and as-built ENERGY STAR homes. This estimate has the highest explained variance of all the scatterplot estimates and most closely adheres to criteria used in the previous study for inclusion of buildings in the adjustment factor. The impact of this estimate—and its application to both UDRH and as-built ENERGY STAR homes—will be to reduce program estimates of energy savings by about 27%.

NMR strongly recommends against using the cooling adjustment factors presented in this report. The factors are based on just 16 or fewer addresses, and the PRISM data that underlie both the scatterplot and ratio approaches do not appear to produce reliable estimates of actual cooling usage in Massachusetts. Furthermore, given that both the previous and current studies encountered similar difficulties using PRISM for estimating cooling usage in Massachusetts, the sponsors may want to seek alternative methods for calculating cooling adjustment factors.

It is also the case that the scatterplot and ratio approaches to calculating adjustment factors produce generally comparable results. However, it should be noted that the scatterplot approach and the application of a regression trend line to the data more explicitly corrects for large errors in each estimate than does the ratio method. Therefore, if feasible, the scatterplot approach should be used instead of the ratio approach, as it is methodologically and statistically superior. The ratio approach should be used only when the scatterplot approach is not reliable and when the data on which the ratio is based are of relatively good quality.

18 Impact Analysis of the Massachusetts 1998 Residential Energy Code Revisions, Xenergy Inc., May 14, 2001 19 For more information on this method see Harley (2002).

Appendix A-2

Updated Billing Analysis

Memorandum: Summary of Revised Billing Analysis Page 1


MEMORANDUM Date: March 19, 2009 To: Joint Management Committee From: Lisa Wilson-Wright (NMR), Lynn Hoefgen (NMR), Matei Perussi (Cadmus), and M. Sami Khawaja (Cadmus) Regarding: Possible Next Steps for the Massachusetts New Homes with ENERGY STAR® Billing Analysis and Adjustment Factor Calculation

The billing analysis team led by Nexus Market Research, Inc. (NMR) submitted the draft Report on PRISM Billing Analysis and Energy Usage Adjustment Factors Developed in Support of the Massachusetts New Homes with ENERGY STAR Program on November 7, 2008. The team received comments from four reviewers of the document. Similar to the team members, the reviewers voiced concern about the size of the adjustment factor which suggested that actual, PRISM weather normalized, heating usage in the sampled homes was approximately 25% lower than the Beacon estimates. Because the adjustment factor will be applied to both ENERGY STAR qualified homes and the User-defined Reference Home (UDRH), it will result in a net decrease in the energy savings claimed by the Massachusetts New Homes with ENERGY STAR Program (the program).

The reviewers provided recommendations that the team could explore in an effort to explain why PRISM estimates of actual usage differ from Beacon estimates and perhaps to develop a revised adjustment factor that would not reduce claimed savings by such a large amount.1 Specifically, a subset of the comments revolved around how PRISM-modeled heating usage may be affected by the role that occupant behavior and characteristics have on usage. Another set of comments questioned the impact of using the 12-year weather series (1996 to 2007) from smaller, local weather stations for the PRISM analysis instead of the 30-year weather series (1961 to 1990) from the larger weather stations used in the Beacon models.

In response to these comments, the billing analysis team has explored the potential implications of modeling occupant behavior and characteristics as a way of understanding actual energy consumption and, possibly, of revising the adjustment factor as a result. The team also has delved more deeply into the actual gas consumption, PRISM estimates of weather normalized heating and whole-house gas usage, Beacon estimates of heating and whole-house gas usage, and the weather data. Ultimately, another PRISM analysis was performed using weather data from 1973 to 2000, a less recent weather series than used in the first set of analyses but one that reflects the warming climate and allows for a longer normalization period. This memorandum summarizes the results of these updated analyses. Finally, in this memo, the team provides a list of actions for finalizing this billing analyses task and for moving forward.

1 Given the inability of PRISM to provide reasonable estimates of cooling usage in the New England climate, the team will not explore revising the billing analysis or adjustment factor for cooling.



Behavioral Component: The Impact of the Omitted Variable The billing analysis team advises against further research into the behavioral component for differences between PRISM and Beacon estimates of heating usage based upon mathematical and statistical simulations and theory. Specifically, in regression analysis, variable bias can arise from different types of errors. The relevant one for the current discussion is the omission of one or more explanatory variables from the regression equation. When relevant variables are omitted, ordinary least squares estimation can result in biased estimates of the model’s coefficients.

To understand how omitted variable bias can occur, suppose that a researcher is interested in knowing the relationship between the Beacon heating usage engineering estimate (EE) and normalized actual heating consumption (NAC). Suppose the “true” model for NAC is given by equation (1):

NAC = β1 * EE + β2 * BC + ε (1)

where BC is a behavioral component affecting NAC and ε is a random error term reflecting unobservable factors affecting NAC. β1 measures the true relationship between EE and NAC and is what we wish to estimate, and it also represents the adjustment factor one would arrive at with the BC factor in the equation.

With information about NAC, EE, and BC, one can estimate equation (1) and obtain an unbiased estimate of β1.

However, when the econometrician does not have information about all of the relevant variables in equation (1), least squares estimates can be biased. To see this, suppose that we are still interested in knowing the relationship between NAC and EE but the researcher has information about only NAC and EE. One cannot estimate equation (1), but instead estimates equation (2):

(2)

α1 is the coefficient to be estimated. This is the model used by the team and α1 represents the estimated adjustment factor. In general, the researcher would like to know under what conditions one can estimate equation (2) by ordinary least squares regression (OLS) and get an unbiased estimate of β1 by estimating α1.

Sparing readers of the mathematical derivation, the above can be transformed to the following equation:

(3)

where bR-B measures the correlation between EE and BC. If bR-B equals or is close to zero, OLS estimation leads to an unbiased estimate of β1 through α1. However, if bR-B is not equal to 0, that is, EE and BC are correlated, OLS estimation of equation (2) will produce a biased estimate of β1.



In English, if there is little or no relationship between the Beacon heating usage engineering estimate (EE) and the behavioral component (BC), then estimating the relationship between NAC and EE without having BC data, will NOT lead to a biased estimate of the correction factor.

There is no doubt that having the data on BC (correlated with EE or not) would improve the model’s explanatory power (higher R2), but adding this variable to the equation would likely do little to change the adjustment factor unless EE and BC were strongly correlated.

To illustrate this, the team ran numerical simulations to illustrate the effect of omitted variable bias on the estimation of a model’s parameters. The simulations vary the correlation between EE and BC to generate different levels of bias. When EE and BC are highly correlated and BC is omitted from the regression equation, omitted variable bias will be severe. Table 1 summarizes the results.

The team simulated two variables in these examples:

1. The BC variable was generated under various degrees of correlation with EE (i.e., several simulations of BC with different correlations to EE).

2. A new NAC such that its value depends on both β2 and EE. Its relationship to EE was created such that the correction factor was 0.734 (as in the original report)

The regression model simulated NAC against the actual EE with BC excluded, the procedure taken to arrive at the adjustment factor recommended in the original report. The difference now is that the team has created hypothetical example data for BC and can control whether it is or is not correlated to the EE actually used in building the model.



The first column in Table 1 shows the number of the model the team ran. The second column displays the “true parameter” value of the adjustment factor recommended in the first draft of this report. The hypothetical data used in these example models were created such that the 0.734 is the true value. Column 3 lists the different correlations between EE and the example value of BC used to simulate NAC. Column 4 shows the regression-based estimate of α1 in equation 2 in which the dependent variable is simulated NAC. Columns 5 and 6 measure the bias in the estimation of the effect of EE on NAC, where the bias is defined as |β1 - α1|. When BC and EE are highly correlated, the estimates of α 1 are quite different than β1. However, as the correlation decreases, the model provides estimates of β1 closer to the true value. Model 4 does not provide an exact estimate of 0.734 for α1 because of random error in the dependent variable.

Table 1: Model Simulation Summary

Model True Parameter:β1

Correlation Between EE and BCa

Estimate of α1 when BC omitted

Bias = |β1 - α1|

% Bias (Bias/β1)

1 0.734 0.99 0.804 0.070 10% 2 0.734 0.40 0.797 0.063 9% 3 0.734 0.20 0.776 0.042 6% 4 0.734 0.002 0.703 0.031 4% a Absolute value of the correlation, which can be positive or negative.

Use of Different Weather Series in Beacon and PRISM Beacon models estimated energy usage based on normalized hourly weather data, which are only available for weather stations located in moderate to large cities. Specifically, ICF reports that they used data from the Albany, Boston, Hartford, Providence, and Worcester weather stations and matched Massachusetts homes to the nearest of those cities when estimating their energy usage. Their models use the NOAA weather data normalization series period from 1961 to 1990.

In contrast, PRISM models base energy usage on normalized daily weather data that are available for smaller, often more local weather stations as well as larger ones. The billing analysis team opted to use the data from these smaller stations because it believed they more accurately described the conditions in many homes, rather than using more distant stations. The data for these smaller stations were available only from 1996 through 2007; prior to then, data for some stations were not available or were not reliable.

Reviewers to the draft report wondered how the analysis results would change if smaller local weather stations with data from a shorter time period were used instead. In order to examine this question, the team began by comparing the Massachusetts state average data on heating degree days (HDD) for the 1961 to 1990 series to that for the data from 1996 to 2007 for smaller stations as well as data from the yearly and entire time period for which we have billing records(Table 2). The table also shows the statewide average HDD for the long-term weather series from 1971 to 2000.



The data in Table 2 demonstrate that the state average number of HDD has generally decreased over time, even though the number varies year to year. Overall, HDD fell from a normalized 6,467 in the 1961 to 1990 series to 6,100 in the short-term 1996 to 2007 weather series and 6,079 for the time period for which we have billing data. The weather in 2006 was particularly warm, while that in 2007 and 2008 was more similar to the 1971 to 2000 time series. In short, the use of the different time series likely contributes to the fact that PRISM estimated lower heating-related usage than did Beacon. Note that the majority of the data in this study come from 2007 and early 2008, a somewhat cooler period than other recent years but warmer than either of the 30-year periods presented.

Table 2: Massachusetts State Average Heating Degree Days Weather Series Annual Heating Degree Days Long-term series: 1961 to 1990 6,467 Long-term series: 1971 to 2000 6,344 Long-term series: 1991 to 2005 6,215 Short-term series: 1996 to 2007 6,100 Annual 2006 5,709 Annual 2007 6,295 Annual 2008 6,232 Billing data period: 2006 to 2008 6,079

Heating Reference Temperature: An Additional Consideration In addition to reviewing the weather data, the team also examined the PRISM data in more detail. The team became concerned with the degree to which the heating reference temperature—known as tau—varied in the models. PRISM uses this reference temperature to determine when an individual home is heating. The analyst has three choices for tau:

1. Allow tau to vary and let PRISM select tau based on the billing data 2. Set tau to a specific temperature 3. Allow tau to vary within a set range

The team originally chose to let tau vary completely, but review of the data revealed that PRISM allowed tau to range from 3o Fahrenheit to 87o Fahrenheit, including models ultimately rejected from the development of the adjustment factor. The average was 55o, and the tau for most records ranged from 45o to 65o.

Next Steps Taken In response to the review of the weather and PRISM data, the analysis team including ICF took three related actions:

1) The billing analysis team ran PRISM using the same weather stations as ICF but using weather data from 1973 to 2000 (1971 and 1972 were not available for all stations)



2) The billing analysis team also ran PRISM using the new weather data but limiting tau to a range of 45 to 65.

3) ICF ran an analysis to assess the impact of using normalized weather data from 1991 to 2005 instead of data from 1961 to 1990 in Beacon models to estimate usage.

This section summarizes the results of each of these steps.

Revised PRISM Analyses Table 3 on the next page summarizes the previously developed adjustment factors and those resulting from the more recent PRISM analyses with new weather data and a fixed range for tau.2 The data are graphed in Figure 1 to Figure 8. In order to develop these new estimates, the billing analysis team downloaded 1973 to 2000 weather data from The National Climate Data Center (NCDC) website for the following weather stations used by ICF in their Beacon models:3

• Albany, NY • Boston, MA • Hartford, CT • Providence, RI • Worcester, MA

The team then assigned each home to a weather station using ICF’s matching method that links cities and towns to the nearest of the five weather stations. Using the same billing data as in the original report, the team ran PRISM models using the different weather series. They applied the same criteria for inclusion of homes in the development of the adjustment factor and produced the estimates listed in Table 3 as “1973 to 2000 weather series, floating tau.” The adjustment factors for these new analyses do not vary as much as those from the previous report, but they fall into a similar range: 0.734 to 0.775 for the recent analyses vs. 0.686 to 0.783 for the previous ones. Tau “floated” from 41o to 73o and averaged 57o. The team then fixed tau to between 45o and 65o and reran the models with the 1973 to 2000 weather series. These results varied from 0.716 to 0.774.

The shaded cells in Table 3 list the originally recommended adjustment factor and the comparable adjustment factors (i.e., models that achieved a PRISM R2 of at least 0.90 and outliers excluded). Relative to the other adjustment factors in the table, these models have moderate standard errors and the estimated factors fall on the higher end of the ranges. The team recommends that the sponsors use the shaded adjustment factor for the fixed-range tau, which improves the adjustment factor from 0.734 in the original report to 0.774 in the current one.

2 The sample sizes for the new analyses are somewhat smaller due to more homes and individual multifamily units not meeting inclusion criteria and, therefore, the team had to exclude the entire building. Furthermore, the team identified additional homes that had billing patterns indicating vacancies or the transition from being unoccupied to occupied. These also had to be excluded. 3 National Climate Data Center National Climate Data On-line. Available at http://www7.ncdc.noaa.gov/CDO/cdoselect.cmd?datasetabbv=GSOD&countryabbv=&georegionabbv=



Table 3: Adjustment Factors: Prior and Current Analysesa

Model Group of Buildings Used n Estimate SE 90% Confidence

Interval Low High

1997 to 2008 weather series, floating Tau

PRISM R2 ≥ 0.70 123 0.742 0.031 0.691 0.793PRISM R2 ≥ 0.70, no outliers 117 0.686 0.023 0.648 0.724PRISM R2 ≥ 0.90 96 0.783 0.032 0.730 0.836PRISM R2 ≥ 0.90, no outliers 91 0.734 0.025 0.693 0.775

1973 to 2000 weather series, floating tau


1973 to 2000 weather series, fixed range tau


a The first four rows of data list the prior adjustment factors from the November 7, 2008 report.



Figure 1: Heating Adjustment Factor – 1973 to 2000 Weather Series (PRISM R2 ≥ 0.70)

y = 0.734x

0

200

400

600

800

1000

1200

1400

1600

1800

2000

0 500 1000 1500 2000 2500




Figure 2: Heating Adjustment Factor – 1973 to 2000 Weather Series Outliers Excluded (PRISM R2 ≥ 0.70)

y = 0.750x

0

200

400

600

800

1000

1200

1400

1600

0 500 1000 1500 2000 2500






Figure 3: Heating Adjustment Factor – 1973 to 2000 Weather Series (PRISM R2 ≥ 0.90)

y = 0.765x

0

200

400

600

800

1000

1200

1400

1600

1800

2000

0 500 1000 1500 2000 2500




Figure 4: Heating Adjustment Factor – 1973 to 2000 Weather Series Outliers Excluded (PRISM R2 ≥ 0.90)

y = 0.775x

0

200

400

600

800

1000

1200

1400

1600

0 500 1000 1500 2000 2500






Figure 5: Heating Adjustment Factor – 1973 to 2000 Weather Data, Fixed-Range Tau (PRISM R2 ≥ 0.70)

y = 0.716x

0

200

400

600

800

1000

1200

1400

1600

1800

2000

0 500 1000 1500 2000 2500






Figure 6: Heating Adjustment Factor – 1973 to 2000 Weather Data, Outliers Excluded, Fixed-Range Tau

(PRISM R2 ≥ 0.70)

y = 0.749x

0

200

400

600

800

1000

1200

1400

1600

0 500 1000 1500 2000 2500






Figure 7: Heating Adjustment Factor – 1973 to 2000 Weather Data, Fixed-Range Tau (PRISM R2 ≥ 0.90)

y = 0.747x

0

200

400

600

800

1000

1200

1400

1600

1800

2000

0 500 1000 1500 2000 2500






Figure 8: Heating Adjustment Factor – 1973 to 2000 Weather Data, Outliers Excluded, Fixed-Range Tau

(PRISM R2 ≥ 0.90)

y = 0.774x

0

200

400

600

800

1000

1200

1400

1600

0 500 1000 1500 2000 2500




ICF Analysis Using 1991 to 2005 Weather Data ICF is better explain the additional analyses they conducted with more recent weather data, but they presented the billing analysis team with the results of one home. The results demonstrated that the Beacon simulated usage for the home using 1961 to 1990 data was four percent higher than actual, non-normalized usage while Beacon simulated usage for the home using 1991 to 2005 data was seven percent lower than actual usage. ICF concluded that the simulations from the two weather series were relatively close and that the earlier data fit the actual usage better; therefore, they did not feel the data warranted additional weather-series related analyses.

The billing analysis team interpreted these results somewhat differently. The simulated usages between the two weather series vary by 11% (4% to -7%) and point in opposite directions. Furthermore, as noted above, the billing data largely come from two cooler years (2007 and 2008) compared to the 1991 to 2005 or 1996 to 2007 time periods. ICF’s results turned out they way they did because the limited data points were from cooler years; when normalized to an even cooler period, the Beacon model will over-predict the actual usage; when normalized to a warmer period, the Beacon model will under-predict the actual usage. PRISM develops models based on the relationship of usage to HDD based on the actual weather during the billing data period and then normalizes it to weather from the time period specified in the model. As with



Beacon, the models are sensitive to differences between weather during the actual billing period and the normalization period.

Conclusions and Next Steps To summarize, the team is faced with a situation in which the billing data come from a time period that is warmer than the 1961 to 1990 normalization period used by Beacon, cooler than the 1997 to 2008 period used in our first PRISM runs, and somewhere in between the 1991 to 2005 weather period used in the new ICF analyses and the 1973 to 2000 period used in this team’s new analyses. The billing analysis team believes that the only way to bring Beacon and PRISM estimates closer together is to coordinate the weather series used in the normalization period. This team strongly recommends that this period be more recent, to take into account the warming of the climate, and likely savings in future years. The period can be either the 1973 to 2000 period or something more recent such as the 1991 to 2005 period.

The billing analyses team believes that it has met the scope of work as outlined in the work plan by providing billing analyses, estimating adjustment factors, and exploring reasons for differences between PRISM and Beacon estimates of usage. This memo provides an updated recommended adjustment factor of 0.774, which represents an improvement from 0.734 in the previous draft.

Table 4: Effect of Adjustment Factors on Energy Savings Estimates Adjustment Factor Beacon Estimate Example Heat

Usage in Therms PRISM

Multiplier Adjusted Usage

Original estimate UDRH 1000 0.734 734 As Built Home 800 0.734 587 Savings 200 0.734 147

Revised estimate UDRH 1000 0.774 774 As Built Home 800 0.774 619 Savings 200 0.774 155

The team is open to entering into a new contract for an analysis that will update the results of this study using the 1991 to 2005 weather data (or any period the sponsors choose) and a longer series of billing data, but this task is only worthwhile if the engineering estimates to which we will compare will also be simulated using the same weather normalization period.

Appendix B

Non-Energy Impact Analysis



Massachusetts New Homes with ENERGY STAR®

Assessment of Non-Energy Impacts

Final Report March 6, 2009

Submitted to: Joint Management Committee

Submitted by:

Nexus Market Research, Inc. Dorothy Conant


Table of Contents Executive Summary ........................................................................................................................ 1 1 Introduction ............................................................................................................................. 2 2 Methodology ........................................................................................................................... 3

2.1 Homeowner Survey ........................................................................................................ 3 2.2 Builder Interviews ........................................................................................................... 4

3 Perception of NEIs .................................................................................................................. 5 4 NEI Value Calculation .......................................................................................................... 12

4.1 Direct Scaling Approach ............................................................................................... 12 4.1.1 Homeowners ......................................................................................................... 12 4.1.2 Builders ................................................................................................................. 12

4.2 NEI Values .................................................................................................................... 13 5 Additional NEIs .................................................................................................................... 17 6 Conclusions and Recommendations ..................................................................................... 18

List of Tables

Table 3-1: Homeowners Who Say Home Provides NEIs ............................................................... 5 Table 3-2: Homeowners Who Say Home Provides NEIs by Spec and Custom Built Homes ........ 6 Table 3-3: Homeowners Who Say Home Provides Positive NEIs by Age of Previous Home ...... 6 Table 3-4: Builders Who Say ENERGY STAR Homes Provide NEIs .......................................... 7 Table 3-5: How Learned of NEI ..................................................................................................... 9 Table 3-6: How Often Discuss NEI with Homebuyers ................................................................ 10 Table 3-7: When Learned of NEI ................................................................................................. 11 Table 3-8: Did the NEI Influence Your Decision to Buy of Build an ENERGY STAR Home? . 11 Table 4-1: Mean NEI Values from the Homeowner Survey ........................................................ 13 Table 4-2: Mean NEI Values for Owners of Spec- and Custom-Built Homes ............................. 14 Table 4-3: Mean NEI Values from the Builder Interviews ........................................................... 15 Table 5-1: Additional NEIs Mentioned by Homeowners ............................................................. 17

List of Figures

Figure 3-1: Builders and Homeowners Who Say ENERGY STAR Homes Provide Positive NEIs................................................................................................................................................. 8

Figure 4-1: Builder and Homeowner Valuation of NEIs .............................................................. 16

Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 1

Executive Summary This study estimates the value of non-energy impacts (NEIs) resulting from the Massachusetts New Homes with ENERGY STAR® program using two sources: a survey of 70 homeowners who have purchased ENERGY STAR-certified homes over the past two years, and in-depth interviews with 30 builders who participated in the program in 2008. The homeowner survey is not intended to be representative of all ENERGY STAR homeowners since it included only those who knew they had purchased ENERGY STAR-certified homes. The NEIs, which had not been previously studied in connection with the Massachusetts program, are:




dust, pollen, humidity, and car exhaust • Safety of the new home’s heating system along with improved ventilation making the

home safer in terms of carbon monoxide levels and gas leaks • Anticipated ease of selling or leasing the home in the future and higher resale/leasing

value • Knowledge of some protection against rising energy prices; that is, as energy prices

increase, energy bill savings will also increase Homeowners overwhelmingly believe their new homes provide positive NEIs in thermal comfort (86%), a higher resale or rental value (80%), and protection against energy bill increases (93%). Majorities also believe their new homes provide noise reduction (67%) and better lighting features (61%), while fewer believe their homes provide better indoor air quality (51%) or more safety (46%). For all the NEIs studied, homeowners most often say they learned of the existence of an NEI before deciding to buy or build the home and that the presence of NEIs influenced their decision to buy or build an ENERGY STAR home. Builders are more likely than homeowners to believe ENERGY STAR homes have positive NEIs in the cases of thermal comfort, outdoor noise reduction, better indoor air quality, safety, and a higher resale or rental value. Builders are more likely to discuss thermal comfort than any other NEI with their customers, and homeowners are more likely to remember hearing about this NEI than any other NEI from the builder. While builders also claim they often discuss noise reduction, lighting, indoor air quality, resale or rental value, and energy bill protection with customers, fewer homeowners remember hearing about these NEIs from builders. Using a direct scaling approach, homeowners place the highest value on thermal comfort, which is almost equal to yearly energy bill savings. After thermal comfort, homeowners place the highest values on increased resale or rental value and energy bill protection—features they say they have largely figured out on their own. Taken together, homeowners value the seven NEIs


Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 2 more than annual savings on energy bills. Even if the survey, despite efforts to the contrary, had some double counting, a conservative estimate would put the value of the seven NEIs at about twice the amount of energy bill savings. The methodology used to calculate these savings is detailed in Chapters 2 and 4. Roughly similar NEI values obtained in 2006 from a survey of New York ENERGY STAR Labeled Home program participants. Overall, builder estimates of the values homeowners place on NEIs are higher than those provided by the homeowners. Builder NEI means are considerably higher than homeowner means for thermal comfort, noise reduction, indoor air quality, and safety. Estimates for resale or rental value are very close for the two groups and builders give considerably lower estimates for lighting features and energy bill protection. The study findings lead to the following recommendations:

• NEIs should be an integral part of marketing for ENERGY STAR homes. Builders should be especially encouraged to talk to prospective buyers about noise reduction, indoor air quality, and safety (or make sure their salespeople do so).

• The sponsors could consider sharing some of the results of this study with builders, in particular to show them the differences between home buyers’ ratings of NEI values compared to how builders think buyers would rate them.

1 Introduction This document presents estimates of non-energy impacts (NEIs) resulting from the Massachusetts New Homes with ENERGY STAR® program (program). NEIs estimation relies on two sources:

• A survey of 70 households who have purchased ENERGY STAR-certified homes over the past two years and have lived in their homes at least since January 2008; and

• In-depth interviews with 30 builders who participated in the program in 2008. ENERGY STAR homes provide a variety of NEIs ranging from those that are relatively easy to quantify, such as water conservation, to those that are much more difficult to measure, such as the homeowner’s sense of satisfaction in helping reduce greenhouse gas emissions. The program sponsors already claim some of the more easily quantifiable NEIs, such as water conservation. This report addresses seven NEIs which had not been previously studied:




dust, pollen, humidity, and car exhaust



• Safety of the new home’s heating system along with improved ventilation making the home safer in terms of carbon monoxide levels and gas leaks

• Anticipated ease of selling or leasing the home in the future and higher resale/leasing value

• Knowledge of some protection against rising energy prices; that is, as energy prices increase, energy bill savings will also increase

NEI quantification for new construction programs inherently faces the issue of participants not being able to make before-and-after comparisons. Thus, ENERGY STAR homeowners are seldom able to compare living in their newly constructed homes to living in newly constructed non-ENERGY STAR homes and will likely compare their new homes with their previous, most likely older, homes. This project has tried to deal with this issue by giving clear instructions in the survey instrument and highlighting instances throughout the report where comparisons to older homes may be occurring. It is also noted, in later chapters, that the Massachusetts findings are in line with values developed by similar studies, thereby increasing confidence in the validity of the results presented.

2 Methodology Two sources of NEI estimation are used: a homeowner survey and in-depth interviews with ENERGY STAR builders.

2.1 Homeowner Survey Seventy homeowners were surveyed via computer-assisted telephone interviewing (CATI) from November 14 through 24, 2008. The sample was obtained from lists of homes certified in 2007. The interviewers confirmed that each house is newly constructed, and that the respondent is one of the buyers. Hence the sample represents only owner-occupied new homes, not newly constructed homes that are for rent. The interviewers further confirmed that each respondent had moved into the home on or before January, 2008 and had thus spent at least part of a winter in the home. Respondents were also asked if they knew that newly constructed homes could be termed ENERGY STAR homes and, if so, whether their new home was an ENERGY STAR home. Again, respondents who had not heard of ENERGY STAR homes or did not know they had bought one were terminated from the survey. It is important to note here that this survey is not intended to be representative of all ENERGY STAR homeowners since the last survey to measure awareness, conducted in January of 2007, found that only 55% of ENERGY STAR homeowners know they have bought an ENERGY STAR home. Thus, the NEI values developed by this project are not appropriate for cost-benefit screening in the absence of additional analyses to account for all program participants. The homeowner survey covered the following areas:

• Whether homeowners believed their new home, because it is ENERGY STAR certified, provides a particular NEI.

• How homeowners learned of a particular NEI



• Whether homeowners learned of a particular NEI before or after purchasing the home and, if before, whether the NEI influenced their purchase decision

• Annual value placed on an NEI in relation to energy bill savings. Values could be expressed in dollars or as a percentage of bill savings

• Demographic and housing characteristics A copy of the survey instrument is found in Appendix A.

2.2 Builder Interviews Thirty in-depth interviews were conducted during the months of December 2008 and January 2009 with builders who participated in the program in 2008. The primary purpose of the builder interviews was to assess the program’s new process for choosing, ordering, and installing CFLs; these findings will be presented in a separate report. The builder interviews contained a section on NEIs paralleling the homeowner survey. Builders were asked

• Whether they believed ENERGY STAR homes provided a particular NEI • If so, how often they or their salespeople talked about the NEI with prospective

homebuyers • If they did not mention the NEI, whether they thought homebuyers were aware of it

anyway • How much they thought homeowners value the NEI in relation to energy bill savings

A copy of the builder interview guide is found in Appendix B.


Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 5 3 Perception of NEIs Table 3.1 shows, first, the percentage of homeowner respondents who say ENERGY STAR homes provide a particular NEI (for example, they are more or less comfortable than other new homes) and then the percentage of the total who say the NEI is positive (for example, ENERGY STAR homes are more comfortable than other new homes). Homeowners overwhelmingly believe their new homes provide positive NEIs in thermal comfort, a higher resale or rental value, and protection against energy bill increases. A majority also believe their new homes provide noise reduction and better lighting features, while only about one-half believe their homes provide better indoor air quality or more safety. These findings are similar to the results of a recent evaluation of the New York ENERGY STAR Labeled Homes (NYESLH) Program where 92% of participants believed they had positive NEIs in thermal comfort; 75% in noise reduction; 67% in better indoor air quality; and 42% in safety.1

Table 3-1: Homeowners Who Say Home Provides NEIs (all homeowner survey respondents)

NEI

Believe new home has NEI(n=70)

Positive NEIs(n=70)

Thermal comfort 87% 86% Noise reduction 70% 67% Lighting life/quality 81% 61% Indoor air quality 53% 51% Safety 49% 46% Resale/rental value 81% 80% Energy bill increase protection 94% 93%

1 Summit Blue Consulting, LLC and Quantec, LLC. Non-Energy Impacts (NEI) Evaluation Final Report Prepared for New York State Energy Research and Development Authority, June 2006.




Homeowners with custom-built homes are more likely to believe their homes provide more thermal comfort, less outside noise, and better indoor air quality while owners of spec-built homes are more likely to consider lighting life and quality a positive NEI. (Table 3-2)

Table 3-2: Homeowners Who Say Home Provides Positive NEIs by Spec and Custom Built Homes

(all homeowner survey respondents) NEI

Spec-built homes(n=51)

Custom-built homes (n=19)

Thermal comfort 84% 90% Noise reduction 63% 79% Lighting life/quality 69% 42%* Indoor air quality 47% 63% Safety 45% 47% Resale/rental value 81% 79% Energy bill increase protection 94% 90%

*Significantly different from spec-built homes at the 90% confidence level. It has been theorized that, despite the instructions given at the time of the survey, homeowners who believe their ENERGY STAR home has a positive NEI are comparing it with their previous, most likely older, home rather than other newly constructed homes. Table 3-3 shows differences by age of previous home among homeowners who say their new home has positive NEIs. There appears to be no consistent relationship between the age of the previous home and having a positive NEI in the new home; in some cases respondents with older previous homes are more likely to say they have a positive NEI; in other cases the opposite is true.

Table 3-3: Homeowners Who Say Home Provides Positive NEIs by Age of Previous Home

(all homeowner survey respondents who know the age of their previous home) NEI

Previous home 10 years old or less

(n=12)

Previous home over 10 years old

(n=45) Thermal comfort 92% 82% Noise reduction 67% 67% Lighting life/quality 83% 53% Indoor air quality 75% 47%* Safety 58% 38% Resale/rental value 67% 80% Energy bill increase protection 92% 96%

*Significantly different from newer homes at the 90% confidence level.


Builders also believe ENERGY STAR homes provide the seven NEIs considered; at least nine-tenths believe ENERGY STAR homes provide more thermal comfort and outdoor noise reduction, a larger number of CFLs, a higher resale or rental value, and some protection against energy price increases compared to newly constructed non-ENERGY STAR homes. Close to three quarters believe ENERGY STAR homes have better indoor air quality and more than one half believe ENERGY STAR homes are safer in terms of carbon monoxide levels and gas leaks. However, less than one half believe the lighting life and quality in ENERGY STAR homes is a positive NEI for homeowners.2(Table 3-4)

Table 3-4: Builders Who Say ENERGY STAR Homes Provide NEIs (all builders interviewed)

NEI

Believe ENERGY STAR homes have NEI(n=30)

Thermal comfort 97% Noise reduction 90% Lighting life/quality 43% Indoor air quality 73% Safety 53% Resale/rental value* 90% Energy bill increase protection 93%

*Based on 29 respondents; one respondent felt this question did not apply to him since he builds affordable homes which may not be resold at market rates.

2 The wording of the builder interviews differed from that of the homeowner survey in that builders were asked if homes provided a positive NEI (e.g., “Are ENERGY STAR homes more comfortable than other new homes” rather than “Are ENERGY STAR homes more or less comfortable than other new homes?” The exception is lighting, concerning which builders were simply asked if they believed ENERGY STAR homes had more CFLs installed than other new homes; almost all builders (97%) agreed that they did. In order to be comparable to homeowner data, Table 3-4 and Figure 3-1 show the percentage of builders who believe homeowners put a positive value on the lighting life/quality.


Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 8 Builders are more likely than homeowners to believe ENERGY STAR homes have positive NEIs in six out of the seven NEIs, as shown in Figure 3-1. Furthermore, the proportion of builders who believe ENERGY STAR homes are quieter and have better indoor air quality is significantly higher than that of homeowners at the 90% confidence level. However, the proportion of builders who believe the lighting life and quality of ENERGY STAR homes is a positive feature for homeowners is significantly lower, at the 90% confidence level, than the proportion of homeowners holding this view.

Figure 3-1: Builders and Homeowners Who Say ENERGY STAR Homes Provide Positive NEIs


Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 9 Sources of information for homeowners who believe their homes provide positive NEIs vary by the type of NEI. Builders are an important source for thermal comfort and noise reduction while many respondents say they simply figured out higher resale/rental values and energy bill protection on their own. However, experience living in the home is an important source of information for lighting life/quality, noise reduction, indoor air quality, safety, and thermal comfort. It may be that a sizable portion of respondents who cite experience living in the home are comparing the performance of their new home against that of their old home. (Table 3-5)

Table 3-5: How Learned of NEI

(all homeowner survey respondents who believe their homes have a positive NEI; multiple response)


Noise

reduction (n=47)

Lighting

life/quality(n=43)

Indoor air

quality(n=36)

Safety(n=32)

Resale/rental

value (n=56)

Energy bill

protection (n=65)

Builder 57% 45% 14% 33% 28% 18% 23% Experience living in home

33%

51%

56%

44%

44%

0%

0%

Media ads 20% 4% 19% 6% 9% 9% 11% Booklet or other literature

12%

4%

12%

6%

19%

4%

5%

Internet 8% 6% 5% 3% 6% 9% 9% Family, friends or co-workers

9%

6%

7%

6%

9%

5%

6%

Realtor or salesperson

7%

9%

12%

6%

6%

9%

6%

Just realized 7% 6% 0% 6% 3% 4% 6% Home show 3% 0% 7% 0% 0% 2% 1% Common knowledge

2%

0%

4%

0%

0%

0%

2%

Program implementer

2%

0%

2%

0%

0%

0%

0%

Architect 0% 0% 2% 6% 0% 0% 0% Store displays

0%

0%

2%

0%

0%

0%

0%

Figured it out

0%

0%

0%

0%

0%

43%

35%

Own research

0%

0%

0%

0%

0%

7%

2%

Don’t know 2% 0% 5% 14% 6% 20% 21%


Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 10 Four-fifths of builders who believe ENERGY STAR homes provide more thermal comfort than other new homes say they or their salespeople always or fairly often discuss this with prospective homebuyers; a majority also say they always or fairly often discuss noise reduction, lighting, indoor air quality, resale or rental value, and energy bill protection. Fewer builders discuss safety. One builder noted that he has to discuss lighting since CFLs often don’t come on right away when customers are going through a home. (Table 3-6)

Table 3-6: How Often Discuss NEI with Homebuyers (all builders who believe their homes have NEIs)


Noise

reduction (n=27)

Lighting

life/quality(n=29)

Indoor air

quality (n=22)

Safety(n=16)

Resale/rental

value (n=25)

Energy bill

protection (n=28)

Always 59% 44% 38% 45% 31% 40% 43% Fairly often

21%

19%

31%

27%

6%

24%

21%

About half the time

7%

7%

7%

14%

19%

8%

7%

Not that often

10%

15%

7%

9%

31%

8%

7%

Never 3% 15% 17% 5% 13% 20% 18% Don’t know

0%

0%

0%

0%

0%

0%

4%

Responses shown in Tables 3-5 and 3-6 confirm that many ENERGY STAR builders are discussing thermal comfort with their customers, and that homeowners remember hearing about this NEI from the builder. While builders also claim they often discuss noise reduction, lighting, indoor air quality, resale or rental value, and energy bill protection with customers, fewer homeowners remember hearing about these NEIs from builders. This may be due, in part, to the fact that homeowners were surveyed at least a year after moving into their new homes.


Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 11 For all the NEIs studied, homeowners most often say they learned of the existence of an NEI before deciding to buy or build the home. (Table 3-7)

Table 3-7: When Learned of NEI (all homeowner survey respondents who believe their homes have a positive NEI)


Noise

reduction (n=47)

Lighting

life/quality(n=43)

Indoor air

quality(n=36)

Safety(n=32)

Resale/rental

value (n=56)

Energy bill

protection (n=65)

Before decided to buy/build

70%

55%

75%

78%

72%

54%

55% After decided to buy/build

28%

43%

23%

19%

28%

32%

37% Don’t know 2% 2% 2% 3% 0% 14% 8% Further, most of those who knew of NEIs at this point say the presence of NEIs influenced their decision to buy or build an ENERGY STAR home. (Table 3-8) Table 3-8: Did the NEI Influence Your Decision to Buy of Build an ENERGY STAR

Home? (all homeowner survey respondents who say they learned their homes have a positive NEI before

deciding to buy or build)


Noise

reduction (n=26)

Lighting

life/quality(n=32)

Indoor air

quality (n=28)

Safety(n=23)

Resale/rental

value (n=30)

Energy bill

protection (n=36)

Yes 86% 65% 59% 71% 61% 67% 72% No 12% 35% 41% 29% 39% 30% 28% Don’t know

2%

0%

0%

0%

0%

3%

0%

Homeowner survey respondents who believe they have positive NEIs thus appear to have learned of these NEIs early in their home buying process and give them some weight in their purchase decision. It is important to remember, however, that this survey is limited to respondents who knew they had bought an ENERGY STAR certified home. It is reasonable to assume that these home buyers are also more likely than others to be aware of NEIs and give them more weight in deciding to purchase an ENERGY STAR home.


Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 12 4 NEI Value Calculation

4.1 Direct Scaling Approach

4.1.1 Homeowners Homeowner survey respondents were asked to estimate an annual monetary value for the NEIs they experience in their homes. The survey used a direct scaling method, asking respondents to value NEIs as a percentage of energy savings.3 Respondents were asked to assume that ENERGY STAR homes saved $400 per year in energy costs compared to similar, newly constructed non-ENERGY STAR homes. This figure is considered an average and reported in both the Massachusetts New Homes with ENERGY STAR program website (http://www.energystarhomes.com/homebuyers/benefits.htm) and the national website (http://www.energystar.gov/index.cfm?c=new_homes.nh_benefits). The survey first asked homeowners if they believed their new home had a particular NEI and whether it was positive or negative. Taking the thermal comfort NEI as an example, respondents were asked if they believed their new home, because it is an ENERGY STAR home, is more comfortable than other new homes, less comfortable, or no different in its comfort level. Those who believed it was more comfortable were asked to place a value per year on this increased comfort either in dollars or as a percentage of energy savings. Those who believed it was less comfortable were asked how much the decreased comfort took away from the value of living in an ENERGY STAR home, either in dollars or as a percentage of energy savings. NEI values for those who believed their new home was no different in comfort level from non-ENERGY STAR new homes were set to zero. Assigning monetary values to intangibles such as comfort is not an easy task and respondents who responded that they did not know were further prompted with the following questions:

“In terms of energy bill savings, would you say increased comfort is worth nothing, about a one fourth of typical annual energy bill savings, about a half of typical annual energy bill savings, about three-fourths of typical annual energy bill savings, about equal to the typical annual energy bill savings, or more than energy bill savings? If the latter, how much more?”

The NEIs for respondents who still could not provide an answer are treated as missing in the calculation of average NEI values.

4.1.2 Builders The builder interviews used a direct scaling approach, similar to that used in the homeowner survey, for estimating NEIs. The builder interviews first briefly described the homeowner survey and asked builders to estimate the value homeowners placed on NEIs relative to estimated annual savings of $400 on their energy bills. Builders were told they could assign a negative

3 A discussion of the various methods used to estimate NEIs in the literature is found in the Memorandum on Non-Energy Impacts for ENERGY STAR Homes submitted on June 30, 2008 for Task 5, Phase One of the 2008 work plan for the Massachusetts New Homes with ENERGY STAR program


http://www.energystarhomes.com/homebuyers/benefits.htm

http://www.energystar.gov/index.cfm?c=new_homes.nh_benefits

Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 13 value to an NEI if they believed homeowners saw it detracting from their experience living in an ENERGY STAR home. NEI estimates from builders who believe ENERGY STAR homes do not provide a particular NEI (for example, are no more comfortable than other new homes) are set to zero.

4.2 NEI Values Table 4-1 presents the mean values assigned to NEIs by homeowner survey respondents who either did not believe their home provided a particular NEI (value set to zero) or who could express a value either in dollars or as a percentage of energy savings. This means that respondents who believe their home provides a particular NEI but who cannot, even after prompting, assign a value to it, are set to missing. The means in Table 4-1 include both positive and negative NEI values. Upper and lower bounds of values are calculated at a 90% confidence level; the lower bounds provide a conservative estimate that may be considered for planning purposes.

Table 4-1: Mean NEI Values from the Homeowner Survey (all homeowner survey respondents except for those who cannot assign an NEI value)


Noise

reduction (n=64)

Lighting

life/quality(n=63)

Indoor air

quality(n=61)

Safety(n=63)

Resale rental value

(n=53)

Energy bill

protection (n=51)

Total Mean NEI Value

Dollars $279 $146 $144 $126 $105 $259 $386 $1445% Bill Savings

70%

37%

36%

32%

26%

65%

97%

361%

Lower Bound NEI Value* Dollars $210 $110 $105 $91 $72 $170 $161 $919 % Bill Savings

53%

28%

26%

23%

18%

43%

40%

230%

Upper Bound NEI Value* Dollars $348 $182 $183 $161 $138 $348 $611 $1971% Bill Savings

87%

45%

46%

40%

35%

87%

153%

493%

*Calculated at a 90% confidence level. The total estimate shown in the last column of Table 4-1 and the last row of Table 4-2, on the next page, is simply the sum of the values placed on the seven NEIs considered. As a check on double-counting, at the end of the NEI section of the survey, homeowner respondents were provided with the sum total of their valuations and asked:

“When we add the values you have placed upon the features of an ENERGY STAR home that we just discussed—we come up with XX% of the home’s annual energy cost savings. Would you say this figure is correct—that these seven features are worth XX% per year of annual energy cost savings to you?”


Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 14 Only one of the 70 homeowner respondents did not agree that the total accurately represented the value of the seven NEIs; he worked with the interviewer to adjust individual values to acceptable levels. It may be that other survey respondents agreed that the totals were accurate since they were close to the end of the survey and did not want to repeat earlier questions. This may have the effect of overstating the totals reported. Similar NEI values were obtained from a survey of NYESLH program participants using a direct scaling approach. Ease of selling the home was valued at just over 60% of energy bill savings with indoor air quality at over 50% and thermal comfort and reduced noise levels at over 40%. This study, however, assumed energy bill savings of $600 per year.4 Overall participant NEI values from zero and low energy homes in New Zealand ranged from 150% to 280% of energy savings. The most highly valued NEIs were increased comfort, helping the environment, and not having to move from the home due to high energy costs.5 Owners of spec-built homes value thermal comfort and resale or rental value more highly, while owners of custom-built homes put higher values on noise reduction, indoor air quality, and energy bill protection. (Table 4-2)

Table 4-2: Mean NEI Values for Owners of Spec- and Custom-Built Homes (all homeowner survey respondents except for those who cannot assign an NEI value)

NEI Spec-built homes Custom-built homes Thermal comfort $298 $226 Noise reduction $123 $204* Lighting life/quality $153 $121 Indoor air quality $111 $170 Safety $98 $120 Resale/rental value $283 $204 Energy bill increase protection $239 $739 Total $1305 $1784

*Significantly different from owners of spec-built homes at the 90% confidence level.

4 Summit Blue, op cit. 5 Stoecklein, Albrecht, & Skumatz, Lisa. (2006). “Using Non-Energy Benefits (NEBs) to Market Zero and Low Energy Homes in New Zealand.” Proceedings from the ACEEE Summer Study Conference 2006. Pacific Grove, CA: American Council for an Energy-Efficient Economy.


Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 15 Builder estimates (means) of how much homeowners value NEIs are shown in Table 4-3.

Table 4-3: Mean NEI Values from the Builder Interviews (all interviewed builders except for those who cannot assign an NEI value)


Noise

reduction (n=28)

Lighting

life/quality(n=28)

Indoor air

quality(n=28)

Safety(n=29)

Resale rental value

(n=27)

Energy bill

protection (n=27)

Total Mean NEI Value

Dollars $371 $284 $86 $271 $224 $267 $301 $1804% Bill Savings

93%

71%

22%

68%

56%

67%

75%

451%


Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 16 Overall, builder estimates are higher than those provided by homeowners, though, at a global level the results are similar enough to reinforce their validity. For individual NEIs, builder means are considerably higher than homeowner means for thermal comfort, noise reduction, indoor air quality, and safety; in the cases of indoor air quality and safety, builder estimates are significantly higher at the 90% confidence level. This is due, in both cases, to the higher proportion of builders than homeowners who believe ENERGY STAR homes are quieter and have better indoor air quality, as shown in Figure 3-1. Estimates for resale or rental value are very close for the two groups and builders give considerably lower estimates for lighting features and energy bill protection. (Figure 4-1)

Figure 4-1: Builder and Homeowner Valuation of NEIs

It should be noted that the lower mean value estimate for lighting life and quality among builders than among homeowners is largely due to the higher proportion of builders who applied a negative value to this NEI. While only four out of 70 homeowners applied negative values to CFLs, four out of 30 builders did so. It would thus appear that builders are more likely to believe that homeowners do not like CFLs than is actually the case. One interviewed builder did acknowledge that he only hears the complaints about CFLs.


Massachusetts New Homes with ENERGY STAR Non-Energy Impact Study - FINAL Page 17 5 Additional NEIs Both homeowners and builders were asked to name any NEIs associated with ENERGY STAR homes that they find valuable in addition to the seven explored and savings on energy bills. Respondents were then asked to value these additional NEIs as they had the seven NEIs under study. Homeowners most often talk about ENERGY STAR homes helping the environment, the quality of the windows, and the heating systems. Additional NEIs and mean values for homeowners are presented in Table 5-1; the small number of responses indicates mean values should be interpreted with caution.

Table 5-1: Additional NEIs Mentioned by Homeowners (homeowner survey respondents with additional NEIs; multiple response; n=18 )

NEI Number of Responses Mean Value Helping the environment; green 5 $220 Quality of windows 5 $285 Heating system (solar, geothermal) 4 $965 Amount of daylight 2 $240 Quality of appliances 2 $400 Vents on timers 1 $120 Tankless water heater 1 $392

Four of the thirty builders interviewed offered an additional NEI; all four cited having a green home or the satisfaction of helping the environment. One of the four says he talks about this with buyers all the time; one talks about it fairly often; the third about one half the time; the fourth says he does not talk about it that often. The mean value to homeowners of this NEI, according to the builders, is $120. The small number of homeowners and builders who could volunteer any additional NEIs would seem to suggest that the survey and interviews captured the most important ones.




6 Conclusions and Recommendations The homeowner survey and builder interviews point to several conclusions about NEIs.

• Thermal comfort is the best known and most highly valued NEI for homeowners. Close to nine-tenths of ENERGY STAR homeowners believe their homes offer more thermal comfort than other new homes and they value this feature, on average, almost as much as yearly energy bill savings.

• Builders are also most likely to talk to homeowners about thermal comfort and homeowners are most likely to remember learning about this NEI from builders.

• Taken together, homeowners value the seven NEIs more than annual savings on energy bills. Even if the survey, despite efforts to the contrary, had some double counting, a conservative estimate would put the value of the seven NEIs at about twice the amount of energy bill savings.

• After thermal comfort, homeowners place the highest value on increased resale or rental value and energy bill protection—features they largely figure out on their own.

• Builders value noise reduction, indoor air quality, and safety much more than homeowners. Homeowners are less likely to believe these aspects of their ENERGY STAR homes are better than in other new homes.

• Builders are much more likely to give a negative value to CFL lighting; it is possible that homeowners don’t mind the light quality and slowness to come to full brightness as much as builders think they do.

These findings lead to the following recommendations:

• NEIs should be an integral part of marketing for ENERGY STAR homes. Builders

should be especially encouraged to talk to prospective buyers about noise reduction, indoor air quality, and safety (or make sure their salespeople do so).

• The sponsors could consider sharing some of the results of this study with builders, in particular to show them the differences between home buyers’ ratings of NEI values compared to how builders think buyers would rate them.

Supplement 1

Homeowner Survey Instrument

Home Buyer NEI Survey Guide Page 2

NMR #2031 October 2008 MA ES Homes—Home Buyer NEI Survey Guide Hello, my name is ______, and I’m calling on behalf of [READ SPONSOR FROM SAMPLE] about the needs and preferences of new home buyers. The survey should take around 15 minutes, and the information you provide will help [READ SPONSOR FROM SAMPLE] improve its energy efficiency programs and services which will, in turn, help keep energy bills as low as possible by reducing consumption. Your responses will be kept strictly confidential. (Note: If the respondent says s/he has been called before, say, “A few homebuyers received calls and participated in a survey we did almost two years ago. We will be asking you many different questions this time and we really do appreciate your time in helping us.”) 1. I just want to confirm that your home is newly constructed—that is, built within the last

few years. Is that correct? 1. Yes [CONTINUE] 2. No [THANK AND TERMINATE] 3. (Don’t know/refused) [THANK AND TERMINATE]

2. Are you the person or one of the people who bought the home? 1. Yes [CONTINUE] 2. No, someone else in home [ASK TO SPEAK TO PROPER PERSON AND BEGIN AGAIN OR RESCHEDULE] 3. No, not available [THANK AND TERMINATE]

4. (Don’t know/refused) [THANK AND TERMINATE]

3. Have you ever seen or heard of a newly constructed home being referred to as an ENERGY STAR home?

1. Yes 2. No [THANK AND TERMINATE] 3. (Don’t know) [THANK AND TERMINATE]

4. Is your new home an ENERGY STAR home? 1. Yes, know for sure 2. Yes, believe so 3. No [THANK AND TERMINATE] 4. (Don’t know) [THANK AND TERMINATE] 5. What year did you move into your new home?

Record year _______________ [RECORD YEAR; IF 2008 ALSO RECORD MONTH; IF MOVED IN AFTER JANUARY 2008, THANK AND TERMINATE]

Home Buyer NEI Survey Guide Page 3 6. Can you tell me approximately what year your previous home, the one you lived in

before moving to your present home, was constructed? Was it... 1. Within the past 10 years 2. 11 to 20 years ago 3. 21 to 30 years ago 4. 31 to 40 years ago 5. 41 to 50 years ago 6. More than 50 years ago 7. (Don’t know/refused) The remainder of our questions are about your new home where you are now living

7. What type of building is your new home? [READ RESPONSES]

1. Detached single-family home 2. Townhouse or duplex, with a wall separating the units from basement to roof, and with separate utilities for each unit 3. Two, three, or four family building—one or more units stacked on top of another or with one water and sewer bill for the whole building

4. Part of a building with five or more units 5. Other [SPECIFY: ________________________] 6. (Don’t know/refused)

8. [IF “DETACHED SINGLE-FAMILY HOME” “TOWNHOUSE OR DUPLEX”OR “TWO- TO FOUR-FAMILY HOME” TO Q.#7] Which of the following best describes how you purchased your home?

1. Purchased land and worked with an architect and/or builder to design and build the home.

2. Had a house plan and a lot and hired a contractor/builder to build the home. 3. Purchased a lot from a builder, selected one of several house plans offered by the builder and selected from various available upgrade options. 4. Purchased a home that was under construction and selected from various available upgrade options.

5. Purchased a finished home 6. (Other [SPECIFY: ______________________]) 7. (Don’t know)

ENERGY STAR certified homes, such as yours, are verified by an independent third party.

They are typically 20 to 30 percent more efficient than standard homes. They include effective insulation, tight sealing of any holes and cracks in the home's "envelope," and may have various other features such as high performance windows and energy efficient heating and cooling systems. ENERGY STAR qualified homes thus use less energy for heating, cooling, and water heating saving owners about $400 per year. In addition to saving energy, ENERGY STAR homes may offer other benefits which you may have learned about when purchasing or building your home or may have experienced after moving into your home.

Home Buyer NEI Survey Guide Page 4 We will be asking you about seven features of ENERGY STAR homes. If you believe your

home has a particular feature, we would like you to think of putting a dollar value on its worth to you over one year compared with the $400 per year in energy bill savings. If you do not like a feature, you may put a negative value on it. If it is easier, you may also express the value as a percentage; for example, an answer of 50% would mean that the value of this feature to you over one year is half as much as the energy bill savings.

9. Some people believe that ENERGY STAR Homes may be more comfortable than other

new homes because they are built tighter, reducing drafts and temperature variations in the home. Do you believe that your home, because it is an ENERGY STAR-certified home, is more comfortable than other new homes, less comfortable, or would you say there is no difference in the comfort level?

1. More comfortable (GO TO Q.#9a) 2. Less comfortable (GO TO Q.#9e) 3. No difference (GO TO Q.#10) 4. (Don’t know) (GO TO Q.#10) 9a. How did you learn about this feature? [DO NOT READ RESPONSES; MULTIPLE

RESPONSE; MAKE SURE TO RECORD FIRST RESPONSE FIRST] 1. (Builder talked about it) 2. (Realtor or person in the sales office talked about it) 3. (Booklet or other literature) 4. (Media ads—newspaper, magazine, or radio) 5. (Internet) 6. (From family, friends, neighbors, or co-workers) 7. (Home show) 8. (Experienced it from living in home) 9. (Just now when you mentioned it) 10. (Other; specify [____________]) 11. (Don’t know refused) 9b. Did you learn about this feature before or after you decided to buy or build an ENERGY

STAR home? 1. Before (GO TO Q.#9c) 2. After (GO TO Q.#9d) 3. (Don’t know) (GO TO Q.#9d)

9c. [IF BEFORE TO Q#9b] Did this feature influence your decision to buy or build an

ENERGY STAR home? 1. Yes 2. No 3. (Don’t know/refused) 9d. [IF MORE COMFORTABLE] Given that an ENERGY STAR home typically saves you

$400 annually on energy bills, what is the value to you per year of this increased comfort from living in an ENERGY STAR home, either in dollars or as a percentage of energy


savings? [RECORD DOLLAR AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#9f; OTHERWISE SKIP TO Q#10; ________]

9e. [IF LESS COMFORTABLE] Given that an ENERGY STAR home typically saves you

$400 annually on energy bills, how much value per year would you say the decreased comfort takes away from the value of living in an ENERGY STAR home, either in dollars or as a percentage of energy savings? [RECORD DOLLAR AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#9g; OTHERWISE SKIP TO Q#10; ________]

9f. In terms of energy bill savings, would you say increased comfort is worth [READ

RESPONSES; IF RESPONDENT DECIDES TO GIVE A DOLLAR AMOUNT, RECORD AND INDICATE IT IS IN DOLLARS]

1. Nothing 2. About a one-fourth of typical annual energy bill savings 3. About a half of typical annual energy bill savings 4. About three-fourths of typical annual energy bill savings 5. About equal to the typical annual energy bill savings 6. More than energy bill savings; How much in total? [RECORD. ______ IF

PERCENTAGE, THE NUMBER NEEDS TO BE OVER 100%. ADD 100% IF RESPONDENT SAYS “XX% MORE THAN BILL SAVINGS” AND CHECK WITH RESPONDENT THAT IT REPRESENTS TOTAL SAVINGS. IF A DOLLAR AMOUNT MAKE SURE IT IS MORE THAN $400; RECORD TOTAL AMOUNT OF WORTH, NOT ADDITIONAL AMOUNT]

7. (Other) [RECORD IF RESPONDENT GIVES A DOLLAR AMOUNT OR PERCENTAGE RATHER THAN PICKING ONE OF THE SIX OPTIONS ABOVE]_______

8. (Have not noticed any increased comfort) [DO NOT READ] 9. (Don’t know) [DO NOT READ] 9g. In terms of energy bill savings, which of the following is closest to the value that the

decreased comfort takes away from living in your home? [READ RESPONSES] 1. Nothing 2. About a one-fourth of typical annual energy bill savings 3. About a half of typical annual energy bill savings 4. About three-fourths of typical annual energy bill savings 5. About equal to the typical annual energy bill savings 6. More than energy bill savings; How much in total? [RECORD. ______ IF

PERCENTAGE, THE NUMBER NEEDS TO BE OVER 100%. ADD 100% IF RESPONDENT SAYS “XX% MORE THAN BILL SAVINGS” AND CHECK WITH RESPONDENT THAT IT REPRESENTS THE TOTAL VALUE. IF A DOLLAR AMOUNT MAKE SURE IT IS MORE THAN $400; RECORD TOTAL AMOUNT OF WORTH, NOT ADDITIONAL AMOUNT]


Home Buyer NEI Survey Guide Page 6 8. (Have not noticed any decreased comfort) [DO NOT READ] 9. (Don’t know) [DO NOT READ] 10. Some people believe the insulation in ENERGY STAR HOMES helps reduce noise. Do

you believe that your new home, because it is an ENERGY STAR-certified home, is quieter than other new homes—with less noise from the outside, less quiet with more noise from the outside, or would you say there is no difference in the noise level? 1. Quieter, with less noise from the outside [GO TO Q#10a] 2. Less quiet, with more noise from the outside [GO TO Q#10e] 3. No difference [GO TO Q#11] 4. (Don’t Know) [GO TO Q#11]

10a. How did you learn of this feature? [DO NOT READ RESPONSES; MULTIPLE




ENERGY STAR home? 1. Yes 2. No 3. (Don’t know/refused) 10d. [IF LESS NOISY] Given that an ENERGY STAR home typically saves you $400 annually

on energy bills, what is the value to you per year of this reduced noise level from living in an ENERGY STAR home, either in dollars or as a percentage of energy savings? [RECORD DOLLAR AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#10f; OTHERWISE SKIP TO Q#11; ________]

10e. [IF MORE NOISY] Given that an ENERGY STAR home typically saves you $400

annually on energy bills, how much value per year would you say the increased noise


level takes away from the value of living in an ENERGY STAR home, either in dollars or as a percentage of energy savings? [RECORD DOLLAR AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#10g; OTHERWISE SKIP TO Q#11; ________]

10f. In terms of energy bill savings per year, would you say a quieter home is worth [READ

RESPONSES; IF RESPONDENT DECIDES TO GIVE A DOLLAR AMOUNT, RECORD AND INDICATE IT IS IN DOLLARS]


PERCENTAGE, THE NUMBER NEEDS TO BE OVER 100%. ADD 100% IF RESPONDENT SAYS “XX% MORE THAN BILL SAVINGS” AND CHECK WITH RESPONDENT THAT IT REPRESENTS TOTAL VALUE. IF A DOLLAR AMOUNT MAKE SURE IT IS MORE THAN $400; RECORD TOTAL AMOUNT OF WORTH, NOT ADDITIONAL AMOUNT]


8. (Have not noticed a decreased noise level) [DO NOT READ] 9. (Don’t know) [DO NOT READ] 10g. In terms of energy bill savings, which of the following is closest to the value that the

increased noise level takes away from living in your home? [READ RESPONSES] 1. Nothing 2. About a one-fourth of typical annual energy bill savings 3. About a half of typical annual energy bill savings 4. About three-fourths of typical annual energy bill savings 5. About equal to the typical annual energy bill savings 6. More than energy bill savings; How much in total? [RECORD. ______ IF



8. (Have not noticed an increased noise level) [DO NOT READ] 9. (Don’t know) [DO NOT READ] 11. Many ENERGY STAR homes in Massachusetts have compact fluorescent lights and

fluorescent fixtures installed through the program before the homebuyer occupies the


house. As a result, some people think these homes may have more compact fluorescent lighting than the typical new home. Do you think your new home has more, fewer, or the same number of compact fluorescent lights and fluorescent fixtures as a typical new home? 1. More [GO TO Q#11aa]

2. Fewer [GO TO Q#12] 3. Same [GO TO Q#12] 4. (Don’t know) [GO TO Q#12] 11aa. In addition to saving energy, these lights generally have a longer lifetime; they may also

have a different lighting quality than incandescents. Do you find the quality of light and longer life of these lights, taken together, to be a positive feature, a negative feature, or one that makes no difference to you? 1. A positive feature [GO TO Q#11a] 2. A negative feature [GO TO Q#11e] 3. Makes no difference [GO TO Q#12] 4. (Don’t know) [GO TO Q#12]

11a. How did you learn of this feature? [DO NOT READ RESPONSES; MULTIPLE RESPONSE; MAKE SURE TO RECORD FIRST RESPONSE FIRST]

1. (Builder talked about it) 2. (Realtor or person in the sales office talked about it) 3. (Booklet or other literature) 4. (Media ads—newspaper, magazine, or radio) 5. (Internet) 6. (From family, friends, neighbors, or co-workers) 7. (Home show) 8. (Experienced it from living in home) 9. (Just now when you mentioned it) 10. (Other; specify [____________]) 11. (Don’t know refused) 11b. Did you learn about this feature before or after you decided to buy or build an ENERGY


11c. [IF BEFORE TO Q#11c] Did this feature influence your decision to buy or build an

ENERGY STAR home? 1. Yes 2. No 3. (Don’t know/refused) 11d. [IF A POSITIVE FEATURE] Given that an ENERGY STAR home typically saves you

$400 annually on energy bills, what is the value to you of the longer life and lighting


quality from compact fluorescents and fluorescent lighting fixtures either in dollars or as a percentage of energy savings? [RECORD DOLLAR AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#11f; OTHERWISE SKIP TO Q#12; ________]

11e. [IF A NEGATIVE FEATURE] Given that an ENERGY STAR home typically saves you

$400 annually on energy bills, how much value would you say the lighting takes away from the value of living in an ENERGY STAR home, either in dollars or as a percentage of energy savings? [RECORD DOLLAR AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#11g; OTHERWISE SKIP TO Q#12; ________]

11f. In terms of energy bill savings, would you say the longer life and lighting quality from

compact fluorescents and fluorescent lighting fixtures is worth [READ RESPONSES; IF RESPONDENT DECIDES TO GIVE A DOLLAR AMOUNT, RECORD AND INDICATE IT IS IN DOLLARS]




8. (Lighting makes no difference) [DO NOT READ] 9. (Don’t know) [DO NOT READ] 11g. In terms of energy bill savings, which of the following is closest to the value that the

lighting issues take away from living in your home? [READ RESPONSES] 1. Nothing 2. About a one-fourth of typical annual energy bill savings 3. About a half of typical annual energy bill savings 4. About three-fourths of typical annual energy bill savings 5. About equal to the typical annual energy bill savings 6. More than energy bill savings; How much in total? [RECORD. ______ IF


Home Buyer NEI Survey Guide Page 10 7. (Other) [RECORD IF RESPONDENT GIVES A DOLLAR AMOUNT OR

PERCENTAGE RATHER THAN PICKING ONE OF THE SIX OPTIONS ABOVE]_______

8. (Lighting makes no difference) [DO NOT READ] 9. (Don’t know) [DO NOT READ] 12. Some people believe ENERGY STAR homes have better indoor air quality because of

reduced infiltration and better ventilation. Do you believe that your new home, because it is an ENERGY STAR-certified home, has better indoor air quality than other new homes, worse indoor air quality, or would you say there is no difference in indoor air quality? 1. Better indoor air quality [GO TO Q#12a] 2. Worse indoor air quality [GO TO Q#12e] 3. No difference [GO TO Q#13] 4. (Don’t Know) [GO TO Q#13]





ENERGY STAR home? 1. Yes 2. No 3. (Don’t know/refused) 12d. [IF BETTER QUALITY] Given that an ENERGY STAR home typically saves you $400

annually on energy bills, what is the value to you of this improved indoor air quality either in dollars or as a percentage of energy savings? [RECORD DOLLAR AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#12f; OTHERWISE SKIP TO Q#13 ________]

Home Buyer NEI Survey Guide Page 11 12e. [IF WORSE QUALITY] Given that an ENERGY STAR home typically saves you $400

annually on energy bills, how much value per year would you say the lower indoor air quality takes away from the value of living in an ENERGY STAR home, either in dollars or as a percentage of energy savings? [RECORD DOLLAR AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#12g; OTHERWISE SKIP TO Q#13; ________]

12f. In terms of energy bill savings, would you say improved indoor air quality is worth

[READ RESPONSES; IF RESPONDENT DECIDES TO GIVE A DOLLAR AMOUNT, RECORD AND INDICATE IT IS IN DOLLARS]




8. (Have not noticed improved air quality) [DO NOT READ] 9. (Don’t know) [DO NOT READ] 12g. In terms of energy bill savings, which of the following is closest to the value that the

lower indoor air quality takes away from living in your home? [READ RESPONSES] 1. Nothing 2. About a one-fourth of typical annual energy bill savings 3. About a half of typical annual energy bill savings 4. About three-fourths of typical annual energy bill savings 5. About equal to the typical annual energy bill savings 6. More than energy bill savings; How much in total? [RECORD. ______ IF



8. (Have not noticed any decreased indoor air quality) [DO NOT READ] 9. (Don’t know) [DO NOT READ]

Home Buyer NEI Survey Guide Page 12 13. Some people believe ENERGY STAR homes are safer in terms of carbon monoxide

levels than other newly constructed homes because of better ventilation and, in some cases, heating system back draft protection. Do you believe that your new home, because it is an ENERGY STAR-certified home, is safer in terms of carbon monoxide levels than other new homes, less safe, or would you say there is no difference in safety? 1. Safer [GO TO Q#13a] 2. Less safe [GO TO Q#13e] 3. No difference [GO TO Q#14] 4. (Don’t Know) [GO TO Q#14]





ENERGY STAR home? 1. Yes 2. No 3. (Don’t know/refused) 13d. [IF SAFER] Given that an ENERGY STAR home typically saves you $400 annually on

energy bills, what is the value to you of this improved safety either in dollars or as a percentage of energy savings? [RECORD DOLLAR AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#13e; OTHERWISE SKIP TO Q#14 ________]

13e. [IF LESS SAFE] Given that an ENERGY STAR home typically saves you $400 annually

on energy bills, how much value per year would you say the lower safety takes away from the value of living in an ENERGY STAR home, either in dollars or as a percentage


of energy savings? [RECORD DOLLAR AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#13g; OTHERWISE SKIP TO Q#14; ________]

13f. In terms of energy bill savings, would you say improved indoor air quality is worth





8. (Have not noticed improved air quality) [DO NOT READ] 9. (Don’t know) [DO NOT READ] 13g. In terms of energy bill savings, which of the following is closest to the value that the

lower safety takes away from living in your home? [READ RESPONSES] 1. Nothing 2. About a one-fourth of typical annual energy bill savings 3. About a half of typical annual energy bill savings 4. About three-fourths of typical annual energy bill savings 5. About equal to the typical annual energy bill savings 6. More than energy bill savings; How much in total? [RECORD. ______ IF



8. (Do not believe home is less safe) [DO NOT READ] 9. (Don’t know) [DO NOT READ]

Home Buyer NEI Survey Guide Page 14 14. Do you believe that your new home, because it is an ENERGY STAR-certified home,

would have a higher resale value or rental value than other new homes, a lower resale or rental value, or about the same resale or rental value?

1. Higher resale or rental value (GO TO Q#14a) 2. Lower resale or rental value (GO TO Q#14e) 3. About the same resale or rental value (GO TO Q#15) 4. (Don’t know) (GO TO Q#15) 14a. How did you learn of this feature? [DO NOT READ RESPONSES; MULTIPLE

RESPONSE; MAKE SURE TO RECORD FIRST RESPONSE FIRST] 1. (Builder talked about it) 2. (Realtor or person in the sales office talked about it) 3. (Booklet or other literature) 4. (Media ads—newspaper, magazine, or radio) 5. (Internet) 6. (From family, friends, neighbors, or co-workers) 7. (Home show) 8. (Just figured it out from the market) 9. (Just now when you mentioned it) 10. (Other; specify [____________]) 11. (Don’t know refused) 14b. Did you learn about this feature before or after you decided to buy or build an ENERGY



ENERGY STAR home? 1. Yes 2. No 3. (Don’t know/refused) 14d. [IF HIGHER VALUE] Given that an ENERGY STAR home typically saves you $400

annually on energy bills, what is the value to you of its higher resale or rental value either in dollars or as a percentage of energy savings? [RECORD DOLLAR AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#14f; OTHERWISE SKIP TO Q#15 ________]

14e. [IF LOWER VALUE] Given that an ENERGY STAR home typically saves you $400

annually on energy bills, how much value per year would you say the lower resale or rental value takes away from the value of living in an ENERGY STAR home, either in dollars or as a percentage of energy savings? [RECORD DOLLAR AMOUNT OR


PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#14g; OTHERWISE SKIP TO Q#15; ________]

14f. In terms of energy bill savings, would you say the higher resale or rental value is worth





8. (Do not believe it will have a higher resale or rental value) [DO NOT READ] 9. (Don’t know) [DO NOT READ] 14g. In terms of energy bill savings, which of the following is closest to the value that the

lower resale or rental value takes away from living in your home? [READ RESPONSES]




8. (Do not believe there is a lower resale or rental value) [DO NOT READ] 9. (Don’t know) [DO NOT READ] 15. Some people believe energy prices will continue to increase in the future and that will

make the size of energy bill savings for ENERGY STAR homes greater. Do you believe


that your new home, because it is an ENERGY STAR-certified home, will provide more protection to you in the face of energy price increases, over other new homes, less protection, or do you believe there is no difference?

1. More protection (GO TO Q#15a) 2. Less protection (GO TO Q#15e) 3. No difference (GO TO Q#16) 3. (Don’t know/refused) (GO TO Q#16) 15a. How did you learn of this feature? [DO NOT READ RESPONSES; MULTIPLE

RESPONSE; MAKE SURE TO RECORD FIRST RESPONSE FIRST] 1. (Builder talked about it) 2. (Realtor or person in the sales office talked about it) 3. (Booklet or other literature) 4. (Media ads—newspaper, magazine, or radio) 5. (Internet) 6. (From family, friends, neighbors, or co-workers) 7. (Home show) 8. (Just figured it out) 9. (Just now when you mentioned it) 10. (Other; specify [____________]) 11. (Don’t know refused) 15b. Did you learn about this feature before or after you decided to buy or build an ENERGY



ENERGY STAR home? 1. Yes 2. No 3. (Don’t know/refused) 15d. [IF MORE PROTECTION] Given that an ENERGY STAR home typically saves you

$400 annually on energy bills, what is the value to you of knowing that your ENERGY STAR home will provide some protection in the face of energy price increases either in dollars or as a percentage of energy savings? [RECORD DOLLAR AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#15f; OTHERWISE SKIP TO Q#16 ________]

15e. [IF LESS PROTECTION] Given that an ENERGY STAR home typically saves you $400

annually on energy bills, how much value per year would you say the lower protection against energy price increases takes away from the value of living in an ENERGY STAR home, either in dollars or as a percentage of energy savings? [RECORD DOLLAR


AMOUNT OR PERCENTAGE. IF RESPONDENT DOES NOT KNOW, GO TO Q#15g; OTHERWISE SKIP TO Q#16; ________]

15f. In terms of energy bill savings, would you say this protection from energy price increases

is worth [READ RESPONSES; IF RESPONDENT DECIDES TO GIVE A DOLLAR AMOUNT, RECORD AND INDICATE IT IS IN DOLLARS]




8. (Do not believe energy prices will increase/energy prices may decrease) [DO NOT READ]

9. (Don’t know) [DO NOT READ] 15g. In terms of energy bill savings, which of the following is closest to the value that the

lower protection against energy price increases takes away from living in your home? [READ RESPONSES]




8. (Do not believe that energy prices will increase/energy prices may go down) [DO NOT READ]

9. (Don’t know) [DO NOT READ]

Home Buyer NEI Survey Guide Page 18 [THE PROGRAM HERE SUMS THE PERCENTAGES GIVEN FOR THE SEVEN

FEATURES, INCLUDING NEGATIVES. VALUES FOR 9e, 9g, 10e, 10g, 11e, 11g, 12e, 12g, 13e, 13g, 14e, 14g, 15e, and 15g SHOULD BE NEGATIVE. RESPONSES GIVEN IN DOLLAR AMOUNTS ARE CONVERTED TO PERCENTAGES BY DIVIDING BY $400; THAT IS, WE ASSUME ENERGY SAVINGS ARE $400 PER YEAR. “DON’T KNOW”/”DON’T BELIEVE HAVE THE FEATURE” RESPONSES AND BLANKS ARE SET TO ZERO.]

16. When we add the values you have placed upon the features of an ENERGY STAR home

that we just discussed—we come up with XX% [IF NEGATIVE READ “MINUS XX%] of the home’s annual energy cost savings. Would you say this figure is correct—that these seven features are worth XX% [IF NEGATIVE READ “MINUS XX%] per year of annual energy cost savings to you? 1. Yes [GO TO Q#17] 2. No [GO TO Q#16a] 3. (Don’t know) [GO TO Q#17]

16a. I will read back to you the individual values you placed on each feature and you can tell me if you would like to change this value. Again, you are comparing your ENERGY STAR home to a typical new home. [RECORD WHETHER THE RESPONDENT WANTS TO LOWER THE VALUE AND THE NEW LEVEL.]

1. Comfort level 2. Noise levels 3. Lighting life and quality 4. Indoor air quality 5. Safety 6. Resale or rental value 7. Protection from energy price increases 17. In addition to the seven features we discussed and the savings on energy bills, are there

other features that are related to your purchasing an ENERGY STAR home that you find valuable?

1. Yes [GO TO Q#17a] 2. No [GO TO INTRO BEFORE Q.#18] 3. (Don’t know) [GO TO INTRO BEFORE Q.#18] 17a. Could you tell me what they are and how valuable you consider them in relation to

savings on energy bills. [ACCEPT UP TO THREE RESPONSES AND PERCENTAGES OR DOLLAR AMOUNTS]

Now I have a few last questions for statistical purposes only.

Home Buyer NEI Survey Guide Page 19 18. Are you a first-time homebuyer, or did you already own a home before you bought this one?

1. First-time homebuyer 2. Already owned home 3. (Don’t know/refused)

19. Including yourself, how many people live in your household most of the year? 1. One 2. Two 3. Three 4. Four 5. Five 6. Six or more 7. None—seasonally occupied 8. (Refused)

20. Approximately how many square feet is your home? 1. Less than 1,500 2. 1,500 – 1,999 3. 2,000 – 2,499 4. 2,500 – 2,999 5. 3,000 – 3,999 6. 4,000 – 4,999 7. 5,000 or more 8. (Don’t know/Refused) [ASK Q.#20a]

20a. [IF Q.#20=8] How many rooms are in your home, not counting bathrooms?

1. 1 2. 2 3. 3 4. 4 5. 5 6. 6 7. 7 8. 8 9. 9 10. 10 or more 11. (Don’t know/refused) 21. What is the highest level of education that you have completed? [READ

CATEGORIES]


1. Less than high school 2. High school graduate 3. Technical or trade school graduate 4. Some college 5. College graduate 6. Some graduate school 7. Graduate degree 8. (Refused)

22. What is your age? Are you . .

1. 18 to 24 2. 25 to 34 3. 35 to 44 4. 45 to 54 5. 55 to 64 [GO TO Q#22a] 6. 65 or over [GO TO Q#22a] 7. (Refused) [GO TO Q#22a]

22a. [IF Q.#22=5, 6, OR 7] Is your new home in an over-55 community? 1. Yes 2. No 3. (Don’t know) 23. How long do you expect to stay in your new home? 1. One year or less 2. Two to three years 3. Four to five years 4. Six to ten years 5. More than ten years 6. (Indefinitely/the rest of my life) 7. (Don’t know) 24. What category best describes your total household income in 2007, before taxes?

1. Less than $35,000 2. $35,000 to $49,999 3. $50,000 to $74,999 4. $75,000 to $99,999 5. $100,000 to 149,999 6. $150,000 or more 7. (Refused)

25. [DO NOT READ] Sex

1. Female 2. Male

Thank you very much!

Supplement 2

Builder Interview Guide (NEI Portion)

Page 2 2008 Massachusetts New Homes with ENERGY STAR® Program

NEI Interview Questions Read: In this set of questions I will be asking you about non energy benefits or impacts of ENERGY STAR homes. If you believe at least some ENERGY STAR homes have these features I would like you to think of putting a dollar value on their worth to homeowners over one year compared with their energy bill savings; we have assumed $400 in energy bill savings each year for ENERGY STAR homes. If you think homeowners do not like a feature, you may put a negative value on it. If it is easier, you may also express the value as a percentage; for example, an answer of 50% would mean that the value of this feature to a homeowner over one year is half as much as the energy bill savings. We are also doing a survey of ENERGY STAR homeowners asking similar questions, so here we are trying to get the builders’ perceptions of how homeowners value certain features. 1. Do you believe ENERGY STAR homes are more comfortable than other new homes because

they are built tighter, reducing drafts and temperature variations in the home? 1.1. If yes, how often do you or your sales people talk about this with homebuyers? If not

much, do you think homebuyers know about this anyway? 1.2. How much do you think homeowners value thermal comfort in relation to savings on

their energy bills? (If don’t know, ask “Is it a positive or a negative for the homeowner?” “Is it one-quarter as much, one half as much, three quarters as much, the same, or more than energy savings?” “If more, how much more?”)

2. Do you believe ENERGY STAR homes are quieter than other new homes because of more

effective insulation? 2.1. If yes, how often do you or your sales people talk about this with homebuyers? If not

much, do you think homebuyers know about this anyway? 2.2. How much do you think homeowners value a quieter home in relation to savings on


3. Do you believe ENERGY STAR homes have more CFLS and fluorescent fixtures than other

new homes because of the program’s installation of CFLs? 3.1. If yes, how often do you or your sales people talk about the greater number of CFLs

and fluorescent fixtures with homebuyers? If not much, do you think homebuyers know about this anyway?

3.2. How much do you think homeowners value the longer life and different light quality, taken together, in relation to savings on their energy bills? (If don’t know, ask “Is it a positive or a negative for the homeowner?” “Is it one-quarter as much, one half as much, three quarters as much, the same, or more than energy savings?” “If more, how much more?”)

4. Do you believe ENERGY STAR homes have better indoor air quality because of reduced

infiltration and better ventilation than other new homes?

Page 3

4.1. If yes, how often do you or your sales people talk about better indoor air quality with homebuyers? If not much, do you think homebuyers know about this anyway?

4.2. How much do you think homeowners value better indoor air quality in relation to savings on their energy bills? (If don’t know, ask “Is it a positive or a negative for the homeowner?” “Is it one-quarter as much, one half as much, three quarters as much, the same, or more than energy savings?” “If more, how much more?”)

5. Do you believe ENERGY STAR homes are safer in terms of carbon monoxide levels

because of better ventilation and, in some cases, heating system back draft protection than other new homes? 5.1. If yes, how often do you or your sales people talk about increased safety with

homebuyers? If not much, do you think homebuyers know about this anyway? 5.2. How much do you think homeowners value increased safety in relation to savings on


6. Do you believe ENERGY STAR homes have higher resale or rental values than other new

homes? 6.1. If yes, how often do you or your sales people talk about this with homebuyers? If not

much, do you think homebuyers know about this anyway? 6.2. How much do you think homeowners value higher resale and rental values in relation

to savings on their energy bills? (If don’t know, ask “Is it a positive or a negative for the homeowner?” “Is it one-quarter as much, one half as much, three quarters as much, the same, or more than energy savings?” “If more, how much more?”)

7. Do you believe the size of energy bill savings for ENERGY STAR homes will increase

because energy prices will continue to go up in the future? 7.1. If yes, how often do you or your sales people talk about this with homebuyers? If not

much, do you think homebuyers know about this anyway? 7.2. How much do you think homeowners value the protection from future energy price

increases in relation to savings on their energy bills? (If don’t know, ask “Is it a positive or a negative for the homeowner?” “Is it one-quarter as much, one half as much, three quarters as much, the same, or more than energy savings?” “If more, how much more?”)

8. Are you familiar with any other non energy benefits or impacts of ENERGY STAR homes that I have not mentioned? 8.1. If yes, what are they? (Ask the same series of questions as above.)

Appendix C

CFL Process

22 Haskell Street, Cambridge, MA 02140

Phone: (617) 497-7544 Fax: (617) 497-7543


2008 MASSACHUSETTS New Homes with ENERGY STAR®

Review of 2008 CFL Installation Process

Final Report May 18, 2009

Submitted to:

The Joint Management Committee

Submitted by:


Contents EXECUTIVE SUMMARY ........................................................................................................... I

CONCLUSIONS AND RECOMMENDATION ....................................................................................... V

1 INTRODUCTION ................................................................................................................. 1

2 ENERGY STAR BUILDER SAMPLE ............................................................................... 3

3 COMPARISON OF 2008 AND PRIOR FREE CFL PROCESS ...................................... 7

3.1 ADVANTAGES OF THE 2008 PROCESS ................................................................................ 7

3.2 ADVANTAGES OF HAVING HERS RATERS INSTALL CFLS ................................................ 9

4 2008 CFL PROCESS ........................................................................................................... 10

4.1 HOW BUILDERS LEARNED ABOUT THE CHANGE IN PROCESS .......................................... 10

4.2 WHO SELECTED, ORDERED AND INSTALLED CFLS ......................................................... 11

4.2.1 Who Selected CFLs .................................................................................................... 12

4.2.2 Who Ordered CFLs .................................................................................................... 14

4.2.3 Who Installed CFLs .................................................................................................... 15

5 CFL ORDERING, DELIVERY AND INSTALLATION ................................................ 16

5.1 ORDERING CFLS ............................................................................................................. 16

5.2 CFL DELIVERY ............................................................................................................... 17

5.3 CFL INSTALLATION ........................................................................................................ 19

5.3.1 Able to Install all Ordered Bulbs ................................................................................ 19

5.3.2 CFLs Replaced Incandescents or Installed in Empty Sockets .................................... 19

5.3.3 Number of CFLs Installed .......................................................................................... 21

6 PROBLEMS WITH CFLS FAILING ............................................................................... 23

7 HERS RATERS VERIFIED CFLS INSTALLED ........................................................... 24

8 FINAL BUILDER COMMENTS....................................................................................... 25

9 CONCLUSIONS AND RECOMMENDATION ............................................................... 26

APPENDIX A CFL PROCESS BUILDER INTERVIEW QUESTIONS ......................... A1



Tables TABLE 2-1: BUILDER SAMPLE STATISTICS ...................................................................... 4

Figures FIGURE 1: NEW OR OLD CFL PROCESS BETTER ........................................................... II

FIGURE 2: WHO INSTALLED CFLS ..................................................................................... iii

FIGURE 2-1: YEARS BUILDING ENERGY STAR HOMES AND QUALIFIED HOMES BUILT ................................................................................................................ 3

FIGURE 2-2: CURRENTLY BUILDING AN ENERGY STAR HOME?.............................. 5

FIGURE 2-3: BUILD NON-ENERGY STAR HOMES ............................................................ 5

FIGURE 3-1: NEW OR OLD CFL PROCESS BETTER ........................................................ 7

FIGURE 4-1: WHO EXPLAINED CHANGES IN CFL PROCESS ..................................... 10

FIGURE 4-2: WHO SELECTED, ORDERED AND INSTALLED CFLS ........................... 11

FIGURE 4-3: WHO SELECTED CFLS................................................................................... 12

FIGURE 4-4: NEEDED HELP SELECTING CFLS .............................................................. 13

FIGURE 4-5: WHO ORDERED CFLS .................................................................................... 14

FIGURE 4-6: WHO INSTALLED CFLS ................................................................................. 15

FIGURE 5-1: SINGLE OR MULTIPLE CFL ORDERS ....................................................... 16

FIGURE 5-2: BACK-ORDERED CFLS ................................................................................. 17

FIGURE 5-3: NEED TO SIGN FOR CFL DELIVERY ......................................................... 18

FIGURE 5-4: ABLE TO INSTALL ALL ORDERED CFLS ................................................. 19

FIGURE 5-5: INSTALLED CFLS IN EMPTY SOCKETS OR REPLACED INCANDESCENTS ........................................................................................................ 20

FIGURE 5-6: INSTALLED MORE OR FEWER CFLS UNDER 2008 PROCESS ............ 21

FIGURE 6-1: PROBLEM WITH CFLS FAILING ................................................................ 23

FIGURE 7-1: HERS RATER VERIFIED CFLS INSTALLED ............................................ 24

MA New Homes with ENERGY STAR: Review of 2008 CFL Installation Process Page i

Executive Summary Builders interviewed about their experiences selecting, ordering and installing the free Compact Fluorescent Lamps (CFLs) available to them for homes participating in the New Homes with ENERGY STAR Program (Program) generally found the 2008 process workable. Prior to 2008 builders could have their HERS raters install free CFLs in all appropriate sockets when the final inspection was conducted; builders did not pay anything for the CFLs and there was no limit to the number of CFLs that could be installed. In 2008, builders became responsible for selecting, ordering and installing the free CFLs and received an incentive of $2 for each CFL they installed. In 2009 the $2 incentive to builders per CFL installed was discontinued and all participating homes are now required to have CFLs installed in at least 50% of available hard-wired screw-based fixtures. Builders have the option of installing the free CFLs themselves or having their HERS rater install them at the final inspection. As in previous years, the free CFLs are available for all appropriate sockets; there is no limit to the number of free CFLs that may be installed per home.

Some builders say that they like the new process better than the pre-2008 process, where the HERS rater installed the CFLs at the final inspection visit, because it allows them to install the CFLs on their schedule or gives them more control over selecting and installing the CFLs. Two builders who like the 2008 process better say that knowing more about the CFLs being installed in their homes is an advantage. Other builders like the previous process because it is less work for the builder and there is no paperwork. These findings support the Program’s plan going forward to give builders the option of selecting, ordering and installing the free CFLs themselves or having their HERS rater select and install them.

This report presents the results of in-depth interviews with 30 ENERGY STAR builders who participated in the free CFL component of the Program in 2008; interviews were conducted by phone in December 2008 and January 2009. Interviewed builders include both experienced ENERGY STAR builders and builders new to the Program. In total, the 30 interviewed builders estimate they have built 1,904 ENERGY STAR-qualified homes and that 59% of these homes (1,126 homes) had free CFLs installed through the Program.


MA New Homes with ENERGY STAR: Review of 2008 CFL Installation Process Page ii

Twenty-two of the thirty interviewed builders have experience under both the old and new CFL processes. Figure 1 shows that just over one-half of these 22 builders (55%) say they think the 2008 process works better, 27% say they think the old process works better and 18% say the two processes work equally well. Eight of the interviewed builders have experience only with the 2008 process; six of these builders commented that they like the 2008 process and two did not comment.

Figure 1: New or Old CFL Process Better

How Builders Learned about the Changes in 2008: All interviewed builders say they learned about the changes in how to get the free CFLs from their HERS rater and/or ICF. Several builders mentioned getting an email, letter or phone call from ICF. HERS raters tended to give builders the forms they needed to fill out—in some cases they emailed the forms so builders would have them on their computer—and/or meet with builders and walk them through the new process. Most interviewed builders (77%) say that they found the instructions for selecting and ordering CFL bulbs easy to understand and follow.

Selecting, Ordering and Installing CFLs: Builders describe a variety of configurations with respect to who selects, who orders and who installs the free CFLs. The most common configurations are: the builder selects and orders the CFLs and the electrician installs them (20%); the builder selects, orders and installs the CFLs (17%); and the builder selects and orders the CFLs and the construction crew installs them (17%).

Who Selected CFLs: Overall, 70% of interviewed builders say they selected what CFLs to order. Several builders commented on why they selected the CFLs rather than asking their electrician to select them:

• I am more familiar with CFLs and/or with what fixtures customers select. • Electricians are still putting some wrong bulbs in fixtures.


MA New Homes with ENERGY STAR: Review of 2008 CFL Installation Process Page iii

• When electricians order fixtures from the supply house, the supply house selects and includes the bulbs—electricians do not know that much about bulbs.

Almost two-thirds of interviewed builders (63%) say that they, or their electrician, did not need help selecting what CFLs to order. Builders who report that they needed help selecting the CFLs also report they received the help they needed from ICF, their electrician, Energy Federation Inc. (EFI), the lighting supply house or the website1.

CFL Orders: Almost all interviewed builders (28 of 30 builders) say they did the actual ordering of the CFLs. Most interviewed builders (77%) ordered all the CFLs they needed in one order, 20% identified additional sockets they needed to order CFLs for after they received their first order and 3% (one builder) usually makes two orders per home so that he does not get too many CFLs at one time. Over three-fourths of builders (77%) received all the CFLs they ordered in one delivery; 23% of builders had at least some CFLs back-ordered. All 29 builders who had received CFLs at the time of the interviews say that they received their CFLs within a week or less of ordering them, or within a week or less of when they were back-ordered. None of the builders say that the back-ordered CFLs delayed their construction schedule.

Installing CFLs: Builders, electricians and construction crew members installed the CFLs. Figure 2 shows that one-third of interviewed builders say their electrician installed the CFLs, just over one-fourth (27%) say their construction crew installed the CFLs and one-fifth say they installed the CFLs. One-fifth of interviewed builders (six builders) gave other responses.

Figure 2: Who Installed CFLs

1 More than one builder mentioned visiting “the website.” They did not say whether they were referring to the Program’s website or EFI’s website.


MA New Homes with ENERGY STAR: Review of 2008 CFL Installation Process Page iv

Most interviewed builders (77%) say they were able to install all the CFLs they ordered. Builders who were not able to install all the CFLs they ordered report some CFLs did not fit, some CFLs were defective, or that they had extra CFLs. If only a few CFLs were defective or did not fit, builders typically reported buying replacements locally. If they have extra CFLs, builders say they install them in their next ENERGY STAR home.

CFLs Installed in Empty Sockets or Replaced Incandescent Bulbs: One-half of the interviewed builders say they installed the free CFLs in empty sockets, just under one-fourth (23%) say they replaced incandescent bulbs, just under one-fourth (23%) say some CFLs were installed in empty sockets and some replaced incandescent bulbs, and one builder says he installed some CFLs in empty sockets and replaced existing CFLs in other sockets. Builders who replaced incandescent bulbs typically said that they saved the incandescent bulbs they removed or gave them to the homeowner. Going forward, all 15 builders who installed CFLs in empty sockets in 2008 and 9 of the 15 builders who replaced incandescent bulbs in at least some sockets say that they plan on asking their electrician not to install incandescent bulbs in sockets for which they have ordered CFLs.

Number of CFLs Installed: Twenty-two interviewed builders had free CFLs installed under both the pre-2008 and 2008 processes. Over one-half (59%) of these builders say that they installed the same number of CFLs in their 2008 homes as HERS raters would have installed under the pre-2008 process and 41% (9 builders) say they installed more CFLs. No builders say they installed fewer CFLs than would have been installed under the pre-2008 process. Eight of the nine builders who say that they installed more CFLs under the 2008 process say that they installed CFLs in applications where they think HERS raters might not have installed them. Seven of the eight builders who report installing CFLs in more applications also say they think they are installing more types of CFLs than their HERS raters might have installed: dimmables, smaller chandelier bulbs, indoor globes over vanities and floodlights.

CFLs Failing: One-half of the 30 interviewed builders report having some CFLs fail. Builders who commented on CFL failures say that that very few fail—only one or two per home. Only four of the builders who report having CFLs fail say that they contacted EFI to get free replacements. When only a few CFLs fail most builders say that they either use CFLs they have on hand or purchase new ones on their own to replace the ones that failed.

Verification of Installed CFLs: Only 10% of builders say that their HERS rater counts the number of CFLs installed; 30% say their HERS rater verifies the CFLs are installed, but do not mention them counting the CFLs; 20% say that their HERS rater verifies CFLs are installed, but does not count them; and 30% say that they are not aware of anyone verifying that the CFLs they ordered were installed. Ten percent of builders reported either that the final inspection on their home was done before the CFLs were installed (one builder) or that their homes had not yet had the final inspection at the time of the interview (two builders).


MA New Homes with ENERGY STAR: Review of 2008 CFL Installation Process Page v

Builder Comments: One builder commented that he would like to see LEDs included, and another would like to see pin-based CFLs and incentives for installing CFL fixtures. A third builder said the following about the 2008 process:

“It worked well, but needs to do a better job of promoting products; it is great that dimmables are available and they should have been promoted to builders since they are a great selling point for CFLs.”

Conclusions and Recommendation The builder interviews point to several conclusions about the 2008 CFL process.

• Most (55%) of the 22 interviewed builders who have installed free CFLs under both the old and new processes like selecting, ordering and installing the free CFLs themselves better than having their HERS rater select and install the CFLs, just over one-fourth (27%) prefer having their HERS raters select and install CFLs and 18% are indifferent, saying both processes work equally well.

• The feature builders like most about the 2008 process is being able to install the CFLs on their schedule, typically at the same time the lighting fixtures are installed.

• The features builders like most about the pre-2008 process are that they do not have to install the CFLs and there is no paperwork.

• Builders who installed CFLs under both the old and new processes report installing the same number of CFLs (13 of 22 builders) or more CFLs (9 of 22 builders) under the 2008 process than they think their HERS raters would have installed under the old process.

• Builders tend to take responsibility for selecting and ordering CFLs rather than having their electricians take on these functions.

• CFLs are delivered within a few days of being ordered; none of the builders interviewed report CFL delivery interfering with their construction schedule.

• Electricians and construction crew tend to install the CFLs. • Very few CFLs fail, and when only a few CFLs fail most builders either use CFLs they

have on hand or purchase new ones on their own to replace the ones that failed rather than return the failed CFLs to EFI.

• Most builders say their HERS rater verifies that CFLs are installed, but very few builders say their HERS rater counts the number of CFLs installed to verify that all ordered CFLs were installed.

These conclusions support the Program’s plan going forward to give builders the option of selecting, ordering and installing the free CFLs themselves or having their HERS rater select and install them. Different builders have different needs and preferences as well as different levels of knowledge about and experience with CFLs. Offering both approaches allows experienced ENERGY STAR builders who are comfortable selecting CFLs on their own and prefer having the CFLs on site to install when the light fixtures are installed to do so. As of 2009,


MA New Homes with ENERGY STAR: Review of 2008 CFL Installation Process Page vi


the Program requires builders to install CFLs in at least 50% of all hard-wired screw-based fixtures. Builders who are new to the Program or who are not very knowledgeable about the variety of CFLs available or which ones work best in specific applications will benefit from being able to have their HERS raters choose and install the CFLs. Builders without much experience with CFLs may, on their own, be likely to install the minimum number of CFLs required by the Program, but if HERS raters select and install the CFLs they can encourage the installation of CFLs in more than 50% of appropriate sockets. Offering both processes will likely maximize the number of CFLs installed through the Program.

MA New Homes with ENERGY STAR: Review of 2008 CFL Installation Process Page 1

1 Introduction This report presents the results of in-depth interviews with 30 ENERGY STAR builders in Massachusetts who participated in the free CFL component of the Massachusetts New Homes with ENERGY STAR Program (Program) in 2008. Builders were recruited from a list provided by ICF, the Program implementation contractor, of ENERGY STAR builders who participated in the free CFL component of the Program in 2008. Interviewed builders received $75 for their participation.

The interviews were conducted by phone in December 2008 and January 2009. Interview questions addressing builders’ experience selecting, ordering and installing CFLs are in Appendix A. In addition to asking builders about their experience with the Program’s free CFL component, the interviews included questions on non-energy impacts, satisfaction with the Program, marketing ENERGY STAR homes, choosing a HERS rater and Program training offerings. Findings related to questions addressing non-energy impacts are in the Assessment of Non-Energy Impacts report.2 Findings related to questions addressing satisfaction with the Program, marketing ENERGY STAR homes, choosing a HERS rater and Program training offerings will be presented in a separate report.

Prior to 2008 builders could have their HERS raters install free CFLs in all appropriate sockets when the final inspection was conducted; builders did not pay anything for the CFLs and there was no limit to the number of CFLs that could be installed. In 2008, builders became responsible for selecting, ordering and installing the free CFLs and received an incentive of $2 for each CFL they installed. In 2009 the $2 incentive to builders per CFL installed was discontinued and all participating homes are now required to have CFLs installed in at least 50% of available hard-wired screw-based fixtures. Builders have the option of installing the free CFLs themselves or having their HERS rater install them at the final inspection. As in previous years, the free CFLs are available for all appropriate sockets; there is no limit to the number of free CFLs that may be installed per home.

The key objectives of the 2008 CFL process review interviews are to assess:

• Builder satisfaction with the change in 2008 to having builders be responsible for selecting, ordering and installing the free CFLs

• How builders were informed of the changes introduced in 2008 • The roles of builders, electricians and construction crews with respect to selecting,

ordering and installing the free CFLs • Impact of having builders select CFLs on the number and type of CFLs installed

2 Nexus Market Research, Inc. Massachusetts New Homes with ENERGY STAR® Assessment of Non-Energy Impacts, submitted to the Joint Management Committee, March 2009.




• Problems encountered during the selection, ordering and installation of CFLs • HERS rater verification of CFL installation

Please note that due to the small sample size the results should not be considered statistically representative of the population of Massachusetts ENERGY STAR builders; rather, the results should be viewed as qualitative in nature.


2 ENERGY STAR Builder Sample The 30 interviewed builders include a mix of experienced ENERGY STAR builders and builders new to the Program; experience building ENERGY STAR homes runs from a builder who has not yet completed an ENERGY STAR-qualified home to a builder who estimates he has completed roughly 600 ENERGY STAR-qualified homes. The interviewed builders build custom homes, spec homes and affordable housing. Figure 2-1 shows the number of years each interviewed builder has participated in the Program and the number of qualifying homes they have built. Six interviewed builders have completed 100 or more ENERGY STAR homes; five of these builders have participated in the Program for at least six years. One interviewed builder, who joined the Program in 2008, has not had a home certified; this builder built two homes in 2008, installed the free CFLs in both homes, and both homes failed to pass the final inspection.3

Figure 2-1: Years Building ENERGY STAR Homes and Qualified Homes Built

3 This builder plans on continuing to participate in the Program and will use a better insulation contractor to ensure meeting all Program requirements for ENERGY STAR certification.



Table 2-1 shows interviewed builders have participated in the Program from one to twelve years; the average is four years and the median is three years. Interviewed builders have had from zero to 600 homes ENERGY STAR-qualified; the average is 63 homes and the median is 19 homes.

Table 2-1: Builder Sample Statistics

Statistics

Years Building ENERGY STAR

Homes (n=30)

Number of ENERGY STAR

Homes Built (n=30)

Minimum 1 0 Maximum 12 600 Average 4 63 Median 3 19

In total, the 30 interviewed builders estimate they have built 1,904 ENERGY STAR-qualified homes and that 59% of these homes (1,126 homes) had free CFLs installed through the Program. Two-thirds of the interviewed builders (20 builders) say that all the ENERGY STAR homes they built had free CFLs installed through the Program. These 20 builders report building a total of 483 ENERGY STAR homes; the average number of years they have participated in the Program is three and the median is two. The ten builders who said that not all their ENERGY STAR homes had free CFLs installed through the Program report building 1,421 ENERGY STAR homes and estimate that 45% of these homes had CFLs installed through the Program. On average, these ten builders have participated in the Program for more than twice as long as the 20 builders who say all their ENERGY STAR homes had free CFLs installed; the average and median number of years these ten builders have participated in the Program is seven. All 30 interviewed builders say it is now their standard practice to install CFLs in all their ENERGY STAR homes.



Figure 2-2 shows that most interviewed builders, 77% or 23 builders, say they are currently building an ENERGY STAR home. Six builders say they are not currently building an ENERGY STAR home; five of these six builders say they build only ENERGY STAR homes and one is a small custom builder currently working on remodeling projects. One builder, who says he may be building an ENERGY STAR home, subbed out work on a home and is not sure it will meet ENERGY STAR requirements.

Figure 2-2: Currently Building an ENERGY STAR Home?

Figure 2-3 shows that only 17% of interviewed builders (five builders) say they build any non-ENERGY STAR homes. One of these builders says his company builds only ENERGY STAR homes, but is taking over a 67 home project in which the first 12 homes are too far along to pass the Thermal Bypass Checklist (TBC); the remaining 55 homes will be ENERGY STAR. Two of these builders say they have built only one or two non-ENERGY STAR homes in the last several years. The remaining two builders say only that they build some non-ENERGY STAR homes.

Figure 2-3: Build Non-ENERGY STAR Homes




When the five builders who say they build some non-ENERGY STAR homes were asked if they install CFLs in their non-ENERGY STAR homes, one says always and two say they leave it up to the owner and/or electrician. One builder, new to the Program in 2008, says he did not install CFLs in his non-ENERGY STAR homes because the homes had recessed lighting and he did not think CFLs could be used in recessed lighting. The fifth builder, who is the builder taking over the 67 home project, says that in the 12 homes that will not be ENERGY STAR he would take advantage of a CFL promotion if it was available.


3 Comparison of 2008 and Prior Free CFL Process In 2008, builders became responsible for selecting, ordering and installing the free CFLs provided by the Program. Previously, HERS raters installed the free CFLs at the final inspection. Interviewed builders were asked, “Overall, how well do you think the process introduced in 2008 works compared to the previous process where your HERS rater ordered and installed the CFLs?”

Figure 3-1 shows that of the 22 interviewed builders who have experience under both the old and new CFL processes, just over one-half (55% or 12 builders) say they think the 2008 process works better, 27% (six builders) say they think the old process works better and 18% (four builders) say the two processes work equally well—there are advantages to each process. Eight of the interviewed builders have experience only with the 2008 process; six of these builders commented that they like the 2008 process; two did not comment.

Figure 3-1: New or Old CFL Process Better

Regardless of which process builders thought worked better, they were asked, “In what ways do you think the 2008 process is better than the previous process?” and “In what ways do you think the old process was better?” Most builders cited advantages to both processes.

3.1 Advantages of the 2008 Process Interviewed builders identified several advantages to having more responsibility for and control over the CFL installation process. The most frequently reported advantage of the new process is that builders are able to install the CFLs on their schedule. Only one builder mentioned the rebate as an advantage of the new process.



When builders have the CFLs ahead of time they can install them when they install the lighting fixtures, which is not only more convenient but eliminates the hassle of having to deal with the incandescent bulbs removed (more than 100 bulbs in a large home) when the HERS rater installs the CFLs. Some builders say that under the old process they did not always install bulbs when they installed the lighting fixtures—they left some sockets empty until the HERS rater installed the CFLs. Other builders say their electricians install bulbs in every fixture, at least temporarily, to make sure the fixture works properly. One builder pointed out that, under the old process, if the electrical inspection took place before the final ENERGY STAR inspection they had to install incandescents because all sockets have to have bulbs installed to pass the electrical inspection. Another builder says that the final inspection may not take place until after a home is being shown to buyers and that all sockets need to have bulbs when he shows a home to potential buyers. Following are comments from two builders on specific advantages of having the CFLs ahead of time and installing them themselves:

• “The bulbs installed in high ceilings used to be a problem, so the new approach is probably a bit better. When the bulbs are installed at the final inspection, the homes are already finished and if you have cathedral ceilings everyone is shuffling around trying to find ladders.”

• “I build large homes that take up to 100 bulbs. Under the old process the HERS rater would not have enough CFLs and would install the remainder in a second trip, which was too close to the time buyers were looking at the homes.”

Several builders say they like having more control over selecting and installing CFLs. They say more types of CFLs are available, including dimmable CFLs, and they can be sure they get the right bulbs. One of these builders says:

“I have more control and the electrician is now responsible for both the fixtures and the bulbs working properly. The electrician can't point at the other guy (HERS rater) who installed the CFL and say he/she didn't put the fixture back together right.”

Two builders say knowing more about the CFLs being installed is an advantage. One of these builders says that when he knows what CFLs are going in he can do a better job of explaining the lighting to buyers. The other builder says, “It gives us better insight into lighting products and it gives my guys a better understanding of why energy-efficient lighting is important.”

Two builders commented on the simplicity of the 2008 process:

• “It's not complicated and doesn't involve a lot of work; you simply order the bulbs, they are delivered and you install them.”

• “In the beginning I thought it wouldn't work well, but I am doing it and now the bulbs are here and available when I need them so it is working well. The paperwork is simple and most of the bulbs we use are listed in bolder print so it is easy to identify them.”




3.2 Advantages of Having HERS Raters Install CFLs As described earlier, most builders identified things they liked about both CFL processes. The most commonly cited advantage of the pre-2008 process, under which HERS raters select and install the CFLs, is that it is less work for the builder. Builders say they do not have to think about what CFLs to order or how many to order, do not have to install the CFLs, and there is no paperwork involved. Only five builders say they can not think of any way in which the pre-2008 process is better.


4 2008 CFL Process This section addresses how builders learned about the changes introduced in 2008 to the process for getting free CFLs and who they had select, order and install the free CFLs.

4.1 How Builders Learned About the Change in Process All interviewed builders say they learned about the changes in how to get the free CFLs from their HERS rater and/or ICF. Figure 4-1 shows that almost one-half (47%) of builders say that their HERS rater explained the changes to the free CFL process, almost a third (30%) say that they learned about the changes from ICF and almost a quarter (23%) say that they received information on the changes from both their HERS rater and ICF. One builder mentioned that he also got a letter from Cape Light Compact.

Figure 4-1: Who Explained Changes in CFL Process

Some builders provided comments on how they learned about the changes to the CFL process. Several builders mentioned getting an email, letter or phone call from ICF. HERS raters tended to give builders the forms they needed to fill out—in some cases they emailed the forms so builders would have the forms on their computer—or meet with builders and walk them through the new process. One builder says he was given brochures and pictures of CFLs and fixtures, which helped him know what bulbs go with what fixtures, wattages, etc. One builder, who says his HERS rater told him what to do, commented, “I also looked on the website. I use the website unless there is something I need to ask the HERS rater—the HERS rater doesn’t volunteer information unless I ask.”

Most interviewed builders (77% or 23 builders) say that they found the instructions for selecting and ordering CFL bulbs easy to understand and follow. Four of the seven remaining builders say the descriptions of the CFLs could have been better; one simply says the descriptions could have been clearer, one says that it was not clear which CFLs would be appropriate for exterior



lighting, one says that because of the lack of information on suitable applications his first three orders were wrong, and another says,

“I would have liked to see good photos to know what the bulbs really look like before ordering. It would be good to have a website that would supply more information on the bulbs, notably dimmables, which are a new product—what they look like and their effectiveness, such as the time to light.”

Of the remaining three builders, one says that the owner handled selecting and ordering the CFLs, one says that he did not see any instructions and one builder reported that there was no choice of CFLs, he just had to say how many CFLs he wanted.

4.2 Who Selected, Ordered and Installed CFLs Interviewed builders describe a variety of configurations with respect to who selected, who ordered, and who installed the free CFLs. Figure 4-2 shows the most common configurations are: the builder selects and orders the CFLs and the electrician installs them (20% or six builders); the builders selects, orders and installs the CFLs (17% or five builders); and the builder selects and orders the CFLs and the construction crew installs them (17% or five builders). An additional five builders (17%) say that they selected and ordered the CFLs and that they and their electrician, they and their construction crew, or their electrician and construction crew worked together to install the CFLs. Four builders (14%) say that their electrician selected the CFLs, they (the builder) ordered the CFLs and either their construction crew or their electrician installed the CFLs. Five builders (17%) described other configurations; in four of these cases the builder, working with the electrician, or the architect, lighting supply house or homeowner selected the CFLs; in one case the electrician selected and ordered the CFLs, but they have not yet been installed.

Figure 4-2: Who Selected, Ordered and Installed CFLs



4.2.1 Who Selected CFLs Most interviewed builders say they selected what CFLs to order. Figure 4-3 shows that 70% of interviewed builders say they selected what CFLs to order, 17% say they asked their electrician to select the CFLs and 13% gave other responses. Of the four builders who gave other responses, one says that he worked with his electrician to select the CFLs, one says the lighting supply house helped select the CFLs, one says he asked the architect who specified the lighting package for the building to select the CFLs, and one says the homeowner selected the CFLs.

Figure 4-3: Who Selected CFLs

Seven builders commented on why they selected the CFLs rather than asking their electrician to select them. Three of these builders say that they are very familiar with CFLs—one says he asked his electrician for feedback on the CFLs he (the builder) selected. Four interviewees commented as follows:

• “I thought I would do a better job than he (the electrician) would. There is no incentive for him to do it. I do the homework for what specific bulbs to install. The electrician is still putting some wrong bulbs in fixtures.”

• “Initially I asked the electrician to select the bulbs, but he wasn't correct for the first couple houses so I started doing it myself. It is an extra step the electrician is not used to doing. When they order fixtures from the supply house they pick the fixture and the supply house will select and include the bulbs—the electrician doesn't have to think about it. Electricians don't know that much about bulbs. Also, they are less familiar with the fixtures in stock and what buyers choose. We order the bulbs at the same time we order fixtures. I am more familiar with what fixtures customers select than the electrician. For example, foyer fixture choices include two-, four- and eight-bulb fixtures, and sometimes buyers change their mind. “



• “Most of the time our homeowners supply the lighting fixtures. They buy them and drop them off in the house. I go through the boxes to see what is needed for when the electrician installs the fixtures. Under the old process we used to have to replace incandescents.

• “I (interviewee responsible for CFLs) had the head builder on our construction crew and our electrician look to see what was needed. Then, I selected the bulbs from the form and went to the website when I had questions about what bulb I needed for a specific application.”

Figure 4-4 shows that almost two-thirds (63%) of interviewed builders say that they, or their electrician, did not need help selecting what CFLs to order and one-third say that they needed some help—one builder had not yet received the CFLs his electrician ordered. One of the builders who says he did not need any help commented, “In hindsight I could have used some help.” Another builder commented that after the first two houses were completed he had a better grasp of what belonged where and that selecting the CFLs was pretty easy.

Figure 4-4: Needed Help Selecting CFLs

Nine of the ten builders who report that they needed help selecting the CFLs also report they received the help they needed. Five of these nine builders say ICF gave them the help they needed to make sure they were selecting the right CFLs, one asked his electrician about wattages, one says the lighting supply house helped choose the size of CFLs to order, one says EFI helped him compile his order, and one went to the website. The builder who went to the website says that he was looking for the right size bulb for a range hood and ended up ordering the wrong thing—a candelabra bulb. Only one builder who needed help says he did not get the help he needed from the Program; this builder says that he learned what CFLs to order by trial and error.



4.2.2 Who Ordered CFLs Almost all interviewed builders (93% or 28 of 30 builders) say they did the actual ordering of the CFLs; one builder says his electrician ordered the CFLs and one says the homeowner ordered them. (Figure 4-5)

Figure 4-5: Who Ordered CFLs

Only one builder says that he had any problems ordering the CFLs. This builder was new to the Program in 2008 and says that he was not able to reach ICF to get help; it is clear from his comments that he got caught in the change of the ICF staff person in charge of the CFL process. Two builders commented on how smooth the ordering process was. One describes the process as very smooth and one says:

“It is seamless to order via email, so fast, I have never seen a delivery as quick as they (EFI) are. It is phenomenal. They could improve one thing. They don't label boxes as to where they are going. I order separately by house; if I order for ten homes I get at least ten boxes of bulbs, but have no clue about what box goes to what house. I need to look at the packing slip and sometimes the lot number is there, but it is a little cumbersome. Sometimes you have to unpack the whole box to find the packing slip. The outside of the box just has your address. If we knew where the box was going we could move it to the house without having to open and unpack it first.”

Another builder, a production builder, commented:

“My HERS rater gave me the forms to send in. I had to order the bulbs and it is difficult to project how many of each bulb type we will need. People sometimes change their mind about what fixtures they want, which changes the number and type of bulbs you need—it is a pain. To do the orders home by home is difficult




for us. We are a production builder. Now I do the orders for ten homes at a time and my site people tell me when they are running low on a specific type of bulb.”

4.2.3 Who Installed CFLs Builders, electricians and construction crew members installed the CFLs. Figure 4-6 shows that one-third of interviewed builders say their electrician installed the CFLs, just over one-fourth (27%) say their construction crew installed the CFLs and one-fifth say they installed the CFLs. One-fifth of interviewed builders (six builders) gave other responses: three say they and their electrician installed the CFLs, one says his electrician and construction crew installed the CFLs, one says he and his construction crew installed the CFLs and one builder had not yet received the CFLs at the time of the interview.

Figure 4-6: Who Installed CFLs


5 CFL Ordering, Delivery and Installation Interviewed builders were asked several questions related to the ordering, delivery and installation of CFLs. These questions addressed:

• The number of CFL orders placed per home • How quickly CFLs were delivered • Ability to install all the CFLs ordered • CFLs installed in empty sockets or replaced incandescent bulbs • Number, type and location of CFLs installed

5.1 Ordering CFLs Most interviewed builders say they ordered all the CFLs they needed in one order. Figure 5-1 shows that 77% of builders (23 builders) ordered all the CFLs they needed in their first order, 20% (6 builders) identified additional sockets they needed to order CFLs for after they received the CFLs from their first order and 3% (1 builder) described a different situation; he says that he usually makes two orders per home so that he does not get too many CFLs at one time.

Figure 5-1: Single or Multiple CFL Orders

One of the builders who ordered all the CFLs he needed in one order says that he orders for multiple units at one time; another reports that he orders four or five extra CFLs to make sure he has enough. Two additional builders who order all the CFLs they need at one time say that they ordered the wrong CFLs in a couple of cases; one of these builders reordered the correct CFLs from EFI and the other says that he bought the right CFLs on his own rather than re-order. The six builders who say they identified additional sockets that needed CFLs after they received their first order all say they ordered the additional CFLs they needed from EFI rather than installing



incandescents. One of these builders says, “Sometimes I buy a CFL locally. When we buy it, it might be a different color—you need to make sure you use the same brand for all CFLs in the same room.”

5.2 CFL Delivery Most interviewed builders received all ordered CFLs in one delivery; one builder had not received the CFLs he ordered at the time of the interview. Figure 5-2 shows that over three-fourths of builders (77% or 23 builders) received all the CFLs they ordered in one delivery and for 23% of builders (7 builders) at least some CFLs were back-ordered. All 29 builders who had received CFLs at the time of the interview say that they received their CFLs within a week or less of ordering them, or within a week or less of when they were back-ordered. None of the builders say that the back-ordered CFLs delayed their construction schedule, though one builder commented:

“It can make it tight for a closing—I sometimes steal from bulbs for another house to close. As long as I have a few orders here it is not a problem, but having an inventory of two or three bulbs of each type would be helpful.”

Figure 5-2: back-ordered CFLs



It is our understanding that EFI sends ordered CFLs via UPS overnight shipping. Figure 5-3 shows that one-half of the builders say someone needed to sign for the CFLs, one-third do not know and 17% say no one needed to sign for the delivery.

Figure 5-3: Need to Sign for CFL Delivery

Regardless of whether or not builders knew if someone needed to sign for the CFL delivery, all 22 builders who commented on how their CFLs were delivered say that needing to sign is not an issue either because they have the CFLs delivered to their office, construction site or home, where someone is always available, or that they get many UPS deliveries and UPS knows where to leave the deliveries and does not ask for a signature.



5.3 CFL Installation

5.3.1 Able to Install all Ordered Bulbs Most interviewed builders say they were able to install all the CFLs they ordered. Figure 5-4 shows that 77% of interviewed builders (23 builders) say they were able to install all the CFLs they ordered, 20% (6 builders) say they were not able to install all the CFLs they ordered and one builder had not received the CFLs he ordered at the time of the interview.

Figure 5-4: Able to Install All Ordered CFLs

Two of the builders who say they were not able to install all the CFLs they ordered say some CFLs did not fit; both of these builders say they saved the CFLs to use in another home. One of two additional builders, both building multiple homes, who say they were not able to install all the CFLs they ordered says he orders for more than one home at a time, so he has extra CFLs; the other builder says that he orders CFLs by the case and that the CFLs he does not install in one home he installs in the next home. One builder who was not able to install all the CFLs he ordered says: “At first I ordered all the wrong kinds; some were too small and meant for candelabras, some were not dimmable and some did not fit into the sockets.” (This builder returned the CFLs he could not use to EFI.) The final builder who says he was not able to install all the CFLs he ordered says that a few of them were defective; he replaced the defective CFLs with ones he purchased locally.

Four of the builders who reported that they were able to install all the CFLs they ordered say that if they are short just a few CFLs they usually have extras on hand to use. One of these builders says that he orders at least one extra CFL for each house because he usually has one or two CFLs per house that are bad. Another builder says that if he is short a few bulbs he buys the ones he needs locally.

5.3.2 CFLs Replaced Incandescents or Installed in Empty Sockets Figure 5-5 shows that one-half of the interviewed builders (15 builders) say they installed the free CFLs in empty sockets, just under one-fourth (23% or 7 builders) say they replaced



incandescent bulbs, just under one-fourth (23% or 7 builders) say that some CFLs were installed in empty sockets and some replaced incandescent bulbs, and 3% (1 builder) says he installed some CFLs in empty sockets and replaced existing CFLs in other sockets.

Figure 5-5: Installed CFLs in Empty Sockets or Replaced Incandescents

One-half of the 14 builders who replaced at least some incandescent bulbs saved the bulbs they removed. On builder says his electricain kept the incandescent bulbs and the other six builders say they:

• Gave them to the homeowner (3 builders) • Made them available to anyone who wanted them (1 builder) • Gave them to the management company that runs the building (1 builder) • Do not remember (1 builder)

Builders who installed all CFLs in empty sockets in 2008 say they will continue that practice going forward. Nine of the fifteen builders who replaced incandescent bulbs in at least some sockets say that in the future they plan on asking their electrician not to install incandescent bulbs in sockets for which they have ordered CFLs. Three builders who replaced incandescent bulbs in at least some sockets say they will continue to install incandescents and then replace them with the free CFLs. The reasons these three builders give for continuing to initially install incandescent bulbs are:

• “There is a timing problem; I have the fixtures in and want to take prospective buyers through the house, so I need light.”

• “I deal with modular homes and the incandescents are installed at the factory.” • “We need the lights for visual accessibility and put in the CFLs before closing on the

home.”



Two builders who say that they replaced at least some incandescent bulbs say “maybe” or “probably” they will ask their electricians to leave sockets empty until the CFLs can be installed; one of these builders says that if he orders the CFLs early and they are there when the fixtures arrive, he will have his electrician install CFLs.

5.3.3 Number of CFLs Installed The 22 interviewed builders who had free CFLs installed in their ENERGY STAR homes by HERS raters prior to 2008 were asked, “Would you say that when you or your electrician selected what CFLs to install in your 2008 home(s) that you installed more or fewer CFLs than would have been installed under the previous process when the Program selected and installed the CFLs?” Figure 5-6 shows that 13 of these builders (59%) say that they installed the same number of CFLs in their 2008 homes as HERS raters would have installed under the old process and 9 of these builders (41%) say they installed more CFLs. No builders say they installed fewer CFLs than would have been installed under the old process.

Figure 5-6: Installed More or Fewer CFLs under 2008 Process

Eight of the nine builders who say that they installed more CFLs under the 2008 process say that they installed CFLs in applications where they think HERS raters might not have installed them. Five of these builders say that they ordered CFLs for every available socket. Builders reported installing more CFLs in the following applications: recessed fixtures, dome-covered fixtures, exterior fixtures, walk-in closets, laundry rooms, basements and attics. One builder commented that his HERS rater missed a lot of ceiling fixtures because he only had a stepstool and needed a ladder to get to the 10 and 12 foot ceilings normally in his homes.

Seven of the eight builders who report installing CFLs in more applications also say they think they are installing more types of CFLs than their HERS rater might have installed. Builders report installing dimmables, smaller chandelier bulbs, indoor globes over vanities, and floodlights. One builder says that the HERS rater installed only the spiral bulbs, but that now he




installs the spirals in basements and other such areas and installs the bulbs that look like incandescents in the main part of the house. Another builder says:

“I am trying to match the output wattage to what was previously there. We had some complaints about CFLs not giving out the same amount of light as the incandescents. If a fixture requires a 60 watt or less I learned you can put in anything up to a 60 watt CFL. So, in some areas I can put in a 100 watt equivalent CFL instead of a 60 watt equivalent because it is really only drawing 23 watts. You have to watch what you are installing. The spiral twist bulbs seem to give off more light per wattage than the globes—the globes are lower light and slower to come on, but they look pretty. I install the higher wattage CFLs where they are most compatible.”


6 Problems with CFLs Failing As Figure 6-1 shows, one-half of the 30 interviewed builders report having some CFLs fail; 47% (14 builders) say that they have not had a problem with CFLs failing and one builder had not yet received the CFLs he ordered.

Figure 6-1: Problem with CFLs Failing

Several of the builders who report having CFLs fail commented that very few fail—only one or two per home. One builder reports that two-thirds of the failures in his homes have been recessed CFLs. Another builder commented:

“One to two bulbs per house fail if we are installing 35 to 50 bulbs per home. It is not that bad a ratio—it is the same with regular bulbs as well. You have to teach people to hold the base, not the spiral, when they install the CFLs or they may break, and sometimes a CFL gets dropped—these things are our fault, not the Program's.”

Only four of the fifteen builders who report having CFLs fail say that they contacted EFI to get free replacements. Three of the eleven builders who had CFLs fail and did not contact EFI for replacments say they did not know about the free replacement option; an additional builder was aware of the warranty, but did not know it was good for two years and could be passed on to the homeowner. The remaining seven builders who had CFLs fail and did not contact EFI for replacments say very few CFLs failed and that they either had some extras on hand or purchased some on their own to replace the failed CFLs.



7 HERS Raters Verified CFLs Installed All interviewed builders were asked, “As far as you know, has your HERS rater or anyone else associated with the Program verified that all the CFLs you ordered through the Program were installed?” Figure 7-1 shows that builders provided a variety of responses. Only 10% of builders (three builders) say that their HERS rater counts the number of CFLs installed; 30% (nine builders) say that their HERS rater verifies CFLs are installed, but do not mention them counting the CFLs; 20% (six builders) say that their HERS rater verifies CFLs are installed, but does not count them; and 30% (nine builders) say that they are not aware of anyone verifying the CFLs they ordered were installed. Ten percent of builders reported either that the final inspection on their home was done before the CFLs were installed (one builder) or that their homes had not yet had the final inspection at the time of the interview (two builders).

Figure 7-1: HERS Rater Verified CFLs Installed



8 Final Builder Comments At the end of the interview builders were asked if there was anything else they would like to say about the free CFL process. Ten builders provided comments; four of these builders had suggestions for the free CFL component of the Program.

• “It worked well, but needs to do a better job of promoting products; it is great that dimmables are available and they should have been promoted to builders since they are a great selling point for CFLs.”

• “It has worked well; it would be nice if the Program tried to get LEDs.” • “I am disappointed that pin-based bulbs are not included—only screw ins. It is too bad

there are no incentives up front for installing fixtures.” • “The bulbs should be on site earlier or the house designed to accommodate the CFL

bulbs.”

Six builders expressed their satisfaction with the free CFL component of the Program:

• “It is a great Program and more economical. It is good from our customers' view point because many of them were already buying CFLs for their fixtures.

• “It is seamless.” • “Once we did it a couple times it was fine. The only ones who don't like it are the electric

supply houses—they want to sell bulbs as well as fixtures.” • “At first, being used to having the HERS rater do it, I was a little scared and didn't want

to do the wrong thing. But, the HERS rater helped and everything has been fine.” • “I am very pleased. It is a bonus to get paid for putting them in—nice, but not necessary.” • “It is a great Program and I much appreciate it as an affordable home provider.”



9 Conclusions and Recommendation The builder interviews point to several conclusions about the 2008 CFL process. .

• Most (55%) of the 22 interviewed builders who have installed free CFLs under both the old and new process like selecting, ordering and installing the free CFLs themselves better than having their HERS rater select and install the CFLs, just over one-fourth (27%) prefer having their HERS raters select and install CFLs and 18% are indifferent, saying both processes work equally well.

• The feature builders like most about the 2008 process is being able to install the CFLs on their schedule, typically at the same time the lighting fixtures are installed.

• The features builders like most about the pre-2008 process are that they do not have to install the CFLs and there is no paperwork.

• Builders report installing the same number of CFLs (13 of 22 builders) or more CFLs (9 of 22 builders) under the 2008 process than they think their HERS raters would have installed under the old process.

• Builders tend to take responsibility for selecting and ordering CFLs rather than having their electricians take on these functions.

• CFLs are delivered within a few days of being ordered; none of the builders interviewed report CFL delivery interfering with their construction schedule.

• Electricians and construction crew tend to install the CFLs. • Very few CFLs fail, and when only a few CFLs fail most builders either use CFLs they

have on hand or purchase new ones on their own to replace the ones that failed rather than return the failed CFLs to EFI.

• Most builders say their HERS rater verifies that CFLs are installed, but very few builders say their HERS rater counts the number of CFLs installed to verify that all ordered CFLs were installed.

These conclusions support the Program’s plan going forward to give builders the option of selecting, ordering and installing the free CFLs themselves or having their HERS rater select and install them. Different builders have different needs and preferences as well as different levels of knowledge about and experience with CFLs. Offering both approaches allows experienced ENERGY STAR builders who are comfortable selecting CFLs on their own and prefer having the CFLs on site to install when the light fixtures are installed to do so. As of 2009, the Program requires builders to install CFLs in at least 50% of all hard-wired screw-based fixtures. Builders who are new to the Program or who are not very knowledgeable about the variety of CFLs available or which ones work best in specific applications will benefit from having their HERS raters choose and install the CFLs. Builders without much experience with CFLs may, on their own, be likely to install the minimum number of CFLs required by the Program, but if HERS raters select and install the CFLs they can encourage the installation of




CFLs in more than 50% of appropriate sockets. Offering both processes will likely maximize the number of CFLs installed through the Program.

MA New Homes with ENERGY STAR: Review of 2008 CFL Installation Process Page A1

APPENDIX A CFL Process Builder Interview Questions 2008 Massachusetts New Homes with ENERGY STAR Program

Review of CFL Installation Process Builder Interview Questions

Builder background questions: First I have a few basic questions about your experience with the Massachusetts New Homes with ENERGY STAR Program: 1. How many years have you been building ENERGY STAR homes? 2. How many ENERGY STAR homes have you built? 3. Are you currently building an ENERGY STAR home? 4. How many of your ENERGY STAR homes included CFL bulbs installed for free through the

program? 5. Is it your standard practice now to install CFL bulbs in all your ENERGY STAR homes? 6. If not, why not? 7. Do you build some homes that you do not have ENERGY STAR-certified?

7.1. If yes, do you install CFL bulbs in those homes? 7.1.1. If yes, always or only some times? 7.1.2. If not always, why not?

CFL Process Questions: READ: As you know, in 2008 the program introduced some changes to the free CFL bulb portion of the program. In 2008, builders became responsible for selecting, ordering and installing the free CFL bulbs. I would like to ask you some questions about how you think this new process works. 8. Overall, how well do you think the process introduced in 2008 works compared to the

previous process where your HERS rater ordered and installed the CFL bulbs? 8.1. In what ways do you think the process in 2008 is better than the previous process? 8.2. In what ways do you think the old process was better?

Ask the following questions if the respondent did not address them in response to the above question. 9. How were the changes in how to get the free CFL bulbs explained to you? 10. Did you find the instructions for selecting and ordering CFL bulbs easy to understand and

follow? 11. If not, what did you find confusing or unclear and how did you get your questions answered? 12. Did you select what CFL bulbs to order or did you ask your electrician to specify what CFL

bulbs should be ordered?



12.1. Did you, or your electrician, need or want any help in deciding what bulbs to order? If yes, did you get the help you needed?

12.2. If yes, who helped you? 13. If you had free CFL bulbs installed in your ENERGY STAR homes completed before 2008,

would you say that when you or your electrician selected what CFL bulbs to install in your 2008 home(s) that you installed more or fewer bulbs than would have been installed under the previous process when the program selected and installed the CFL bulbs? 13.1. If installing more CFL bulbs, are you installing CFL bulbs in applications where you

think HERS raters might not have installed CFL bulbs? 13.1.1. If so, what applications?

13.2. If installing more CFL bulbs, are you installing more types of CFL bulbs than you think HERS raters might have installed?

13.2.1. If so, what types of CFL bulbs? 13.3. If installing fewer bulbs, did you decide not to install CFL bulbs in some applications

where you think HERS raters would have installed CFL bulbs? 13.3.1. If so, what applications?

13.4. If installing fewer bulbs, did you decide not to install certain types of CFL bulbs that you think HERS raters would have installed?

13.4.1. If so, what types of CFL bulbs? 14. Did you order the CFL bulbs from EFI or did you have your electrician order the CFL bulbs? 15. Did you run into any problems ordering the CFL bulbs?

15.1. If so, what problems? (Probe for problems such as confusion about who to send the order to—some builders sent their orders to ICF instead of EFI.)

16. It is our understanding that bulbs were sent via UPS overnight shipping. Did you receive the CFL bulbs within a day or two of ordering them?

17. Did someone need to be on site to sign for the CFL bulbs when they were delivered? 17.1. If so, was someone there to sign for the CFL bulbs when they were delivered?

17.1.1. If not, what did you have to do to reschedule the delivery? 18. Did you get all the CFL bulbs you ordered or were some back-ordered?

18.1. If some were back-ordered, how long did you have to wait for the CFL bulbs to be delivered?

18.2. Did this delay affect your construction schedule? 19. Were you able to use all the bulbs you ordered?

19.1. If not, why not? (Probe to find out if they ordered bulbs that did not fit where they thought they would, etc.)

19.2. What did you do with the bulbs you could not use? (Probe to find out if they returned unused bulbs to EFI, asked their HERS rater or ICF account manger to pick them up and return them, kept them to use in their next ENERGY STAR home etc.)

20. Did you order all the CFL bulbs you needed in your first order or did you identify additional sockets where you could use CFL bulbs after you made the first order?




20.1. If did not order all they could use in the first order ask: Did you order more CFL bulbs or did you install standard incandescent bulbs?

20.1.1. If installed standard incandescent, ask why they did not order more CFL bulbs?

Now I would like to ask you some questions about installing the CFL bulbs. 21. Who installed the CFL bulbs you ordered—you or your electrician, or someone else? 22. When the CFL bulbs were installed, were they installed in empty sockets or did you have to

replace already installed incandescent bulbs? 23. If the CFL bulbs replaced existing incandescent bulbs, what did you do with the

incandescent bulbs? 24. In the future, do you plan on asking your electrician not to install incandescent bulbs in

sockets that you have ordered CFL bulbs for? 24.1. If not, why not?

25. Have you had any problems with CFL bulbs failing? 25.1. If yes, did you contact EFI to get free replacement bulbs? 25.2. If not, why not? (Probe if they are familiar with the free replacement policy for CFL

bulbs that fail within the first two years.) 26. As far as you know, has your HERS rater or anyone else associated with the program verified

that all the CFL bulbs you ordered through the program were installed? 26.1. If so, how was this done?

Final Question 27. Is there anything else you would like to say about the free CFL bulb process?

27.1. If yes, what would you like to say?

Appendix D

Builder Interviews



2008 Massachusetts New Homes with ENERGY STAR®

Builder Interview Report

Final REPORT June 14, 2009

Submitted to:

The Joint Management Committee

Submitted by:



Contents EXECUTIVE SUMMARY ........................................................................................................... I

MARKETING ENERGY STAR HOMES .......................................................................................... I

Homebuyer Awareness of and/or Interest in Energy Efficiency ............................................. II

Homebuyer Awareness of and/or Interest in ENERGY STAR Homes .................................. II

Use of Marketing Support Offered by Program ...................................................................... II

Need for Additional Marketing Support ................................................................................ III

CHOOSING A HERS RATER .........................................................................................................III

TRAINING AND TECHNICAL SUPPORT ..........................................................................................III

CONCLUSIONS ............................................................................................................................ IV

RECOMMENDATIONS .................................................................................................................... V

1 INTRODUCTION ................................................................................................................. 1

2 ENERGY STAR BUILDER SAMPLE ............................................................................... 2

3 MARKETING ENERGY STAR HOMES .......................................................................... 5

3.1 VALUE OF BUILDING AND MARKETING ENERGY STAR HOMES .....................................5

3.2 HOMEBUYER AWARENESS OF AND/OR INTEREST IN ENERGY EFFICIENCY ..........................6

3.3 HOMEBUYER AWARENESS OF AND INTEREST IN ENERGY STAR HOMES ........................8

3.4 USE OF MARKETING SUPPORT OFFERED BY PROGRAM ....................................................11

3.5 NEED FOR ADDITIONAL MARKETING SUPPORT ................................................................14

4 CHOOSING HERS RATERS ............................................................................................ 16

5 TRAINING ........................................................................................................................... 18

6 FINAL BUILDER COMMENTS....................................................................................... 21

7 CONCLUSIONS AND RECOMMENDATIONS ............................................................ 24

7.1 CONCLUSIONS ..................................................................................................................24

7.2 RECOMMENDATIONS ........................................................................................................25

APPENDIX A BUILDER INTERVIEW QUESTIONS ..................................................... A1


Tables TABLE 2-1: BUILDER SAMPLE STATISTICS ...................................................................... 3

Figures FIGURE 2-1: YEARS BUILDING ENERGY STAR HOMES AND QUALIFIED

HOMES BUILT ................................................................................................................ 2

FIGURE 2-2: CURRENTLY BUILDING AN ENERGY STAR HOME?.............................. 3

FIGURE 2-3: BUILD NON-ENERGY STAR HOMES ............................................................ 4

FIGURE 3-1: HOW VALUABLE IS BEING ABLE TO MARKET HOMES AS ENERGY STAR QUALIFIED? ...................................................................................... 5

FIGURE 3-2: GREATER HOMEBUYER AWARENESS OF AND/OR INTEREST IN ENERGY EFFICIENCY ............................................................................................ 6

FIGURE 3-3: HOMEBUYERS NOW AWARE OF AND/OR INTERESTED IN ENERGY EFFICIENCY ................................................................................................. 7

FIGURE 3-4: HOMEBUYERS MORE AWARE OF AND/OR INTERESTED IN PURCHASING AN ENERGY STAR HOME ............................................................... 8

FIGURE 3-5: HOW MANY HOMEBUYERS NOW AWARE OF OR INTERESTED IN PURCHASING AN ENERGY STAR HOME ............................... 9

FIGURE 3-6: USING MARKETING SUPPORT OPTIONS OFFERED BY PROGRAM ..................................................................................................................... 11

FIGURE 3-7: MARKETING SUPPORT OPTIONS BEING USED..................................... 12

FIGURE 3-8: USE OF MARKETING SUPPORT BY VALUE OF BEING ABLE TO MARKET HOMES AS ENERGY STAR QUALIFIED ...................................... 13

FIGURE 3-9: LIKE BEING ABLE TO CHOOSE HERS RATER ....................................... 16

FIGURE 3-10: KEPT ASSIGNED RATER OR CHOSE A DIFFERENT RATER ............ 16

FIGURE 3-11: SATISFIED WITH LEVEL AND AMOUNT OF TRAINING/ SUPPORT AVAILABLE ............................................................................................... 18

FIGURE 3-12: WOULD LIKE ADDITIONAL TRAINING OFFERINGS ......................... 19

MA New Homes with ENERGY STAR: 2008 Builder Interview Report Page I


Executive Summary This year’s builder interviews focused on three basic topics: marketing ENERGY STAR-qualified homes, choosing HERS raters and satisfaction with the training and support provided by the Program. Thirty ENERGY STAR builders were interviewed in December 2008 and January 2009. The interviewed builders were randomly recruited from a list ICF provided of 66 ENERGY STAR builders who ordered CFLs through the Program in 2008. Although the questions in this year’s interviews are not identical to the questions asked in interviews with 40 ENERGY STAR builders1 conducted in November and December 2007, they are similar enough to make comparisons that suggest several very positive trends.2

• In the 2007 interviews, 78% of builders called for the Program to conduct more consumer marketing because of low homebuyer interest in ENERGY STAR homes. This year, 93% of interviewed builders (28 out of 30 builders) say that building and marketing ENERGY STAR-qualified homes in the current housing market is very valuable (70%; 21 builders) or somewhat valuable (23%; 7 builders).

• In the 2007 interviews, only five percent of builders said almost all homebuyers were aware of ENERGY STAR homes. This year, over one-half (57%) of interviewed builders say awareness of and/or interest in buying ENERGY STAR homes has increased, and the percentage of builders saying almost all homebuyers are aware of ENERGY STAR homes doubled to 10%.

• In the 2007 interviews only seven builders (18%) said they received help marketing their homes from the program. This year, one-half of the 30 interviewed builders say they are taking advantage of marketing support options offered by the Program to help them market their ENERGY STAR homes.

Marketing ENERGY STAR Homes As described above, 93% of interviewed builders say that building ENERGY STAR homes and being able to market them as ENERGY STAR qualified is very or somewhat valuable in today’s depressed and very competitive housing market.

1 Because a key purpose of the 2007 builder interviews was to assess the impact of the transition to a new implementation contractor and marketing approach it was important to have a pool of builders to draw from that had completed homes under ICF and/or signed agreements in 2007. Interviewed builders were randomly recruited from lists provided by ICF of builders falling into the following four targeted groups: 1) builders who signed up for the first time in 2007 and completed a home, 2) builders who signed a project before 2007 under CSG and completed a home after March 2007 under ICF, 3) builders who previously completed qualified homes under CSG and signed up a new project under ICF, and 4) builders who had projects that needed to be re-signed by ICF because their agreement expired before their project was completed. 2 Please note that due to the small sample sizes the results of neither the 2007 nor the 2008/2009 interviews should not be considered statistically representative of the population of Massachusetts ENERGY STAR builders; rather, the results should be viewed as qualitative in nature.

MA New Homes with ENERGY STAR: 2008 Builder Interview Report Page II


Homebuyer Awareness of and/or Interest in Energy Efficiency Almost all interviewed builders, 90% or 27 of 30 builders, say that homebuyers are showing more awareness of and/or interest in energy efficiency. Roughly two-thirds of the 27 interviewed builders who say homebuyers are showing more awareness in and/or interest in energy efficiency say that almost all homebuyers are now aware and/or interested. On the downside, five builders say that homebuyer interest in energy efficiency rises and falls with fuel prices.

Some builders commented that homebuyers are most interested in heating and cooling costs— efficient heating systems and good insulation. A builder of low-income housing says the participants in a focus group he conducted were very excited when they learned they would be buying high-quality energy-efficient homes—they had assumed low-income housing was low-cost construction.

Homebuyer Awareness of and/or Interest in ENERGY STAR Homes A majority of interviewed builders (57%) report homebuyers are showing more awareness of and/or interest in buying an ENERGY STAR-certified home. Four builders say buyers came to them specifically looking for an ENERGY STAR home and another two builders say that certification helped sell some of their homes. Several builders also say that even though many customers are aware of ENERGY STAR homes they (the builder) need to educate them on what is involved in building an ENERGY STAR home and the benefits of living in one.

Builders who do not report increased homebuyer interest in ENERGY STAR homes tend to say homebuyers have not heard of ENERGY STAR homes—that homebuyers know about ENERGY STAR appliances, but not ENERGY STAR homes. One builder of very large homes says, “ENERGY STAR isn't what my buyers come looking for.”

Use of Marketing Support Offered by Program One-half of the 30 interviewed builders say they are taking advantage of marketing support options offered by the Program to help them market their ENERGY STAR homes. Of the 15 builders taking advantage of marketing support, 67% (10 builders) used ENERGY STAR signage, 60% (9 builders) got help incorporating the logo and messaging in their model homes (one builder had a Program representative attend an open house), 53% (8 builders) got help incorporating the logo and messaging into their advertising and websites, 47% (7 builders) got help incorporating the logo and messaging into their sales process, 27% (4 builders) had sales staff training, 27% (4 builders) used brochures available through the Program, 7% (1 builder) had Program staff talk to the Realtors they work with and 7% (1 builder) had Program staff talk with a group of homebuyers.

Eleven of the 15 builders taking advantage of marketing support from the Program say that they think it is helping them sell their ENERGY STAR homes by letting buyers know they build energy-efficient homes and moving people to ask questions about what is involved in building an ENERGY STAR home.

MA New Homes with ENERGY STAR: 2008 Builder Interview Report Page III


When the builders who are not taking advantage of marketing support offerings were asked why, three say that they build low-income housing and marketing is not an issue, a custom-home builder says he does very little marketing, one builder converted his building to rentals, one builder says his company has their own way of doing things and two say they were not aware of the marketing support available. Three builders say they plan to take advantage of marketing support on their next projects and two say they have not taken the time to look into what is available.

Need for Additional Marketing Support All builders were asked if there if there is something not currently offered that they would like to see the program do to help them market their ENERGY STAR homes. Builders provided the following suggestions:

• “Match a portion of advertising dollars—we participated in this in the past when it was offered by the Program.” (market-rate builder)

• “Something that shows how savings relate to the HERS scale—something visual and not vague.” (one market-rate builder and one builder of both market-rate and low-income housing)

• “Maybe videos or DVDS we can play for or give to homebuyers.” (market-rate builder)

• “Bring back the HERS folders given to homeowners.” (market-rate builder) • “More giveaways etc.” (market-rate builder) • “A blue ENERGY STAR flag that would move like a balloon to let people know

when the model home is open.” (market-rate builder)

Choosing a HERS Rater Interviewed builders like being able to choose their HERS raters and most builders who had a HERS rater assigned to them in 2007 continued to work with the same rater in 2008. Only 3 of the 22 interviewed builders who built ENERGY STAR homes in both 2007 and 2008 chose a different rater to work with in 2008. Their reasons for changing raters all address the need to have a good working relationship with their rater. Most importantly, all three builders who changed raters are satisfied with the raters they are now working with.

Training and Technical Support Roughly three-fourths (73%) of the 30 builders interviewed in late 2008 or early 2009 say they are satisfied with the level and amount of training/support available to them through the Program and/or their HERS rater. This is consistent with what the 40 ENERGY STAR builders interviewed in 2007 said; 78% of builders interviewed in 2007 said they were either extremely satisfied (25%) or satisfied (53%) with the technical support they received.

MA New Homes with ENERGY STAR: 2008 Builder Interview Report Page IV


The 30 builders interviewed in late 2008 or early 2009 are virtually split between whether or not they think the Program needs to offer additional training—15 say no, 14 say yes and one does not know. One production builder who commented on why he does not think additional training is needed says, “We build the same house over and over, so once we have it down the only time we need more training is when the rules change.” Four other builders who commented say that there is help available if you ask for it or study what is available on the Program’s website.

The 14 builders who would like to see additional training offerings provide a variety of suggestions including: refresher courses; annual review meetings with small groups of ten or fewer builders to go over Program changes and what they (the builders) are doing well and how they could improve; seminars on the HERS system, the certification process and Program changes; seminars or training on new technologies and educational marketing; and training on a variety of specific technical topics.

Conclusions Builders’ perception of the value of having the ENERGY STAR label on a home has increased. Almost all (93%) of interviewed builders say that building ENERGY STAR homes and being able to market them as ENERGY STAR qualified is very or somewhat valuable in today’s housing market.

The Program’s increased marketing to consumers is very likely a factor contributing to increased homebuyer awareness of ENERGY STAR homes. Almost all (90%) of interviewed builders say that homebuyers are showing more interest in and/or awareness of energy-efficiency and 57% say that homebuyers are showing more awareness of and/or interest in buying an ENERGY STAR home. One of the most encouraging findings from this year’s interviews is that four builders reporting homebuyers looking for an ENERGY STAR home and another two builders saying that certification helped sell some of their homes.

Listing marketing support offerings on the application form and asking builders to check the options they are interested in appears to have increased awareness of and the likelihood of builders taking advantage of available marketing support. The percentage of interviewed builders taking advantage of available marketing support options increased from 18% of builders interviewed in 2007 to 50% of builders interviewed in late 2008 and early 2009.

Builders like being able to choose their HERS rater. The few interviewed builders who did not like working with the HERS rater originally assigned to them have successfully selected and established good working relationships with different raters.

Satisfaction with the training and technical support provided by the Program and HERS raters remains high at 73%. The need for training and support clearly varies from builder to builder. A challenge for the Program will be to identify and prioritize training and support offerings, beyond the on-site support HERS raters offer, that will provide the most overall value to participating builders.

MA New Homes with ENERGY STAR: 2008 Builder Interview Report Page V


Positive trends in homebuyer awareness of and interest in energy efficiency and ENERGY STAR homes combined with 93% of interviewed builders saying that building ENERGY STAR homes and being able to market them as ENERGY STAR qualified is very or somewhat valuable in today’s housing market bodes well for the Program as it strives to achieve broader participation and deeper savings.

Recommendations Based on the above conclusions, the NMR team makes the following recommendations.



• Produce more case studies appropriate for use in marketing to builders and homebuyers. There are now several builders saying they have homebuyers coming to them looking for an ENERGY STAR home and several builders saying buyers are very happy with the low operating cost of their ENERGY STAR homes. One builder has documented that the over-55 ENERGY STAR homes he builds command a higher resale price. These are messages likely to be effective in attracting new builders to the Program and generating homebuyer interest in ENERGY STAR homes.

• Review the training and support suggestions provided by interviewed builders and incorporate those that will provide the most overall value to the Program. Builders provided a variety of suggestions for additional training/support. Given that the sample of interviewed builders is small and individual builder training and support needs vary greatly, it is important that Program staff factor in their understanding of the full population of participating builders and their training/support needs in prioritizing and selecting training options that will facilitate the Program’s goals to move builders toward building zero energy homes.

• Start planning now for training builders to meet ENERGY STAR 2011 requirements. Builders like to know ahead of time about changes to the Program. The NMR team believes that educating builders in 2009 about changes likely to be implemented in 2011 and assuring them that in 2010 the Program will provide the training needed to ensure they can meet the new requirements will help maintain builder participation and the increasing penetration rate of ENERGY STAR homes in the single family housing market.

MA New Homes with ENERGY STAR: 2008 Builder Interview Report Page 1


1 Introduction This report presents findings related to questions addressing satisfaction with the Program’s marketing support, process for selecting Home Energy Rating System (HERS) raters, and technical support and training that were asked during in-depth interviews with 30 ENERGY STAR builders in Massachusetts. The interviews, conducted by phone in December 2008 and January 2009, also addressed non-energy impacts (NEIs) and the builders’ experience with the free compact fluorescent lamp (CFL) component of the Program. Findings related to questions addressing NEIs can be found in the Assessment of Non-Energy Impacts report3 and findings related to the free CFL component of the Program can be found in the Review of 2008 CFL Installation Process report4.

Interviewed builders were recruited from a list provided by ICF, the Program implementation contractor, of ENERGY STAR builders who participated in the free CFL component of the Program in 2008. The list consisted of 66 builders who had ordered CFLs through September 2008. The NMR team randomly called the builders and, in some cases, emailed them as well. Interviewed builders received $75 for their participation.

The interview questions addressing builders’ experience and satisfaction with available marketing support, choosing a HERS rater, and the training offered by the Program are in Appendix A. The key objectives of these questions are to assess:

• Perceived value of being able to market homes as ENERGY STAR qualified • Builder assessment of homebuyer awareness of and interest in energy efficiency and

ENERGY STAR-qualified homes • Builder use of and experience with marketing support options offered by the Program • Satisfaction with the change in 2008 to having builders select their HERS rater • Satisfaction with the level and amount of training/support available from the Program

and HERS raters • Additional builder training needs

Please note that due to the small sample size the results should not be considered statistically representative of the population of Massachusetts ENERGY STAR builders; rather, the results should be viewed as qualitative in nature.

3 Nexus Market Research, Inc. Massachusetts New Homes with ENERGY STAR® Assessment of Non-Energy Impacts, submitted to the Joint Management Committee, March 2009. 4 Nexus Market Research, Inc. Massachusetts New Homes with ENERGY STAR® Review of 2008 CFL Installation Process, submitted to the Joint Management Committee, May 2009.



2 ENERGY STAR Builder Sample The 30 interviewed builders include a mix of experienced ENERGY STAR builders and builders new to the Program; experience building ENERGY STAR homes runs from a builder who has not yet completed an ENERGY STAR-qualified home to a builder who estimates he has completed roughly 600 ENERGY STAR-qualified homes. The interviewed builders build custom homes, spec homes and low-income housing. Figure 2-1 shows the number of years each interviewed builder has participated in the Program and the number of qualifying homes they have built. Six interviewed builders have completed 100 or more ENERGY STAR homes; five of these builders have participated in the Program for at least six years. One interviewed builder, who joined the Program in 2008, has not had a home certified; this builder built two homes in 2008, installed the free CFLs in both homes, and both homes failed to pass the final inspection.5

Figure 2-1: Years Building ENERGY STAR Homes and Qualified Homes Built

5 This builder plans on continuing to participate in the Program and will use a better insulation contractor to ensure meeting all Program requirements for ENERGY STAR certification.



Table 2-1 shows interviewed builders have participated in the Program from one to twelve years; the average is four years and the median is three years. Interviewed builders have had from zero to 600 homes ENERGY STAR-qualified; the average is 63 homes and the median is 19 homes.

Table 2-1: Builder Sample Statistics

Statistics

Years Building ENERGY STAR

Homes (n=30)

Number of ENERGY STAR

Homes Built (n=30)

Minimum 1 0 Maximum 12 600 Average 4 63 Median 3 19

Figure 2-2 shows that most interviewed builders, 77% or 23 builders, say they are currently building an ENERGY STAR home. Six builders say they are not currently building an ENERGY STAR home; five of these six builders say they build only ENERGY STAR homes and one is a small custom builder currently working on remodeling projects. One builder, who says he may be building an ENERGY STAR home, subbed out work on a home and is not sure it will meet ENERGY STAR requirements.

Figure 2-2: Currently Building an ENERGY STAR Home?

Figure 2-3 shows that only 17% of interviewed builders (five builders) say they build any non-ENERGY STAR homes. One of these builders says his company builds only ENERGY STAR homes, but is taking over a 67 home project in which the first 12 homes are too far along to pass the Thermal Bypass Checklist (TBC); the remaining 55 homes will be ENERGY STAR. Two of



these builders say they have built only one or two non-ENERGY STAR homes in the last several years. The remaining two builders say only that they build some non-ENERGY STAR homes.

Figure 2-3: Build Non-ENERGY STAR Homes



3 Marketing ENERGY STAR Homes Builders questions addressed the value of building ENERGY STAR homes and being able to market them as ENERGY STAR-qualified homes in the current housing market as well as homebuyer awareness of, and interest in, energy efficiency and buying an ENERGY STAR-qualified home.

3.1 Value of Building and Marketing ENERGY STAR Homes Almost all interviewed builders say that building ENERGY STAR homes and being able to market them as ENERGY STAR qualified is valuable in today’s depressed and very competitive housing market. Figure 3-1 shows that 93% of interviewed builders (28 out of 30 builders) say that building and marketing ENERGY STAR-qualified homes in the current housing market is very valuable (70%; 21 builders) or somewhat valuable (23%; 7 builders). One builder (3%) says building and marketing ENERGY STAR homes is not valuable at all to him because no one has heard of ENERGY STAR homes—this is a builder of market-rate homes who reports building 20 ENERGY STAR homes in the last 4 years. Another builder (3%), a non-profit builder of low-income housing, says that building and marketing ENERGY STAR-qualified homes is not very valuable to him because he does not have to market his homes—people are lined up to purchase them.

Figure 3-1: How Valuable is Being Able to Market Homes as ENERGY STAR Qualified?

Several of the builders who say building and marketing ENERGY STAR homes is very valuable commented on their responses. Examples include four builders of market-rate housing saying it



is “very, very valuable,” “extremely valuable” or “incredibly important.” Three additional builders of market-rate housing comment:

• “We actually had two sales because of the homes being ENERGY STAR. Everyone is heading toward the green side. For sales purposes and marketing, the impact of ENERGY STAR is huge. People are now thinking more about saving energy.”

• “All builders are going to green. When talking to customers, ENERGY STAR gives you a competitive edge over the builder down the street who is not ENERGY STAR.”

• “It is very valuable for the benefits we get back (rebates etc.) because it helps. Everybody now, with today's housing market, is looking for energy efficiency and ways to save money so the ENERGY STAR sign makes a difference.”

Two builders of market-rate housing who say that building and marketing ENERGY STAR homes is somewhat valuable comment:

• “I don't think people go out and look for an ENERGY STAR home, but when they see one, it has value. They still respond more to granite countertops etc. than energy efficiency. They were more interested when fuel prices spiked—after the fuel spike faded, interest in energy efficiency faded away, but ENERGY STAR does help justify their purchase.”

• “I am surprised it is not more important to people. They [homebuyers] are surprised there even is a program.

3.2 Homebuyer Awareness of and/or Interest in Energy Efficiency Almost all interviewed builders, 90% or 27 of 30 builders, say that homebuyers are showing more awareness of and/or interest in energy efficiency. (Figure 3-2)

Figure 3-2: Greater Homebuyer Awareness of and/or Interest in Energy Efficiency



One builder says that he does not know if homebuyers are showing more interest in energy efficiency because when oil prices fall their interest evaporates. A builder of low-income housing who does not think homebuyers are more interested in energy efficiency says:

“I think low-income housing buyers are just beginning to be interested in energy efficiency. It is a very good program in that it helps them afford to stay in their homes; but most low-income housing buyers are so stretched that all they can think about is the price of the home.”

Figure 3-3 shows that roughly two-thirds of the 27 interviewed builders who say homebuyers are showing more awareness of and/or interest in energy efficiency say that almost all homebuyers are now aware and/or interested, 22% say about one-half and 7% say less than one-half of homebuyers are aware and/or interested. One builder, the “other” category, says, “We make them aware.”

Figure 3-3: Homebuyers Now Aware of and/or Interested in Energy Efficiency

Five builders say that homebuyer interest in energy efficiency rises and falls with fuel prices. Four other builders say homebuyers are not that interested in energy efficiency until they (the builder) bring it up. As described by one builder, “On their own, people are not construction savvy and the builder needs to bring up energy efficiency.” One builder says, “Half are just aware—the other half is not only aware but also interested.” Another builder, who says that about one-half of homebuyers are now aware of and/or interested in energy efficiency, says that in the past almost nobody was interested.

Builders were asked to provide examples of homebuyer awareness of and interest in energy efficiency. Examples of their responses are:



• “Everyone is interested but fewer are willing to pay more for energy efficiency. Generally, savvy homebuyers are most interested in good insulation and efficient heating systems and less in CFLs.” (market-rate builder)

• “We did a focus group of buyers who would qualify for low-income housing. When we brought up the subject of energy efficiency, the participants were very excited; they had assumed low-income housing was low-cost construction and we told them that they were buying high-quality energy-efficient homes.” (non-profit low-income housing builder)

• “They're interested and builders have to do what they can—as long as it doesn't add too much to costs—to move houses in this market.” (market-rate builder)

• “Buyers ask about heating and cooling cost. We say we are doing our best, but can't quote specific numbers because everyone uses energy differently. We have meet and greet parties with interested buyers and people who already live in our homes; this allows people to talk about what their actual heating and cooling bills are and they tell prospective buyers how low they are.” (market-rate builder)

3.3 Homebuyer Awareness of and Interest in ENERGY STAR Homes A majority of interviewed builders (57%) report homebuyers are showing more awareness of and/or interest in buying an ENERGY STAR-certified home. (Figure 3-4)

Figure 3-4: Homebuyers More Aware of and/or Interested in Purchasing an ENERGY STAR Home

Twelve of the thirteen builders who say homebuyers are not showing more awareness of and/or interest in buying an ENERGY STAR-certified home commented on why they say this. Ten of these builders say homebuyers have not heard of ENERGY STAR homes; six of these ten



builders say homebuyers know about ENERGY STAR appliances, but not ENERGY STAR homes. Two builders commented in more detail as follows:

• “I don't think any of my buyers were aware that homes could be ENERGY STAR qualified. However, I do think that ENERGY STAR qualification makes buyers a lot happier about their purchase in this market and that the certification may have helped close the deal for a couple of homes.” (market-rate builder)

• “I haven't had any feedback yet. No one has come looking for an ENERGY STAR home. I build monster homes and ENERGY STAR isn't what my buyers come looking for.” (market-rate builder)

The 17 builders who say homebuyers are showing more awareness of and/or interest in buying an ENERGY STAR-qualified home were asked how much more awareness and/or interest they were seeing. Figure 3-5 shows that 18% of these 17 builders say almost all homebuyers are now aware of and/or interested in buying an ENERGY STAR home, 7% (one builder) says 75% of homebuyers, 47% say almost one-half of homebuyers, and 29% say fewer than one-half of homebuyers are aware of and/or interested in buying an ENERGY STAR home.

Figure 3-5: How Many Homebuyers Now Aware of or Interested in Purchasing an ENERGY STAR Home

Thirteen of the 17 builders who say they are seeing more awareness of and/or interest in purchasing ENERGY STAR homes provided examples of what they are seeing. One builder says that his company has been advertising their homes as ENERGY STAR for several years. A builder of modular homes says that he thinks buyers of modular homes tend to be more attuned to ENERGY STAR because they tend to do more web research than other home buyers. A



builder of low-income housing says that he is building ENERGY STAR homes because it is a requirement to get funding, but buyers are happy they are getting a well constructed home.

Four builders provided examples of buyers looking for ENERGY STAR homes:

• “Some people are aware [of ENERGY STAR homes] when they come to see our homes. One man is really aware and interested—he wants us to build using stress skin panels, etc.” (market-rate builder)

• “People know we are ENERGY STAR builders so we get the ones who are interested in ENERGY STAR homes.” (market-rate builder)

• “Buyers are excited to learn that our homes will be ENERGY STAR qualified; one buyer came to us specifically looking for an ENERGY STAR home.” (market-rate builder)

• “People say they noticed we build ENERGY STAR homes and came to us because of that.” (market-rate builder)

Six builders provided examples addressing marketing ENERGY STAR homes and the need to educate buyers:

• “We have ENERGY STAR signs on the property. I don't think they [buyers] quite comprehend what ENERGY STAR means in construction—they don’t come in with knowledge of that. It is up to us to explain why our homes get HERS ratings in the 60s.” (market-rate builder)

• “Most of my customers are interested in ENERGY STAR homes since they are buying in an ENERGY STAR development, though some are probably just there on price. We try to educate buyers, but most still don't understand ENERGY STAR homes.” (builder of market-rate and low-income housing)

• “Homebuyers don't know much about the Program, but they have an inkling—they may be aware there is a program, but not much more than that. The majority of homebuyers know about ENERGY STAR appliances, but not much about homes. If you explain it to them—that homes can be ENERGY STAR labeled—it is pretty easy for them to add two and two together and understand.” (market-rate builder)

• “We are trying to become more visible—added the logo to our website and a link to the ENERGY STAR Home website, but we do not have much signage for our models. We are trying to make some brochures—they are a hot commodity right now. We made some signs on sticks that we pushed into the grass that said we are an ENERGY STAR partner. A customer came in Saturday who didn't know the home they were looking at was ENERGY STAR and now they are thrilled. They talked to the neighbors, who are friends of theirs, and can't believe how low they said their utility bills are.” (market-rate builder)

• “Our signs out front saying we are an ENERGY STAR partner draw attention. I would say about 50% [of buyers] are aware of ENERGY STAR homes, but not informed



about what goes into the construction of an ENERGY STAR home, and the other 50% are both aware of and informed about what makes a home ENERGY STAR.” (market-rate builder)

• “The ENERGY STAR label on a home is becoming known. When buyers ask about energy efficiency we tell them what the ENERGY STAR label on a home means. Our sales office has a lot of posters and displays, and our sales people are trained to walk people through the process of making a home ENERGY STAR and how important that is.” (market-rate builder)

3.4 Use of Marketing Support Offered by Program One-half of the 30 interviewed builders say they are taking advantage of marketing support options offered by the Program to help them market their ENERGY STAR homes. (Figure 3-6) One of the two interviewees in the “other” category in Figure 3-6 is a job superintendent who says that he does not know about marketing because it is handled in a different part of the company. The second builder in the “other” category was new to the Program in 2008 and built two homes that failed to qualify for the ENERGY STAR label—going forward, this builder says that she plans on using a better insulation contractor to be assured that her homes meet the Thermal Bypass Checklist (TBC) etc.

Figure 3-6: Using Marketing Support Options Offered by Program

Forty-three percent of interviewed builders (13 builders) say they do not take advantage of marketing support offered by the program. When asked why, three of these builders say that they build low-income housing and marketing is not an issue; two of these three builders say their units are sold through a lottery and one of these two builders says that they had 100 families



interested in 11 units. Three builders say they plan on taking advantage of marketing support options on their next projects. Two builders report that they are not aware of marketing support offerings. One builder only does custom homes and says that he does very little marketing.

Four builders give the following reasons for not taking advantage of marketing support offered by the Program:

• “We have our own way of doing things, but we do display the ENERGY STAR logo on our materials and offer information about ENERGY STAR houses.” (market-rate builder)

• “I haven't but I should—it is foolish not to.” (market-rate builder) • “About a year ago, I had trouble getting brochures off the website; I haven’t tried to

do anything since then.” (builder of market-rate and low-income housing) • “I tried to sell and found it would not make sense in this market so I converted the

building to rentals.” (market-rate builder)

Figure 3-7 shows the marketing support options being used by builders. As shown, 67% of the 15 builders taking advantage of marketing support offered by the Program (10 builders) used ENERGY STAR signage, 60% (9 builders) got help incorporating the logo and messaging in their model homes (one builder had a Program representative attend an open house), 53% (8 builders) got help incorporating the logo and messaging into their advertising and websites, 47% (7 builders) got help incorporating the logo and messaging into their sales process, 27% (4 builders) had sales staff training, 27% (4 builders) used brochures available through the Program, 7% (1 builder) had Program staff talk to the Realtors they work with and 7% (1 builder) had Program staff talk with a group of homebuyers.

Figure 3-7: Marketing Support Options Being Used



Three builders say that they downloaded the ENERGY STAR logo and incorporated it into their advertising, websites and sales processes on their own. In addition to the four builders who say they took advantage of sales staff training, one builder, who had Program staff talk to his sales staff several years ago, says, “Our sales people are now experienced and we train new people. We take all new staff to see a final inspection so they know what is involved.” The construction manager of another company says the owner “hammers ENERGY STAR to everyone on his staff at every weekly meeting.”

All 15 builders who say they take advantage of Program marketing support also say that they think that building and marketing ENERGY STAR homes in the current housing market is very or somewhat valuable. Figure 3-8 shows that builders who say that building and marketing ENERGY STAR homes is very valuable in today’s housing market are more than twice as likely as builders who think it only somewhat valuable to be taking advantage of marketing support through the Program (62% vs. 29%).

Figure 3-8: Use of Marketing Support by Value of Being Able to Market Homes as ENERGY STAR Qualified



Eleven of the 15 builders taking advantage of marketing support from the Program say that they think it is helping them sell their ENERGY STAR homes by letting buyers know they build energy-efficient homes and moving people to ask questions about what is involved in building an ENERGY STAR home. Five market-rate builders commented:

• “It gives us an edge over non-ENERGY STAR builders. We need even better marketing. We are looking for recognition/acknowledgement that we are an ENERGY STAR builder, but the owner gets the benefit.”

• “Once buyers realize what an ENERGY STAR home is, compared to a non-ENERGY STAR home, we win out every time.”

• “The price tag is still paramount, as well as the street scene etc., but ENERGY STAR adds substance.”

• “One of our biggest marketing things is the sign out front—it is more important than advertising in the Globe and elsewhere. Seventy-five percent of our business comes from having the ENERGY STAR sign out front.”

• “Media messages and articles on "green" and saving energy and dollars are all over the place (TV, news items, etc.). They have pounded home the message and consumers are waking up a little bit. Our marketing of ENERGY STAR has helped a little—more this year than in the past.”

Two builders commented on why they do not think the marketing support has helped them sell their homes; one builds low-income housing and says that the demand for the homes he builds is so high that marketing is not an issue, and the other, a market-rate builder, says, “I think it will help, but I haven’t enough experience yet to tell.”

3.5 Need for Additional Marketing Support All builders were asked if there is something not currently offered that they would like to see the Program do to help them market their ENERGY STAR homes. Several builders suggested options already offered by the Program: general advertising to the public, making builders aware of what is available and testimonials/case studies showing what builders are doing. Builders suggested the following marketing support options not currently offered:

• “Match a portion of advertising dollars—we participated in this in the past when it was offered by the Program.” (market-rate builder)

• “Something that shows how savings relate to the HERS scale—something visual and not vague.” (one market-rate builder and one builder of both market-rate and low-income housing)

• “Maybe videos or DVDS we can play for or give to homebuyers.” (market-rate builder)

• “Bring back the HERS folders given to homeowners.” (market-rate builder) • “More giveaways etc.” (market-rate builder)



• “A blue ENERGY STAR flag that would move like a balloon to let people know when the model home is open.” (market-rate builder)

The market-rate builder wanting more information on savings expanded on why he thinks this is important:

“I am trying to get information through our HERS rater about what the utility bills for a home would look like if it was built to standard code requirements. I know it depends on lifestyle, but I would like to say more than, ‘You will save 40% on gas bills etc.’ The HERS rater should be able to provide me with that. A modeling estimate from a third party is very helpful. This is a lynch pin for the program—putting money in now to save later. Put it to the test—you can't be vague on something like this. Here is the model home, here is what the estimated cost to operate the home is. What would the cost be if it was the same house but non-ENERGY STAR, with warm air oil heat, regular light bulbs, lower insulation etc.? I want to be able to say the difference is anticipated to be X dollars a year. We need something tangible.”



4 Choosing HERS Raters Interviewed builders like being able to choose their HERS raters and most builders who had a HERS rater assigned to them in 2007 continued to work with the same HERS rater in 2008. Figure 4-1 shows that 87% of the 30 interviewed builders (26 builders) say they like being able to choose their HERS rater. Of the four builders in the “other” category, two say it does not matter to them and two say they were not aware they could choose their HERS rater.

Figure 4-1: Like Being Able to Choose HERS Rater

The 22 builders who built ENERGY STAR homes in both 2007 and 2008 were asked if they decided to stay with the rater they were assigned in 2007 or to choose a different rater to work with in 2008. Figure 4-2 shows that 77% of these builders (17 builders) decided to keep working with the HERS rater they were assigned and 14% (3 builders) chose a new HERS rater.

Figure 4-2: Kept Assigned Rater or Chose a Different Rater



The two builders in the “other” category say that they have different HERS raters in 2008, but that they did not choose them. In one case, the builder knows the rater he originally worked with no longer works in Massachusetts.

The three builders who chose to work with a different HERS rater in 2008 explained their reasons for changing raters. Their reasons, presented below, all address the need for builders to establish positive working relationships with their raters. Most importantly, all three builders who changed raters are satisfied with the new raters they are now working with.

• “My foreman is more comfortable working with new person than the old one. I left it up to him. The new rater is also from a different company.”

• “I look at it as teamwork. Attitudes make all the difference. We will do all we need to do to pass. I wrote a nasty letter to the rater we were assigned about the lack of teamwork. Three houses failed and we had to go back in after people had moved into the home for another testing. He [the rater] wouldn't let me do it during the final inspection—they were things I could have fixed on the spot. He sometimes came, did the testing and left without telling us something needed to be fixed. The rater we chose works with us—he no longer does the actual testing, but his team works with us. They tell us about problems in enough time to fix them. It is a good teamwork relationship.”

• “Attitude—the assigned rater found issues and didn't work with us on how to change them, he just dropped the ball. There was a go around, but we got it resolved. We stayed with the same organization, but are now working with a different person and it is better.”



5 Training Figure 5-1 shows that 73% of the 30 builders interviewed in late 2008 or early 2009 say they are satisfied with the level and amount of training/support available to them through the Program and/or their HERS raters. This is consistent with 78% of builders interviewed in 2007 saying that they were either extremely satisfied (25%) or satisfied (53%) with the technical support they received.6

Only one builder, who was new to the Program in 2008, reports not being satisfied; this builder says that he put down on his application that he was interested in training, but no one ever contacted him. However, he also says that he is very happy with his HERS rater, who has spent a lot of time going over the requirements for certification with him.

The 24% (7 builders) in the “other” category include a modular builder who says that he has not needed much training; two builders who say they are not aware of what training is available beyond the support they get from their HERS rater; two builders who are not dissatisfied with the technical support available, but would like to see more seminars and information/training on recent changes to Program requirements; a builder who describes his satisfaction with available support as “so-so” and a builder who rates his satisfaction five on a scale of one to ten.

Figure 5-1: Satisfied with Level and Amount of Training/Support Available

Figure 5-2 shows that the 30 interviewed builders are virtually split between whether or not they think the Program needs to offer additional training. Five of the 15 builders who say they do not think additional training is needed commented on why they say this. Three of these builders say there is help available if you ask for it. One is a production builder who says, “We build the same 6 Nexus Market Research Inc. and Dorothy Conant. 2007Massachusetts ENERGY STAR® Homes Builder Interview Report, submitted to the Joint Management Committee, April 2008.



house over and over so once we have it down the only time we need more training is when the rules change.” Another builder says, “You need to be a self starter—go to the website, print it out, study it, it is all there.”

Figure 5-2: Would Like Additional Training Offerings

The 14 builders who would like to see additional training offerings provide a variety of suggestions. One builder says, “Refresher courses are good in any profession.” Another says that seminars are good for builders who do not like to read technical materials. Another says that he likes ongoing, on-site training correcting mistakes as they are found because you can't grasp everything on your first house. One builder proposes annual review meetings:

“I would like to sit down once a year with someone to review what we did in the current year and give me a preview of what is going to happen/change in the next year—talk about what we did that was good and what could be improved. I would like to hear about changes in the program—what we need to be prepared for or what changes we need to make. An annual review and discussion would be helpful. This could be done one on one or with a group of builders, but the group would need to be small (not more than ten builders) to be useful.”

Other builders suggested training on Program basics and changes to requirements as well as specific topics for technical seminars and training. Suggestions for training on program basics and changes include:

• More materials and training on recent changes to Program requirements • All aspects of the HERS system • Seminars providing a basic overview of the program, lighting and solar options



• An annual ENERGY STAR seminar appropriate for construction foremen • “More on the overall process of certification; when does the HERS rater get called;

when and how CFLs get ordered; right now it is piecemeal, I find out from the HERS rater when he is supposed to return, but not how it fits into the overall process.”

Suggestions for training on specific topics include:

• Trainings or seminars on new technologies, ideas, and educational marketing • “HVAC contractor training worked well. Insulation contractors need on-site training

for new crews/subs. There is a big turnover in insulation contractor crews—it is hard to maintain trained crews.”

• Seminars on sick air and how it is addressed by ENERGY STAR specifications • Courses on how to make homes more efficient, particularly addressing framing and

the building envelope • Training on using LEDs • Training on addressing ventilation in a ductless system



6 Final Builder Comments The final interview question asked builders if there was anything else they would like to add—any suggestions they might have for the Program in addition to what they had already said. Also, several builders digressed during the interviews to express their views about various aspects of the Program. The comments presented here draw from both sources.

A builder of market-rate homes addressed expanding the Program to include renovations and additions:

“It is too bad a program couldn't be developed for additions and renovations—I do lots of these. I still do my work to ENERGY STAR standards, but do not do a blower door test. Good air sealing etc. is all part of the package. All my clients hire me because of the way I do things. If an addition or renovation met certain requirements it would be good to bring a HERS rater in. Potentially, this could create more work for the builder on the house as well—updating insulation etc. From the builder's point of view we could team with ENERGY STAR to set our work apart from Joe down the street who is not doing it—put a sign out front when doing a big renovation or addition. The homeowner, the utilities and I all win because we reduce the carbon footprint and electrical load.”

Another builder of market-rate homes says, “What the subs learn they carry over to other jobs—it helps them gain more business. Hopefully, one day we will all be thinking the same way.”

Several builders commented on their positive experiences participating in the Program:

• Until more builders do ENERGY STAR we like the edge we have—it is a distinction from other builders. (market-rate builder)

• You would be foolish not to do the program. You get local town and building officials that add zero to the benefit of the house, where the program adds so much more substance and useful information. What the program provides far exceeds anything you would be getting from a building department that is supposed to be looking out for you.” (market-rate builder)

• “The program has helped us stay ahead of the game and the last two years we met the tax credit.” (market-rate builder)

• “The testing gets done when I request it—I am never denied. We've got it to where the HVAC company we hired brings their own testing equipment and tests everything before they finish to make sure it will pass the final inspection. It is harder and harder to achieve ENERGY STAR—just when you get it down they come up with new rules. It is a good program as long as it stays within our reach. It does cost money, but the benefits outweigh the cost.”

• I like field people having the ability to ask questions and learn more. On-site training it is good, educating framers to use less lumber and more insulation to make the house tighter. I am happy, but would like even more. Also, training of the subs and sales



people is important. They need to know why we do things because they are telling the customers. People don't appreciate the engineering that is behind a product unless it is explained. Why is one facial cream better than another—it is not just packaging and advertising. Buyers need to know all the things going into why one home is better than another. (market-rate builder)

• “We are expanding/broadening our business to work with after-market, older homes we built earlier to add insulation etc. because we now have experience with ENERGY STAR. There are programs out there to do this (he did not name MassSAVE) and it is worthwhile to save energy.” (market-rate builder)

• We had a meet and greet Caribbean party with a band and food etc. Current owners discussed with prospective buyers why they like to live here and it gives them an opportunity to share ideas. I had a mole attend and talk to people to get feedback—they all tell us they like their homes, but you never know what they say behind your back. I was happy that the mole said no one had anything bad to say about their home. (market-rate builder)

• “Buyers are proud to show their friends and neighbors that they have an ENERGY STAR home.” (market-rate builder)

• “I tell my people that in a tough economy we should promote ENERGY STAR because it puts us one up on ‘equivalent’ homes somewhere else in the city.” (market-rate builder)

Several builders provided suggestions for improving the Program:

• “Bring more awareness to the public about ENERGY STAR homes.” (market-rate builder)

• “The program should take a more proactive approach marketing to builders—reaching them before the permitting stage. Builders can take more advantage of the Program if they are exposed to the Program and involved in the planning stage. In a basic level home you usually have one can fixture in the middle of the room. There might be a better design when using CFLs—using different numbers or types of fixtures to maximize the benefits of CFLs.” (low-income housing builder)

• “Part of the issue with the HERS process was that we had made some changes since the initial application was sent in and we didn't send the new information in. It might be good to make it part of the HERS rater process to have the rater send out a copy of what they have on file before they come to do an inspection and ask us to check to see if it is all accurate—that nothing has changed.” (builder of both market-rate and low-income housing)

• “I am looking to confirm what the rebate checks are for. I get checks from ICF but they don't say what home the rebate check is for or whether it is for thermostats, heating systems, bulbs etc. I would like to know who to contact to find out. There should be a number you can call to say, ‘I got a check for x dollars, what is it for.’



When I get the check for ENERGY STAR participation the street address is on the check and I can marry up the amount to what I submitted.” (market-rate builder)

• When the change in the presidency takes place we don't know what programs will continue or be cut back, and I hope they keep this program going. If they are going to make standards tighter then they need to increase rebates—building costs are not going down. I almost considered dropping out of program a year ago because of the new air infiltration changes etc., but decided to try and stay in and it worked out. However. they had also cut rebates and you can't do both. For us it is more extra labor cost than anything else—it is time consuming. The Central Massachusetts Home Builders' Association had a meeting where some people from ICF came to talk about ENERGY STAR. There weren't that many builders there, but I am glad we were there.” (market-rate builder)

Several builders commented on green building.

• “If you are not doing LEED or ENERGY STAR you are behind. You need to pass the Green Rating tool EPA has now to say you are a green builder.” (market-rate builder)

• “People are concerned about global warming and want green elements in their homes. However, if they ask about, or you mention, PV panels etc. they are not interested once they know the cost. When the green comes out of their wallet, then it isn't green.” (market-rate builder)

• “Clients are more green conscious than ENERGY STAR aware. They want green products—cellulose and fiber cement siding etc. They are socially conscious and want to be politically correct.” (market-rate builder)

Several builders commented on the resale value of ENERGY STAR homes:

• “I have seen some of our homes being resold and the price never reflects the benefits of it being ENERGY STAR. I find this frustrating because they should. But, the owners are not marketing them as ENERGY STAR so I assume they don't give it much value in terms of pricing their home for resale.” (market-rate builder)

• “I tell owners to hold on to the ENERGY STAR certificate because it could be valuable when they go to sell their home.” (market-rate builder)

• “We have proven statistics that the prices on the units that are resold are better. People know when they move into an over 55 community that they will not be living in the home for 50 years and that the unit will be sold.” (market-rate builder)

• “When I introduce the ENERGY STAR certificate one of the first things I tell buyers is they can say the home is ENERGY STAR when it is sold. Today's younger couples are always thinking of resale value.” (market-rate builder)



7 Conclusions and Recommendations

7.1 Conclusions Builders’ perception of the value of having the ENERGY STAR label on a home has increased. Almost all (93%) of interviewed builders say that building ENERGY STAR homes and being able to market them as ENERGY STAR qualified is very or somewhat valuable in today’s housing market.

The Program’s increased marketing to consumers appears to have increased homebuyer awareness of ENERGY STAR homes based on the increase from 2007 in the percentage of builders saying that homebuyers are aware of ENERGY STAR homes. This is consistent with 90% of interviewed builders saying that homebuyers are showing more interest in and/or awareness of energy-efficiency and 57% saying that homebuyers are showing more awareness of and/or interest in buying an ENERGY STAR home. One of the most encouraging findings from this year’s interviews is that five builders, 17% of interviewed builders, report homebuyers looking for an ENERGY STAR home and another builder saying that certification helped sell some of his homes.

Listing marketing support offerings on the application form and asking builders to check the options they are interested in appears to have increased the likelihood of builders taking advantage of available marketing support. The percentage of interviewed builders taking advantage of available marketing support options increased from 18% of builders interviewed in 2007 to 50% of builders interviewed in late 2008 and early 2009.

Builders like being able to choose their HERS rater. The few interviewed builders who did not like working with the HERS rater originally assigned to them have successfully selected and established good working relationships with different raters.

Satisfaction with the training and technical support provided by the Program and HERS raters remains high at 73%. The need for training and support clearly varies from builder to builder. A challenge for the Program will be to identify and prioritize training and support offerings, beyond the on-site support HERS raters offer, that will provide the most overall value to participating builders.

Positive trends in homebuyer awareness of and interest in energy efficiency and ENERGY STAR homes combined with builders’ perception of the value of building and marketing ENERGY STAR homes growing bodes well for the Program as it strives to achieve broader participation and deeper savings.



7.2 Recommendations Based on the above conclusions, the NMR team makes the following recommendations.



• Produce more case studies appropriate for use in marketing to builders and homebuyers. There are now several builders saying they have homebuyers coming to them looking for an ENERGY STAR home and several builders saying buyers are very happy with the low operating cost of their ENERGY STAR homes. One builder has documented that the over-55 ENERGY STAR homes he builds command a higher resale price. These are messages likely to be effective in attracting new builders to the Program and generating homebuyer interest in ENERGY STAR homes.

• Review the training and support suggestions provided by interviewed builders and incorporate those that will provide the most overall value to the Program. Builders provided a variety of suggestions for additional training/support. Given that the sample of interviewed builders is small and individual builder training and support needs vary greatly, it is important that Program staff factor in their understanding of the full population of participating builders and their training/support needs in prioritizing and selecting training options that will facilitate the Program’s goals to move builders toward building zero energy homes.

• Start planning now for training builders to meet ENERGY STAR 2011 requirements. Builders like to know ahead of time about changes to the Program. The NMR team believes that educating builders in 2009 about changes likely to be implemented in 2011 and assuring them that in 2010 the Program will provide the training needed to ensure they can meet the new requirements will help maintain builder participation and the increasing penetration rate of ENERGY STAR homes in the single family housing market.

MA New Homes with ENERGY STAR: 2008 Builder Interview Report Page A1


Appendix A Builder Interview Questions

2008 Massachusetts New Homes with ENERGY STAR Program Program Satisfaction Builder Interview Questions

Builder background questions: First I have a few basic questions about your experience with the Massachusetts New Homes with ENERGY STAR Program: 1. How many years have you been building ENERGY STAR homes? 2. How many ENERGY STAR homes have you built? 3. Are you currently building an ENERGY STAR home? 4. How many of your ENERGY STAR homes included CFL bulbs installed for free through

the program? 5. Is it your standard practice now to install CFL bulbs in all your ENERGY STAR homes? 6. If not, why not? 7. Do you build some homes that you do not have ENERGY STAR-certified?

• If yes, do you install CFL bulbs in those homes? • If yes, always or only some times? • If not always, why not?

Questions Addressing Satisfaction with 2008 Program I would now like to ask you a few questions about other areas of the 2008 program. Marketing 8. How important, or valuable, is it to you to build ENERGY STAR homes and be able to

market them as ENERGY STAR-certified homes in the current housing market? 9. Are buyers showing more awareness of and/or interest in energy efficiency? 10. If yes, how much more awareness/interest? 11. Can you give an example? 12. Are buyers showing more awareness of and/or interest in buying an ENERGY STAR-

certified home? 13. If yes, how much more awareness/interest? 14. Can you give an example? 15. Are you taking advantage of any of the marketing support options offered by the program to

help you market your ENERGY STAR homes? 16. If not, why not? 17. If yes, what marketing support options offered by the program are you using? (Following

list is from 2008-09 application form.) 18. Sales staff training on the ENERGY STAR benefits and features

MA New Homes with ENERGY STAR: 2008 Builder Interview Report Page A2


19. New home buyer seminars on the benefits of ENERGY STAR 20. Real estate agent trainings at your model home(s) on the benefits of ENERGY STAR for

their clients 21. Help incorporating the ENERGY STAR logo and/or benefits messaging in your model

home(s) 22. Help incorporating the ENERGY STAR logo and/or benefits messaging in your advertising 23. Help incorporating the ENERGY STAR logo and/or benefits messaging into your web site 24. Help incorporating the ENERGY STAR benefits message into your sales process 25. Other1—Describe 26. Other2—Describe 27. Other3—Describe 28. Do you think the marketing support is helping you sell your ENERGY STAR homes? 29. If yes, can you give some examples? 30. Is there something not currently offered that you would like to see the program do to help

you market your ENERGY STAR homes? 31. If so, what?

Choosing HERS Rater 32. Do you like being able to choose your HERS rater? 33. If you built ENERGY STAR homes in both 2007 and 2008, did you decide to stay with the

rater you worked with in 2007 or choose a different rater to work with in 2008? 34. If chose a different rater, why?

Training 35. Are you satisfied with the level and amount of training/support available to you through the

program and/or your HERS rater? 36. If not, what additional training would you like? 37. Have you paid for extra training or support from your HERS rater? 38. Is there any training you think builders need or would like to have the program offer? 39. If yes, what training would you like to see the program offer?

Closing 40. That's all the questions I have. Is there anything else you would like to add—any

suggestions you might have for the program in addition to what you have already told me?

Documents

Evaluation of the Massachusetts New Homes with ENERGY … · 2020-07-15 · The evaluation work conducted for the 2008 Program includes: • PRISM billing analyses of homes built