SEEM 94 Calibration to Single Family RBSA Data

Regional Technical ForumJanuary 23, 2013

Analysis Performed By Adam Hadley

Cursory Reviews By Tom Eckman, Ben Larson, David Baylon, Nick O’Neil, Josh Rushton

Presentation By Adam Hadley

Introduction• SEEM is used to estimate energy savings for most space-heating-

effected residential UES measures• Goal

– Ensure SEEM’s results are grounded in measured space heating energy use.

• Method1. Run SEEM using characteristics data from billing analyses and/or

metering reports to define inputs.2. Compare SEEM’s heating energy use outputs to the reports’ findings.3. Modify unknown characteristics data (usually T-stat setting) until a

reasonable match is found.4. Standardize the “calibration inputs” (from step 3) for use in RTF

measure analyses.

Introduction - 2

RTF Guidelines3.3.3.2 Model CalibrationIn most cases, calibrated engineering procedures will involve at least one stage of modeling in which baseline and efficient case energy consumption are estimated for the measure-affected end use. For example, the heating load for single-family homes is estimated as part of the derivation of UES for ductless heat pump conversion. A simulation model is used to derive the heating end use for typical homes in different climate zones. Ideally, the model would be calibrated to measured heating end use for a sample of homes. If end use data are not available, the model should at least be calibrated to metered total use for the sample. Calibration should also be performed for samples that have adopted the measure, i.e., the efficient case. For measures that affect new buildings the calibration may be limited to the efficient case or to comparable buildings of recent vintage.

Introduction - 3

History• 11/9/2009

– SEEM92 Single Family Calibration Approved• 70°F/64°F for Gas FAF and HP’s• 66°F for Electric FAF and Zonal

• 12/13/2011– RTF adopted SEEM 94

• Infiltration Calculation now physics-based (previously assumed at a steady rate)– SEEM94 Manufactured Home Calibration Approved

• 69.4°F/61.9°F for all heating system types

• 9/18/2012– SEEM94 Multi-Family Calibration Approved

• 68°F for walk-up and corridor buildings• 66°F for townhouses

Introduction - 4

Data Sources• SF Data Sources used in previous calibration:

• Data Source used in this calibration:– Underlying database** for the Single Family Residential Building

Stock Assessment (2012)• RBSA study’s database offers recent billing analyses results and detailed

house characteristics on 1404 houses in the Region.• This allows well-defined SEEM runs for each individual house.

Report Date Type n*

Single-Family Residential New Construction Characteristics and Practices Study (RLW for NEEA)

2007 Billing Analysis 114

Analysis of Heat Pump Installation Practices and Performance (Ecotope for NEEA)

2005 Billing Analysis 381

Super Good Cents Metered Data ~1994 Billing Analysis 740

Method - 5

*Sample size used in the calibration exercise (study sample size may have been larger).** Using a pre-release version of the database for this analysis .

Key Model Input Parameters

RBSA Data Availability

UA Available for each house.

Weather Zip code (available for each house) linked to nearest TMY3 weather station.

Gas Heating Efficiency Available for some houses; used average for remaining houses.

HP Operation & Efficiency Not readily available. Used ARI control & 7.9 HSPF.

Duct System Leakage and Surface Area

Available for some houses; used average for remaining houses with ducts.

Duct System Insulation and Location

Available for each house.

Infiltration Available for some houses; used a floor area-scaled average (by foundation type) for remaining houses

Mechanical Ventilation Not available. Assumed 2 hours /day at 50 cfm.

Non-Lighting Internal Gains Not available. See next slide for details.

Lighting Internal Gains LPD available for each house; assumed 1.5 hours/day.

T-stat Setting Available based on interviews, but used this as the “calibration knob”.

Method - 6

Non-Lighting Internal Gains• Equation:

• Based loosely on Building America Benchmark*– Used the original equation and values (averaged) to determine average internal

gains for RBSA homes.• Original equation also includes Number of Bedroom and Finished Floor Area terms

– Set Number of Bedrooms and Finished Floor Area terms to zero and adjusted Number of People term to achieve same average internal gains for RBSA homes.

• Building America Benchmark based on– “The appliance loads were derived by NREL from EnergyGuide labels, a Navigant

analysis of typical models available on the market that meet current NAECA appliance standards, and several other studies. ”

– “The general relationship between appliance loads, number of bedrooms, and house size, was derived empirically from the 2001 RECS. ”

Method - 7

*Hendron, Robert. "Building America Research Benchmark Definition, Updated December 20, 2007." NREL/USDOE EERE. January 2008. NREL/TP-550-42662

Some Houses Unable (or unwilling) to Run with SEEMIssue Count

More than one foundation type 331

25% > Ceiling Area to Floor Area > 200%, or Missing Ceiling U-value 36

Footprint Area to Floor Area < 20% 36

30% > Wall Area to Floor Area > 200%, or Missing Wall U-value 24

Missing Floor U-value for Crawlspace Foundation 5

Window Area = 0 3

Window u-value = 0 3

• Resulting House Count: 1011– These issues overlap on some houses, so the sum of

the counts cannot be subtracted from 1404 to get 1011.

Method - 8

Data FiltersVariable Filter

Value(s)Notes Count

(filtered out)SEEM Run Valid SEEM run must be valid (> 0 kWh/yr). 4

Billing Energy Use > 1,500 kWh/yr

Intends to screen out partially used or unused houses

38

eRsq and gRsq = 0 or ≥0.45

Screens out houses with poor billing analysis results (0.45 per David Baylon)

398

Non-natural-gas & non-electric Fuel Use

0 Screens out houses with wood, oil, propane, etc. consumption because billing analysis not performed.

352

Primary Heating System

eZonal, eFAF, gFAF, HP

Removes gas boilers, wood stoves, etc. 216

Secondary Heating System Fuel

Electric or Gas

Removes wood stoves, propane heaters, etc. 274

• Gas Billing converted to kWh/year using reported AFUE• Resulting House Count: 289• (The counts for each item overlap here, too)

Method - 9

Results

Results - 10

T-Stat Setting “Calibrated” to:Heating System Type

Heating High °F(day)

Heating Low °F(night)

Electric Zonal64 64

Electric FAF

Gas FAF 68.6 63.9

Heat Pump 69.6 65.4

Results - 11

• Electric Zonal and FAF based on results• Gas FAF and Heat Pump based on average

from RBSA (n=1011 subset).

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000

Nor

mal

ized

Billi

ng H

eatin

g U

se (k

Wh/

hr)

SEEM Heating Use (kWh/yr)

80,000 40,000 30,000

Avg. 12,332 11,560 10,868

St. Dev.

9,060 7,205 6,189

Avg. 11,348 11,054 10,819

St. Dev.

7,086 6,783 6,600

kWh 984 506 49

% 9% 5% 0%

289 283 274

P-Value

1.6% 16.0% 87.9%

>.05 ? No Yes Yes

SEEM Heating Use less than

SEEM kWh/yr

Billing kWh/yr

Difference of

Averages

Count

Student's T-test

eZonal, eFAF, gFAF, Heat Pump

Results - 12

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000

Nor

mal

ized

Billi

ng H

eatin

g U

se (k

Wh/

hr)

SEEM Heating Use (kWh/yr)

80,000 40,000 30,000

Avg. 8,876 8,876 8,399

St. Dev.

5,757 5,757 4,838

Avg. 7,922 7,922 7,881

St. Dev.

4,638 4,638 4,681

kWh 954 954 518

% 12% 12% 7%

91 91 89

P-Value

9.7% 9.7% 29.4%

>.05 ? Yes Yes Yes

SEEM Heating Use less than

SEEM kWh/yr

Billing kWh/yr

Difference of

Averages

Count

Student's T-test

eZonal

Results - 13

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000

Nor

mal

ized

Billi

ng H

eatin

g U

se (k

Wh/

hr)

SEEM Heating Use (kWh/yr)

80,000 40,000 30,000

Avg. 14,071 12,471 12,471

St. Dev.

9,558 6,292 6,292

Avg. 12,103 11,963 11,963

St. Dev.

4,872 4,955 4,955

kWh 1968 508 508

% 16% 4% 4%

21 20 20

P-Value

31.2% 69.5% 69.5%

>.05 ? Yes Yes Yes

SEEM Heating Use less than

SEEM kWh/yr

Billing kWh/yr

Difference of

Averages

Count

Student's T-test

eFAF

Results - 14

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000

Nor

mal

ized

Billi

ng H

eatin

g U

se (k

Wh/

hr)

SEEM Heating Use (kWh/yr)

80,000 40,000 30,000

Avg. 17,244 15,909 14,829

St. Dev.

9,863 7,010 5,627

Avg. 15,841 15,365 15,039

St. Dev.

7,098 6,728 6,516

kWh 1402 544 -210

% 9% 4% -1%

125 120 113

P-Value

6.4% 41.2% 72.5%

>.05 ? Yes Yes Yes

SEEM Heating Use less than

SEEM kWh/yr

Billing kWh/yr

Difference of

Averages

Count

Student's T-test

gFAF

Results - 15

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000

Nor

mal

ized

Billi

ng H

eatin

g U

se (k

Wh/

hr)

SEEM Heating Use (kWh/yr)

80,000 40,000 30,000

Avg. 5,873 5,873 5,873

St. Dev.

2,913 2,913 2,913

Avg. 6,239 6,239 6,239

St. Dev.

4,062 4,062 4,062

kWh -366 -366 -366

% -6% -6% -6%

52 52 52

P-Value

48.9% 48.9% 48.9%

>.05 ? Yes Yes Yes

SEEM Heating Use less than

SEEM kWh/yr

Billing kWh/yr

Difference of

Averages

Count

Student's T-test

Heat Pump

Results - 16

Alternative Method• This is how the RTF has performed SEEM calibrations in the past

– Results shown here for comparison; not intended to be used as part of calibration

– Modeling each house is considered a better approach• Compare Averages: RBSA average billing data vs. SEEM runs with

Average RBSA characteristics– Using only RBSA data from the report to define SEEM run characteristics

for the 3 prototypes• Exception: Used database to determine how many R0 duct insulation cases had

ducts inside– Note: Used T-stat setting per RBSA report

• 68.7°F with 65% of homes using a setback to 62.2°F

• Results…

Alternative Method - 17

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

18,000

20,000

OR(PDX/SPK)

WA(SEA/SPK)

ID(BOS)

MT(KAL)

Region

Annu

al H

eatin

g U

se (k

Wh/

yr)

City for SEEM TMY3 weather file shown in parenthesis

Electric Space Heating

RBSA Billing90% CI

SEEM

Alternative Method - 18

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

OR(PDX/SPK)

WA(SEA/SPK)

ID(BOS)

MT(KAL)

Region

Annu

al H

eatin

g U

se (k

Wh/

yr)

City for SEEM TMY3 weather file shown in parenthesis

Gas Space Heating

RBSA Billing90% CI

SEEM

Alternative Method - 19

What are the Takeaways?• Key SEEM inputs RTF carries forward

for future Single Family space heating modeling:– Tstat settings (see table)– Internal Gains Method

• Non-lighting: Use modified equation with RBSA average # of people/house for each prototype

• Lighting: Use RBSA average LPD & 1.5 hours/day

– Baseline Ventilation• 2 hours/day at 50 cfm

– Baseline InfiltrationConclusion - 20

Heating System Type

Heating High °F(day)

Heating Low °F(night)

Electric Zonal 64 64Electric FAFGas FAF 68.6 63.9Heat Pump 69.6 65.4

Remaining Issues• Wood/Other Heat– By ignoring the significant fraction of wood/other heated

homes in this analysis, future RTF analyses will likely overstate the electric space heat savings.• From a cost-effectiveness perspective, this may be ok if we consider

the value of wood/other fuels similar to the value of electricity.• From an electric savings perspective, it’s important to remember

some fraction of the stated electric savings will actually be wood/other savings, not actual electric savings.

• Unused Homes– By ignoring unused homes in this analysis, we have a similar

issue as with wood/other heat (except there won't be wood/other savings in these cases).

Discussion - 21

Some Options

1. Leave as is; note caveats regarding stated savings2. Determine a “grid savings” and a “TRC savings” for each

measure– Grid Savings

• Savings would account for wood/other and unused houses. – This will take more work & discussion.

• Used to report electric savings.– “TRC Savings”

• Savings don’t include the effects of wood/other heat. – Savings would be based on the calibration presented today.

• Used to report TRC – Value of electricity used as a proxy for value of wood/other heat

Discussion - 22

Decision

• Approve SEEM94 calibration for use in estimating space heating energy savings in single family homes.

Decision - 23