SBEED Validation Against ASHRAE Standard 140-2014

June 2017

Murray Milne

UCLA Department of Architecture and Urban Design

Professor Murray Milne

For the California Energy Commission Contract PIR-12-032:

Tools and Materials for Zero Net Energy California Buildings,

SBEED: Small Building Energy Efficient Design

INTRODUCTION

SBEED (Small Building Energy Efficient Design) is an easy-to-use day-one design tool that

helps owners, builders, and architects create a more energy efficient non-residential building.

ASHRAE Standard 140-2014 is a method for evaluating a building energy analysis computer

program by running 35 variations of a small building design then comparing the results of using

eight different energy performance programs. The current release of SBEED 1.0 (Build 4) could

run 27 of these cases and the heating and cooling loads were reported and compared for each.

BACKGROUND

SBEED uses the Solar-5 computation engine, developed at UCLA beginning in 1978 for its

thermal analysis kernel. Solar-5 calculates an hourly heat balance similar to the method used in

EnergyPlus. It finds the heat gain or heat loss for each of the 8760 hours in a year using standard

ASHRAE algorithms, the Mackey and Wright time lag and decrement factor method of

accounting for heat flow through external mass walls, the Admittance Factor Method to account

for internal thermal mass, and the California Energy Commission’s ACM method to calculate the

performance of basements. To find the hourly heat balance it uses a successive approximation

method to calculate the indoor air temperature. Thus it can integrate loads and energy

calculations at each hourly time step, which means that the HVAC system only adds heating or

cooling energy if the indoor air temperature has floated beyond the upper or lower comfort

limits.

ANSI/ASHRAE Standard 140: Standard Method of Test for the Evaluation of Building Energy

Analysis Computer Programs was developed by the American National Standards Institute

(ANSI) and the American Society of Heating, Refrigerating and Air-Conditioning Engineers

(ASHRAE). This standard is updated periodically, and the version from 2014 has been used to

test the recently released version of SBEED 1.0 Build 4.

Test Procedures:

Standard 140 specifies test procedures to evaluate the results produced by software designed to

calculate the thermal performance of a building and its environmental control systems. The tests

are based on the principle of comparing the performance of one program against the performance

of other programs, and while the tests are not intended to evaluate all aspects of the software,

they are designed to indicate any serious flaws or limitations.

Standard 140 uses a small reference building of 48 square meters (26.25 by 19.69 feet) that has

35 variations of envelope, windows, internal loads, and infiltration. For each case its

performance was reported for eight simulation programs: ESP, BLAST, DOE2, SRES/SUN,

SERIRES, S3PAS, TRANSYS, and TASE. For each case the overall minimum and maximum

value is reported for heating load and for cooling load among these eight programs. The annual

heating and cooling load was also calculated for each case using SBEED, and whether it falls

within the minimum and maximum acceptance range of all the other eight simulation programs.

SBEED, like most U.S. energy simulation programs, uses Inch-Pound units, while Standard 140

is reported in SI (metric) units of MWh per year, which in this report is converted to MBTU per

year. The annual performance of SBEED is reported in MBTU/year on the Building Energy

Performance (BEPS) screen and is reported here in Table 1 and 2. These results are also plotted

graphically in Figures 1 and 2.

The 35 cases in Standard 140 range from quite realistic to extremely abstract. Eight cases are not

included here because using SBEED some test variables can not be changed: interior infrared

emittance, interior shortwave absorption, exterior combined radiative and convective surface

coefficients, cavity albedo, solid opaque windows, and adding a sunspace. The results of the

remaining 27 cases are plotted graphically in Figure 1 for Annual Heating Loads and in Figure 2

for Annual Cooling Loads, and are reported numerically in Table 1 and Table 2.

RESULTS

It is important to emphasize that in Standard 140 no formal validation criteria are established to

determine the range of acceptable results (ANSI/ASHRAE 2007, Section 4.4.1). Thus while this

study does not demonstrate official acceptance ranges, it does show that SBEED closely follows

the same pattern of performance as these other eight simulation programs (Fig. 1 and 2).

Standard 140 is used here in part because it was also used previously in the development of our

earlier design tool HEED (Home Energy Efficient Design) which also uses the Solar-5

computation engine, but for residential buildings. Comparing the performance reported for both

SBEED and HEED also shows that they both closely follow the same pattern of performance

(Henkhaus, 2012).

SBEED fell within the normative range on 33 of the 54 cases that it ran. This includes 18 of the

27 heating load comparisons (Figure 1), and 15 of 27 cooling load comparisons (Figure 2). Note

that in all cases SBEED tends to be conservative, in that it estimates that a higher heating or

cooling load is needed compared to the average of the other eight programs. Thus, an actual

building would likely use less annual energy than SBEED predicted. This means that when

designing a Zero Net Energy building SBEED will have a higher probability of meeting that

goal.

Compared to the eight reference programs, SBEED showed the same magnitude and direction of

performance of in each successive case. This implies that when SBEED is used as a design tool,

each incremental design change to a building should produce changes in heating and cooling

energy that are of the correct magnitude and direction.

Figure 1: Annual Heating Loads Comparison

Mbt

u/ye

ar

TESTCASE

ASHRAE Standard 140-2014 ComparisonSBEED 1.0 Build 5: ANNUAL HEATING

ASHRAE Minimum ASHRAE Maximum SBEED Value

Figure 2: Annual Cooling Loads Comparison

DISCUSSION OF RESULTS

Reference Programs:

Note that Standard 140 is based on results of eight reference computer programs that are now

20 or more years old, most of which are no longer considered state of the art in building

simulation. Missing from this list is EnergyPlus, which is now considered by many in the

U.S. to be the standard of the industry. In 2006 and 2010 EnergyPlus was run against these

same eight reference simulation programs, plus three more that were added (BLAST 3.0,

DOE2.1E, and DOE2.1E-RevWindow). In the 2010 test series using EnergyPlus 6.0.0.023

the test files generate results which lay outside bounds for eight the 62 cases. To date

0.000

5.000

10.000

15.000

20.000

25.000

30.000

35.000

40.000

Mb

tu/y

ea

r

TEST CASE

ASHRAE Standard 140-2014 Comparison SBEED 1.0 Build 6: ANNUAL COOLING

Range Minimum Range Maximum SBEED Value

Standard 140 has not been revised to include these three new versions of the original eight

programs or to include EnergyPlus or to include SBEED or HEED..

Weather Data:

All the eight original reference programs were run using an 8760 hour climate data file called

DRYCOLD.TMY, but since then the weather data formats have been revised, corrected, and

updated to TMY2 and now to TMY3. This file apparently originally used the Denver-Stapleton

Airport, Colorado TMY data.

SBEED could not use the original DRYCOLD.TMY file because it is not in EPW format, so

instead used the Denver-Stapleton Airport, Colorado TMY data in its currently published EPW

format (USA_CO_Denver-Stapleton.724690_TMY) available from the EnergyPlus Weather site.

(https://energyplus.net/weather). Thus weather data used in this current SBEED study may be

slightly revised from what was used in the original eight simulation programs.

EnergyPlus validation analysis was originally done using a BLAST weather file which in turn

had been converted into EnergyPlus format using the EnergyPlus weather converter. Since then,

the DRYCOLD.TMY weather file provided with Standard 140 has been directly converted into

the EnergyPlus format using the EnergyPlus weather converter. This produced significant

changes in results for some test cases using EnergyPlus with both the originally converted

weather file and results with the new weather file.

The EnergyPlus validation study reported that a comparison of the two weather files shows

several differences. First, the BLAST version has Daylight Savings Time option turned on while

the EnergyPlus version of the BESTEST weather file has the Daylight Savings Time option

turned off. This created differences in results for those test cases which have schedules which

change throughout the day, i.e. thermostat setback and nighttime ventilation cases (Cases 640,

650). Secondly, there were differences in the hourly outdoor wet-bulb temperature, sky

temperature, and diffuse and direct solar radiation. These changes are undoubtedly due to

differences in the psychrometric and solar radiation routines between the BLAST and the

EnergyPlus weather conversion programs.

Modeling Issues:

The specifications for Case 220 say that the opaque surface radiation properties should be

applied to all exterior opaque surface solar and infrared absorptances, and infrared emittances.

However the SOLAR5 engine in HEED and in SBEED applies these to the roof only.

Also in ASHRAE standard 140 the thermostat specifications say that heat shall be on when the

indoor temperature is less than 68 degrees, but SBEED turns heat on when interior temperature is

less than or equal to 68 degrees, so this will add a small amount of energy consumption to

Heating Energy.

CONCLUSIONS

SBEED (Small Building Energy Efficient Design) Version 1.0 Build 4 was used to model a

range of buildings as specified in ANSI/ASHRAE Standard 140-2014, Standard Method of Test

for the Evaluation of Building Energy Analysis Computer Programs. The ability of SBEED to

predict heating and cooling loads was tested using a test suite of 54 cases which included

buildings with both low mass and high mass construction, without windows and with windows

on various exposures, with and without exterior window shading, with and without temperature

setback, with and without night ventilation, and with and without free floating space

temperatures. The annual heating and cooling loads predicted by SBEED were compared to

results from eight other whole building energy simulation programs. SBEED was within the

normative range for 18 of the 27 annual heating load cases and within the normative range for 15

of the 27 annual cooling load cases. The nine heating cases that were out of range averaged less

than 6.5% overheating. The twelve cooling cases that were out of range averaged less than

10.4% overcooling. Thus 61% of the test cases were within the normative range, and all cases

that fell outside the range were on the safe side. This means that an actual building analyzed

using SBEED would likely use slightly less heating energy and slightly less cooling energy than

predicted.

Acknowledgements:

SBEED (Small Building Energy Efficient Design) was developed under contract with the

California Energy Commission by Murray Milne and Robin Liggett, Principal Investigators, with

Carlos Francisco Gomez, Senior Research Associate, and Donald Leeper, Senior Systems

Specialist. Testing and Evaluation was by Tim Kohut and Pablo LaRoche plus dozens of

colleagues around the country.

References:

ANSI/ASHRAE Standard 140-2014, Standard Method of Test for the Evaluation of Building

Energy Analysis Computer Programs, American Society of Heating, Refrigerating, and

Air-Conditioning Engineers. Atlanta, GA., 2014

HEED Validation Against the ASHRAE/BESTEST Standard (ASHRAE Standard 140), Grace

Tsai and Murray Milne, UCLA Department of Architecture and Urban Design,

http://www.energy-design-tools.aud.ucla.edu/HEED, 2003

HEED Validated Using HERS Bestest Tier 1 and 2: 2008, Murray Milne, UCLA Department of

Architecture and Urban Design, http://www.energy-design-tools.aud.ucla.edu/HEED,

2008

HEED Validation Reports: HEED 4.0, Build 27 and Build 29, Alicyn Henkhaus, EIT, UCLA

Department of Architecture and Urban Design, October 2012, HEED Summary

Validation Report 2012 includes the following validation reports:

HEED Validation Against ASHRAE /BESTEST Standard 140, 2012

HEED Validation Against HERS BESTEST Standard, 2012

Comparison of HEED and EnergyPlus, 2012

Validation Results Validation of PV Power Simulation in HEED, 2012

HERS BESTEST 1995. Home Energy Rating System Building Energy Simulation Test., R

Judkoff and J Neymark, National Renewable Laboratory, 1617 Cole Blvd, Golden

Colorado, 80401, 1995

EnergyPlus Testing with Building Thermal Envelope and Fabric Load Tests from

ANSI/ASHRAE Standard 140-2004. U.S. Department of Energy, Energy Efficiency and

Renewable Energy, Office of Building Technologies, 2006

EnergyPlus Testing with Building Thermal Envelope and Fabric Load Tests from

ANSI/ASHRAE Standard 140-2007. U.S. Department of Energy, Energy Efficiency and

Renewable Energy, Office of Building Technologies, 2010

SBEED is available at no cost from http://www.energy-design-tools.aud.ucla.edu/SBEED

Table 1: SBEED Annual Heating Loads Tested Using ASHRAE Standard 140-2014 ASHRAE Note: The statistics in the tables below are based on the Standard 140 Informative example results. These

statistics do not have any substantial importance and are not to be interpreted as acceptance criteria.

Eight cases are not included in this table because SBEED does not allow changes in the variables

being tested: Solid Opaque Windows, Surface Convection Coefficient, Interior Surface

Radiation, Cavity Albedo, and Added Sunspace.

Table 1: ANNUAL HEATING LOADS

Min Max Mean Min Max

Case MWh/yr MWh/yr MBTU/yr MBTU/yr kBTU/sf MBTU/yr

600: BASECASE Low Mass Building 4.296 5.709 14.659 19.480 37.200 19.195

610: 600 w/ South Window Overhang 4.355 5.786 14.860 19.743 37.360 19.278

620: 600 w/ East and West Windows 4.613 5.944 15.740 20.282 38.400 19.814

630: 620 w/ E&W Window Overhang+Fins 5.050 6.469 17.231 22.073 40.100 20.692

640: 600 w/ Night Setback Thermostat 2.751 3.803 9.387 12.976 28.070 14.484 no no

650: 600 w/ Night Ventilation 0.000 0.000 0.000 0.000 0.000 0.000

900: BASECASE High Mass Building 1.170 2.041 3.992 6.964 13.230 6.827

910: 900 w/ South Window Overhang 1.575 2.282 5.374 7.787 14.750 7.611

920: 910 w/ East and West Windows 3.313 4.300 11.304 14.672 27.930 14.412

930: 920 w/ E&W Window Overhang+Fins 4.143 5.335 14.136 18.204 33.780 17.430

940: 900 w/ Night Setback Thermostat 0.793 1.411 2.706 4.815 12.610 6.507 no no

950: 900 w/ Night Ventilation 0.000 0.000 0.000 0.000 0.000 0.000

220: 600 w/ Opaque Window Low Mass 6.944 8.787 23.694 29.982 56.420 29.113

230: 220 w/ Infiltration Restored 10.376 12.243 35.404 41.775 79.750 41.151

240: 220 w/ Internal Gains Restored 5.649 7.448 19.275 25.414 47.390 24.453

250: 220 w/ Exterior Shortwave Absorptance 4.751 7.024 16.211 23.967 45.430 23.442

270: 220 w/ Clear South Windows 4.510 5.920 15.389 20.200 40.760 21.032 no

290: 270 w/ South Window Overhang 4.577 5.942 15.617 20.275 40.900 21.104 no

300: 270 w/ East and West Windows 4.761 5.964 16.245 20.350 41.620 21.476 no

310: 300 w/ E&W Window Overhang+Fins 5.221 6.165 17.815 21.036 43.98 22.694 no

320: 270 w/ Thermal Deadband Restored 3.859 5.141 13.167 17.542 34.630 17.869 no

395: 400 w/ No Windows, Low Mass Building 4.799 5.835 16.375 19.910 36.370 18.767

400: Opaque Windows, Low Mass, No Loads 6.900 8.770 23.544 29.924 55.500 28.638

410: 400 w/ Infiltration Restored 8.596 10.506 29.331 35.848 67.140 34.644

420: 410 w/ Internal Gains Restored 7.298 9.151 24.902 31.224 58.010 29.933

430: 420 w/ Extior Shortwave Absorptance 5.429 7.827 18.525 26.707 53.830 27.776 no

800: 430 w/Opaque Windows, High Mass 4.868 7.228 16.610 24.663 51.210 26.424 no

yes/no

SBEED TestASHRAE 140 Example Results

Table 2: SBEED Annual Sensible Cooling Loads Using ASHRAE Standard 140-2014 ASHRAE Note: The statistics in the tables below are based on the Standard 140 Informative example results. These

statistics do not have any substantial importance and are not to be interpreted as acceptance criteria.

Eight cases are not included in this table because SBEED does not allow changes in the variables

being tested: Solid Opaque Windows, Surface Convection Coefficient, Interior Surface

Radiation, Cavity Albedo, and Added Sunspace.

Table 2: ANNUAL SENSIBLE COOLING

LOADS

Min Max Mean Min Max

Case MWh/yr MWh/yr MBTU/yr MBTU/yr

600: BASECASE Low Mass Building 6.137 7.964 20.940 27.174 58.820 30.351 no

610: 600 w/ South Window Overhang 3.915 5.778 13.359 19.715 45.500 23.478 no

620: 600 w/ East and West Windows 3.417 5.004 11.659 17.074 32.400 16.718

630: 620 w/ E&W Window Overhang+Fins 2.129 3.701 7.264 12.628 20.940 10.805

640: 600 w/ Night Setback Thermostat 5.952 7.811 20.309 26.652 58.820 30.351 no

650: 600 w/ Night Ventilation 4.816 6.545 16.433 22.332 44.250 22.833 no

900: BASECASE High Mass Building 2.132 3.415 7.275 11.652 28.030 14.463 no

910: 900 w/ South Window Overhang 0.821 1.872 2.801 6.388 17.730 9.149 no

920: 910 w/ East and West Windows 1.840 3.092 6.278 10.550 20.180 10.413

930: 920 w/ E&W Window Overhang+Fins 1.039 2.238 3.545 7.636 11.610 5.991

940: 900 w/ Night Setback Thermostat 2.079 3.241 7.094 11.059 28.030 14.463 no

950: 900 w/ Night Ventilation 0.387 0.921 1.320 3.143 2.440 1.259 no

220: 600 w/ Opaque Window Low Mass 0.186 0.835 0.635 2.849 4.670 2.410

230: 220 w/ Infiltration Restored 0.454 1.139 1.549 3.886 6.470 3.339

240: 220 w/ Internal Gains Restored 0.415 1.246 1.416 4.252 7.130 3.679

250: 220 w/ Exterior Shortwave Absorptance 2.177 3.380 7.428 11.533 18.570 9.582

270: 220 w/ Clear South Windows 7.528 10.350 25.687 35.316 72.320 37.317 no

290: 270 w/ South Window Overhang 5.204 8.089 17.757 27.601 58.370 30.119 no

300: 270 w/ East and West Windows 4.302 7.100 14.679 24.226 43.290 22.338

310: 300 w/ E&W Window Overhang+Fins 2.732 5.471 9.322 18.668 28.320 14.613

320: 270 w/ Thermal Deadband Restored 5.061 7.304 17.269 24.922 51.450 26.548 no

395: 400 w/ No Windows, Low Mass Building 0.000 0.016 0.000 0.055 0.060 0.031

400: Opaque Windows, Low Mass, No Loads 0.000 0.061 0.000 0.208 0.240 0.124

410: 400 w/ Infiltration Restored 0.000 0.084 0.000 0.287 0.370 0.191

420: 410 w/ Internal Gains Restored 0.011 0.189 0.038 0.645 0.970 0.501

430: 420 w/ Extior Shortwave Absorptance 0.422 0.875 1.440 2.986 2.710 1.398 no

800: 430 w/Opaque Windows, High Mass 0.055 0.325 0.188 1.109 0.690 0.356

yes/no

SBEED Test

MBTU/yrkBTU/sf

ASHRAE 140 Example Results