Upload
clayton-ayala
View
30
Download
2
Embed Size (px)
DESCRIPTION
NJ Comfort Partners Evaluation. Jackie Berger August 21, 2014. Program Goals And Approach. 2. NJ Comfort Partners Utility Partners. 3. Evaluation Questions. To what extent does the program achieve its goals? What opportunities are available for increased effectiveness or reduced costs? - PowerPoint PPT Presentation
Citation preview
NJ Comfort Partners Evaluation
Jackie Berger
August 21, 2014
Program Goals And Approach
Goals•Cost-effective energy savings•Improved bill payment•Target assistance program participants•Improve comfort, health, and safety
Approach
•Joint delivery for electric and gas customers•Comprehensive measure installation•Comprehensive customer education
2
NJ Comfort PartnersUtility Partners
3
Electric Utilities
Atlantic City Electric
Jersey Central Power & Light
PSE&G
Evaluation Questions
1. To what extent does the program achieve its goals?
2. What opportunities are available for increased effectiveness or reduced costs?
3. How can the program be refined to achieve improved outcomes?
5
Evaluation Overview
Tracking Process Affordability
Impact
Energy Saving Protocols
6
Tracking System
Information Needs
• Management• Reporting• Operations• Evaluation
Elements to Review
• Program reports• Protocol savings
calculations• Data fields• System use
Assessment Issues
• Content• Quality• Sufficiency• Accuracy• Efficiency
• Invoices• Messaging• Retrieving job data• Job file attachments• BPU reporting• Management reporting
Functions
• Messaging capabilities• Ad hoc reports require request• No ability to import or export data• Balancing accuracy checks and
complexity• Data timeliness• Electronic audit tool• Improved attachment feature
Challenges and Needs
7
Process Research
Background and Procedures Review
Needs Assessment
Manual
Marketing
Staff Training
Energy Education
Data Forms
Utility Interviews
6 Utilities
Managers and Staff
Program Design
Challenges and Successes
Contractor Interviews and On-
Site Assessment
5 Prime Contractors
Quality Control
Contractor
Program Procedures
Data Management
On-Site
Observations Audits
InstallationsFinal
Inspections
Inspections of Completed Jobs
Needs Assessment
8
Service TypeLow-Income Households with Bills
Low-Income Households
with High Bills
Percent of Low-Income Households with High Bills
AllMeet all Eligibility
Criteria
Electric Baseload 623,382 313,335 50% 41%
Electric Heating 92,871 22,417 24% 19%
Gas Heating 462,013 255,523 55% 53%
Low-Income Households
Number Percent
Income at or below 225% 812,370 100%
Heat with Gas or Electric 693,604 85%
Direct Payment of Electric or Gas Bill 730,725 90%
Eligible Housing Type 631,109 78%
Meet All Eligibility Criteria 503,242 62%
Observe
• Systematic and consistent approach
• Use of detailed data collection forms
• Quality rating scales
• Experts trained to conduct assessment
• Quantitatively document findings
ObservedAudits-80 Installs-41
Final Inspections-13
9
Observation Procedures
• Purpose: Observe and record process• Observers should not train, recommend, correct, or otherwise influence the process• Observers should only intervene if they see serious issues
Inspect
• Diagnostics – Tests include blower door, ducts, safety testing, IR, and appliance metering
• Measures – Inspect all installed measures for quality and completeness
• Missed Opportunities – Identify any measures that could have been installed, but were not.
• Client Interview – Discussion of health, comfort, safety issues, and client-related factors
InspectedRepresentative Sample
of 288 Jobs
Inspection Procedures
• Selected from customer survey respondents• Work completed approximately one year ago• $50 incentive offered to participants
10
Observation Assessments
11
Action Completeness of Inspection
1 Auditor inspected every accessible room
2 Auditor measured and recorded dimensions of building
3 Auditor did outside home walk-around
4 Auditor physically checked insulation levels in the attic(s), walls, and basement/crawl
5 Auditor talked with customer about comfort issues
6 Auditor talked with customer about energy issues
7 Auditor talked with customer about health and safety issues
8 Auditor assessed all accessible attics
9 Auditor assessed all accessible basement/crawl space areas
10 Auditor inspected the garage
11 Auditor conducted blower door test
Observation Assessments
12
Action Quality of Air Sealing Full
Scale
Exclude Safety Issues
Impact Usage Scale
1 Fire rated materials used around chimneys, flue pipes, recessed lighting, etc. ü ü
2 Materials integrated with existing finishes ü ü
3 Used supply-air respirators when using two-part foams ü
4 Blower door used to target air sealing ü ü ü
5 Blower door and smoke used to verify air sealing effectiveness ü ü ü
6 Caulk not used to seal gaps larger than 1/8” ü ü ü
7 Large holes and gaps covered with rigid materials ü ü ü
8 Area cleaned before caulk or foam applied ü ü ü
9 All material scraps and packaging removed from home ü ü
10 Materials installed consistently and are uniform in appearance ü ü
11 Gloves and eye protection used when cutting, or using caulk or foam ü
12 Used hard hats when working in confined spaces ü
13
Affordability Research
Customer Survey
• Understanding• Energy Education• Impact• Satisfaction
Impact
• Bills• USF Subsidy• Energy Burden• Payment
Findings
• Program serves needy households including veterans, customers receiving assistance, and the unemployed
• Some contractors provide more effective education than others
• Customers are most interested in reducing bills
• Small but positive impacts on energy affordability
• Positive impact on ratepayers, as USF subsidy declined
Customer Feedback
14
All
Implementation Contractor
1 2 3 4 5
Surveys 977 459 203 194 101 20
Provider Reviewed Energy Bill 37% 43% 35% 24% 34% 45%
Provide Explained how Energy Use is Measured 48% 50% 50% 40% 47% 45%
Customer Reported Good Understanding Of Energy Bill 78% 81% 79% 74% 70% 80%
Contractor Provided Written Action Plan 54% 58% 58% 44% 46% 45%
Contractor Provided Estimate of Money Savings from Actions 27% 27% 32% 18% 27% 30%
Customer Took Energy Saving Actions 60% 64% 60% 56% 52% 70%
15
Impact Research
Program Data
Analysis
• Customer Characteristics• Housing• Measure Installation• Measure Costs• Health and Safety Costs• % in Spending Tiers• Inspection Pass Rates• Inspection Failure Reasons
Usage Impact
• Data from 6 Utilities• Weather Normalized• Comparison Group
Adjusted• All Treated Customers
with Sufficient Data• Savings Realization Rates• Cost Effectiveness
Protocol Savings Category
Treatment Group
# %
CFL 5,100 79%
Air Sealing 4,201 65%
Hot Water 3,926 61%
HVAC 2,991 46%
Refrigerator 2,797 43%
Thermostat 2,436 38%
Duct Sealing 2,061 32%
Insulation 2,029 32%
Pre Post Change Measured
Treatment Group
Year Before
Services
Year After Services
After - Before
Program Impact
and Other Factors
Comparison Group
2 Years Before
1 Year Before
2 Years Before – 1 Year Before
Other Factors
Treatment - Comparison 2009-2011 2010-2012 Program
Impact
16
Energy Saving Protocols
Review and Verify
Recommen
ded Chang
esProtocols for Additional
Measures
Engineering Estimates