View
932
Download
0
Embed Size (px)
DESCRIPTION
One of the most challenging evaluation questions for residential lighting energy efficiency programs in the U.S. is the identification and correction for net-to-gross (NTG) effects such as free ridership and spillover. Over the last twenty years, considerable effort and financial resources have been directed toward accurately measuring these effects. Furthermore, the correction for these NTG effects has direct, and sometimes, drastic impact on program savings. APT and Opinion Dynamics discuss a new framework for the estimation of free ridership in upstream lighting programs grounded in sound, economically rational decision making on the part of retail partners. This approach, built on functional retail behavior, provides a clearer more insightful look into the elements comprising the retail sales environment thus providing program implementers with a more predictable outcome of end results – up front.
Citation preview
MEEA Technical Webinar Series:
A New Approach to Estimating
Free Ridership for Upstream
Lighting Programs
Presenters: Stan Mertz - Applied Proactive Technologies &
Tami Buhr - Opinions Dynamics
Thursday, October 18th, 2012
MEEA’s Role in the Midwest
• Nonprofit serving 13 Midwest states
• 10+ years serving states, energy offices, utilities and communities
• Staff of 25 in Chicago
• Actions
– Designing & Administering Energy Efficiency Programs
– Evaluating & Promoting Emerging Technologies
– Regional Voice for DOE/EPA & ENERGY STAR
– Coordinating Utility Program Efforts
– Delivering Training & Workshops
– Advancing Energy Efficiency Policy
– Promoting Best Practices
REVENUE NEUTRAL MODELA New Approach to Estimating Free
Ridership for Upstream Lighting Programs
October 2012
Agenda
1. Challenges associated with estimating lighting program NTG
2. Theoretical background underlying Revenue Neutral Sales Model
3. Example of how the model works
4. Questions
Revenue Neutral Model 2
Lighting Program Evaluation
Challenges and Traditional Methods
Revenue Neutral Model 3
Estimating Lighting Program Free Ridership is Challenging
• Programs usually delivered through an upstream markdown method
• Customers purchase discount lighting, walk out the store and disappear• Often unaware of the discount
• Note that this does not mean customer would have purchased bulbs at full price
• Retailers are fully aware of their participation
• Traditional methods use data from both customers and retailers to estimate free ridership
Revenue Neutral Model 4
What Evaluators Really Want: Full Sales Data
• Data that tracks sales of EE lighting with and without program pricing would provide the best estimate of program impact
• Could see the actual lift in sales when program in effect
• Depending on how long program has been running, measurement approaches could include:
• Pre-program sales, sales when programs turn pricing on and off, sales of like products that are not discounted, sales in comparison areas that do not have programs
• Unfortunately, retailers will not provide sales of EE lighting at regular price. Will only provide sales of products lighting programs discount.
Revenue Neutral Model 5
Sales Data Alternatives for Estimating Free Ridership
• Participant self-reports• Common method for estimating program free ridership for rebate programs• Difficult with upstream programs where customers purchase program incented product • Difficult with upstream programs where customers purchase program incented product
and disappear
• Two survey methods commonly used:• General population telephone surveys
• Call utility customers and ask detailed questions about past lighting purchases• Results are of questionable validity due to timing of survey , small nature of purchase, and difficulty
identifying program purchasers
• In-store customer interviews• Interview customers in store immediately after they make purchase decision• Greater confidence in self-report results• Usually make use of non-probability samples• Expensive and challenging to conduct
Revenue Neutral Model 6
Sales Data Alternatives for Estimating Free Ridership (2)
• Retailer Interviews• Conduct interviews with corporate or store level retailers and ask for
ti t f g i t lestimate of program impact on sales• Are no more likely to give up the numbers in an interview than a request for
data. • Store level staff often do not know sales• Store level staff often do not know sales• Corporate level do not know for a specific utility territory• At best, get rough estimates• May have vested interest in seeing programs continue • May have vested interest in seeing programs continue
• Modeling Techniques• Many require use of self-report data in addition to other data (e.g. multi-
state model revealed preference models)state model, revealed preference models)• Suffer from the same validity problems
Revenue Neutral Model 7
Another Look at Sales Data
• Existing FR methods are challenging, expensive and produce questionable results
• We keep going back and asking for complete sales data. • Maybe if we ask again, or ask nicer, or ask a different person, we’ll get it.
• We have asked everyone for sales data including program implementersI l t l h l l t g t t t t d t i t t • Implementers also help evaluators get access to stores to conduct intercept interviews
• We started talking about the challenges associated with our existing free ridership methods and alternativesp
• Is there something we can do with the sales data we do have?• Have program sales data. Also have program prices, regular prices, and sales goals
for each retailer for each product
• Estimate non-program sales using data we do have and a model of retailer behavior
Revenue Neutral Model 8
Revenue Neutral Model
The Alternative Sales Data Method
Revenue Neutral Model 9
Retailer Behavior
• Retailers will only participate in utility lighting programs if their participation is revenue neutral• Their “top line sales” remain the same or increase; cannot decrease
• But why are “top line sales” so important to a retailer?
• How exactly do retailers factor topline sales into their decision to participate in utility programs?
Revenue Neutral Model 10
What is Topline Sales?
• “Top Line Sales” is a reference to the gross sales or revenues of a company.
• The "top" reference relates to the fact that on a company's income statement, the first line at the top of the page is generally reserved for gross sales or revenue. • Program reimbursements for sales are not included in revenue
• A company that increases its revenues is said to be "growing its top line", or "generating top-line growth".
• This contrasts with net income (or net earnings per share), which is usually the bottom line of the company's income statement.
Revenue Neutral Model 11
Topline Sales Impact Example
• Retailer must sell a minimum of 254 additional units just to get back to $697 total sales dollars generated before program
$697.00 $697.00
Top Line Sales $
Incremental sales lift+254 units
R l R il $6 97
Sales $ Generated $197.00
+254 units
Regular Retail $6.97
Promo Retail $1.97Discount $5.00 100 Units
sold100 Units
sold354 Units
sold
Revenue Neutral Model 12
Gross Margin Impact Example
• Retailer is made whole on the Gross Margin line after discounts have been reimbursed on first 100 units and all additional units provide incremental GM $ G th Growth.
• Unfortunately, the rebate dollars are not able to be credited to Top Line Sales dollars.
IncrementalGross
Margin $ Generated
Incremental Gross
Margin lift on
+254 units
100 Units sold before
If only 100 Units sold
354 Units sold
Revenue Neutral Model 13
Program during Program
Retailer Decision Tree: Year 1 (SKU Level)
Less than i t
Decline to participate ORRevise Strategy of Incentives to
How many units can be sold in the
Promotional time period at the
prior to Program
gymeet Necessary Revenue Dollar
level
preduced price? Same or Above
prior to Program
Same Above
Participate atAgreed Upon Incentive level and Allocation
Amount
Participate atAgreed Upon Incentive level and Negotiate Additional
Allocation Amount
Revenue Neutral Model 14
Retailer Decision Tree: Additional Years (SKU Level)
Was the Previous Year
promotion revenue
Discontinue OR
No
promotion revenue neutral or better? Participate with Revised Incentive
Discontinue OR
How many units can be sold in the
Yes Less than Previous Year
Revise Strategy of Incentives to meet Previous Year Revenue or
Additional Promotion Opportunity
Next Year? Same or Above Previous Year
Continue atCurrent Incentives
d
Continue at Current Incentives
d
Same Above
Revenue Neutral Model 15
and Same Allocation
and Negotiate Additional Allocation
Model Implementation
Revenue Neutral Model 16
Required Data
• Need program tracking data at sku level for:• Sales goals
• MSRP
• Incentive amount (and any changes in incentive amount over the promotional timeframe)
• Program sales• Program sales
• Use the first three to estimate sales without program discount pricing
• Use program sales to calculate free ridership
Revenue Neutral Model 17
Model Implementation
• Can estimate the program’s likely free ridership before the program year
• Likely free ridership is what the program will receive if each sku meets its sales goal and the program does not allow sales to exceed goals
• Actual free ridership is based on final program sales• Will be higher for skus that do not meet sales goal
• Will be lower for those that exceed goals
Revenue Neutral Model 18
Use in Actual Evaluations
• Have used the model in two evaluations so far
• Results are not yet publicly available
• Used multiple methods for one evaluation and all came up with similar p pnumbers
• We are pursuing a patent on the model. We are pursuing a patent on the model.
Revenue Neutral Model 19
Model Benefits
• Can calculate free ridership by• Bulb type (standard, specialty, fixtures, CFLs, LEDs)
• Retailer
• During special promotions compared to regular program pricing
• None of the existing lighting free ridership methods provide this level of detail
Revenue Neutral Model 20
Program Design
• Can use results to identify changes to program implementation to minimize free ridership within existing program budgets.
• Tailor the product mix by bulb type
• Vary incentive levels by retailer and bulb type• Some retailer and bulb types have higher FR rates
• Need greater incentives to encourage purchase by non-free riders
• Maximize incentive dollars by matching incentive amount to bulb and retailer types
Revenue Neutral Model 21
Questions or Comments?
Tami Buhr
Director of Survey Research
Opinion Dynamics
617-301-4654
Stan Mertz
Director of Retail OperationsDirector of Retail Operations
Applied Proactive Technologies
413 731 6546413-731-6546
Revenue Neutral Model 22
The EE Story (Future)
• Future: Finding a new portfolio
– Lighting savings going down
– Some program saturation
– Need ‘new’ programs
• Whole home (HPwES, air sealing, etc)
• Systems work (HVAC systems, smart homes, etc)
• Behavior programs (changing the customer habit)
• Education
• Building Energy Codes (adoption, training and
compliance)
– Challenges
• Cost effectiveness (non-energy benefits not counted)
• More complex (contractors, systems, etc)
Presenter Contact Information
Stan Martz, Applied Proactive Technology –
Tami Buhr, Opinion Dynamics –