Using Market Segmentation to Track Program Success_ADwelley

Using Market Segmentation to Track Program Success

Amanda Dwelley

AESP EM&V Online ConferenceDecember 4, 2013

About Opinion Dynamics

Established in 1987

Leader in market research for utilities

Offices in Massachusetts, California & Wisconsin

Custom approach —We work with utilities and implementers to use all available data to develop tailored solutions

Energy Advising

Market Research

Energy Efficiency Evaluation

Smart Grid, DR, and Behavior

AESP EM&V Online Conference 2

Key Points


There are many ways to segment utility customer populations Some are more meaningful than others for program design, portfolio

planning and/or EM&V

Implementers are already using segmentation to improve program targeting (and uptake)

The EM&V community (us!) does analyze results by customer group/segment …But often not in a cohesive or consistent way

Consistently integrating segmentation in to EM&V will: Deliver insights that help programs improve faster

Get stakeholders thinking about (a) how results can be used/extrapolated, and (b) if/how programs should be tailored/targeted to different segments

1

Program implementers use segmentation all the time

Segmentation defines and divides a large population into identifiable groups based on similar characteristics

• High summer usage targeted for HVAC rebate

• High annual usage targeted for behavioral programs

Multi-family middle-income targeted for audits / weatherization

0%

5%

10%

15%

20%

25%

Summer kWh

Experian Mosaic Segment

AESP EM&V Online ConferenceUrbanites targeted for HEMS / IHD

Historical approach of “equal access” to programs, and undifferentiated marketing, hasn’t yielded equal impacts

5AESP EM&V Online Conference

What were the drivers of these differences? Targeted marketing? Awareness/knowledge? Qualification criteria? Interest?

Cumulative EE Participation vs. Assessed Home Value (among the

50% of customers with assessor data)

0%

2%

4%

6%

8%

10%

12%

1 2 3 4 50%

2%

4%

6%

8%

10%

12%

1 2 3 4 50%

2%

4%

6%

8%

10%

12%

14%

16%

1 2 3 4 5Income Quintile

EE P

artic

ipat

ion

Rat

e

EE P

artic

ipat

ion

Rat

e

Income Quintile Home Value Quintile

For this utility, there’s a strong relationship between wealth quintile (measured three ways) and long-term EE program participation:

Cumulative EE Participation vs. Per Capita Income as % Poverty

Line (modeled value)

Cumulative EE Participation vs. Pct of Neighborhood with Income

>$75k (from secondary data)

EE P

artic

ipat

ion

Rat

e

We’re leaving opportunity on the table, but don’t know where or how much


2.50% 2.55% 2.60%

0.0%

0.5%

1.0%

1.5%

2.0%

2.5%

3.0%

3.5%

PY 2013 PY 2014 PY2015

Three-Year Plan vs. Statewide Goals Segmented program evaluation and opportunity studies can uncover how/why:• Moderate income status?• House type (SF/MF)?• Seasonal/vacation homes?• Channel preferences vs.

implementation channels?• Baseline efficiencies

already high?

“Our customers are unique – So we can’t

reach statewide goals”

Evaluators do report on differences by customer group, but sometimes we only look within a program

1.2%1.6%

1.8%

0.0%

0.5%

1.0%

1.5%

2.0%

2.5%

LowConsumption

MediumConsumption

HighConsumption

Annu

al P

erce

nt

Savi

ngs


Annual Percent Savings by Consumption Tertile

Top 20-30%

Top 10-20%

Top 10%

Make sure segment “membership” we report is relative to the customer population; use the same data source

• Misleading to report, because the program targeted high users!

• Difficult for planners/evaluators to understand how to use findings

Metrics Measurement Opportunities

Segment-level insights are useful across the program lifecycle


General population and non-part surveys

Inquiries, leads, incomplete applications that link to customer database by account #

Ex ante: Filter database by qualifying criteria Ex post: Program qualification rates

Program participation rates Portfolio-level participation: What % of all segment

members have participated in any EE?

Online / HEMS / IHD device tracking Participant surveys

Realization rates by segment Savings “depth” by segment (% savings) Measure mix by segment

Awareness / Knowledge

Intention

Qualification

Participation

Engagement

Impacts

So, what segmentation is “good” for EM&V purposes?


1. Segment membership must be identifiable ex ante for all customers Rate code (Low income, SF/MF, Small/large commercial) Psychographic “lifestyle segment” available through data providers

(e.g., Experian) Usage characteristics (L/M/H; summer load; load shape)

1. Segments should distinguish between meaningful differences that affect program outcomes

Energy opportunity Barriers to participation (own/rent; income) Motivation to participate Channel/communication preferences (on-bill, web, phone) Impacts!

1. Segments should be “consumable” by readers/regulators: Easy to understand / well-named Manageable number

1

2

3

Identification: Tracking by segment requires defining segments based on readily-available data – And we have a lot!

Customer characteristics from CIS data – e.g., rate class, time-as-customer

Customer characteristics from CIS data – e.g., rate class, time-as-customer

New 1-4 yrs 5-9 yrs 10-19yrs

20+ yrs

Energy indicators –e.g., seasonal usage, load shape

Energy indicators –e.g., seasonal usage, load shape

0 2 4 6 8 10 12 14 16 18 20 22

Past program participation –DSM and non-DSMPast program participation –DSM and non-DSM

AccountTOURate

Energy Audit

Ref. Rebate

A B C

Customer engagement – e.g., online activity,payment preferences

Customer engagement – e.g., online activity,payment preferences

Secondary demographic/ housing data – e.g., age, income, home value

Secondary demographic/ housing data – e.g., age, income, home value

11

1

AESP EM&V Online Conference

Meaningful Differences: Segment membership should correlate with savings opportunities, program propensity, barriers and preferences

Demographically-Based “Lifestyle” Segmentation

Custom Psychographi Segmentation

Energy Usage Patterns

0 2 4 6 8 10 12 14 16 18 20 22

HighestHighest

MediumMedium

LowestLowest

Past Participation

• May correlate well with:• Ability to

participate• Channel/

marketing affinity• Heterogeneous in

terms of:• Savings

opportunities

• May correlate well with:• Savings

opportunities• Heterogeneous in

terms of:• Ability to

participate• Channel/

marketing affinity

• May correlate well with:• Ability to

participate• Motivation

• Heterogeneous in terms of:

• Savings opportunities

• Channel/ marketing affinity

Dim. 2

Dim. 1

2

Have AMI data? Clustering customers into Load Shape Segments could enable long-term impact tracking


0

500

1000

1500

2000

2500

3000

3500

4000

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Whole‐House Load Shapes

cluster similar patterns

0

500

1000

1500

2000

2500

3000

3500

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

High Peak / Low Baseload

Extended Peak

High Baseload

Low Users

Best target for DR and conservation

programs?Relatively high

baseload - many EE/Wx opportunities

Non-HVAC EE and behavioral interventions

Low-cost conservation and

behavior\

Identify highest-impact equipment, envelope and

behavioral opportunities for each segment

“Consumable” segments: Easy to explain and interpret; manageable number


Single dimensions (single-family / multi-family) or 2X2 matrices have merit

But they leave a lot of heterogeneity undescribed

Complex segmentation schemes quickly go un-used

Reviewers don’t have background/knowledge of approach

Imagine 70 Experian lifestyle segments!

Cost implications to what we choose

Segment quotas

3

We can start by reporting savings at a segment level


SegmentPercent of Customers

Percent of WxParticipants

Wx Savings per Household

(kWh)

Wx Savings Total

(MWh)

A 25% 28% 180 81.0

B 15% 14% 150 33.8

C 40% 34% 100 56.0

D 20% 24% 80 32

Total 100% 100% 124 202.8

End game: Identify and track program opportunities and success metrics specific to each segment


SegmentPercent of Customers

Percent of Wx

Participants

n Targeted for Wx

Wx Uptake (among those

targeted)

Wx Savingsper Household

(kWh)

Wx Opportunity per Household

(kWh)

% of Opportunity

Achieved

A 25% 28% 5,000 9% 180 200 90%

B 15% 14% 3,000 7.5% 150 300 50%

C 40% 34% 8,000 7% 100 150 75%

D 20% 24% 4,000 10% 80 100 80%

Total 100% 100% 20,000 8.2% 124 172 72%

Participation rate among encouraged

Savings depth or realization rate

Thank You!


Amanda Dwelley

Associate Director

617-301-4629

[email protected]

Visit us at www.opiniondynamics.com

Marketing

Using Market Segmentation to Track Program Success_ADwelley