Factored Customer Models for Agent-based Smart Grid Simulation

Prashant Reddy Manuela VelosoPrashant Reddy Manuela VelosoMachine Learning Department Computer Science Department

S h l f C t S iSchool of Computer ScienceCarnegie Mellon University

8th Annual CMU Conference on the Electricity IndustryMarch 14, 2012

Smart Grid DistributionSmart Grid Distribution

Source: EPRI

2

Distribution Grid ComplexityDistribution Grid Complexity

Source: IEEE

3

OutlineOutline

1. Agent-based Smart Grid simulation Power Trading Agent Competition

2. Intermediary agent strategies Strategy learning for broker agents

Interactions of multiple learning broker agents

3. Factored customer models Timeseries simulation using Bayesian learning

Decision-theoretic demand side management

4

OutlineOutline

1. Agent-based Smart Grid simulation Power Trading Agent Competition

2. Intermediary agent strategies Strategy learning for broker agents

Interactions of multiple learning broker agents

3. Factored customer models Timeseries simulation using Bayesian learning

Decision-theoretic demand side management

5

Agent based Smart Grid SimulationAgent-based Smart Grid Simulation

Di t ib ti id d l d lti t t Distribution grid modeled as a multi-agent system Focus on emergent economics of self-interested behavior 1

Do not assume rationality nor determinismy Agents contributed by independent research teams Competitive benchmarking to drive innovation

6

1 Leigh Tesfatsion, ACE: A Constructive Approach to Economic Theory. Ch. 16, Handbook of Computational Economics, 2005.

Agent based Smart Grid SimulationAgent-based Smart Grid Simulation

Di t ib ti id d l d lti t t Distribution grid modeled as a multi-agent system Focus on emergent economics of self-interested behavior 1

Do not assume rationality nor determinismy Agents contributed by independent research teams Competitive benchmarking to drive innovation

Power Trading Agent Competition (Power TAC) Annual tournament at major AI or MAS conferencej

Builds upon experience with other TAC domains Simulation platform available for offline research

Assumes liberalized retail markets Assumes liberalized retail markets Customers have choice of “broker agents”

7

1 Leigh Tesfatsion, ACE: A Constructive Approach to Economic Theory. Ch. 16, Handbook of Computational Economics, 2005.

Power TAC ScenarioPower TAC Scenario

CompetitionParticipants

SimulationInfrastructure

http://www.powertac.org

8

p // p g

9

OutlineOutline

1. Agent-based Smart Grid simulation Power Trading Agent Competition

2. Intermediary agent strategies Strategy learning for broker agents

Interactions of multiple learning broker agents

3. Factored customer models Timeseries simulation using Bayesian learning

Decision-theoretic demand side management

10

Strategy Learning for Broker AgentsStrategy Learning for Broker Agents

Reddy & Veloso. Strategy Learning for Autonomous Agents in Smart Grid Markets.Twenty-Second Intl Joint Conf on Artificial Intelligence (IJCAI) Barcelona 2011

11

Twenty Second Intl. Joint Conf. on Artificial Intelligence (IJCAI), Barcelona, 2011.

Interactions of Multiple Learning Broker AgentsInteractions of Multiple Learning Broker Agents

Reddy & Veloso. Learned Behaviors of Multiple Autonomous Agents in Smart Grid Markets.Twenty-Fifth AAAI Conf on Artificial Intelligence San Francisco 2011

12

Twenty Fifth AAAI Conf. on Artificial Intelligence, San Francisco, 2011.

OutlineOutline

1. Agent-based Smart Grid simulation Power Trading Agent Competition

2. Intermediary agent strategies Strategy learning for broker agents

Interactions of multiple learning broker agents

3. Factored customer models Timeseries simulation using Bayesian learning

Decision-theoretic demand side management

13

Factored Customer ModelsFactored Customer Models

P TAC i l d f d t l d t ti ti l d l Power TAC includes fundamental and statistical models Trade-off on behavioral accuracy vs. scalability

14

Factored Customer ModelsFactored Customer Models

P TAC i l d f d t l d t ti ti l d l Power TAC includes fundamental and statistical models Trade-off on behavioral accuracy vs. scalability

Goals for statistical models:1. Representation

a. Represent diverse types of consumers and producersb. Represent varying levels of granularity

2 Automated learning2. Automated learning Learn parameters from “real-world” data

3. Facilitate agent algorithms Develop algorithms that can be applied in real-world

15

Factored Customer Model RepresentationFactored Customer Model Representation

{ }N { }N { }MC = h{B i}Ni= 1,{Si}

Ni= 1,U i, B i = {O ij}

M ij= 1

16

Factored Customer Model RepresentationFactored Customer Model Representation

{ }N { }N { }MC = h{B i}Ni= 1,{Si}

Ni= 1,U i, B i = {O ij}

M ij= 1

Factored Customer

Load BundleLoad Bundle

Load Originator

Load Originator

Tariff Subscriber

Load Bundle

Utility Optimizer

Load OriginatorLoad OriginatorTariff Subscriber

17

Factored Customer Model RepresentationFactored Customer Model Representation

{ }N { }N { }MC = h{B i}Ni= 1,{Si}

Ni= 1,U i, B i = {O ij}

M ij= 1

Factored Customer

Load Bundle

Calendar- Time of day- Day of week

Tariff Terms- Fixed Payments- Variable Rates

Contract Time Load Bundle

Load Originator

Load Originator

Tariff SubscriberDay of week

- Month of year

Weather- Temperature

Cloud Cover

- Contract Time- Exit Penalties- Reputation

Load Bundle

Utility Optimizer

Load Originator

- Cloud Cover- Wind Speed- Wind Direction

Tariff Terms

Tariff Evaluation- Inertia- Rationality

Load OriginatorTariff Subscriber - Price Elasticity- Control Events

Load Origination- Reactivity- Receptivity- Rationality

18

OutlineOutline

1. Agent-based Smart Grid simulation Power Trading Agent Competition

2. Intermediary agent strategies Strategy learning for broker agents

Interactions of multiple learning broker agents

3. Factored customer models Timeseries simulation using Bayesian learning

Decision-theoretic demand side management

19

Timeseries Simulation using Bayesian LearningTimeseries Simulation using Bayesian Learning

Gi ll l f b d d t fit d l th t Given small samples of observed data, fit a model that can generate a long range time series forecast Use “similar” samples to improve the fitp p

20

Timeseries Simulation using Bayesian LearningTimeseries Simulation using Bayesian Learning

Gi ll l f b d d t fit d l th t Given small samples of observed data, fit a model that can generate a long range time series forecast Use “similar” samples to improve the fitp p

ARIMA forecasting over long range is poor

Consumption

Time (Hours)

Consumption

21

Time (Hours)

Bayesian Timeseries Simulation MethodBayesian Timeseries Simulation Method

Use similarUse similartraining data

Fit hierarchicalBayesian model

usingGibbs samplingG bbs sa p g

Latent factorLatent factoroptimization to boost fit

Forecast from small sample

22

Boosted Bayesian Timeseries SimulationBoosted Bayesian Timeseries Simulation

Consumption

Time (Hours)

Consumption

Time (Hours)

23

Boosted Bayesian Forecasting Accuracy Boosted Bayesian Forecasting Accuracy

24

Boosted Bayesian Forecasting Accuracy Boosted Bayesian Forecasting Accuracy

25

OutlineOutline

1. Agent-based Smart Grid simulation Power Trading Agent Competition

2. Intermediary agent strategies Strategy learning for broker agents

Interactions of multiple learning broker agents

3. Factored customer models Timeseries simulation using Bayesian learning

Decision-theoretic demand side management

26

Decision theoretic DSMDecision-theoretic DSM

M lti l d i i ki i t di i Multi-scale decision-making in two dimensions1. Temporal: Metering period vs. tariff contract period2. Contextual: Individual load vs. bundle/customer/co-op/ / p

27

Decision theoretic DSMDecision-theoretic DSM

M lti l d i i ki i t di i Multi-scale decision-making in two dimensions1. Temporal: Metering period vs. tariff contract period2. Contextual: Individual load vs. bundle/customer/co-op/ / p

argm axyt

U S (pt,yt,UN (yt))

argm axz2 Z t0

U S� (P

zt0,Yt0,U

N� (Yt0))

Self Utility Price Profile Load Profile Neighborhood Utility

28

Decision theoretic DSMDecision-theoretic DSM

M lti l d i i ki i t di i Multi-scale decision-making in two dimensions1. Temporal: Metering period vs. tariff contract period2. Contextual: Individual load vs. bundle/customer/co-op/ / p

argm axyt

U S (pt,yt,UN (yt))

P b bili ti lti tt ib t tilit d l

argm axz2 Z t0

U S� (P

zt0,Yt0,U

N� (Yt0))

Probabilistic multi-attribute utility model:

P r(z) =eλ U

S⌧(z)

PZ eλ U S

⌧(z)

29

Decision theoretic DSMDecision-theoretic DSM

M lti l d i i ki i t di i Multi-scale decision-making in two dimensions1. Temporal: Metering period vs. tariff contract period2. Contextual: Individual load vs. bundle/customer/co-op/ / p

argm axyt

U S (pt,yt,UN (yt))

P b bili ti lti tt ib t tilit d l

argm axz2 Z t0

U S� (P

zt0,Yt0,U

N� (Yt0))

Monte Carlo Sampling Probabilistic multi-attribute utility model:

o e Ca o Sa p g

P r(z) =eλ U

S⌧(z)

PZ eλ U S

⌧(z)

30

Peak Shifting (Herding) BehaviorPeak Shifting (Herding) Behavior

31

Ramchurn, et al. Agent-Based Control for Decentralised Demand Side Management in the Smart Grid. Autonomous Agent and Multi-Agent Systems (AAMAS), 2011.

Household Demand ShiftingHousehold Demand Shifting

B d d t f G ’ M R i j t Based on data from Germany’s MeRegio project

nC

onsu

mpt

ion

C

Time (Hours)

32

( )

Household Demand ShiftingHousehold Demand Shifting

L i d t i f 10% Lower variance and cost savings of ~10%

Balancing

Combined

Temporal

g

Original

Consumption

33

DSM Deployment OptionsDSM Deployment Options

ARM C t A8 d Zi b S CARM Cortex-A8 and Zigbee SoC

34

ConclusionConclusion

S Summary Versatile customer model representation Decision-theoretic algorithms for DSMg Bayesian learning algorithms for timeseries simulation

F t W k Future Work Customer type-specific factor modeling Non-cooperative decision-making modelsg

Participating in Power TACHosted at AAMAS Valencia June 2012 Hosted at AAMAS, Valencia, June 2012

More information at http://www.powertac.org

35

Factored Customer Models for Agent-based Smart Grid Simulation

Prashant Reddy Manuela Velosoppr@cs.cmu.edu mmv@cs.cmu.edu

School of Computer SciencepCarnegie Mellon University