High-dimensional Data Analysis in PowerSystem Monitoring

Dr. Meng Wang

Rensselaer Polytechnic Institute

CURENT CourseNovember 11, 2015

Big Data Issues

Large in quantity or complex instructure.The curse of dimensionality.Challenges in acquisition,storage, and processing.Applications: Medical imaging,genomic data analysis, socialnetwork analysis, powersystem monitoring, Internetmonitoring, etc.

1 / 46

Power System Monitoring

http://www.nepower.org/npa-on-energy-issues/transmission/

Example of PMUhttp://www.macrodyneusa.com/

model_1690.htm

SCADA (supervisory control and data acquisition) Systemprovides power measurements every five seconds.PMUs (Phasor Measurement Units) can provide synchronizedphasor measurements of the power system at a sampling rate of30 samples per second or more.

2 / 46

Phasor Measurement Units

After the DOE smart grid investment program starting in 2009, theinstallation of PMUs in the North American power system will beclose to 2,000 PMUs soon.Multi-channel PMUs can measure bus voltage phasors, linecurrent phasors, and frequency.

http://www.riteh.uniri.hr/zav_katd_sluz/zee/nzz/klub/images/IEEE14.JPG

Current Installation of PMUshttps://www.naspi.org/documents3 / 46

PMU Data Analysis

Applications:I State estimation,I Oscillation detection and electromechanical mode identification,I Stability analysis and post-event analysis,I Disturbance detection and location, dynamic security assessment,

Task specific methods, developed when the coverage of PMUs ona power network was quite sparse.Mostly exploit the spatial and the temporal correlations in themeasurements separately. Lack a common framework.

4 / 46

Spatial-temporal Blocks of PMU data

As the coverage of PMUs becomes denser, one can analyze PMUdata collectively from PMUs located in electrically close regionsand distinct control regions.Processing spatial-temporal blocks of PMU data for tasks such asmissing data recovery, data compression and storage, disturbancetriggering, and detection of cyber data attacks.

How to process the high-dimensional PMU data blocks?

5 / 46

Outline

1 Introduction

2 High-dimensional Data Analysis Using Low-dimensional Models

3 Missing PMU Data Recovery

4 Detection of Cyber Data attacks

Outline

1 Introduction

2 High-dimensional Data Analysis Using Low-dimensional Models

3 Missing PMU Data Recovery

4 Detection of Cyber Data attacks

Exploiting Low-dimensional Structures inHigh-dimensional Datasets

It is relatively easy to acquire, store and process high-dimensionaldatasets if they have low-dimensional structures.Examples: sparse signals, low rank matrices.Wide existence of low-dimensional structures: images, ratingmatrices, network measurements,...Recent arts: compressed sensing theory, low rank matrix theory.

6 / 46

Wide Existence of Low-dimensional Structures

Sparse signalsa vector in Rn

k nonzero coefficients undercertain basis. (k << n)Imaging,Communication,Biology

Low-rank matricesrank << dimension.Collaborative filtering (Netflixproblem)System identificationInternet traffic analysis

http://www.inkfarm.com/Image-File-Extensions

http://www.netflix.com

User ratings

7 / 46

Compressed Sensing for Sparse Signals

y A x

m = n

Unknown sparse signal x, observation y, measurement matrix A.y = Ax, m n.

Given A, sparse x can be recovered from y!

significant reduction in the required number of measurements.k-sparse n-dimensional signals, m = O(k log n

k ).simple compression methods. A could be random matrices. Wideexistence of good A’s.efficient recovery methods. e.g., `1 minimization.

8 / 46

Compressed Sensing for Sparse Signals

y A x

m = n

`0-minimization

minz‖z‖0 s.t. Az = y,

where ‖z‖0 = number of non-zero entries of z.

bad news: computationally hard.

9 / 46

Compressed Sensing for Sparse Signals y A x

m = n

`1-minimization`1-minimization

minz‖z‖1 = ∑

i|zi| s.t. Az = y,

where ‖z‖1 = ∑i |zi|.

The sparse signal x is the solution to `1 problem!

computationally efficient.theoretical guarantee. (Candès and Tao 2006, Donoho 2006)

10 / 46

Low-rank Matrix Completion

Netflix Problem

Low rank matrix completion problem.

11 / 46

Low-rank Matrix Completion

Low-rank Matrix with Missing Entries

?

?

? ?

? ?

?

How shall we recover the missing entries of a low-rank matrix?

minX

Rank(X)

s.t. Xi j = Mi j,∀i ∈Ω.

Ω: locations of the observed entries.NP hard.

12 / 46

Low-rank Matrix Completion

Nuclear norm minimization (Fazel 2002, Candés & Recht 2009),recover the missing data by solving a convex program.

minX‖X‖∗ = sum of the singular values of X

s.t. Xi j = Mi j,∀i ∈Ω.

Convex relaxation of Rank minimization problem.computationally efficient. Can be solved by SemidefiniteProgramming.Theoretical guarantee. (Candés & Recht 2009)

13 / 46

Low-rank Matrix Completion

Theorem (Candés & Recht 09, Gross 11, Recht 11)All entries of a rank-r matrix L ∈ Cn1×n2 can be corrected recovered, aslong as O(rn log2 n)(n = max(n1,n2)) randomly selected entries of L areobserved.

Significant saving in the number of observations when r is small.

Theorem (Candés & Tao 09)If each entry is observed with prob. p = m

n1n2, no method whatsoever

can succeed with

m≤Cnr logn, for some fixed constant C

Near-optimal recovery via Nuclear norm minimization.

14 / 46

Connection to Compressed Sensing

General setupRank minimization

minX

Rank(X)

s.t. 〈Ai,X〉= Trace((Ai)T X) = bi,

∀i = 1, ...,m

Convex relaxation.

minX‖X‖∗

s.t. 〈Ai,X〉= Trace((Ai)T X) = bi,

∀i = 1, ...,m

Consider the special case X = diag(x),x ∈ Rn, then

Rank(X) = ‖x‖0,

‖X‖∗ = ‖x‖1.

Reduce to compressed sensing!

15 / 46

Principle Component Analysis: Classical Method

widely used for dimension reduction.convert observations to a set ofvalues of linearly uncorrelatedvariables, called principlecomponents.

Stack all the data points as column vectors of a matrix M, PCA seeksthe best rank-k approximation by solving

minL‖M−L‖2

s.t. Rank(L)≤ k

Not rubust to even a few outliers in M.

16 / 46

Robust PCA

The observation matrix is the sum of a low-rank matrix L0 (actualdata) and a sparse matrix (corruptions).

M = L0 + S0

= +

low rank sparse Measurements

Decomposition of a low-rank matrix and a sparse matrix.Applications in computer vision, image processing, network trafficanalysis, etc.

17 / 46

Robust PCA

(Hard) Optimization problem

minL,C

Rank(L)+λ‖C‖0

s.t. L+C = M

‖C‖0: the number of nonzero entries in C.not tractable.

Convex relaxation

minL,C‖L‖∗+λ‖C‖1

s.t. L+C = M

‖C‖1 = ∑i j |Ci j|Theoretical guarantee:Candés & Li & Ma & Wright 11, Chandrasekaran & Sanghavi &Parrilo & Willsky 11, Recht & Fazel & Parrilo 10.

18 / 46

Does low-dimensionality exist in PMU measurements?

19 / 46

Low-rank Property of PMU data

PMUs in Central NY PowerSystems

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200

5

10

15

20

Time(s)

Cu

rre

nt

Ma

gn

itu

de

Current magnitudes ofPMU data

5 10 15 20 25 30 350

200

400

600

800

1000Distribution of singular value

Singular value index

Singular values of thePMU data matrix

6 PMUs measure 37 voltage/current phasors. 30 samples/secondfor 20 seconds.Singular values decay significantly. Mostly close to zero. Singularvalues can be approximated by a sparse vector.Low-dimensionality (also observed in Chen & Xie & Kumar 2013,Dahal & King & Madani 2012)

20 / 46

How could we use the low dimensionality in PMU data managementfor power system monitoring?

21 / 46

Publications

Missing data recoveryGao, Wang, Ghiocel, Chow, IEEE Trans. Power Systems 2015.Cyber data attacksWang ,Gao, Ghiocel, Chow, Fardanesh, Stefopoulos,Razanousky,IEEE SmartGridComm 2014Low rank approach to data analysisWang, Chow, Gao, Jiang, Xia, Ghiocel, Fardanesh, Stefopoulos,Kokai, Saito, Razanousky, HICSS 2015

22 / 46

Outline

1 Introduction

2 High-dimensional Data Analysis Using Low-dimensional Models

3 Missing PMU Data Recovery

4 Detection of Cyber Data attacks

Missing Data Recovery

We observe partial entriesof the complex matrix(rectangular form of voltageand current phasors).How shall we recover themissing points for offlineapplications like systemidentification and past eventanalysis?

Low-rank Matrix Completion Problem!

23 / 46

Low-rank Matrix Completion

Wide applications in collaborative filtering, computer vision,machine learning, remote sensing, and system identification.Problem formulation: given part of the entries of a matrix, recoverthe remaining entries.the rank of the matrix is much less than its dimension.

24 / 46

Low-rank Matrix Completion

Low-rank Matrix with Missing Entries

?

?

? ?

? ?

?

Nuclear norm minimization

minX‖X‖∗ = sum of singular values of X

s.t. X is consistent with the observed entries,

Quite a few recovery algorithms exist, e.g., singular valuethresholding (SVT) (Cai et al. 2010), information cascading matrixcompletion (ICMC) (Meka et al. 2009).

25 / 46

Low-rank Matrix Completion

Theorem (Candés & Recht 09, Gross 11, Recht 11)All entries of a rank-r matrix L ∈ Cn1×n2 can be corrected recovered, aslong as O(rn log2 n)(n = max(n1,n2)) randomly selected entries of L areobserved.

Significant saving in the number of observations when r is small.Existing analysis assumes that the locations of missing points areselected randomly.

26 / 46

Our Results

The locations of missing PMU data are usually correlated.I temporal correlation: loss of consecutive measurements in one

PMU channel.I channel correlation: loss of measurements in multiple PMU

channels simultaneously.

Theorem (Gao & Wang & Ghiocel & Chow 14)Although the locations of the missing entries of a rank-r matrix aretemporally or spatially correlated, all missing entries can be correctlyrecovered as long as O(n2− 1

r+1 r1

r+1 log1

r+1 n) entries are observed.

The first theoretical guarantee of low-rank matrix completion whenthe locations of missing entries are correlated.

27 / 46

Simulation

6 PMUs measuring 37 busvoltage phasors and linecurrent phasors.30 samples per second.

0 2 4 6 8 10 12 14 16 18 200

5

10

15

20

Time (second)

Curr

ent M

agnitude

Dataset #1

0 2 4 6 8 10 12 14 16 18 200

2

4

6

8

10

Time (second)

Curr

ent M

agnitude

Dataset #2

0 2 4 6 8 10 12 14 16 18 200

5

10

15

Time (second)

Curr

ent M

agnitude

Dataset #3

0 2 4 6 8 10 12 14 16 18 200

5

10

15

20

Time (second)

Curr

ent M

agnitude

Dataset #4

Current magnitudes in four datasets.

28 / 46

Simulation Results

Recover missing data points that are temporally correlated.I the first 5-second PMU data of dataset #1.

0.1 0.2 0.3 0.4 0.50

0.05

0.1

Average erasure rate

Rel

ativ

e re

cove

ry e

rror

τ = 1τ = 3τ = 5τ = 7τ = 9

Relative recovery error of the SVTalgorithm

0.1 0.2 0.3 0.4 0.5 0.6 0.70

0.2

0.4

0.6

0.8

1

Average erasure rate

Rel

ativ

e re

cove

ry e

rror

τ = 1τ = 3τ = 5τ = 7τ = 9

Relative recovery error of the ICMCalgorithm

29 / 46

Simulation Results

Recover missing data points that are spatially correlated.I the first second PMU data of dataset #1.

0.1 0.2 0.3 0.4 0.5 0.6 0.70

0.1

0.2

0.3

0.4

0.5

Average erasure rate

Rel

ativ

e re

cove

ry e

rror

q = 0.6q = 0.7q = 0.8q = 0.9q = 1.0

Relative recovery error of the SVTalgorithm

0.1 0.2 0.3 0.4 0.50

0.2

0.4

0.6

0.8

1

Average erasure rate

Rel

ativ

e re

cove

ry e

rror

q = 0.8q = 0.85q = 0.9q = 0.95q = 1.0

Relative recovery error of the ICMCalgorithm

30 / 46

Online Missing Data Recovery

We also develop an online missing data recovery algorithm thatcould fill in the missing data instantaneously and detect systemdisturbance.

Table: Relative recovery error of OLAP on fourdatasets

pavg Dataset #1 Dataset #2 Dataset #3 Dataset #40.05 0.0082 0.0026 0.0006 0.01860.10 0.0089 0.0039 0.0009 0.02350.15 0.0119 0.0053 0.0011 0.04420.20 0.0145 0.0060 0.0013 0.04940.25 0.0153 0.0066 0.0014 0.07260.30 0.0191 0.0075 0.0017 0.07790.35 0.0204 0.0082 0.0018 0.11370.40 0.0227 0.0091 0.0019 0.1189

2 3 4 50

5

10

15

20

25

Time (second)

Cur

rent

Mag

nitu

de

Original dataErasure estimation by SVTErasure estimation by ICMCErasure estimation by OLAP

Missing data recovery byOLAP algorithm

31 / 46

Outline

1 Introduction

2 High-dimensional Data Analysis Using Low-dimensional Models

3 Missing PMU Data Recovery

4 Detection of Cyber Data attacks

Example of Cyber Data Attacks

Bus 1

Bus 2

Bus 3 𝑍!"

𝑍!"

Bus 1

Bus 2

Bus 3

𝐼!"

𝐼!! 𝑉!

𝑉!

𝑍!" 𝑍!"

Bus 1

Bus 2

Bus 3

𝐼!"

𝐼!! 𝑉!

𝑉!

𝑉! = 𝑉! − 𝐼!"𝑍!" = 𝑉! − 𝐼!!𝑍!!

𝑍!" 𝑍!"

Bus 1

Bus 2

Bus 3 𝑉!

𝑉!

𝐼!" +𝛽𝑍!"

𝐼!! +𝛽𝑍!!

Bus 1

Bus 2

Bus 3 𝑉!

𝑉!

𝐼!" +𝛽𝑍!"

𝐼!! +𝛽𝑍!!

𝑉! + 𝛽

Bus 1

Bus 2

Bus 3 𝑉!

𝑉!

𝐼!" +𝛽𝑍!"

𝐼!! +𝛽𝑍!!

𝑉! + 𝛽

Measurements V1, V2, I12, and I13.Estimate V3.Redundancy in measurementscan be used to detect bad data.Cyber data attack: manipulate I12and I13 simultaneously.Can result in significant error in V3.The removal of attackedmeasurements will make V3unobservable.

32 / 46

Cyber Data Attacks

The worst interacting bad data. (Liu & Ning & Reiter 11).An intruder with the system topology information cansimultaneously manipulate multiple measurements so that theseattacks cannot be detected by any bad data detector.Cyber data attacks can potentially lead to significant errors to theoutcome of state estimation.Also termed as “unobservable attacks”, because the removal ofaffected measurements would make the system unobservable.(Kosut & Jia & Thomas & Tong 10).

33 / 46

Existing Approaches

Cyber attacks in SCADA system. A protection perspective.Usually protect key PMUs to avoid these attacks.(Kosut & Jia &Thomas & Tong 10, Kim & Poor 11, Bobba et al. 10, Dán &Sandberg 10)Sedghi & Jonckheere 13: Detection of cyber data attacks inSCADA system. Assume the measurements at different timeinstants are i.i.d. distributed.

34 / 46

Our Contributions

A new detection method of cyber data attacks in PMU measurements.no other assumptions expect for the low-rankness of the PMUdata.The detection method can identify the attacks even when thesystem is under disturbance.The PMU channels under attack can be identified by solving aconvex optimization problem.

35 / 46

System Model

π model of a transmission line.

π model of a transmission line

Current Ii j from bus i to bus j is related to bus voltage V i and V j by

Ii j =V i−V j

Zi j +V iYi j

2.

Voltage and current phasor measurements can be represented bylinear functions of state variables.

36 / 46

Attack Model

The intruder can only attack a small number of PMUscontinuously.The intruder injects an unobservable attack at each time instant.An unobservable attack is a linear combination of errors in statevariables.

37 / 46

Mathematical Formulation

t: total number of sampling time instants.p: total number of PMU channels.n: total number of buses. n < pL ∈ Ct×p: the actual voltage and current phasors in t instants.D ∈ Ct×n: the additive error in state variables.N ∈ Ct×p: the noise.M ∈ Ct×p: the obtained measurements that are under attack.W ∈ Ct×p: relates state variables with phasors.

M = L+CW T +N

= + × +

low rank column sparse

Measurements

channel

time Noise Actual phasors Errors in

bus voltages

Measurements under attack 38 / 46

Assumptions

= + × +

low rank column sparse

Measurements

channel

time Noise Actual phasors Errors in

bus voltages

Measurements under attack

L: low-rank. From correlations in measurements.C: column sparse. The intruder has limited access to the system.N: ‖N‖F ≤ ε.

Given M and W , how could we identify L and C?Decomposition of a low-rank matrix and a transformed column-sparsematrix.

39 / 46

Matrix Decomposition

Measurements

= + +

Noise Low Rank Sparse

Decomposition of a low-rank matrix and a sparse matrix.Through convex optimization.

minL∈Ct×p,C∈Ct×n

‖L‖∗+λ ∑i, j|Ci j| s.t. ‖L+C−M‖F ≤ ε (1)

Theoretical guarantee:Candés & Li & Ma & Wright 11, Chandrasekaran & Sanghavi &Parrilo & Willsky 11, Recht & Fazel & Parrilo 10.Applications: Internet traffic analysis, image processing, medicalimaging, etc.

40 / 46

Connection to Related Work

Xu & Caramanis & Sanghavi 12: Decomposition of a low-rankmatrix and a column-sparse matrix.

Measurements

= + +

Noise Low Rank Column Sparse

Our methods and proofs are built upon those in Xu & Caramanis &Sanghavi 12. Extension to general cases.

= + × +

low rank column sparse

Measurements

channel

time Noise Actual phasors Errors in

bus voltages

41 / 46

Connection to Related Work

Mardani & Mateos & Giannakis 13: Decomposition of a low-rankmatrix plus a compressed sparse matrix. Internet traffic anomalydetection.

Measurements

= x + +

Noise WT Low Rank Sparse

Our focus: column-sparse matrices, W is arbitrary.

= + × +

low rank column sparse

Measurements

channel

time Noise Actual phasors Errors in

bus voltages

42 / 46

Our Approach

Find (L∗, C∗), the optimum solution to the following optimizationproblem

minL∈Ct×p,C∈Ct×n

‖L‖∗+λ‖C‖1,2 s.t. ‖L+CW T −M‖F ≤ ε (2)

Compute the SVD of L∗ =U∗Σ∗V ∗†.Find column support of D∗ =C∗W T , denoted by I ∗.Return L∗I ∗c , U∗ and I ∗.

(2) is convex and can be solved efficiently.

43 / 46

Theoretical Guarantee

Theorem (Noiseless measurements, N = 0)With a properly chosen λ , the solution returned by our method

1 identifies the PMU channels under attack.2 identifies the measurements that are not attacked.3 recovers the correct subspace spanned by actual phasors.

Theorem (Noisy measurements, N 6= 0)With a properly chosen λ , the solution returned by our method issufficiently close (with distance depending on the noise level) to asolution such that 1-3 hold.

44 / 46

Numerical Results

Simulate the case that theintruder alters the PMUchannels that measure I12, I52,I13 and I43.The voltage phasor estimatesof Buses 2 and 3 are corrupted

0.5 1 1.5 20

5

10

15

Time (second)(b) Disturbance data

Cu

rre

nt

Ma

gn

itu

de

Actual PMU dataPMU data under attack

0.5 1 1.5 20

5

10

15

Time (second)(a) Ambient data

Cu

rre

nt

Ma

gn

itu

de

Actual PMU dataPMU data under attack

Actual values and corrupted values

0 10 20 300

1

2

3

4

Column index(a) Ambient data

l2 n

orm

of

the

co

lum

n

0 10 20 300

1

2

3

4

Column index(b) Disturbance data

l2 n

orm

of

the

co

lum

n

Column norm of the recovered errormatrix

45 / 46

Conclusion

Leverage the low-dimensional structures in high-dimensional datato address the challenges in data acquisition, storage, andinformation extraction.Connection of low-rank methods with power system monitoring.Analysis of spatial-temporal blocks of PMU measurements.Missing data recovery: theoretical guarantee of successfulrecovery when the locations of the missing points are correlated.Detection of cyber data attacks: decompostion of a low-rank andtransferred column-sparse matrix.

46 / 46