The Implication of Faulty Sensors to Asset Analyticscigre-usnc.tamu.edu/wp-content/uploads/2015/06/The-Implication-of... · The Implication of Faulty Sensors to Asset Analytics Oct

©2013 Doble Engineering Company. All Rights Reserved

The Implication of Faulty

Sensors to Asset Analytics

Oct. 21-22, 2013

Matt Garber

Doble Engineering Company

Don Angell


Lee Margaret Ayers


©2013 Doble Engineering Company. All Rights Reserved Asset Management Meeting, dobleARMS™ Consortium 2

• Implication of faulty

devices to the Analytic

Process

• The importance of data

validation within the

Analytic Process

• Where/how this is being

tested today

• Tangible examples

Key Takeaways


• Analytic results give us insight into asset health.

• Writing an analytic is relatively easy. – E.g. calculating abnormal OLTC activity

– The result should also inform us about:

• Who, what, where, when, and how long do I have to respond?

• The challenge is communications and monitoring errors can result in false positives – Diverts $$$ from key areas that need

attention

• Analytic results – Too many alerts, people will stop listening

– Not enough alerts, people will not trust the system

– Storing a “bad” result muddies our understanding of asset history and analytics for determining asset life

Determining Asset Health?

• Woman: I'm not a witch! I'm not a witch!

• Valdimir: ehh... but you are dressed like one.


• Supervisory Control and Data Acquisition (SCADA)

– Very reliable data source

– w/ 1-2 sec timestamps

– SCADA represents an amalgam of sources:

• RTU’s/IED’s/Relays/Transducers

• When linked with SCADA, data sources are typically dropped

– SCADA might see “DGA Alarm,” but not whether the alarm

is from a Delphi, a Severon, or a Hydran, etc. • 1:n vs. 1:1 relationship

• “n” represents a variety of reliable and non-reliable devices in the field.

What happens if the sensor behind a good data

source, is bad?


• Those with tribal knowledge learn to ignore bad sources of

data

• Those with less knowledge, but with responsibility must

respond

• Can we program/teach an automated system to do this?

– And can the automated system be used to debug

questions humans cannot?

Humans Filter


4-A—Data Pathways

6

Member Data

4-A NOC & Data Center


Data Pathways and Relationships

DNP 3 4-A

1:1 relationship DGA Monitor Vendor A (via cellular/DNP) directly into AAA

DNP 3 RTU/IED/

Relay/ Gateway

SCADA 4-A Vendor OPC Historian OPC

1:N relationship DGA Monitor Vendor A to IED/RTU/Relay/Gateway to SCADA to AAA

RTU/IED/Relay/

Gateway SCADA DNP 3 4-A Vendor Vendor

Custom DB

Vendor

N:N relationship DGA Monitor Vendor A to ED/RTU/Relay/Gateway to SCADA to The PI System to AAA DGA Monitor Vendor B to ED/RTU/Relay/Gateway to SCADA to Custom DB to AAA

DGA A

DGA A

DGA A

DGA A


Data Sources Currently Feeding Analytics

Conditional Alerts

Criticality & Impact

Risk

Error Trapping/ Validation

Analytics

Asset Registry/ CIM/ IEC 61850 Co

nd

itio

nal

Eve

nts

Co

nd

itio

nal

Eve

nts

M5300

Sweep Frequency

M4000

Insulation Analyzer

Partial Discharge

PDS100

2. Online Diagnostics

IDD

Moisture in Oil

Domino

Continuous Monitoring

PD-Guard

dobleDATABASE™

SCADA/ Relay/

Gateway

Oil Sample

**Inspection

1. Offline Diagnostics

3. Real-time

5. Maintenance

4. Lab

IEDs, RTUs, Transducers

1:1

1:N

Delphi

Bushing DGA

**Manual Errors

6. Apparatus test DATABASE


• Complex, multilayered process – Math, Science & Art

• Establish behavior

• Error trapping, Points of failure – From Source to Target

– Device/System, Hardware/Software, Communications/Network

• Validation

• Analytics-Basic & Advanced

• Complex Analytics – Tap changes SCADA sees versus

TAP changes reported during physical inspection.

• Rules for how we handle bad data

The Analytics Before the Analytics

• Health & Risks • Rolling up results • Relevance • Confidence • Complex Algorithms • Rules • Math/Algorithms • Data Validation • Error Trapping • Behavior • Data sources

Math?


Tuning the Data: Establishing Expected Ranges & Behavior

• Timestamps (**need common)

• Update Rates (.5 sec. to 5 yrs.)

• Expected Range (0-span)

• Exception (when data changes)

• Compression (when stored)

• Engineering units (W or MW)

• Scaling (x10)

• Type-digital, integer, float

3500 points

72 points

No sense going any further if this

is not well understood!


Deviations were tied to filtering on relay

• Station to Station with PT in between

• Same filter on back-end, different filters on the front-end

• We can resolve during the tuning phase


Front-End: Points of Failure or Delay

•Source, intermediary, target (& then source, intermediary & target)

•Hardware shut down, or hardware/ interface not rebooted

•Memory, CPU, Speed?

•Comms loss or network delay

•Ability to generate alerts—hardware, comms, & network

•Watchdog tags = 0 or 1


• Devices: Transducers/Meters/RTU’s/IED’s can be faulty from the start

– Missing data

– Spikes (is the new meter under warranty?)

– Data not changing or are out of range

– Data are changing, within range, but abnormal read

– Improper reconfiguration of relay

• Known to degrade over time

– Unreliable with intermittent good and bad values

• Are the data good or bad?

– And what should we write to database if bad—policies?

– If data source is available, but a point is not = Not a Number (NaN)

• Analytic has to watch for initiation

Once We Establish Consistent Source, Validate the Data


Temperature Scenario

• Three temperature monitors (bad (blue), went bad (yellow)& good (red)

• Identification of faulty sensor (blue & yellow trend) is straightforward

• Business must decide how we handle the alert- built into analytic rules


• Data-source dependent (uses behavior constraints)

Validation Rules


Summer Peak

• A=data.rt.scada.3Phs.mva.meas

• B=data.rt.SummerPeak

• [if badval(B/A) then NaN else B/A]

– NaN= Not a Number

Front-end Validation Routines

• A “zero” value would be confusing

• Not an error. Data source is available, but not yet!

• Analytic must initiate when data is available

• Must consider viewer in the whole data validation process

If data.rt.scada.3Phs.mva.meas.val = good, then calculate summer

Peak….


Advanced Analytic: Overly active OLTC:

SCADA Analytic Results 1.

A look at the actual data


Correlation Using Maintenance Records

• Can we correlate with maintenance period?

• Compare number of operations

• Or maybe compare Average daily operations

2. Compare with

maintenance

records

start_date end_date type counter_ltc ltc_position_max

7/23/2012 9:07 7/23/2012 9:43 Patrol 50105 6R

6/6/2012 8:46 6/6/2012 9:14 Patrol 49284 4R

4/23/2012 11:39 4/23/2012 12:07 Patrol 48547 7R

2/29/2012 8:22 2/29/2012 8:47 Patrol 47761 4R

1/11/2012 8:34 1/11/2012 9:07 Patrol 47219 3R

11/4/2011 9:09 11/4/2011 9:57 Patrol 46522 4R

08/03 07/23 08/09 06/06


• Front-end error-trapping and validation processes are essential to a producing valuable analytic – Over time we will have a history of characteristics for data sources to add to

the historical database (80 years, 30M records)

– Additional Tuning—need analytics for expected variations & pattern recognition

– **Some sources can have a “Quality bit”

• Codes are custom and range from 1 to 200

• Correlation Analytics will improve Confidence – Users will likely tune their Alerts to see higher Confidence.

• Bad data = Alerts – Provides a good test that analytics are working

– Start-up is typically about correcting configuration, comms issues or data errors

– Asset analytics infrastructure offers the same opportunity to prioritize, sort and address data errors as it does asset health issues.

Summary


• Company preferences and procedures need to become

part of analytic rules when alerting on data errors

– Do we send an alert to repair, ignore or flag issue?

• Ad hoc tools to query & perform analysis of like devices

– Only the trained few should only see comms alerts

– People tend to trust data, so users should see: “Watchdog reports

an error on this page”

• Asset Management offers a real business case for

standards • When a data source normalizes the origin of data (turns 1:n into 1:1

relationships), standards (CIM/IEC 61850) give us a path forward

• Re-infuse the Asset Registry with intelligence during integration

Summary


• Today data origins have to be manually re-infused in the asset model (if it is an option).

• Example – Before: data.rt.scada.3Phs.mva.meas

– After: data.rt.pi.scada.SEL287E.3Phs.mva.meas • Allows us to troubleshoot comms issues and points of failure

• PI-to-PI between ARMS has inherent comms troubleshooting

• Can correlate other data issues related to the relay

– Before: data.rt.scada.DGAAlarm.status • Is it a high alarm or low alarm?

• Does the alarm refer to a specific gas?

• Analytics can only be broad.

– After: data.rt.pi.scada.IDD.Delphi.DGAAlarm.status • Asset Analytics can specific

Proposal—Extending Data Path Names to the

Asset Risk Management System


Matt Garber

[email protected]

Lee Margaret Ayers

[email protected]

+1-808-333-9845

Don Angell

[email protected]

+1-508-887-1945

Contacts

22

mailto:[email protected]



Documents

The Implication of Faulty Sensors to Asset Analyticscigre-usnc.tamu.edu/wp-content/uploads/2015/06/The-Implication-of... · The Implication of Faulty Sensors to Asset Analytics Oct