Stephen BayStephen BayPat LangleyPat Langley

Mei Wang MarkerMei Wang MarkerDaniel ShapiroDaniel Shapiro

Institute for the Study of Learning and ExpertiseInstitute for the Study of Learning and Expertise2164 Staunton Court, Palo Alto, California2164 Staunton Court, Palo Alto, California

http://www.isle.org/http://www.isle.org/

{sbay,langley,mei,dgs}@isle.org{sbay,langley,mei,dgs}@isle.org

Filtering Information inFiltering Information inComplex Temporal DomainsComplex Temporal Domains

SSUs (2): primary power supply

Batteries (6): Current,Voltage (38 cells)Pressure, Temp DDCUs (6):

secondary power demand, all loads

Monitoring the Space Station Power GridMonitoring the Space Station Power Grid

• Given:Given: thousands of variables measured every ten seconds; thousands of variables measured every ten seconds;

• Detect:Detect: any significant anomalies as soon as possible. any significant anomalies as soon as possible.

Our approach to filtering high-dimensional temporal data relies Our approach to filtering high-dimensional temporal data relies on five key ideas:on five key ideas:

Model-Driven Anomaly DetectionModel-Driven Anomaly Detection

This means combining techniques from model-based reasoning, This means combining techniques from model-based reasoning, simulation languages, and human-computer interaction. simulation languages, and human-computer interaction.

1. use models and schedules to predict quantitative values;1. use models and schedules to predict quantitative values;

2. compare predictions and observations to detect anomalies;2. compare predictions and observations to detect anomalies;

3. provide graphical aids that depict functional modules;3. provide graphical aids that depict functional modules;

4. support modeling at multiple levels of abstraction;4. support modeling at multiple levels of abstraction;

5. give users control over level of detail and thresholds.5. give users control over level of detail and thresholds.

In our first twelve months on this research project, we have:In our first twelve months on this research project, we have:

Project Accomplishments (through 2/2002)Project Accomplishments (through 2/2002)

The main limitation involves the need to manage model complexity. The main limitation involves the need to manage model complexity.

• formulated a set of generic research problems in monitoring;formulated a set of generic research problems in monitoring;

• studied the structure and function of the Space Station power grid;studied the structure and function of the Space Station power grid;

• examined the actual telemetry stream from this system;examined the actual telemetry stream from this system;

• designed an approach to fault detection and event filtering;designed an approach to fault detection and event filtering;

• developed a process modeling language for numeric prediction;developed a process modeling language for numeric prediction;

• used this language to model power system at various levels of detail;used this language to model power system at various levels of detail;

• developed a method to compare predicted and observed values;developed a method to compare predicted and observed values;

• implemented a technique for displaying anomalies graphically;implemented a technique for displaying anomalies graphically;

• demonstrated fault detection and event filtering using these tools;demonstrated fault detection and event filtering using these tools;

• developed a method that uses machine learning to improve models;developed a method that uses machine learning to improve models;

• used this approach to construct accurate models of battery behavior.used this approach to construct accurate models of battery behavior.

In the most recent six months on the research project, we have:In the most recent six months on the research project, we have:

Project Accomplishments (through Project Accomplishments (through 10/2002)10/2002)

This approach provides a principled way to manage model complexity. This approach provides a principled way to manage model complexity.

• extended the modeling language to represent hierarchical models;extended the modeling language to represent hierarchical models;

• constructed detailed models of batteries and solar wing array;constructed detailed models of batteries and solar wing array;

• extended the modeling environment to simulate hierarchical models;extended the modeling environment to simulate hierarchical models;

• extended anomaly detection from variables to faulty processes;extended anomaly detection from variables to faulty processes;

• implemented hierarchical propagation and filtering of alerts;implemented hierarchical propagation and filtering of alerts;

• implemented hierarchical display of both models and alerts;implemented hierarchical display of both models and alerts;

• implemented a color coding scheme to signify severity of alerts;implemented a color coding scheme to signify severity of alerts;

• implemented ability to replay telemetry data and graph values;implemented ability to replay telemetry data and graph values;

• developed componential and causal views of hierarchical models.developed componential and causal views of hierarchical models.

To represent knowledge about the power grid, we use a modeling To represent knowledge about the power grid, we use a modeling formalism that describes a system in terms of:formalism that describes a system in terms of:

Hierarchical Quantitative Process ModelsHierarchical Quantitative Process Models

This process modeling language borrows ideas from research in This process modeling language borrows ideas from research in qualitative physics and model-based reasoning. qualitative physics and model-based reasoning.

But it adapts them to domains that involve numeric variables. But it adapts them to domains that involve numeric variables.

• the physical subsystems from which it is composed;the physical subsystems from which it is composed;

• the quantitative variables associated with each system;the quantitative variables associated with each system;

• a set of causal processes and their effects on variables;a set of causal processes and their effects on variables;

• cast as either instantaneous or differential equations;cast as either instantaneous or differential equations;

• with conditions on when each process will be active;with conditions on when each process will be active;

Partial Hierarchical Model of the Power GridPartial Hierarchical Model of the Power Grid

system PowerStore;system PowerStore; components ba1, ba2, ba3; components ba1, ba2, ba3; variables I, maxPower, Power, charging;variables I, maxPower, Power, charging; measurables I, Power, charging;measurables I, Power, charging; equalities charging = b1.charging = ba2.charging = ba3.charging;equalities charging = b1.charging = ba2.charging = ba3.charging;

process distributePowerCharging;process distributePowerCharging; conditions charging > 0;conditions charging > 0; equations ba1.Power = Power * ba1.maxPower / maxPower;equations ba1.Power = Power * ba1.maxPower / maxPower; ba2.Power = Power * ba2.maxPower / maxPower;ba2.Power = Power * ba2.maxPower / maxPower; ba3.Power = Power * ba3.maxPower / maxPower;ba3.Power = Power * ba3.maxPower / maxPower;

process totalMaxPower;process totalMaxPower; equations maxPower = ba1.maxPower + ba2.maxPower + ba3.maxPower; equations maxPower = ba1.maxPower + ba2.maxPower + ba3.maxPower;

process totalCurrent;process totalCurrent; equations i = ba1.i + ba2.i + ba3.I; equations i = ba1.i + ba2.i + ba3.I;

Partial Hierarchical Model of the Power GridPartial Hierarchical Model of the Power Grid

system ba1;system ba1; variables charging, maxPower, Power, Vcb, i, soc, Vt, Rs;variables charging, maxPower, Power, Vcb, i, soc, Vt, Rs; measurables charging, Power, Vcb, i, soc, Vt, Rs;measurables charging, Power, Vcb, i, soc, Vt, Rs; parameters Rp = 100, Rload = 2.6, Icharge = 12;parameters Rp = 100, Rload = 2.6, Icharge = 12;

process ChargeDischarge;process ChargeDischarge; equations d[soc,t,1] = 0.001 * (36.2 + 76.2 * soc) / Rp);equations d[soc,t,1] = 0.001 * (36.2 + 76.2 * soc) / Rp);

process FullCharge;process FullCharge; conditions soc < 0.96, charging > 0;conditions soc < 0.96, charging > 0; equations maxPower = Icharge * (VCb + Icharge * Rs);equations maxPower = Icharge * (VCb + Icharge * Rs);

process MaintainCharge;process MaintainCharge; conditions soc > 1.0, charging > 0;conditions soc > 1.0, charging > 0; equations maxPower = 1.1 * (36.2 + 76.2 * soc);equations maxPower = 1.1 * (36.2 + 76.2 * soc);

process VtCharge;process VtCharge; conditions charging > 0;conditions charging > 0; equations Vt = Vcb + I * Rs;equations Vt = Vcb + I * Rs;

Graphical Display of Model StructureGraphical Display of Model Structure

Quantitative Model-Based MonitoringQuantitative Model-Based Monitoring

quantitativequantitativeprocess modelprocess model

schedule of powerschedule of powergeneration/usagegeneration/usage

initial systeminitial systemconditionsconditions

predipredicted valuescted valuessimulationsimulation

oobserved valuesbserved values(telemetry)(telemetry)

anomalyanomalydetectiondetection

GUI with GUI with visvisual ual alertsalerts

To demonstrate our approach to model-based monitoring and get To demonstrate our approach to model-based monitoring and get initial feedback on our interface design, we:initial feedback on our interface design, we:

Evaluation of the ApproachEvaluation of the Approach

Our experience with these runs has suggested revisions to both Our experience with these runs has suggested revisions to both the monitoring method and the user interface.the monitoring method and the user interface.

• selected parts of the power grid for our initial study;selected parts of the power grid for our initial study;

• developed partial models at multiple levels of detail;developed partial models at multiple levels of detail;

• used mutated models to generate “observed” values; used mutated models to generate “observed” values;

• ran the monitoring system on these data to detect anomalies;ran the monitoring system on these data to detect anomalies;

• displayed detected faults in our graphical user interface.displayed detected faults in our graphical user interface.

Related Work on Filtering and MonitoringRelated Work on Filtering and Monitoring

Previous research on intelligent filtering and monitoring includes:Previous research on intelligent filtering and monitoring includes:

However, few of these efforts address issues in human-centered However, few of these efforts address issues in human-centered computing and information overload. computing and information overload.

• plan monitoringplan monitoring

- for military plans (e.g., Shapiro et al., 1985)- for military plans (e.g., Shapiro et al., 1985)

- for robotic plans (e.g., Washington et al., 1999)- for robotic plans (e.g., Washington et al., 1999)

• fault detectionfault detection

- in manufacturing systems (e.g., GenSym) - in manufacturing systems (e.g., GenSym)

- in space operations (e.g., Config, CRANS)- in space operations (e.g., Config, CRANS)

• activity monitoring activity monitoring

- for detecting fraud (e.g., Fawcett & Provost, 1997) - for detecting fraud (e.g., Fawcett & Provost, 1997)

- for detecting computer intrusion (e.g., Maloof, 1995)- for detecting computer intrusion (e.g., Maloof, 1995)

In future work on filtering temporal information, we plan to:In future work on filtering temporal information, we plan to:

Plans for Future ResearchPlans for Future Research

These extensions will involve integrating ideas from model-based These extensions will involve integrating ideas from model-based reasoning, HCI, machine learning, and intelligent simulation. reasoning, HCI, machine learning, and intelligent simulation.

• develop even more extensive models of the power grid;develop even more extensive models of the power grid;

• run models and monitoring method on more telemetry data;run models and monitoring method on more telemetry data;

• augment and improve the interface to serve users better;augment and improve the interface to serve users better;

• evaluate the resulting system on human test subjects; evaluate the resulting system on human test subjects;

• predict possible future faults through forward simulation;predict possible future faults through forward simulation;

• develop methods for handling missing values in data; develop methods for handling missing values in data;

• use telemetry data to further improve models via learning;use telemetry data to further improve models via learning;

• combine monitoring method with interactive scheduling. combine monitoring method with interactive scheduling.

Our research goal is to design, implement, and evaluate intelligent Our research goal is to design, implement, and evaluate intelligent assistants for this task.assistants for this task.

Humans often encounter domains involving many variables that Humans often encounter domains involving many variables that change rapidly over time.change rapidly over time.

• Given: Given: a domain with thousands of continuous variablesa domain with thousands of continuous variables;;

• Given: Given: values for these variables as a function of time;values for these variables as a function of time;

• Given: Given: knowledge about the domain and the user’s goals; knowledge about the domain and the user’s goals;

• Find: Find: events interesting to the user as they occurevents interesting to the user as they occur..

The Problem of Filtering Temporal DataThe Problem of Filtering Temporal Data

Lacking the ability to process all these data, they need aids that Lacking the ability to process all these data, they need aids that detect interesting events and filter out the rest. detect interesting events and filter out the rest.

We have selected this domain as our main testbed for research on We have selected this domain as our main testbed for research on intelligent filtering assistants. intelligent filtering assistants.

Staff at Mission Control monitor the state of the electrical power Staff at Mission Control monitor the state of the electrical power grid for the International Space Station. grid for the International Space Station.

• Given: Given: ~50,000 variables in the Space Station power grid~50,000 variables in the Space Station power grid;;

• Given: Given: observed values for these variables every ten seconds;observed values for these variables every ten seconds;

• Given: Given: schedules for usage/generation and expected effects; schedules for usage/generation and expected effects;

• Find: Find: significant divergences from expected values.significant divergences from expected values.

The Task of Power Grid MonitoringThe Task of Power Grid Monitoring

Clearly, they would benefit from computational aids that helped Clearly, they would benefit from computational aids that helped them detect anomalies in this complex system. them detect anomalies in this complex system.

Partial Process Model of the Power GridPartial Process Model of the Power Grid

process PowerGeneration;process PowerGeneration;variables TimeOnOrbit;variables TimeOnOrbit;conditions sin(TimeOnOrbit) < 0.5;conditions sin(TimeOnOrbit) < 0.5;equationsequations Supply = 20000;Supply = 20000;

process PowerDistribution;process PowerDistribution;variables Demand, Supply;variables Demand, Supply;equationsequations BatteryPower = Supply – Demand;BatteryPower = Supply – Demand;

process Charge;process Charge;variables BatteryPower, Q;variables BatteryPower, Q;conditions BatteryPower >= 0;conditions BatteryPower >= 0;# The battery saturates at full charge # The battery saturates at full charge equationsequations d[Q, t, 1] = (100000 – Q) * (1 – e^(– BatteryPower / (100000 – Q)));d[Q, t, 1] = (100000 – Q) * (1 – e^(– BatteryPower / (100000 – Q)));

Graphical Display of Model StructureGraphical Display of Model Structure