Monitoring IVHM Systems using a Monitor-Oriented Programming Framework S. Ghoshal, S. Manimaran - QSI G. Rosu, T. Serbanuta, G. Stefanescu - UIUC

Monitoring IVHM Systems using a Monitor-Oriented Programming

Framework

S. Ghoshal, S. Manimaran - QSI

G. Rosu, T. Serbanuta, G. Stefanescu - UIUC

IVHM System Analysis

• IVHM systems pose significantly higher safety and dependability requirements than most other systems

• Formal analysis of IVHM systems is therefore highly desirable …

• … but also challenging, due to their highly integrated nature (different technologies, hardware, software, sensors, etc.) and combined complexity

Overview

• Our Approach

• MOP (University of Illinois at Urbana)

• TEAMS (Qualtech Systems Inc.)

• Project Research Plan

• Conclusion and Future Work

Our Approach to IVHM Analysis

• Separation of concerns

1. State “health” assessment, or diagnosis

2. Temporal behaviors of state sequences

Steering / Recovery

IVHM System

Model-basedobservation

Temporal behavior monitor

Violation /Validation

Abstract events/statesTEAMS

MOP

Overview

• Our Approach





Monitoring-Oriented Programming (MOP)http://fsl.cs.uiuc.edu/mop

- proposed in 2003 –RV’03, ICFEM’04, RV’05, CAV’05, TACAS’05,

CAV’06, CAV’07, OOPSLA’07, ICSE08, …

ERE LTL ptLTL ptCaRet

logic plugins

…

…JavaMOP

BusMOP

MOP CFG

lang

uage

s

What is MOP?

• Framework for reliable software development– Monitoring is basic design discipline

• … rather than “add on” grafted onto existing code

– Recovery allowed and encouraged– Provides to programmers and hides under the hood a

large body of formal methods knowledge/techniques• Monitor synthesis algorithms

– Generic for different languages and application domains

Example: Correct and efficient sorting

Heapsort

O(n log(n))

Monitor ifvector issorted

yes

O(n)

Insertionsort

no

O(n2)provablycorrect

Works in MOP

We have an efficient and provably correct sorting algorithm!We avoided proving heap sort correct, which is hard!

Need to show it does not

destroy the multiset

MOP Example: “Authentication before use”

Execution 1, correct

authenticate

begin end

use

begin end

Execution 2, incorrect

class Resource {/*@ class-scoped SafeUse() { event authenticate : after(exec(* authenticate())) event use : before(exec(* access())) ptltl : use -> <*> authenticate }@*/

void authenticate() {...}void access() {...}...}

MOP Example: “Authentication before use”

MOP Example: “Enforce authentication before use”

Execution 1, correct

authenticate

begin end

use

begin end

Execution 2, incorrect but corrected

Call authenticate()

class Resource {/*@ class-scoped SafeUse() { event authenticate : after(exec(* authenticate())) event use : before(exec(* access())) ptltl : use -> <*> authenticate violation { @this.authenticate(); } }@*/void authenticate() {...}void access() {...}...}

MOP Example: “Enforce authentication before use”

/*@ class-scoped SafeClose() { event openRegKey : after(exec(* openRegKey())) event closeHandle : before(exec(* closeHandle())) event closeRegKey : before(exec(* closeRegKey())) ere : any* openRegKey closeHandle validation { @this.closeRegKey(); return; } }@*/

Method openRegKey should be followed

by closeRegKey, not by closeHandle

MOP Example: “Correcting method matching”

/*@ class-scoped FileProfiling() { [ int count = 0; int writes = 0;] event open : after(call(* open(..))) {writes = 0;} event write : after(call(* write(..))) {writes ++;} event close : after(call(* close(..))) ere : (open write+ close)* violation { @RESET; } validation { count ++; File2.log(count + ": " + writes); } }@*/

MOP Example: ProfilingHow many times a file is open, written to, and then closed?

Fail Fast Iterators

Vector v = new Vector();v.add(new Integer(1));Iterator i = v.iterator();v.add(new Integer(2));

• Following code throws exception in Java:

• No exception raised if one uses Enumeration instead of Iterator– Java language decision, showing that properties

referring to sets of objects are important

MOP Example:Safe Enumeration

• Basic safety property:– If nextElement() invoked on an enumeration

object, then the corresponding collection (vector) is not allowed to change after the creation of the enumeration object

/*@global validation SafeEnum(Vector v, Enumeration+ e) { event create<v,e> : after(call(Enumeration+.new(v, ..))) returning e event updatesource<v> : after(call(* v.add*(..))) \/ … event next<e> : before(call(Object e.nextElement())) ere : create next* updatesource+ next)}@*/

MOP Example:Safe Enumeration

AspectJ code generated from the above: ~700 LOC

MOP Example:Safe Locking Policy

Each lock should be released as many times as it was acquired

/*@method-scoped SafeLock(Lock l) { event acquire<l> : before(call(* l.acquire())) event release<l> : before(call(* l.release()))

cfg : S -> epsilon | S acquire S release}@*/

MOP Example:Safe Locking

MOP Approach to Monitoring

Keep the following distinct and generic:• specification formalisms• event definitions• validation handlers

MOP Distinguished Features: Extensible logic framework

• Observation: no silver-bullet logic for specs• MOP logic plugins (the “How”): encapsulate

monitor synthesizers; so far we have plugins for– ERE (extended regular expressions), PtLTL (Past-time

LTL), FtLTL (Future-time LTL), ATL (Allen temporal logic), JML (Java modeling language), PtCaRet (Past-time Call/Return), CFG (Context-free grammars)

• Generic universal parameters– Allow monitor instances per groups of objects

MOP Distinguished Features: Configurable monitors

Working scope– Check point: check spec at defined place– Method: within a method call– Class: check spec everywhere during obj lifetime– Interface: check spec at boundaries of methods– Global: may refer to more than one object

Running mode– Inline: shares resources with application– Outline: communicates with application via sockets– Offline: generated monitor has random access to log

MOP Distinguished Features: Decentralized Monitoring/Indexing

• The problem: how to monitor a universally quantified specification efficiently!

create<v,e>udatesource<v>next<e>

create next* updatesource+ next

( v,e)

Decentralized Monitoring

Monitor instances(one per parameter instance)

Mp1

Mp2

Mp3

… Mp1000

Indexing …

• The problem: how can we retrieve all needed monitor instances efficiently?

Mp1

Mv,e1

Mv,e2

… Mp1000

udatesource<v>

Naïve implementation very inefficient (both time- and memory-wise)

MOP – Grigore Rosu 26

MOP’s Decentralized Indexing

• Monitors scattered all over the program

• Monitor states piggybacked to object states

• Weak references

SafeEnum events

create<v,e>udatesource<v>next<e>

MOP: Evaluation• More than 100 program-property pairs

– Dacapo benchmark, Tracematches benchmark, Eclipse, …

• Overhead < 8% in most cases; close to hand-optimized

4.52.21.71.52.80.06.54.75.46.74.43.5xalan

0.00.08.05.413.725.40.00.011.344.80.00.0pmd

0.10.00.61.10.00.30.00.00.00.00.00.5lusearch

1.11.72.23.20.00.31.81.20.51.90.21.6luindex

0.20.42.30.00.60.00.30.20.50.80.00.6jython

0.00.01.41.40.00.82.10.01.20.90.03.3hsqldb

0.00.01.00.51.50.80.00.00.01.50.61.2fop

2.42.23.13.01.53.80.53.71.40.02.84.1eclipse

0.00.00.00.50.00.04.83.60.00.00.00.0chart

0.05.80.00.00.00.41.10.00.00.01.50.0antlr

ClosedReaderLeakingSyncHasNextHashMapSafeIteratorSafeEnum

Overhead in % MOP monitors VS. hand-optimized monitors

MOP: Evaluation (cont.)

• Even significantly faster than logic specific solutions

N/A63.520.211.151.2KABCReweave

N/A124.323.921.2438.7KAproveHashSet

N/A15.23.33.39.9KWekaHashtable

N/A4522322101.4KCerRevSimNullTrack

708415091360.19.5KjHotDrawSafeEnum

21933546.6021.1KajHotDrawListener

PQLTracematchesMOPHand Optimize

d

LOCProgramProperty

Results for Tracematches benchmarks, Overhead in %

Overview

• Our Approach





QSI’s TEAMS

• Model-based diagnosis system– TEAMS model = dependency model capturing

relationships: failure modes observable effects

• QSI’s TEAMS Tool Set– TEAMS Designer: help create models– TEAMS-RT: processing data in real time– TEAMATE: infers health status + optimal tests– TEAMS-RDS: remote diagnostic server

TEAMS Designer

• Help users create models– (model can also be imported)– Capture component and data

dependency + other aspects that allow efficient diagnosis

• Model = hierarchical multi-layered directed graph– Node: physical component– Test-point: “observation” node– Edge: cause-effect dependency

Overview

• Our Approach





Project Objectives

1. Develop tools, techniques and ultimately an integrated framework for IVHM system monitoring, control and verification

2. Show that runtime verification and monitoring can play a crucial role in the development of safe, robust, reliable, scalable and operational IVHM systems

Project Plan

• TEAMS: capture system “health”

• MOP: generate and integrate monitors

• Integrated system: check IVHM system at runtime, steering if failures are detected

Steering / Recovery

IVHM System

Model-basedobservation

Temporal behavior monitor

Violation /Validation

Abstract events/statesTEAMS

MOP

What is done: TEAMS side Case study: B-737 Autoland

• With data provided by Celeste M. Belcastro and Kenneth Eure, a model for B-737 is being developed

What is done: MOP side Case study: B-737 Autoland

• Two new logic plugins– Context-free patterns– Past-time LTL with Calls/Returns– (still missing timed logic plugins)

• Improved monitor garbage collection– Current MOP more than an order of

magnitude faster than other RV systems

Overview

• Our Approach





Conclusion and Future Work

• Discussed initial steps towards integrated framework for IVHM system monitoring, control and verification– Separation of concerns

• Observation / diagnosis of system “health”• Monitoring of temporal behaviours

• A lot to be done– Complete TEAMS model for B-737 autoland– Automate integration of TEAMS and MOP

Documents

Monitoring IVHM Systems using a Monitor-Oriented Programming Framework S. Ghoshal, S. Manimaran - QSI G. Rosu, T. Serbanuta, G. Stefanescu - UIUC