MaCS: Monitoring, Checking and Steering
O. Sokolsky, S. Kannan, I. Lee, U. Sammapun,
J. Shin, M. Viswanathan
CIS, Penn
M. Kim
SECUi.com, Korea
Motivation
Two traditional approaches to certify correctness of systems Testing cannot guarantee correctness of
application completely and lacks of formality. Formal verification lacks scalability and does not
apply to implementation directly gap between models and implementations
We need a new approach - run-time formal analysis
Run-time Formal Analysis Run-time formal analysis ensures the run-time
compliance of an execution of a system with its formal requirement.
The analysis validates properties on the current execution of application.
The analysis can detect incorrect execution of applications predict error and steer computation collect statistics of actual execution
Run-time Formal Analysis using MaCS
To detect incorrect execution, we need Monitor: monitors changes in an
application by instrumenting monitor code into an application. This will create an abstract view of the application.
Checker: uses monitor information and requirement spec to detect violation
To steer, we need The violation information from Checker and
the Feed-back to the application
System SpecSystem
Spec
RequirementSpec
RequirementSpec
Formal verification
Design
SystemImplementation
SystemImplementation
MonitoringData
MonitoringData
Implementation
Monitor +CheckerMonitor +Checker
SystemSystem FilterCommunication
Run-time Check
MaCS Methodology
Issues in Run-time Formal Analysis
Side effect of instrumentation to a target system
An expressive formal language describing correctness criteria
A proper granularity of monitoring
Automatic v.s. Manual instrumentation
Synchronous v.s. Asynchronous monitoring
MaCS languages
PEDL: abstraction
Run-time state:•control locations•object state•local variables
Abstract state:•events•conditions•auxiliary variables
MEDL
PEDL
SADL
MEDL: abstract transformation SADL: feedback
Events and Conditions Information about the system
comes in two different forms: Conditions, which are true, false, or
undefined for a finite duration of time
Events, which are either present or absent at some instant of time
Examples of Events and Conditions
C3
C3. condition safeVolts = ( -10 < IP.volts ) && ( IP.volts < 10 );
IP.volts = 6
IP.volts = 15
E1 E2 E2
IP.init()
E1. event startProg = startM( IP.init() );E2. event voltsChange= update( IP.volts);
Logic for Events and Conditions
Let c be a primitive condition Let e be a primitive event Define Condition (C) and Event (E) as:
<C> ::= c | [ <E>, <E> ) | ! <C> | <C> && <C> | <C> || <C> | <C> => <C> <E> ::= e | start( <C> ) | end( <C>) | <E> && <E> | <E> || <E> | <E> when <C>
Conditions
<C> ::=
[ <E1>, <E2> )
! <C><C1> && <C2>
<C1> || <C2>
<C1> => <C2>
c |
|
|
|
|
Primitive condition
T from E1 until E2
T if C is F
T if C1 and C2 are T
T if C1 or C2 are T
T if C2 is T whenever C1 is T
Events
<E> ::=
start( <C> )
<E1> && <E2>
<E1> || <E2>
<E> when <C>
e |
|
|
|
|
Primitive event
An instant when C becomes T
An instant when C becomes F
An instant when both events happen
An instant when either events
happenAn instant when E
happens and C holds T
end( <C> )
The language that creates an abstract view of the application
It maps the low-level state information of the system to high-level events/conditions used in describing the requirements.
Primitive Event Definition Language (PEDL)
Primitive Event Definition Language (PEDL)
x=0,y=0
x=1,y=0
x=2,y=0
x=2,y=1
ProgramExecution
Abstract ViewMonitor Sees
x=3,y=1
x < 2
x =2
x> 2
x=3,y=2
InformationExtraction
Primitive Event Definition Language (PEDL)
Provides primitives to refer to values of variables and to certain points in the execution of the program. condition safeVolts =
( -10 < IP.volts ) && ( IP.volts < 10 ); event startProg = startM( IP.init() ); event voltsChange= update( IP.volts);
Provides primitive “time” to refer to time when events happen condition slowDoor =
( time(doorClose) - time(doorOpen) ) > 3000;
Meta Event Definition Language (MEDL)
Expresses requirements using the events and conditions, sent by a monitor. Defines Auxiliary Variables, which may be
used to store history. RaiseGateEvent -> { num_train_pass++; }
Defines more events and conditions using auxiliary variables and events/conditions sent by a monitor.
event safeStart = startProg when (safeVolts)
Meta Event Definition Language (MEDL) (Cont. )
Using auxiliary variables, events and conditions, we describes the safety requirements of the system, in terms of conditions that must always be true, and alarms (events) that must never be raised. safeprop safeIP = safeTrack && safeVolts; alarm violation = start (!safeIP);
Specifies updates to variables and steering invocations in response to events violation -> { controller = 0;
invoke change2SafetyController(); }
Property checking
A MEDL specification can also be seen as an automaton running on a stream of events provided by the monitor and the auxiliary variable
aux. variables
Steering Action Definition Language(SADL)
What: Identifies object used in steering// the target of steering is the object dm of type
// DecisionModule located in the class IP
DecisionModule IP:dm;
How: Defines steering actions// setSC() method of dm is invoked
steering action change2SC = { call (IP:dm).setSC(); }
Where: Specifies steering conditions locations in the code where the actions can be
executedbefore read DecisionModule:volts;
Steering process
system
checker
violation
action invoked
action executed
detection
steeringconditionsatisfied
actioninvocationreceived
event received
MaCS Toolset
Program (.class)(Java byte code)
Monitoring Script(PEDL)
Requirements(MEDL)
PEDLPEDLCompilerCompiler
MEDLMEDLCompilerCompiler
Instrumented Code
Filter Generator(JTREK)
InstrumentationInformation
Compiled PEDL
Compiled MEDL
Event Recognizer Checker
Steering Script(SADL)
SADLSADLCompilerCompiler
InstrumentationInformation
Injector class(Java byte code)
Filter (or an instrumented class)
Initializes the connection between application and monitor.
For each monitored object: for each variable update or start/end method,
send events/conditions to the monitor.
Filter (or an instrumented class)
IP.init()
Instrumented class
IP.volts = 6
IP.volts = 15
send to monitor
Implementation of monitor
EventReconizer class: Listener (i.e. server)
accepts events and conditions, specified in PEDL, from filter.
Notify (i.e. client) sends events and conditions to runtime checker
Implementation of Monitor
IP.init()
Instrumented class Monitor
IP.volts = 6
IP.volts = 15
startProg
safeVolts = T
voltsChange
voltsChange
safeVolts = F
send to checker
Implementation of checker
RuntimeChecker class: Listener (i.e. server)
accepts events and conditions, specified in MEDL, from monitor.
Check for any violation If violation occurs, check if any action needs to
be invoked by Steering Notify (i.e. client)
sends action to be invoked to steerer.
Implementation of Checker
IP.init()
Instrumented class Monitor
IP.volts = 6
IP.volts = 15
startProg
safeVolts = T
voltsChange
voltsChange
safeVolts = F
startProg
alarm violation = voltsChange when !
safeVolts
ALARM
OK
Checker
send to steering component
Implementation of steering
Injector class: Action bodies as methods Listener thread
accepts action invocations and sets invocation flags Calls to action bodies at fixed locations
defined in the steering script, added by instrumentation
Implementation of Checker
IP.init()
Instrumented class Monitor
IP.volts = 6
IP.volts = 15
startProg
safeVolts = T
voltsChange
voltsChange
safeVolts = F
startProg
alarm violation = voltsChange when !
safeVolts
ALARM
OK
Checker Injector
Action change2SC
IP.setSC()
Implementation Issues
How close should the application and the monitor be? Tight coupled
Pro: Decrease Steering Latency Con: Compete for resources
Implementation Issues
Conservativeness How to ensure that the application
does not suffer a significant performance degradation as long as steering is not invoked?
Implementation Issues
Effectiveness How to ensure that the steering
action cannot be ignored by the application?
How to ensure that after the action is invoked, the application will not “undo” the action?
Simplex Architecture (courtesy L. Sha)
Safety
Experimental
DecisionModule
PhysicalSystem
us
ue
xu
SC
EXx0
Equilibriumstate
Experimental controllers provide improved performance but uncertain stability properties Can be dynamically added or replaced
Safety controller has the largest stability region
ExperimentalController
ExperimentalController
Inverted Pendulum in MaCS
DeviceDrivers
angle,track
volts
Decision Module
ExperimentalController
Switching logic
SafetyController
JNI
monitor
steer
Case study results
It works! Without disturbance, there is slightly
more jitter partly due to instrumentation but mostly due to
Java
Switch to the safety controller takes 0.32s (16 control cycles) bad values compensated by the safety controller in general may not be good enough
Future Work
Synchronous vs. asynchronous steering currently: the application and may produce
erroneous outputs when checking is slow May be unacceptable in critical situations
alternative: before producing critical output, pause and wait for checker feedback
IP case study suggests it is possible in some cases Need predictable checker turnaround
Currently work in progress
Future Work
Static analysis to enhance steering currently, user is responsible for specifying
steering locations – error prone control flow analysis can identify locations by
code-independent criteria e.g. “steering location is visited once in each
main loop iteration” or even “if action changes variable x, do not
place before the system writes x” What are the right criteria?
Study other uses of steering