Verification Case Studies with · PDF file– Limited computation ability and energy supply; – High concurrency requirements; – Diversities in designs and usages; – High reliability

Verification Case Studies with ObjectCheck

Fei Xie

(Joint work with James C. Browne, Robert P. Kurshan, and Vladimir Levin)

Presentation at Microsoft Research, April 10, 2003

2

Presentation Outline

• Overview and Architecture of ObjectCheck •• Modeling and Verification of a TinyOS Modeling and Verification of a TinyOS

RunRun--time Image with ObjectChecktime Image with ObjectCheck•• More Case StudiesMore Case Studies•• Summary and Future WorkSummary and Future Work• Work Built on ObjectCheck

3

ObjectCheck Overview

• An integrated development, validation, and mode-checking environment for software system designs;– System designs are specified in xUML;– xUML is an executable subset of UML;

• Developed in conjunction with – FormalCheck (Robert P. Kurshan, et. al.);– SDLCheck (Vladimir Levin and Husnu Yenigun);– Goal: Software/hardware co-design and co-verification;

• Sub-goal: Model checking of software system designs in xUML.

4

Executable UML (xUML)

• Has well-defined Execution Semantics;

• Utilizes UML Action Semantics recently adopted by OMG;

• Can be compiled to procedural codes;

• Tools provided by:– Project Technologies;– Kennedy Carter;– Hyperformix (SES);– …

5

Architecture and Workflow of ObjectCheck

Property Specification Interface xUML IDE Error Visualizer

xUML-to-S/R Translator

COSPAN Model Checker

S/R ModelS/R Query

Error Report

Error Track

Designer

xUML ModelProperty

Error Report Generator

6



RunRun--time Image with ObjectChecktime Image with ObjectCheck•• More Case StudiesMore Case Studies•• Summary and Future Work Summary and Future Work • Work Built on ObjectCheck

7

Case Study on TinyOS

• A run-time image of TinyOS, a component-based operating system for networked sensors;

• xUML models for TinyOS components have been constructed from their C source codes to enable:– Model-driven development on design level;– Software/hardware co-design and co-verification;

• Two properties are to be checked:– Repeated transmission on physical network;– No consecutive 0’s in any sequence-number sequence.

8

More on TinyOS

• An OS for networked sensors that have– Limited computation ability and energy supply;– High concurrency requirements;– Diversities in designs and usages;– High reliability requirement;

• Is component-based– Run-time images are specialized for different sensors;– Only necessary components are selected and composed.

• More information -- http://webs.cs.berkeley.edu/tos

http://webs.cs.berkeley.edu/tos

http://webs.cs.berkeley.edu/tos

9

Step-by-Step DemonstrationDesigner

Property Specification Interface xUML IDE Error Visualizer

Property xUML Model Error Report

xUML-to-S/R Translator Error Report Generator

S/R Query S/R Model Error Track


10

11

12

13

14

15

16

17

18

Step-by-Step DemonstrationDesigner

Property Specification Interface

Property

xUML IDE Error Visualizer

xUML Model Error Report

xUML-to-S/R Translator Error Report Generator



19

20

Step-by-Step Demonstration


Property


Designer






21

22

23

24



Property


Designer






25

26

27

28

29

30

31

32



Property


Designer






33

34

35


Property Specification Interface Error Visualizer

Property


Designer

xUML IDE





36

37

38

39

40

41

42

43

Statistics from Model Checking “Repeated Output” Property

• Four model checking runs with different combinations of reduction algorithms – Start at the same time on the same host;– The host is a SUN with 8 CPUs and 2GB memory.

1384S80MOnOff

1379S102MOnOn17438S596MOffOn

19370S596MOffOffTime UsageMemory UsageSMCSPOR

Conclusion: SMC helps and SPOR doesn’t.

44




45

Model Checking of NASA robot Controller

• A typical control-intensive embedded system;• Conducted by Natasha Shyrigina using ObjectCheck;

– 37 properties were checked.– 22 properties were successfully checked.– 6 bugs were found.

consists of

defines position of

specifies

is a part of

R15

GlobalRepresentation

. number_of_joints

. type of search

. abort_var….

R1

R11

Follows to thespecified

R2

R5

R4

Kinematics

InverseKinematics

OSCAR Library

R7

R6

Trajectory (TJ)* TJ_ID. position. final_position. EE_ID(R3)

TrajectoryPoint (TP)*TP_ID. TJ_ID (R4). point

R3

R9

R10

has

can havedifferent

can have anumber of

Is A

is a component of

requires

OSCAR Libraries* OS_ID. TC_ID (R10). status* link-ID

R8

ForwardKinematics

Arm (A)* Arm_ID. arm_status

is configured by

End-Effector (EF)* EE_ID. Current position. Limit. status. end_position. ee_reference

JointConfiguration (JC)*JC_ID. Joint_ID(R7). direction. trial_angle. type. NoinSet. TS_counter. Status

Joint (J)* Joint ID. current_angle. limit_Max.. limit_Min. EE_ID (R2). reference. Arm_ID (R1) . DT_ID (R11). JointStatus

is interfaced with

TrialConfiguration (TC)*TC_ID. DT_ID(R8). configuration. validSolutions. SS_ID(R13). status. LockConfiguraion. TP_ID(R5). JC_ID(R6)

Is evaluated byDecision Tree

Joint motion iscontrolled byDecision Tree

R13Is a componentof Search Space

R14

Checker (Ch)* Ch_ID. counter. recovery_status. abort_var

is checked by

R16

R15

Is A

R19

Performance Criterion (PC)

* PC ID . average value . status . FC_ID (R16) . scale . PC_name .TS_ID (R15)

Kinetic energy distortion Criterion

* PC ID (R19) *Criteria name .criterion value

System Compliance Criterion

* PC ID (R19) *Criteria name .criterion value

Inertia Criterion * PC ID (R19) *Criteria name .criterion value

Geometric Criterion* PC ID (R19) *Criteria name .criterion value

Constraint Criterion * PC ID (R19) * Criteria name . Error . criterion value

R17

Performance Monitoring

Decision Making

DecisionTree (DT)*DT_ID . optimal_solution . status R12

R13

R18

has

has

has

creates

Search Space (SS) * SS_ID . SS_size . DT ID (R12)

FactorialSearchSpace (FSS)

* FSS_ID . SS_ID (R17) . condition

SimpleSearchSpace(SSS)* SSS_ID . SS_ID (R18) . condition

Fused Criterion (FC)* FC ID . PI . TS_ID (R14)

Consists of a number of trial configurations

Evaluates a trial configuration

R11Controls motion of each joint

R8

Is used for evaluation of a trial configuration

NASA Robot Controller

Properties to be Model checked

48

Model Checking of Online Ticket Sale System

• A typical commercial transaction system;• Presented at FASE 2002;• Focus: Integrated state space reduction.

49

An Online Ticket Sale System (Class Diagram)

50

An Online Ticket Sale System (A State Model)

51

Some Verification Statistics of Online Ticket Sale System

• Verification of a liveness property– After an agent is assigned to a customer,

eventually the agent will be released.• Statistics related to state space reductions

1450.3S74.0MOnOn6668.3S17.3MOffOn44736.S113.73MOnOff

-Out of MemoryOffOffTime UsageMemory UsageSMCSPOR

52




53

Summary and Future Work

• ObjectCheck– Integrates industrial software development environments

and model checkers with research tools;– Provides comprehensive automation for development,

validation, and model checking of xUML models;– Has enabled verification of non-trivial software system

designs modeled in xUML.

• Future work is focused on– State space reduction capability of ObjectCheck;– Hardware/software co-design and co-verification.

54

Related Work

• Most closely related work– UML Model Checking toolset from University of

Michigan;– vUML tool from Åbo Akademi University;

• There is also related work on model checking of statecharts with different semantics.

55

Additional Information

• http: //www.cs.utexas.edu/users/ObjectCheck• Selected publications:

– Fei Xie and James C. Browne. Verified Systems by Composition from Verified Components. Submitted for review.

– Fei Xie, James C. Browne, and Robert P. Kurshan. Translation-based Compositional Reasoning for Software Systems. Submitted for review.

– Fei Xie and James C. Browne. Integrated State Space Reduction for Model Checking Executable Object-oriented Software System Designs. In Proc. of FASE 2002.

– Fei Xie, Vladimir Levin, and James C. Browne. ObjectCheck: A Model Checking Tool for Executable Object-oriented Software System Designs. In Proc. of FASE 2002.

– Fei Xie, Vladimir Levin, and James C. Browne. Model Checking for an Executable Subset of UML. In Proc. of ASE, 2001.

56




57

Work Built on ObjectCheck

• An integrated state space reduction framework;• Integration of model checking into component-

based development of software.

58

Integrated State Space Reduction Framework

xUML-to-S/R Translation

S/R Model S/R Level Query

Model Checking with COSPAN

Success Report / Error TrackBasic Model Checking Process

Symbolic VerificationLocalization Reduction

Partial Order Reduction

xUML Model xUML Level Query

ReducedxUML Model

Reduced xUML Level Query

User-Driven State Space Reduction

Verification Task

Verification Subtasks

DecompositionAbstractionSymmetry Reduction

59

Reduction Steps for Checking P0

Customers, DispatcherAgents, Ticket Sever

Step 1: Symmetry Reduction

Step 2: Decomposition

Step 3: Symmetry Reduction

Step 4: Decomposition

Step 5: Case SplittingStep 6: Symmetry Reduction

Customers, DispatcherAgents, Ticket Sever

P1

Customers Dispatcher Agents, Ticket ServerP21 , P22 P31 , P32

P33 , P23

Agents, Ticket ServerP41 , P42 P43 , P44

Ticket ServerAgentsP41 , P42 P43 , P44

P5

Ticket ServerP6AgentP41 , P42 P43 , P44

P0

60

Evaluation of User-driven State Space Reduction

• Directly model checking P0 on OTSS – Two customer instances and two agent instances;– SPOR and SMC are both applied.– Memory usage: 152.79M– Time usage: 16273.7S

• Memory and time usages for discharging subtasks at the leaf nodes of the reduction tree.

0.63S0.04S0.01S0.04S0.01S1.81S0.02STime

0.35M0.29M0.28M0.29M0.28M0.95M0.30MMemory

P6P44P43P42P41P22P21

61

Integration of Model Checking into Component-based Development

• Temporal properties of a component are – Established with assumptions on the environment of the

component; – Verified under these assumptions and then packaged with

the component.• Selecting a component for reuse considers not only

its functionality but also its temporal properties. • Properties of a composed component are verified

by reusing verified properties of its sub-components and applying compositional reasoning.

62

Sensor Component

63

Properties of Sensor ComponentProperties:

Repeatedly (Output);After (Output) Never (Output) UntilAfter (OP_Ack);After (Done) Eventually (Done_Ack);Never (Done_Ack) UntilAfter (Done);After (Done_Ack) Never (Done_Ack) UntilAfter(Done);

Assumptions:After (Output) Eventually (OP_Ack);Never (OP_Ack) UntilAfter (Output);After (OP_Ack) Never (OP_Ack) UntilAfter (Output);After (Done) Never (Done) UntilAfter (Done_Ack);Repeatedly (C_Intr);After (C_Intr) Never (C_Intr + A_Intr + S_Schd) UntilAfter (C_Ret);After (ADC.Pending) Eventually (A_Intr);After (A_Intr) Never (C_Intr + A_Intr + S_Schd) UntilAfter (A_Ret);After (STQ.Empty = FALSE) Eventually (S_Schd);After (S_Schd) Never (C_Intr + A_Intr + S_Schd) UntilAfter (S_Ret);

Documents

Verification Case Studies with · PDF file– Limited computation ability and energy supply; – High concurrency requirements; – Diversities in designs and usages; – High reliability