CSCI1600: Embedded and Real Time SoftwareLecture 28: Verification I

Steven Reiss, Fall 2015

Requirements

Requirements are central to embedded systems Failure might now be an option

Types of Requirements Safety: the car won’t crash

Timing: the solenoid will go on within 10ms of bumper switch

Fairness: as long as the heating system is on, the room will eventually be comfortable

Requirements: Goal

Show the system meets its requirements That the system is safe

That the system works correctly

Have confidence in the system’s safety

Show the system meets its requirements Under all possible circumstances

Helpful Techniques: SE Good requirements analysis

Understanding all the possible problems and solutions

Good specifications Accurate modeling

Showing the models are correct

Petri net and FSA validation

Design for security Defensive coding

Sanity checks

Helpful Techniques: Monitors

Add monitor tasks to the code These check that the system is operating correctly

Detect if something is going wrong

Too many solenoids turned on

Train moving onto a block containing another train

Lights have been on for > 10 ms

If something is detected (unusual or wrong) Move the system to a “safe” state

Helpful Techniques: Validation Testing

Use lots of test cases

Simulate the hardware and ensure system runs

Simulate potential failures and see what the system does

Multiple simultaneous failures as well as single ones

Run time assertion checking

Preconditions and postconditions Defensive coding

Sanity checking

But what do these tell you How sure can you be?

Verification: Basic Idea

Mathematically prove the program is correct

Show the program works for all possible runs Not just the ones covered by the test cases

All possible (and impossible) inputs

Show the timing works for all possible schedules

Verification: Issues

What does it mean to be correct If we are going to prove something, it has to be precise

How can you state precisely what the program should do?

Especially for a complex system

How can you model all possible runs? There are an infinite number of these (nonterminating)

Or a very large finite number

Verification: Issues

Programs are very complex Proving anything about a program is undoable

Almost anything you want to prove is undecidable

Halting problem

Actual not true since machines are finite, but effectively true

Even restricting values to finite ranges, etc.

Can involve huge numbers of program states and value sets

Verification: Issues Proofs are very complex

The mathematics involved is complex

Would you believe a 1000 page proof of a program

Suppose it is done by a computer

How long is the program doing the proof

Has it been shown to be correct

What about the compiler and the hardware?

Yet the work has to be done And techniques have been developed to do it

Verification: Basic Idea Steps

Model the program in a checkable representation

Prove properties of the program in that representation

Prove that the program satisfies the model

We’ve seen approximations before

Petri nets: only consider elements affecting synchronization and task coordination

Queueing theory: only consider tasks and queues

Program

Model Proof in Model

Proof in Program

Modeling the Program

Goal: Map the program into a finite state automata Why finite state automata?

It already is, but the number of states is very large

Reduce the number of states Restrict variable values to reduce the number of states

Restrict the number of threads, tasks, etc.

Show that the restriction doesn’t affect proof

Proving the Model is Correct

Provide a mapping from the code to the model Automatically generate the model from the code

Prove the code implements the model

Show that other aspects of the program don’t matter Consider all possible interpretations of those properties

Model Checking

Program

Model Proof in Model

Proof in Program

Model Checking: Properties First we need to be able to state the property

In a form that is understandable

In a form that can be checked

Represent the property to be proved in finite form As a finite automata

As properties over a finite automata

States in a finite automata are labeled with propositions

Then you want to state properties over those propositions

Taking time into account

Model Checking: Program Model the program as a generalized finite automata

Need to generalize to handle infinite programs

Automatically map code into model To ensure the mapping is correct

With some help from the programmer

For example, providing set of legal values for an int

The mapping has to be conservative

Cover all possible executions

Might allow “impossible” executions as well

Model Checking: Proofs

Show that all executions of the finite program Satisfy the property

Can be done in terms of executions

Can be done in terms of languages (symbolic math)

Defining Properties

Modularize correctness Rather than trying to show the whole program is correct

Break correctness down into smaller pieces

Define correctness in terms of specific program properties Prove each of these separately

This is often easier than trying to show everything at once

Example: HVAC System

My heating system 3 floors

Basement and 2nd floor share common water unit

Basement is radiant floor heating, 2nd floor is water->air

These have separate thermostats

Want to ensure that not both heating and cooling the water

How might you express this?

Example: HVAC System Think about the HVAC controller as a FSA

Transitions between states determined by events

Including timer events and random events (outside temp)

We can define different properties of interest

Heating On (H), Air Conditioning On (AC)

Build an FSA with states based on these properties

~H,~AC

H, ~AC

~H, AC

H, AC

Example How would you model the program

to check this

Look only at elements that affect turning on heat and air conditioning and their interaction

Assume the rest of the system can do whatever it wants

Arbitrary setting of thermostats, temperatures, etc.

Based on what program actually does

Example: HVAC System Other Examples

Show the heat is on if the room is too cool

Show that air conditioning is on if the room is too hot

Show neither is on if the room is comfortable

Show that air conditioning doesn’t come on within 5 minutes of its last being on

Show temperature will be near comfortable levels at a designated time

Are these going to be true? What would you really show

Example: Your Systems

How would you express your requirements

Example: Pinball

Show that you don’t blow fuses No more than 2 solenoids on at once

No more than 8 lights on at once

Show that each element scores correctly

Show ball logic is correct (number, free balls, …)

Next Time

We’ll look at formalizing this process

And how it can be made practical