Lecture 1: Specifying and verifying state-transition models

General Introduction Kripke Models CTL ATL Model checking

Social Laws for Multi-Agent Systems: Logic and Games

Lecture 1: Specifying and verifyingstate-transition models

Thomas Ågotnes1

1Department of Information Science and Media StudiesUniversity of Bergen, Norway

NII Tokyo 13 December 2011

Contents

1 General Introduction

2 Kripke Models

5 Model checking

Lecturer

Professor Thomas ÅgotnesUniversity of Bergen, Norway

14500 students, 3200 faculty and staff

Visiting NII 3 December 2011 – 4 February 2012Contact: thomas.agotnes@infomedia.uib.no

About the lecture series

Goals:To motivate and introduce the idea of social laws , ornormative systems, as a coordination mechanism formulti-agent systemsTo motivate the idea of using logic in this context, and inparticular temporal logics and cooperation logics in theformal specification and verifaction of social laws

A social law is a restriction on the behaviour of individualagents, which ensures some desired global properties of thebehaviour of the system

About the lecture series

1 Specifying and veryfying state-transition models formulti-agent systems

2 Social laws for coordination3 Dealing with non-compliance4 Coordinating self-interested agents5 Social laws design as an optiomisation problem, and as an

optimisation problem6 Reasoning about social laws7 Strategic reasoning under imperfect information

Contents

2 Kripke Models

5 Model checking

Multi-agent systems

Consists of several autonomous agents that interactAn agent is an entity that perceives the enviroment, andactsAn agent can be an (artificially) intelligent agent programBut can also be a simple component of some system, likea thermostate

Formal models of multi-agent systems

State-transition modelsQuite common abstraction

Ex.: UML

Example: microwave oven

b startederror c

closedd closed

startedclosederror

startedclosed

startedclosedwarm

Computations

Every path in the tree obtained from a given initial staterepresents a possible computation

Computations: example

closed

started, closed

started, closed, warm

closed, warm

closed closed, warm

started, error

started, closed,error

started, error closed

Adding agents

If the system is a multi-agent system, the states are globalstatesWe label the transitions with the name of the agent thatcauses the transition by executing some actionThis assumes asynchronous action

Formally

An agent-labelled Kripke structure (over Φ) is a 6-tuple:

K = 〈S,S0,R,Ag, α,V 〉, where

S is a finite, non-empty set of states,S0 ⊆ S (S0 6= ∅) is the set of initial states;R ⊆ S × S is a total (each state has a successor) binarytransition relation on S;Ag = {1, . . . ,n} is the set of agents;α : R → Ag labels each transition in R with an agentV : S → 2Φ labels states with a set of propositional atoms

Note: simplifying assumption: single agent execute singleaction in each state (interleaved concurrency)

Contents

2 Kripke Models

5 Model checking

Introduction

Consider this statement:The microwave can only start without error if it isclosed

Is it true in our microwave model?This statement is true in some models (including ours), butfalse in othersWe want to be able to check whether such properties holdor not automaticallyThus we need a precise way of writing down statementsFor that we can use modal logic, and in particularComputation Tree Logic (CTL)

CTL: language

The language of CTL (CTL formulas) is defined as follows:Propositional atoms such as p or started are formulasFormulas can be combined using propositional connectivessuch as ∧ (and), ∨ (or), ¬ (not),→ (implication), etc.We can construct new formulas by putting temporalconnectives in front of an existing formula. If ϕ and ψ areformulas, then the following as also formulas:

E iϕ on some path, ϕ is true nextE(ϕU ψ) on some path, ϕ until ψE♦ϕ on some path, eventually ϕE ϕ on some path, always ϕA iϕ on all paths, ϕ is true nextA(ϕU ψ) on all paths, ϕ until ψA♦ϕ on all paths, eventually ϕA ϕ on all paths, always ϕ

CTL: language

The language of CTL (CTL formulas) is defined as follows:Propositional atoms such as p or started are formulasFormulas can be combined using propositional connectivessuch as ∧ (and), ∨ (or), ¬ (not),→ (implication), etc.We can construct new formulas by putting temporalconnectives in front of an existing formula. If ϕ and ψ areformulas, then the following as also formulas:

E iϕ on some path, ϕ is true nextE(ϕU ψ) on some path, ϕ until ψE♦ϕ on some path, eventually ϕE ϕ on some path, always ϕA iϕ on all paths, ϕ is true nextA(ϕU ψ) on all paths, ϕ until ψA♦ϕ on all paths, eventually ϕA ϕ on all paths, always ϕ

Examples

E ierror (it is possible that there is an error in the nextstate)¬A ierror (it is not necessary that there is an error in thenext state)E♦warm (it is possible that the oven will eventually bewarm)A ¬(warm ∧ error) (it is necessary that the microwavecan never be both warm and have an error)A♦closed (it is necessary that the microwave willeventually be closed)A(¬warmU closed) (it is necessary that the microwave iscold until it is closed)A (¬closed → ¬E i(start ∧ ¬error)) (the microwavecan only start without error if it is closed)

Examples

A (A♦enabled) (the process is infinitely often enabled)A (A♦deadlock) (the process will eventually be in apermanent deadlock)A (E♦restart) (it is always possible to get to therestart-state)A ¬(c1 ∧ c2) (safety)A (t1 → ♦c1) (liveness)

Examples

Formal interpretation

Given a Kripke model K , a state s in K and a CTL formula ϕ,

K , s |= ϕ

means that ϕ is true (or satisfied) in state s of K .

E mE i= for some path, in the next state

Example: E ma

closed

started, closed

closed, warm

closed closed, warm

started, error

K , s |= E ierror

A mA i= for all paths, in the next state

Example: A ma

closed

started, closed

closed, warm

closed closed, warm

started, error

¬warm¬warm

K , s |= A i¬warm

E♦ = for some path, in some future state

Example: E♦

closed

started, closed

closed, warm

closed closed, warm

started, error

K , s |= E♦warm

A♦ = for all paths, in some future state

Example: A♦

closed

started, closed

closed, warm

closed closed, warm

started, error

K , s |= A♦closed

E = for some path, in all future states

Example: E

closed

started, closed

closed, warm

closed closed, warm

started, error

¬warm

K , s |= E ¬warm

A = for all paths, in all future states

Example: A

closed

started, closed

closed, warm

closed closed, warm

started, error

K , s |= A ¬(warm ∧ error)

AU = for all paths, ψ becomes true in some future states and ϕis true in all states before that

Example: U

closed

started, closed

closed, warm

closed closed, warm

started, error

¬warm

K , s |= A(¬warm U closed)

Example: nesting

closed

started, closed

closed, warm

closed closed, warm

started, error

EO (closed ∧ error)

K , s |= E iE i(closed ∧ error)

Notation

We useK |= ϕ

to denote the fact that K , s |= ϕ for all initial states s ∈ S0.

Some equivalences

ϕ ∨ ψ ≡ ¬(¬ϕ ∧ ¬ψ)

A iϕ ≡ ¬E i¬ϕE♦ ≡ E(> U ϕ)

A ϕ ≡ ¬E♦¬ϕA(ϕU ψ) ≡ ¬( E(¬ψ U ¬(ϕ ∨ ψ)) ∨ E ¬ψ )

That means that we only need the operators{>,¬,∧,E ,EU ,E i}!

Some equivalences

ϕ ∨ ψ ≡ ¬(¬ϕ ∧ ¬ψ)

A iϕ ≡ ¬E i¬ϕE♦ ≡ E(> U ϕ)

Some equivalences

ϕ ∨ ψ ≡ ¬(¬ϕ ∧ ¬ψ)

A iϕ ≡ ¬E i¬ϕE♦ ≡ E(> U ϕ)

Some equivalences

ϕ ∨ ψ ≡ ¬(¬ϕ ∧ ¬ψ)

A iϕ ≡ ¬E i¬ϕE♦ ≡ E(> U ϕ)

Some equivalences

ϕ ∨ ψ ≡ ¬(¬ϕ ∧ ¬ψ)

A iϕ ≡ ¬E i¬ϕE♦ ≡ E(> U ϕ)

Some equivalences

ϕ ∨ ψ ≡ ¬(¬ϕ ∧ ¬ψ)

A iϕ ≡ ¬E i¬ϕE♦ ≡ E(> U ϕ)

Contents

2 Kripke Models

5 Model checking

Alternating-time Temporal Logic (ATL) is an agentizedextension of CTL introduced by Alur and colleagues (1997)The language of ATL is obtained by replacing A and E with〈〈C〉〉 where C ⊆ Ag and Ag is the finite set of all agents inthe systemIntuitively,

〈〈C〉〉♦ϕmeans that

C can cooperate to ensure that ϕ becomes true sometimein the future no matter what the other agents do (andsimilarly for h, , U )C has a strategy to enforce that ϕ becomes true sometimein the futureIs used to reason about game-like distributed systems

Example

〈〈merkel , sarkozy〉〉♦¬crisis

Merkel and Sarkozy can cooperate to ensure that at some pointin the future the crisis is over

Example

〈〈Ann〉〉〈〈Bob〉〉♦win

ATL and CTL

CTL is contained in ATL:A ≡ 〈〈∅〉〉E ≡ 〈〈Ag〉〉

Definition (ATL models)

A concurrent game structure is a tuple M = 〈Ag,S, π,Act ,d ,o〉,where:

Ag: a finite set of all agentsS: a set of statesπ: a valuation of propositionsAct : a finite set of (atomic) actionsd : Ag × S → ℘(Act) defines actions available to an agentin a stateo: a deterministic transition function that assigns outcomestates q′ = o(q, α1, . . . , αk ) to states and tuples of actions

A strategy for agent i is a function f : S → Act such that

f (s) ∈ d(i , s)

Example: Robots and Carriage

pos1pos2

wait,wait

wait,wait wait,wait

push,push

push,push push,push

push,wait

wait,push

wait,pushw

wait,wait

wait,wait wait,wait

push,push

push,push push,push

push,wait

wait,push

push,wait

wait,push

pos0 → 〈〈1〉〉 ¬pos1

wait,wait

wait,wait wait,wait

push,push

push,push push,push

push,waitpush,w

aitwait,push

wait,pushw

pos0 → 〈〈1〉〉 ¬pos1

wait,wait

wait,wait wait,wait

push,push

push,push push,push

push,wait

wait,push

push,wait

wait,push

waitpush

pos0 → 〈〈1〉〉 ¬pos1

wait,waitwait

wait,wait wait,waitwait

push,push

push,push push,push

push,wait

wait,push

push,wait

wait,push

pos0 → 〈〈1〉〉 ¬pos1

wait,waitwait

wait,wait wait,waitwait

push,push

push,push push,push

push,wait

wait,push

push,wait

wait,push

pos0 → 〈〈1〉〉 ¬pos1

wait,waitwait,wait

wait,wait wait,waitwait,wait

push,push

push,pushpush,push push,push

push,wait

wait,push

push,wait

wait,push

pos0 → 〈〈1〉〉 ¬pos1

q2 q1q1

pos1pos1

wait,waitwait,wait

wait,wait wait,waitwait,wait

push,push

push,pushpush,push push,push

push,wait

wait,push

push,wait

wait,push

pos0 → 〈〈1〉〉 ¬pos1

Contents

2 Kripke Models

5 Model checking

Verification

Verification means to make sure that the design of a(hardware or software) system is correctAriane 5: the world’s most expensive software bug(exploded 37 seconds after takeoff)

Formal specification and verification

Traditional verification methods such as simulation ortesting are not exhaustive, they don’t explore all possiblebehaviours of the systemFormal verification methods do, and they can thereforegive a guarantee that the design does not have any errorsWe specify the properties we want to check that thesystem has as a formula in some formal logic

Example: A (t1 → A♦c1) (liveness)Two main techniques:

Proof-based: describe also the system using formal logic,and try to find a formal proof that the property followsModel-based: describe the system using a mathematicalstructure, and use an algorithm to check whether theproperty holds

Reasoning: satisfiability

The satisfiability problem is as follows:Given a formula ϕ is there some interpretation that makesϕ true?How hard is the satisfiability problem?

For Coalition Logic: PSPACE-complete (Pauly, 2001).For CTL: EXPTIME-complete – a lower bound for ATL.

Reasoning with ATL: Satisfiability

Theorem (van Drimmelen, 2003)For any fixed set of agents Ag, satisfiability for ATL formulaeover Ag is EXPTIME-complete.

But if the set of agents is not fixed, van Drimmelen’s algorithmis 2EXPTIME.

Theorem (Walther, Lutz, Wolter, Wooldridge, 2006)

The satisfiability problem for arbitrary ATL formulae isEXPTIME-complete (and hence no harder than CTL).

Model checking

Successfull model-based verification techniqueInput: K , ϕOutput: yes if K |= ϕ

Effective algorithms exist

Model checking

Modelchecker

A□ (A◇ enabled)

Input: model and formula Output: "no" and a counterexample(sequence of states where theproperty does not hold)

Model checking: complexity

Model checking CTL and ATL can be done in time polynomial inthe size of the formula and the number of states in the model

... if the model is explicitly represented

Model checking: complexity

Model checking CTL and ATL can be done in time polynomial inthe size of the formula and the number of states in the model

... if the model is explicitly represented

Explicit representation

Explicit representation is often not feasibleNumber of states increase exponentially with the numberof independent variables

v1=0v2=0

v1=1v2=0

v1=0v2=1

v1=1v2=1

2 independent variables number of variables

number of states

Model representation languages

Practical model checkers use high-level modelspecification languages.Reactive modules: a rule-based language for modelspecification. . .

module toggle controls xinit[]> -> x ′ := >[]> -> x ′ := ⊥update[]x-> x ′ := ⊥[](¬x)->x ′ := >

Practical model checking

The complexity of model checking depends on the specificationlanguage and the representation of the model.

Theorem (van der Hoek, Lomuscio, Wooldridge, AAMAS06)For Reactive Modules models, model checking is exactly ashard as theorem proving in the corresponding language:

ATL EXPTIME-completeCoalition Logic PSPACE-completeprop logic co-NP-complete

Lecture 1: Specifying and verifying state-transition models

Documents

Specifying and verifying a transformer station in Signal and … · 2016-12-27 · Herv e Marchand, Eric Rutten, Mazen Samaan To cite this version: Herv e Marchand, Eric Rutten, Mazen

Specifying and verifying programs in Spec# K. Rustan M. Leino Microsoft Research, Redmond, WA, USA Invited talk, PSI 2006 Novosibirsk, Russia 27 June 2006

Dijkstra Monads for All - arXiv › pdf › 1903.01237.pdf1 INTRODUCTION The aim of this paper is to provide a semantic framework for specifying and verifying programs with arbitrary

Verifying Funds

Formal Methods for Specifying, Validating, and Verifying … · Key Words: Formal Methods, Requirements, SCR Toolset Category: D.2.4, F.3.1 1 Introduction During the past three decades,

Specifying and Verifying Properties of Space

Specifying and Verifying Program Transformations with PTRANS

1 Specifying and Verifying Hardware Support for Copy and Tamper-Resistant Software David Lie, John Mitchell, Chandramohan Thekkath and Mark Horowitz Computer

Specifying stone

On Library Correctness under Weak Memory Consistency · 2018-12-15 · 68 On Library Correctness under Weak Memory Consistency Specifying and Verifying Concurrent Libraries under

TTM/PAT: A Tool for Modelling and Verifying Timed ... · TTM/PAT: a Tool for Modelling and Verifying Timed Transition Models Jonathan Ostro , Chen-Wei Wang, Simon Hudon EECS, York

WELL BEHAVED TRANSITION SYSTEMSinfo.usherbrooke.ca/mblondin/papers/BFM17.pdf · 2017. 5. 2. · WELL BEHAVED TRANSITION SYSTEMS 3 domain. Verifying this condition for weighted VASS

Specifying and verifying requirements of real-time systems · Specifying and Verifying Requirements of Real-Time Systems Anders P. Ravn, Member, IEEE, Hans Rischel, Member, IEEE,

Secure Software Architecture Design Multidatabse · 2005. 2. 2. · distributed environment, multilevel security, and the difficulties in modeling, specifying and verifying data security

Verifying DMI pool data._ Verifying DMI pool data

Specifying and Verifying Requirements for Election Processeslaser.cs.umass.edu/techreports/08-21slides.pdf · To detect potential defects in election processes and identify ways to

Specifying and Verifying Interaction Protocols in a ...argo/papers/GMS-JAL.pdf · Abstract In this paper we ... tributed systems and they have recently gained attention in the area

Specifying Data

On Library Correctness under Weak Memory Consistencyplv.mpi-sws.org/yacovet/paper.pdf · 68 On Library Correctness under Weak Memory Consistency Specifying and Verifying Concurrent

Standards We Love for assuring and verifying safety ... · Evaluating, designing, specifying, and verifying safety-critical systems Safety & Security Assessments Carrying out safety