Verifying Epistemic Properties of Multi-agent Systems via Action-basedTemporal Logic

Marina BagicUniversity of Zagreb

Faculty of Electrical Engineering and ComputingUnska 3, HR-10000 Zagreb, Croatia

marina.bagic@fer.hr

Aleksandar BabacCroatia-Pumpe Nova d.o.o.

Mala Svarca 155, HR-47000 Karlovacaleksandar.babac@croatia-pumpe.com

Mojca CiglaricUniversity of Ljubljana

Faculty of Computer and Information ScienceTrzaska 25, SI-1000 Ljubljana, Slovenia

mojca.ciglaric@fri.uni-lj.si

Abstract

This paper provides a specifying and verifying frame-work of a multi-agent system, with the emphasis on theirepistemic features. We use an epistemic transition systemto specify the agents and an epistemic synchronous productto specify the multi-agent system.We verify the system bymeans of a special action-based temporal logic - ACTLWfor Epistemic Reasoning (ACTLW stands for Action Com-putation Tree Logic with Unless Operator). Using temporaland epistemic operators we create the appropriate formu-lae to perform model checking for the system. We test ourmethod by the example of security communication protocolcalled Dining Cryptographers.

1. Introduction

This paper investigates the problem of a multi-agent sys-

tem specification within the formal framework for verifica-

tion of its epistemic properties. We use the approach sim-

ilar to [9], i.e. we use symbolic model checking to verify

the system. Our basis is the formal framework of Epis-

temic Transition System (ETS) and Epistemic Synchronous

Product (ESP) while [9] uses the interpreted deontic system

which relies on Kripke structure.

Another direction in our work is to extend the ACTLW

(An Action-based Computation Tree Logic with unless op-

erator)logic [4, 5, 6] for epistemic reasoning. ACTLW was

developed for the verification of generic systems based on

Labelled Transition System (LTS). We add epistemic opera-

tors to it, and also introduce colours to actions. Colours are

dedicated to each action in order to manipulate over atomic

propositions which hold knowledge provided by the agents.

In our previous work [1] we have given some epistemic -

temporal operators. Now, for efficiency reasons we use only

temporal operators, but we add separately two epistemic op-

erators to ACTLW. These are K ϕ - ”an agent knows ϕ” and

EG ϕ - ”each agent in the group G knows ϕ”.

The paper is divided in the following sections; The

first chapter introduces epistemic structures to support the

idea of colours which is presented within the semantics of

ACTLW-ER (ACTLW for Epistemic Reasoning). The for-

mal framework for an agent and a multi-agent system is

given by means of the epistemic constructs (ETS and ESP).

Then, we explain the syntax and the semantics of ACTLW-

ER, a special logic for model checking multi-agent systems.

In the end, we provide the example of Dining Cryptogra-

phers to verify this protocol with our approach.

2. Epistemic Structures

2.1 Epistemic Particles and Atoms

Inspired by [3] and [8] (discussions on the first order

logic) we introduce some definitions in order to reason

about agent’s knowledge:

Definition. A Epistemic Particle ωi A Epistemic Par-

ticle ωi is an abstract piece of information, and the smallest

meaningful piece of an intelligent agent’s knowledge.

CIMCA 2008, IAWTIC 2008, and ISE 2008

978-0-7695-3514-2/08 $25.00 © 2008 IEEEDOI 10.1109/CIMCA.2008.221

470

In the formal language of predicate logic this piece of in-

formation is called the term. We can think of the examples

of particles as the abtractions of entites from the real world,

i.e. an abstraction of a circle. Also, we can think of particles

as elementary words from the sentence.

Definition. An Epistemic Atom Δj A Epistemic

Atom Δj is a Boolean product of epistemic particles ωi,

i ≤ n, which constitutes a particular domain D of an

agent’s raison d’etre with the order n denoting the number

of knowledge particles within a domain D.

Δj =∏

i

ωi (1)

Epistemic atoms can be imagined as sentences with a se-

mantics valid under the agent’s domain D.

We can think of an agent’s temporarily knowledge as a col-

lection of epistemic atoms. They are always set initially to

some values which are than changed according to the per-

formance of the agent. New atoms can be added or the ex-

isting ones can change their values.

Definition. Agent’s Knowledge (Ks, ·) Agent’s

Knowledge Ks is a Boolean product of epistemic atoms Δj .

Ks =∏

j

Δj (2)

2.2 Epistemic Operators

Definition. κ-operator under the structure Ks

κ(Ks) = (∃Δj .Ks) · Δj

κ-operator adds new Δj to the agent’s knowledge Ks. Epis-

temic atom added by κ-operator remains the same, e.g. does

not depend on the current agent’s state, until another opera-

tor changes it.

Definition. χ-operator under the structure Ks

χ(Ks) = (∃Δj .Ks) · Δj

χ-operator adds new Δj to the agent’s knowledge Ks but

only to the current state of the knowledge, e.g. the knowl-

edge in the next step Ks+1 does not depend on this change.

This is the main difference beween the two operators;

while κ affect the next states, or permanently changes the

knowledge of the agent, χ has the same effect only to the

current state of the knowledge and nothing else.

Definition. ε-operator under the structure Ks

εΔ(Ks) = ∃Δ.Ks

ε-operator forgets the specified agent’s. This operator suits

well for the purpose of creating the finite buffer for knowl-

edge storage. The infinite one could not be applicable to

real-time systems. If the system is returned to a previous

state in the system by this action, that means that it forgets

the current knowledge but begins to collect new one from

the position of this new-old state, i.e. it immediately learns

epistemic atoms from that state.

Definition. ρ-operator under the structure Ks

ρ(Ks) = Ks

ρ-operator does affect the knowledge of agent at all. It is

used, e.g. when one agent asks another about a particular

atom, so the asked agent cannot change its state of knowl-

edge.

3. Formal Specification of a Multi-agent Sys-tem

3.1. Epistemic Transition System

Here we describe the event-based approach to modeling

knowledge, one that is typically used in the work on knowl-

edge in game theory and mathematical economics. This ap-

proach in [2] uses Aumann structures while we define our

own structure in order to extend Aumann structure with data

and also retaining some properties of Kripke structure to

reason on atomic propositions, i.e. epistemic properties of

states.

Formally, we define Epistemic Transition System (ETS),

combining constructs from Labelled Transition System -

LTS [5] and Mixed Transition system - MTS [7] as follows.

Definition. Epistemic Transition System.Epistemic Transition System (ETS) is a 9-tuple:

A = (S,S0,A, δ,K, C, fs , fa , fc) (3)

where there are:

• S, a set of states where Si ∈ S• S0, a set of agent’s initial states

• A, a set of actions

• δ ⊆ S ×A× S, the transition relation

• K, a set of (non-)epistemic atoms (or atomic proposi-

tions)

• C, a set of actions’ colours: {α, κ, ρ, ε}• fa , a function mapping a set of atoms to each action

fa : ai → 2K

• fs , a function mapping a set of atoms to each state fs :Si → 2K

• fc , a function mapping an action to its colour fc : ai →c ∈ C.

471

3.2. Epistemic Synchronous Product

According to the previously defined agent as ETS (Defi-

nition 3.1), we define now a multi-agent system (MAS) as a

collection of ETSs. It is defined as synchronous product of

individual ETSs - Epistemic Synchronous Product (ESP).

Definition. Epistemic Synchronous Product.Epistemic Synchronous Product (ESP) is a 9-tuple:

M = (S,S0,A, δ,K, C, fs , fa , fc) (4)

where there are:

• S ⊆ S1 × · · · × Sn, a non-empty set of states of MAS

• S0 ⊆ S0,1 × · · · × S0,n, a set of MAS’ initial states

• A = A1 ∪ · · · ∪ An, a finite, non-empty set of actions

• δ ⊆ S ×A× S, the transition relation;

– (Si, a, S′i) ∈ δi ∧ ∀j = i : a ∈ Aj :

((S1, ..., Si, ..., Sn), a, (S1, ..., S′i, ..., S

′n)) ∈ δ

– (Si, a, S′i) ∈ δi ∧ (Sj , a, S′

j) ∈ δj :

((S1, ..., Si, ..., Sj , ..., Sn), a,

(S1, ..., S′i, ..., S

′j , ..., S

′n))

∈ δ

• K, a set of (non-)epistemic atoms

• C, a set of actions’ colours: {α, κ, ρ, ε}• fa , a function mapping a set of atoms to each action

fa : ai → 2K

• fs , a function mapping a set of atoms to each state fs :Si → 2K

• fc , a function mapping an action to its colour fc : ai →c ∈ C.

Here, the index i relates to each of the agent in the multi-

agent system. Using these constructs we reason on agent’s

knowledge. In further sections we will give a precize defi-

nition of what it means for an agent to ”know” something.

So far, when speaking of knowledge, we think of a set of

atomic propositions that an agent is aware of in its particu-

lar state in time.

Let us here define some notation to be used in further

sections.

An element (p, a, q) is called an a-transition or shortly

a transition from state p to state q. If there exists an a-

transition from a given state, we say that in this state the

ETS can perform a-transition or that it can perform action

a.

A sequence of transitions (p0, a1, p1), (p1, a2, p2), ...

where ∀i > 0 : (pi, ai+1, pi+1) ∈ δ is called a path π.

Moreover, pi and ai are called the i-th state and the i-th ac-

tion on this path, respectively, and the transition ending in

the i-th state is called the i-th transition on this path. We will

also use notations st(π, i) and act(π, i) for identification of

particular states and transitions on paths.

A sequence of transitions starting and ending in the same

state is called a cycle. If a path is infinite or ends in a dead-

locked state, it is called an infinite fullpath or a finite full-

path, respectively. The empty fullpath is a finite fullpath

with one state and no transitions. The number of transitions

in finite fullpath π will be denoted with len(π).

3.3. Colours of agents’ actions

Actions of ETS or ESP can be either epistemic or non-

epistemic. In both cases they can carry a package of data,

i.e. a set of atomic propositions which are true for that ac-

tion. We have chosen to dedicate a colour (or type ) to each

of a (non-)epistemic action. A colour of an action defines a

special operator to manipulate over the agent’s knowledge.

Figure 1. Effects of actions’ colours

We have introduced a set C consisting of four colours (or

types) of actions, denoted α, ρ, κ and ε. The first two, αand ρ are non-epistemic actions. It means that they have

only slight or no impact on the current state of the agent’s

knowledge.

When carrying atomic propositions α-action is true if

these atomic propositions are true only for that action and

for the state in this (α, s)-transition (s is a incoming state for

this α-action). So, α-action comes from LTS [6] where an

action is true only for the one particular transition, but car-

472

ries no data. We call this kind of action a weak epistemicaction.

ρ-action it a kind of question-action asking whether

a particular atomic propositions hold or not, or similar.

Therefore, ρ-action has no any impact on agent’s knowl-

edge.

κ and ε actions are epistemic actions. They affect the

current state of the knowledge in the diametrical manner.

κ-action adds new knowledge to the agent’s state, i.e. κ-

action is true for the (κ, s)-transition (s is a incoming state

for this κ-action), but atomic propositions which an action

is carrying migth also be true for the next few states, until

e.g. an agent ”forgets” them. We call this kind of action astrong epistemic action.

ε-action is true if it deletes the specified atomic propo-

sitions from the incoming state. It is an epistemic action

since it changes the current state of knowledge. Performing

ε-action we say that an agent ”forgets” some of its knowl-

edge (e.g. redundant information).

The effects of coloured actions are depicted in Figure 1

where Δ is a portion of atomic propositions. Cycles and

arrows represent states and actions, respectively.

Meaning of the actions’ colours are better explained in

the context of the actions and paths rather then only by ac-

tions semantics. So, we give a formal definitions of actions’

colours semantics in Table 2.

The logical interpretation of the colours is given by the

epistemic structures in Section 2.

3.4. Syntax of ACTLW for Epistemic Rea-soning

Action Computation Tree Logic for Epistemic Reason-

ing (ACTLW-ER) syntax and semantics is defined over the

ETS and ESP.

Let χ, ϕ, and γ be a data-action formula, a state formula,

and a path formula, respectively, iff they meet the following

syntactic rules:

χ ::= true |α |α(Δ) |κ |κ(Δ) | ρ | ρ(Δ) | ε | ε(Δ) |

τ | τ(Δ)|¬χ |χ ∨ χ (5)

ϕ ::= true | k | ¬ϕ |ϕ ∧ ϕ′ |Eγ |Aγ |Kϕ |EG ϕ (6)

γ ::= {χ}ϕU {χ′}ϕ′ | {χ}ϕW {χ′}ϕ′ (7)

A data-action formula χ (3.4) may be constructed by send-

ing or receiving (! or ?) message to or from port p or may be

a Boolean negation (¬χ) or a Boolean composition (either

product or addition) of more than one data-action formulae

χi. Formal definition of the data-action formula semantics

is given in Table 1.

A state formula (6) is either Boolean value, or contains a

path operators A or E denoting all or some of the states at a

lifecycle path of an agent. Formal definition of the colour-

action formula semantics is given in Table 3.

A path formula (7) contains the constructs from the

above (actions, states) and temporal operators (U, W and K)

while the other temporal operators are derived from these

ones.

3.5. ACTLW for Epistemic Reasoning Se-mantics

Let M = (S,S0,A, δ,K, C, fs , fa , fc) be a multi-agent

system. Satisfaction of data-action formula χ by an action

a ∈ A (written a |= χ), state formula ϕ by a state s ∈ S(s |= ϕ), a path formula γ by a finite fullpath π (written

π |= γ), and a path formula γ by an infinite fullpath σ(written σ |= γ) in a ESP M is given inductively by the

semantic rules given in tables 1, 3 and 4.

a |= true always

a |= χ iff a = χa |= χ(Δ) iff a = χ ∧ Δ = truea |= τ(Δ) iff a = τ ∧ Δ = truea |= ¬χ iff a |= χa |= χ ∨ χ′ iff a |= χ ∨ a |= χ′

Table 1. Data-action Semantic Rules ofACTLW for Epistemic Reasoning

a, π |= κ(Δ) iff ∃i ∈ [1, |π|] : act(π, i) |= κ(Δ)∧st(π, i) |= Δ ∧ ∀j > i ∧ j ≤ |π| :st(π, j) |= Δ

a, π |= ε(Δ) iff ∃i ∈ [1, |π|] : act(π, i) |= ε(Δ)∧st(π, i) |= Δ ∧ ∀j > i ∧ j ≤ |π| :st(π, j) |= Δ

a, π |= α(Δ) iff ∃i ∈ [1, |π|] : act(π, i) |= α(Δ)∧st(π, i) |= Δ

a, π |= ρ(Δ) iff ∃i ∈ [1, |π|] : act(π, i) |= ρ(Δ)∧st(π, i) |= true

Table 2. Colours Semantics of ACTLW forEpistemic Reasoning

4. The Dining Cryptographers Protocol

In this chapter we give an example of inter-agent com-

munication protocol - Dining Cryptographers. We specify

473

and verify the system by the formal tools we have described

in previous chapters. First, we define the problem of Dining

Cryptographers from the literature [10];

Three cryptographers are sitting down to dinner at their

favorite three-star restaurant. Their waiter informs them

that arrangements have been made with the maitre d’hotel

for the bill to be paid anonymously. One of the cryptogra-

phers might be paying for the dinner, or it might have been

NSA (U.S. National Security Agency). The three cryptog-

raphers respect each other’s right to make an anonymous

payment, but they wonder if NSA is paying. They resolve

their uncertainty fairly by carrying out the following proto-

col.

s |= true always

s |= k iff k ∈ fs(s) ∈ Ks |= ¬ϕ s |= ϕs |= ϕ ∧ ϕ′ s |= ϕ ∧ s |= ϕ′

s |= Eγ iff ∃π : s = st(π, 0) ∧ π |= γor ∃σ : s = st(π, 0) ∧ σ |= γ

s |= Aγ iff ∀π and ∀σπ : s = st(π, 0)σ : s = st(π, 0) ∧ σ |= γ

s |= Kϕ iff s |= ϕs |= EGϕ iff ∀i ∈ G : s |= ϕ

Table 3. State Semantic Rules of ACTLW forEpistemic Reasoning

Each cryptographer flips an unbiased coin behind his

menu, between him and the cryptographer on his right, so

that only the two of them can see the outcome. Each cryp-

tographer then states aloud whether the two coins he can

see–the one he flipped and the one his left-hand neighbor

flipped–fell on the same side or on different sides. If one

of the cryptographers is the payer, he states the opposite of

what he sees. An odd number of differences uttered at the

table indicates that a cryptographer is paying; an even num-

ber indicates that NSA is paying (assuming that the dinner

was paid for only once). Yet if a cryptographer is paying,

neither of the other two learns anything from the utterances

about which cryptographer it is [10].

4.1. Specification of Agents in Dining Cryp-tographers System

Specification of a multi-agent system of dining cryptog-

raphers requires at least three agents for the cryptographers

and one for the environment. Therefore, we denote each one

of them as agent A, B, C and E, respectively. Each Cryptog-

rapher Agent has no any prior knowledge before the system

startup. Initial knowledge is given to them by the Environ-

π |= ϕ{χ(Δ)}U {χ′(Δ′)}ϕ′

iff st(π, 0) |= ϕ ∧ ∃i ∈ [1, |π|] : (act(π, i) |= χ′(Δ′)∧st(π, i) |= ϕ′) ∧ ∀j ∈ [1, i − 1] : (act(π, j) |= χ(Δ)∧st(π, j) |= ϕ)

σ |= ϕ{χ(Δ)}U {χ′(Δ′)}ϕ′

iff st(σ, 0) |= ϕ ∧ ∃i ∈ [1, |σ|] : (act(σ, i) |= χ′(Δ′)∧st(σ, i) |= ϕ′) ∧ ∀j ∈ [1, i − 1] : (act(σ, j) |= χ(Δ)∧st(σ, j) |= ϕ)

π |= {χ}ϕW {χ′}ϕ′

if π |= {χ}ϕU {χ′}ϕ′ or if

∀i ∈ [1, len(π)]st(π, i) |= ϕ∧ act(π, i) |= χ

σ |= {χ}ϕW {χ′}ϕ′

if σ |= {χ}ϕU {χ′}ϕ′ or if

∀i ≥ 1 : st(σ, i) |= ϕ∧ act(σ, i) |= χ

Table 4. Path Semantic Rules of ACTLW forEpistemic Reasoning

ment Agent. The Environment Agent supplies each Cryp-

tographer Agent with the necessary information and then

lets it communicate to the other agents in the system.

We have explicitly denoted the port’s (or channel’s)

names in order to give precise address of the message des-

tination, e.g. a-c, env-c and b-c meaning Agent A to AgentC, Environment Agent to Agent C and Agent B to Agent C,

respectively.

To specify the communication between the agents in the

system we have explicitly denoted the port’s (or channel’s)

names in order to give precise address of the message des-

tination, i.e. a-c stands for a communication point between

the Cryptographer Agent A and the Cryptographer Agent

C, or env-c stands for a communication point between the

Environment Agent and the Cryptographer Agent C.

The main difference between the three cryptographers

agents’ specifications are in the epistemic atoms and the ad-

dresses of the adjacent agents.

The first two actions that each Cryptographer Agent per-

forms are synchronized with the Environment Agent. They

learn about their status of paying for the dinner or not and

also the status of the coin on their right to notify the adjacent

agent on their left (Tables 5 and 6).

474

CRYPTOGRAPHER AGENT AINIT = env-a ? inform (NOTpayingA);

env-a ? inform (rightTailA);

a-b ! tell (leftTailB);

b-c ? tell (leftHeadB);

a-b ! inform (differentA);

a-c ! inform (differentA); WAIT

WAIT = a-c ? inform (differentC); WAIT B

+ a-b ? inform (differentB); WAIT C

WAIT C = a-c ? inform (differentC);

env-a ? inform(odd);

return (differentB, differentC); INIT

WAIT B = a-b ? inform (differentB);

env-a ? inform (odd);

return (differentB, differentC); INIT

Table 5. Dining Cryptographer Agent A

ENVIRONMENT AGENTINIT = env-a ! inform (NOTpayingA);

env-b ! inform (NOTpayingB);

env-c ? inform (payingC);

env-a ! inform (rightTailA);

env-b ? inform (rightHeadB);

env-c ? inform (rightHeadC);

env-a ! inform (odd);

env-b ! inform (odd);

env-c ! inform (odd); INIT

Table 6. Environment Agent

4.2. Verification of Dining CryptographersMulti-agent System

We now have a strong and full equipment to verify the

system. We reason on specification of a multi-agent system

by using different ACTLW-ER formulae.

(F1) There is no path such that Cryptographer Agent A can

decide on his statement until he collects the information

from other agents.

¬E {inform(differentA)}U {infrom(rightTailB)}(F2) If the number of differences in the utterances is odd,

then Cryptographer Agent A knows that either Cryptogra-

pher Agent B or Cryptographer Agent C paid for the dinner.

AG {inform (odd)} KA (payingB ∨ payingC)

(F3) All cryptographers know that either one of them or

NSA is paying for the dinner.

AG {true} EG (payingA ∨ payingB ∨ payingC ∨ pay-

ingNSA)

(F4) If the number of differences in the utterances is even,

then Cryptographer Agent A knows that non of the cryp-

tograpers paid.

AG {inform (¬odd)} KA (payingNSA)

All the actions are coloured as strong epistemic actions

since they carry new knowledge to the agent.

All the formulae are true for our system.

5. Conclusions

This paper gives a formal framework for specification

and verification of a multi-agent systems with the empha-

sis on their epistemic properties. Our contribution in this

paper compared to the previous ones is the introduction of

epistemic operators Kϕ and EGϕ to ACTLW logic for Epis-

temic Reasoning. Also, we have added colours and data to

the actions of ACTLW to enrich the modelling of the system

and we interpret these according to the epistemic structures,

i.e. atoms, knowledge. We have shown our approach on the

example of Dining Cryptographers.

References

[1] M. Bagic and M. Kunstic. Verification of intelligent agents

with actl for epistemic reasoning. Proceedings of the Inter-national Conference on Intelligent Agents, Web Technolo-gies and Internet Commerce - IAWTIC06, page 76, 2006.

[2] R. Fagin, J. Y. Halpern, Y. Moses, and M. Y. Vardi. Rea-soning About Knowledge. The MIT Press, Cambridge Mas-

sachusetts, London England, 2003.[3] L. A. Kaluzanin. Sta je matematicka logika. Skolska knjiga

Zagreb, 1971.[4] R. Meolic. An Action Computation Tree Logic With Unless

Operator. Doctoral thesis (in Slovene), Faculty of Electrical

Engineering and Computer Science, University of Maribor,

Slovenia, 2005.[5] R. Meolic, T. Kapus, and Z. Brezocnik. Verification of con-

current systems using actl. Proceedings of the IASTED in-ternational conference AI’2000, IASTED/ACTA Press, Ana-heim, Calgary, Zurich, pages 663–669, 2000.

[6] R. Meolic, T. Kapus, and Z. Brezocnik. An action compu-

tation tree logic with unless operator. Proceedings of the 1stSouth-East European workshop on formal methods SEEFM2003, pages 100–114, 2003.

[7] C. Pecheur and F. Raimondi. Symbolic model checking of

logics with actions. Proceedings of the Fourth Workshopon model checking artificial intelligence (MoChArt 2006),Springer Verlag LNAI, 2000.

[8] M. R, A. Huth, and M. D. Ryan. Logic in Computer Sci-ence: Modelling and reasoning about systems. Cambridge

University Press Cambridge, England UK, 2000.[9] F. Raimondi and A. Lomuscio. Automatic verification of

deontic and epistemic properties of multi-agent systems by

model checking via obdd’s. Proceedings of ECAI 2004, Va-lencia, 2004.

[10] F. Raimondi and A. Lomuscio. A tool for specification and

verification of epistemic properties in interpreted systems.

Electronic Lecture Notes of Theoretical Computer Science,

vol. 85, 2004.

475