Towards a Unied Model of Sociality in Multiagent · PDF fileTowards a Unied Model of Sociality in Multiagent Systems ... of CS-based applications are described which illustrate the

International Journal of Computer & Information Science, Vol. 5, No. 1, March 2004

Towards a Unified Model of Sociality in Multiagent Systems

Matthias Nickles and Michael Rovatsos and Wilfried Brauer and Gerhard WeißDepartment of Informatics, Technical University Munich

Abstract

This paper presents communication systems (CS) as thefirst available unified model of socially intelligent systemsdefined in terms of communication structures. It combinesthe empirical analysis of communication in a social systemwith logical processing of social information to providea general framework for computational components thatexploit communication processes in multiagent systems. Thetwo main contributions offered by this paper are as follows:First, a formal model of CS that is based on an improvedversion of expectation networks and their processing ispresented. This formal model is based on a novel approachto the semantics of agent communication languages whichcontrasts with traditional approaches. Second, a numberof CS-based applications are described which illustrate theenormous potential and impact of a CS-based perspective ofsocially intelligent systems.

Keywords: Artificial Agents, Multiagent Systems, AgentCommunication Languages, Agent-Oriented Software Engi-neering, Socionics.

1. Introduction

The crucial property of artificial agents is their autonomy,and since communication is the only autonomy-preservingway for agents to interact, it can be argued [1, 2] that anykind of social relationship among agents (constituted throughe.g. virtual organizations, interaction protocols, social norms,common ontologies...) can be described in form of commu-nication structures. Traditional attempts to model the seman-tics of agent communication languages (ACLs), which consti-tute communication structures, are mostly based on describ-ing mental states of communicating agents [3, 4, 5, 6] or onpublicly available (usually commitment-based) social states[7, 8, 9]. The theoretical advantages of the former approachare that it could be able to fully expose the whole mecha-nism of utterance and utterance understanding (provided thatthe agents are equipped with social intelligence). But it hastwo obvious shortcomings, which eventually led to the de-velopment of the latter “objectivist” approach: At least inopen multiagent systems, agents appear more or less as au-tonomous black boxes, which makes it impossible in generalto impose and verify a semantics described in terms of cog-nition. Furthermore, the description of interaction scenariosin terms of the cognition of individuals tends to become ex-

tremely complicated and intractable for large sets of agents,even if one could in fact “look inside the agents’ heads”. Thisis not so much the case due to the complexity of communica-tion processes themselves, but due to the subjective, limitedperspective of the involved individual agents, which makesit hard to conclude a concise picture of supra-individual pro-cesses. Current mentalistic approaches either lack a conceptfor preventing such complexity at all, or they make simpli-fying but unrealistic assumptions, for example that all inter-acting agents were benevolent and sincere. The “objectivist”semantics, in contrast, is fully verifiable, and it achieves abig deal of complexity reduction by limiting itself to a smallset of normative semantical rules. Therefore, this approachhas been a big step ahead. But it oversimplifies social pro-cesses neglecting pragmatics in favor of traditional sentencesemantics, and it does not have a sufficient concept of mean-ing dynamics and generalization, and social rationality likeargumentation and sanctioning [10]. Communication alwayshas an implicit social semantics prior to any normative defi-nition, and this semantics needs to be exploited. In general,we doubt that the predominately normative, static and defi-nite concepts of most of the current ACL research, borrowedfrom the study of programming languages and protocol se-mantics, are really adequate to cope with concepts like agentautonomy, agent opaqueness and the emergence and vague-ness of socially constructed meaning, which communicationis in fact about. Therefore, both these traditional views fail torecognize that communication semantics evolve during op-eration of a multiagent system (MAS), and that they alwaysdepend on the view of an observer who is tracking the com-municative processes of black- or gray-box agents in the sys-tem [10, 11]. Yet communication dynamics and observer-dependency are crucial aspects of inter-agent communica-tion, especially in the context of open systems in which apre-determined semantics cannot be assumed, let alone thecompliance of agents’ behavior with it.In response to these issues, in [1, 12] we have – influenced bysociological systems-theory [13] – introduced expectationsregarding observable communications as a universal meansfor the modelling of emergent sociality in multiagent systems,and in [11, 10, 14], we have presented – influenced by socio-cognitive and socio-systems theories [15, 16, 13] – formalframeworks for the semantics of communicative action thatis empirical, constructivist and consequentialist in nature andanalyzed the implications of this model on social reasoningboth from an agent (bottom-up) and a systemic (top-down)perspective.


Thereby, we have suggested that recording observationsof message exchange among agents in a multiagent system(MAS) empirically is the only feasible way to capture themeaning of communication, if no a priori assumptions aboutthis meaning can be made. Being empirical about meaningnaturally implies that the resulting model very much dependson the observer’s perspective, and that the semantics wouldalways be the semantics “assigned” to the utterances by thatobserver, hence this view is inherently constructivist. Since,ultimately, no more can be said about the meaning of a mes-sage than that it lies in the expected consequences that thismessage has, we also adopt a consequentialist outlook onmeaning.

In this paper, we present a framework for the formal de-scription of socially intelligent multiagent systems based onthe universal, systems-theoretical concept of communicationsystems (CS). Following sociological systems theory, com-munication systems (also called social systems) are systemsthat consist of interrelated communications which describetheir environment [13]. We use this term to denote computa-tional models of such systems that process empirical informa-tion about observed communication which takes place withina MAS of artificial agents (either with the CS as an MAS-external MAS-observer or as a component of an agent whichparticipates in the observed communication himself). Theirdistinct features are (i) that they only use data about com-munication for building models of social processes, the un-derlying assumption being that all relevant aspects of agentinteraction are eventually revealed through communication,and (ii) that, if the respective CS is part of an agent, theresults of the observations are suitable to take action in theMAS to influence its behavior; in other words, there might bea feedback loop between observation and action, so that anCS-based agent becomes an autonomous component in theoverall MAS.

CSs might be (part of) socially intelligent software agents.Note, however, that this is not necessarily the case. Althoughtheir autonomy presumes some agentification in the tradi-tional sense, their objectives need not be tied to achievingcertain goals in the physical (or virtual simulation) world as isthe case with “ordinary” agents. Thus, CS are best character-ized in a abstract fashion as components used to (trans)formexpectations (regardless of how these expectations are em-ployed by agents in their reasoning) and are autonomous withrespect to how they perform this generation of expectations.

Thereby, the “semantics” aspect mentioned above plays acrucial role, because the meaning of agent communicationlies entirely in the total of communicative expectations in asystem [1], and CS capture precisely these expectations andhow they evolve.

The remaining sections are structured as follows: We startby introducing expectation networks in section 2, which con-stitute our formal model for describing communicative ex-pectations. Then, we formally define communication systemsand their semantics (section 3). Section 4 discusses applica-

tions and extensions of the CS, and section 5 concludes.

2. Expectation Networks

It is widely accepted in Distributed Artificial Intelligencethat the most important property of intelligent agents istheir autonomy. The major consequence of the autonomousbehavior of agents is that a certain agent appears to otheragents and observers more or less as a black box whichcannot fully be predicted and controlled. This obscurity anduncontrollability is particularly salient in the open multiagentsystems we are focussing on. Because only the actions of theagent in its environment can be observed, while its mentalstate keeps obscure, beliefs and demands directed to therespective agent can basically be stylized as action expec-tations, which are fulfilled or disappointed in future agentactions1. To overcome the situation of mutual indeterminismamong black-box agents (the so-called situation of doublecontingency [13]), i.e. to determine the respective otheragent and to achieve reasonable coordination (including“reasonable” conflicts), the agents need to communicate.A single communication is the whole of a message act asa certain way of telling (not necessarily via speech, butalso e.g. as a demonstrative gesture or other semioticallyrelevant manipulations of the “physical” domain), plus acommunicated information, plus the understanding of thecommunication attempt. Communication is observable as acourse of related agent interactions. Because communica-tions are the only way to overcome the problem of doublecontingency (i.e. the isolation of single agents), they are thebasic constituents of sociality, and the relation of subsequentcommunications forms the social system for the participatingagents. And if action expectations are related to message actsas parts of communications (i.e. sociality), social structurescan be modeled as expectation structures [13].By processing existing expectations, agents determine theirown actions, which, then, influence the existing expectationsin turn. So communication is not only structured by indi-vidual agent goals and intentions, but also by expectations,and the necessity to test, learn and adapt expectationsfrom observed interactions and in order to optimize futurecommunications.Expectations regarding agent behavior can be formed notonly by peer agents (as an aspect of their mental state),but also by observers with a global view of the multiagentsystem. Such system-level expectations are called emergentif they are formed solely from the statistical evaluation ofthe observed communications. In contrast, e.g. the systemdesigner forms expectations not only from her knowledgeabout the existing multiagent system “under construction”,but also from her design goals (in 4.2 we will present a designmethod for multiagent system based on this principle). Ineither case, interrelated expectations regarding future agent

1This view of expectations and sociality follows the Theory of SocialSystems (“systems theory”) of the sociologist Niklas Luhmann [13] and isdescribed in detail in [1].


can(B,X)

reject(B,A,X)

[X/deliver_goods]

price>0

[A/agent_1], [B/agent_2]

[A/agent_1],[B/agent_3]

can(A,Y)can(C,Y)

request(A,B,X) [Y/pay_price]

acceptproposal(A,B,Y)

rejectproposal(A,B,Y)

0.50.5

propose(B,A,Y)

do(B,X)

0.2

can(B,X)

0.1

0.5

accept(B,A,X)

0.5?

delegate(A,C,Y)

do(C,Y)

can(A,Y)

do(A,Y)

0.30.8

0.7

do(B,X)

0.9

price=0

[C/agent_3]

[A/agent_3], [B/agent_2]

Figure 1.An expectation network. Nodes are labelled with message templates in typewriter font and the special symbols ., ⊥ and ?;they are connected by (solid) edges labelled with numerical expectabilities in italic font. Substitution lists/conditions belongingto edges appear in rounded/edged boxes near the edge. If neighbouring edges share a condition this is indicated by a drawnangle between these edges. This example network shows a short communication course of three agent roles, A, B and C, whichare instantiated with three agents through substitution lists (A = agent 1, B = agent 2, C = agent 3). The provision ofsuch substitution lists is optional. The EN starts with a request to do action X (deliver goods) of A directed to B. In casecondition price = 0 is fulfilled, B is expected to always accept this request and to perform the requested action X in case he isable to do it (condition can(B,X) is true). Otherwise, with probability 0.5 the request will be rejected and the communicationends (⊥). With probability 0.5, B answers with a proposal Y (where Y = pay price), which is accepted by A with probability0.5. After rejection, the further course of communication is unknown (?), whereas the acceptance leads to the fulfillment of Xthrough B (do(B,X)). In the latter case, A in turn does Y (i.e., pays the price for X), or delegates this to C if he is not ableto pay (can(A, Y ) is false).

behavior are the only modeling means that are universallysuitable for the description of black-box agents’ interactionbehavior from an external observers point of view as ourcomputational models of communication systems.

Expectation networks (ENs) [1] are the graphical datastructures used in this work for the formal representation ofsuch social expectations. They capture the regularities in theflow of communication between agents in a system by in-terconnecting message templates (nodes) that stand for utter-ances via links (edges) which are labelled with (i) probabilis-tic weights called expectabilities, (ii) a logical condition and(iii) lists of variable substitutions. Roughly speaking, the se-mantics of such a weighted edge is as follows: If the vari-ables in the messages have any of the values in the (optional)substitution lists, and the logical condition is currently satis-

fied, then the weight of this edge reflects the probability withwhich a message matching the label of the target node is ex-pected to follow the utterance of a message matching the labelof the source node of the edge. Before presenting a full def-inition of ENs, we have to introduce some basic notions andnotation we use, and to make certain auxiliary definitions andassumptions. The example network in figure 2 will be usedthroughout the discussion of ENs to illustrate the purpose ofdefinitions and assumptions.

2.1 Basics

A central assumption that is made in ENs is that observedmessages may be categorised as continuations of other com-munications, or may be considered the start of a new inter-action that is not related to previous experience. So an edgeleading from message m to message m′ is thought to reflect


the probability of communication being “continued” fromthe observer’s point of view. Usually, continuation dependson temporal and spatial proximity between messages, but itmight also be identified through a connection about “subject”,or, for example, through the use of the same communicationmedium (m′ was shown on TV after m was shown some timeearlier on).

Apart from “ordinary” node labels denoting messages, weuse three distinct symbols “.”, “⊥”, and “?”. “.” is the la-bel occurring only at the root node of the EN. Whenever amessage is considered a start of a new conversation insteadof continuing previous sequences, it is appended to this “.”-node. Nodes labelled with “⊥” denote that a course of com-munications is expected to end with the predecessor of thisnode. The label “?”, finally, indicates that there exists no ex-pectation regarding future messages at this node. Nodes withsuch “don’t know” semantics are usually messages that occurfor the first time – the observer knows nothing about whatwill happen after them being uttered.

To define the syntactic details of EN, we introduce for-mal languages L and M used for predicate-logical expres-sions and for message templates. L is a simple logicallanguage consisting of propositions Statement potentiallycontaining (implicitly universally quantified) variables andof the usual connectives ∨, ∧, ⇒ and ¬, the logical con-stants “true” and “false”, and braces () for grouping sub-expressions together (the language is formally given by thegrammar in table 1). Given the set of all possible interpreta-tions I = {I : Statement → {true, false}} we define therelation |=⊆ I × L in the usual way by induction over for-mulae ϕ ∈ L and interpretations I ∈ I:

I |= ϕ iff ϕ ∈ Statement and I(ϕ) = trueI |= ϕ iff ∃ϑ : ϕ′ϑ = ϕ and I |= ϕ′

I |= ¬ϕ iff I 6|= ϕ

I |= ϕ ∨ q iff I |= ϕ or I |= q

where ϑ = 〈[v1/t1], . . . , [vk/tk]〉 is a variable substitution.As usually, ∧ and⇒ can be defined as abbreviations throughthe other operators. Also, we write |= ϕ if ϕ is a tautologythat is satisfied by any I ∈ I. A knowledge base KB ∈ 2L

can be any finite set of formulae from L. For simplicity, wewill often write KB |= ϕ to express |= (

∧

ϕ′∈KB ϕ′ ⇒ ϕ).As for M, this is a formal language that defines the mes-

sage patterns used for labelling nodes in expectation net-works. Its syntax is given by the grammar in table 1. Mes-sages observed in the system (we write Mc for the lan-guage of these concrete messages) can be either physicalmessages of the format do(a, ac) where a is the executingagent and ac is a symbol used for a physical action, or anon-physical message performative(a, b, c) sent from a tob with content c. (Note that the symbols used in the Agent

and PhysicalAction rules might be domain-dependent sym-bols the existence of which we take for granted.) The node

Var → X | Y | Z | . . .

AgentVar → A1 | A2 | . . .

PhysicalActVar → X1 | X2 | . . .

Expect ∈ [0; 1]

Agent → agent 1 | . . . | agent n

Head → it rains | loves | . . .

Performative → accept | propose | reject | inform

| . . .

PhysicalAction → move object | pay price

| deliver goods | . . .

Message → Performative(Agent ,Agent ,LogicalExpr)

| do(Agent ,Agent ,PhysicalAction)

MsgPattern → Performative(AgentTerm,AgentTerm,

LogicalExpr)

| do(AgentTerm,AgentTerm,

PhysicalActTerm)

| . | ⊥ | ?

PhysicalActTerm → PhysicalActVar | PhysicalAction

AgentTerm → AgentVar | Agent

LogicalExpr → (LogicalExpr ⇒ LogicalExpr)

| (LogicalExpr ∨ LogicalExpr)

| (LogicalExpr ∧ LogicalExpr)

| ¬LogicalExpr

| Statement

Statement → Head | Head(TermList) | true

| false

TermList → TermList ,Term | Term

Term → Var | AgentTerm | MsgPattern

| Graph

EdgeList → (MsgPattern,Expect ,MsgPattern,

LogicalExpr ,SubstList) EdgeList | ε

Graph → 〈EdgeList〉

SubstList ′ → SubstList ′ Subst | ε

SubstList → 〈SubstList ′〉

Subst → [AgentV ar/Agent]

| [PhysicalActV ar/PhysicalAction]

| [V ar/Term]

Table 1.A grammar for messages, generating the languagesM (the language of mes-sage patterns, using MsgPattern as starting symbol),Mc (the language ofconcrete messages, using Message as starting symbol) and the logical lan-guage L (using LogicalExpr as starting symbol).


labels (type MsgPattern) used in the expectation networksmay also contain variables for agents and physical actions(though not for performatives). Following the concept ofagent roles introduced in [1, 12], the variables for agents arecalled (agent) roles. These variables are useful to general-ize over different observed messages, and can optionally befurther specified by adding variable substitution lists. Thecontent c of a non-physical action, finally, is given by typeLogicalExpr . It can either be (i) an atomic proposition, a(ii) message term or physical action term, (iii) an expectationnetwork2, or (iv) a logical formula containing these elementsaccording to table 1. Syntactically, expectation networks arehere represented as lists of edges (m, p, n, c, l) where m andn are message terms, p is a transition probability (expectabil-ity) from m to n, c is a logical condition, l is a list of variablesubstitutions. We use functions in : V → 2C , out : V → 2C ,source : C → V and target : C → V which return the in-going and outgoing edges of a node and the source and targetnode of an edge, respectively, in the usual sense. C is the setof all edges, V the set of all nodes in the EN. cond : C → Lreturns the conditions of edges, subst : C → SubstList

(with SubstList as in table 1) returns the edges’ substitutionlists. Edges denote correlations in observed communicationsequences. Each cognitive edge is associated with an ex-pectability (returned by Expect : C → [0; 1]) which reflectsthe probability of target(e) occurring shortly after source(e)in the same communicative context (i.e. in spatial proximity,between the same agents, etc.).

The full meaning of these ingredients will be further clari-fied once the full definition of expectation networks has beenpresented. Note that according to table 1 expectation net-works are allowed to be contained within message termsof expectation network nodes themselves to allow the mod-elling of the communication of complex expectation struc-tures among agents.

2.2 Edge Conditions

As a final ingredient to network edges, we briefly discussedge conditions. The idea is that these conditions should fur-ther define the scope of validity to cases in which a formulacan be shown to hold using the observer’s knowledge base.So, if ϕ = cond(e), then e is only relevant iff KB |= ϕ.

Because all conditions for outgoing edges of a certain nodeshould be mutually exclusive to ensure that later the seman-tics of a certain message trajectory can be calculated unam-biguously, we want the sum of expectabilities of all out-edgesof a node to be one for a certain knowledge base content3. In

2Such expectation networks within messages are useful to replace perfor-matives of agent communication languages. We have to refer to [10, 14] fordetails of this concept.

3From a probabilistic point of view, it would be sufficient to demand asum lower or equal one, but a sum of exactly one (which is practically al-ways feasibly through insertion of a “dummy” ’?’-edge) formally ensuresthe exhaustiveness of the set of outgoing edges.

other words, the condition

∀v∑

e∈out(v),KB|=cond(e)

Expect(e) = 1

should hold.This can be ensured, for example, by guaranteeing that the

following condition holds through appropriate constructionrules for the EN. Assume the outgoing links out(V ) of ev-ery node v are partitioned into sets O1, O2, . . . Ok where thelinks’ expectabilities in each Oi are non-negative and sum upto one4. Now let all edges in Oi share the same edge condi-tion, i.e. ∀i∃ϕ∀o ∈ Oi.(cond(o) = ϕ) and define cond(Oi)as precisely this shared condition ϕ. (The Oi sets are pre-cisely those sub-sets of out(v) connected by a drawn anglein figure 2.)

If we make sure that the outgoing links of every node arepartitioned in this way, we can assign mutually exclusive con-ditions to them, i.e. ensure that

∀i 6= j.cond(Oi) ∧ cond(Oj ) ≡ false

and ∨i cond(Oi) ≡ true

This way, it is not only guaranteed that we can derive un-ambiguous probabilities directly from the Expect values, butalso that we can do so for any knowledge base contents (cf.2.4)5.

2.3 Formal Definition

Having discussed all the prerequisites, we can now defineENs formally:

Definition 1. An expectation network is a structure

EN = (V,C,M,L, H,mesg , cond , subst ,Expect)

where

• V with |V | > 1 is the set of nodes,

• C ⊆ V × V are the cognitive edges (or edges for short)of EN . (V,C) is a tree called expectation tree.

• M is a message term language6, L is a logical language,cond : C → L returns the conditions of edges,

• mesg : V →M is the message label function for nodessuch that

- mesg(v) = . exactly for the root node of (V,C),

4Formally, out(v) = ∪1≤i≤kOi and ∀1 ≤ i < j ≤ k.Oi ∩ Oj = ∅,and ∀i ≤ k.(∀o ∈ Oi.Expect(o) ≥ 0 ∧

P

o∈OiExpect(o) = 1).

5This comes at the price of having to insert redundant edges in somesituations. For example, insertion of a new edge e with cond(e) = ϕ ifout(v) = ∅ necessitates insertion of another edge e′ with cond(e) = ¬ϕ.

6All languages as defined in the previous sections.


- ∀v ∈ V.∀e, f ∈ out(v).¬unify(mesg(target(e)),mesg(target(f)))(where unify shall be true iff its arguments aresyntactically unifiable, i.e., target node labels ofoutgoing links never match),

• H ∈ N is a finite communication horizon,

• Expect : C → [0; 1] returns the edges’ expectabilities,

• subst : C → SubstList (with SubstList as in table 1)returns the edges’ substitution list.

Our full formal framework [17] also defines so-called norma-tive edges, which have been omitted here for lack of space.In contrast to cognitve edges, the expectabilities of normativeedges are only seldomly adapted by the communication sys-tem according to newly observed messages, and under veryspecific circumstances. In the tradition of systems theory,here the term “cognitive” means “adaptable through cogni-tion about observations”.

The only element of this definition that has not been dis-cussed so far is the communication horizon H , which denotesthe scope of maximal message sequence length for which theEN is relevant. It is necessary for defining the semantics ofthe EN, and will be discussed in detail in the following sec-tion.

2.4 Formal Semantics of Message Sequences

The purpose of an EN is to provide a semantics for mes-sages. For an arbitrary set S, let ∆(S) be the set of all (dis-crete) probability distributions over S with finite support. Wedefine the semantics IEN (KB,w) of an observed messagesequence w in a network EN as a mapping from knowl-edge base states and current message sequence prefixes to theposterior probability distributions over all possible postfixes(conclusions) of the communication. Formally,

IEN (KB , w) = fw, fw ∈ ∆(M∗c) (1)

where

fw(w′) =

gw(w′⊥)

∑

v∈M∗

c

gw(v⊥)(2)

is defined as the normalized value of gw(w′⊥). gw(w

′⊥) rep-resents the probability that w will be concluded by messagesequence w′, for any w,w′ ∈ M∗. We compute the prob-ability for w′⊥ to make sure w′ is followed by a node withlabel ⊥ in the network, because the probability of w′ is theprobability with which the communication sequence will endafter w′

|w′| (and not that w′ will simply be the prefix of somelonger sequence). Also note that the sum in the denominatoris not, as it may seem, infinite, because fw has finite supportand the length of the considered message sequences is limitedby means of the communication horizon H (see below), andthat the semantics of w depends on KB, because only those

edges which have conditions that are true according to KBare used for calculating the semantics of w.

Informally, the probability of w′ should be inferred frommultiplying all the expectability weights along the path thatmatches w′ (if any). Before presenting the top-level formulafor gw(w

′), we need some auxiliary definitions:Firstly, we need to determine the node in a network EN

that corresponds to a word w, which we denote by mesg−1:

mesg−1(ε) = v :⇔ mesg(v) = .

mesg−1(wm) =

v′ if ∃(v, v′) ∈ C(KB).

∃ϑ ∈ subst((v, v′)).

(mesg(v′) · subst(w)ϑ = m

∧mesg−1(w) = v)

⊥ if no such v′ exists

(3)

if w ∈M∗c , m ∈Mc

7. The first case states that the node cor-responding to the empty sequence ε is the unique root nodeof (V,C) labelled with .. According to the second case, weobtain the node v′ that corresponds to a sequence wm if wetake v′ to be the successor of v (the node reached after w)whose label matches m under the following condition:

There has to be a substitution ϑ ∈ subst((v, v′)) which,when composed with the substitution subst(w) applied sofar to obtain the messages in w1 to w|w| from the respectivenodes in EN , will yield m if applied to mesg(v′). This isexpressed by mesg(v′) · subst(w)ϑ = m. In other words,there is at least one combined (and non-contradictory) vari-able substitution that will make the node labels along thepath mesg−1(wm) yield wm if it is applied to them (con-catenating substitutions is performed in a standard fashion).Thereby, the following inductive definition can be used to de-rive the substitution subst(w) for an entire word w:

w = ε : subst(w) = 〈〉

w = w′m : subst(w) =

subst(w′) · unifier(mesg(mesg−1(wm)),m)

where · is a concatenation operator for lists and unifier(·, ·)returns the most general unifier for two terms (in a standardfashion). Thus, subst(w) can be obtained by recursively ap-pending the unifying substitution of the message label of eachnode encountered on the path w to the overall substitution.With all this, we are able to compute gw(w

′) as follows:

gw(w′) =

| ∪Hi=1 M

ic|−1

if ∃v ∈ out(mesg−1(w)).mesg(v) =?∏

i

(

∑

e∈pred(ww′,i) S(e))

else(4)

which distinguishes between two cases: if the path to nodemesg−1(w) whose labels match w (and which is unique, be-cause the labels of sibling nodes in the EN never unify) ends

7For convenience, let C (KB) be the set of nodes within the sub-networkof expectation network EN where the edge set is reduced to those edgeswhose conditions are satisfied under KB .


in a “?” label, the probability of a w′ is simply one over thesize of all words with length up to the communication hori-zon H (hence its name). This is because the semantics of “?”nodes is “don’t know”, so that all possible conclusions to ware uniformly distributed. Note that this case actually onlyoccurs when new paths are generated and it is not knownwhere they will lead, and also that if an outgoing link of anode points to a node with label “?”, then this node will haveno other outgoing links.

In the second case, i.e. if there is no “?” label on the pathp from mesg−1(w) to mesg−1(ww′), then the probability ofw′ is the product of weights S(e) of all edges e on p. Thereby,S(e) is just a generalized notation for expectability or norma-tive force depending on the typed edge, i.e. S(e) = Exp(e)for e ∈ C. The sum of these S-values is computed for all in-going edges pred(ww′, i) of the node that represents the theith element of w′, formally defined as

∀w ∈M∗c .pred(w, i) =

in(mesg−1(w1 · · ·wi))

if mesg−1(w1 · · ·wi) 6= ⊥

∅ else(5)

3. Communication Systems

A communication system can be seen as a description ofthe social dynamics of a multiagent system. The two mainpurposes of a CS are i) to capture the social expectations (rep-resented as an EN) in the current state of a multiagent systemunder observation, and ii) to capture changes to these expecta-tion structures. Whereas the EN models the current meaningof communicative action sequences (i.e., their expected, gen-eralized continuations in a certain context of previous mes-sage utterances), the CS models the way the EN is build up,and, if necessary, adapted according to new statistical obser-vations. As already mentioned in section 1, in contrast toagents who reason about expectations (such as InFFrA agents[18]), a CS need not necessarily be an active agent who takesaction in the MAS itself.

Describing how communication systems work should in-volve (at least) clarifying:

• which communicative actions to select for inclusion inan EN,

• where to add them and with which expectability (in par-ticular, when to consider them as “non-continuations”that directly follow “.”),

• when to delete existing nodes and edges (e.g. to “forget”obsolete structures), and how to ensure integrity con-straints regarding the remaining EN.

A formal framework for specifying the details of the above isgiven by the following, very general, definition:

Definition 2. A communication system at time t is a structureCSt = (L,M, f,$t, κ)

where

• L, M are the formal languages used for logical expres-sions and messages (according to table 1),

• f : EN (L,M) ×Mc → EN (L,M) is the expectationstructures update function that transforms any expecta-tion network EN to a new network upon experience ofa message m ∈Mc,

• $t = m0m1...mt ∈ M∗c is the list of all messages

observed until time t. The subindexes of the mi imposea linear order on the messages corresponding to thetimes they have been observed8.

• κ : 2L×Mc → 2L is a knowledge base update functionthat transforms knowledge base contents after a messageaccordingly,

and EN (L,M) is the set of all possible expectation networksover L andM. The intuition is that a communication systemcan be characterized by how it would update a given knowl-edge base and an existing expectation network upon newlyobserved messages m ∈ Mc. The EN within CSt can thusbe computed through the sequential application of the expec-tation structures update function f for each message within$, starting with an empty expectation network (i.e., an ENwhich contains only the start node). $t−1 is called the con-text of the message observed at time t, and IEN (KB , $t)computes the semantics of this message within this context.

This definition of CS is very general, as it does not pre-scribe how the EN is modified by the CS. However, someassumptions are reasonable to make, although not obligatory(see [14] for a concrete EN-/CS-learning algorithm):

• If KB is the current knowledge base, κ(KB ,m) |=KB(m) should hold, so that all facts resulting fromexecution of m are consistent with the result of theκ-function.

• An EN should predict the future of the respective observ-able communication sequences as accurately as possi-ble. Although there is no canonical method a CS shoulduse to construct and update ENs, we propose the fol-lowing very general heuristic: if any message sequencew′ has occurred with frequency Pr(ww′) as a contin-uation of w in the past, and EN ′ is the same as EN ,IEN ′(KB , w)(w′) = Pr(ww′) should be the case, i.e.the expectabilities along a certain path within the expec-tation tree shall reflect the frequencies with which therespective message sequences have been occurred.

8For simplicity, we assume a discrete time scale with t ∈ N, and that nopair of messages can be uttered at the same time.


In addition to these basic assumptions, we propose the fol-lowing functionality a CS shall provide to be of practical use:

3.1 Message Filtering and Syntax Recognition.

Depending on its goals and the application domain, theCS as an autonomous observer might not be interested in allobservable messages. Since ENs may not provide for a pri-ori expectations, the discarding of such “uninteresting” mes-sages can only take place after the semantics (i.e., the ex-pected outcome) of the respective messages has already beenderived from previous observation. Because discarding mes-sages bears the risk that these messages become interestingafterwards, as a rule of thumb, message filtering should bereduced to a minimum. More particularly, messages shouldonly be filtered out in cases of more or less settled expecta-tions. Paying attention to every message and filtering unin-teresting or obsolete information later by means of structurereweighting and filtering (cf. below) is presumably the morerobust approach.

3.2 Structure Expansion and Generalization.

Structure expansion is concerned with the growth of anEN in case a message sequence is observed which has no se-mantics defined by this EN yet. In order to do so, we couldstart with an empty EN and incrementally add a node foreach newly observed message. But this would be not smartenough, because it does not take advantage of generalizablemessage sequences, i.e. different sequences that have approx-imately the same meaning. In general, such a generalizationrequires a relation which comprises “similar” sequences. Theproperties of this relation of course depends on domain- andobserver-specific factors. A quite simple way of generalizingis to group messages which can be unified syntactically, usingthe message patterns introduced in table 1.

In theory, the expansion of the EN would never be neces-sary if we could a-priori generate a complete EN, i.e. an ENwhich contains dedicated paths for all possible message se-quences. In this case, the CS would just have to keep track ofthe perceived messages using $ and to identify this sequencewithin the EN to derive its semantics, with no need for f . Forobvious reasons, such a complete EN cannot be constructedin practice.

3.3 Pruning the EN.

Several further methods of EN processing can be con-ceived of that aid in keeping the computation of (approx-imate) EN semantics tractable. This can be achieved bycontinuously modifying expectation structures using certainmeta-rules, for example:

1. “fading out” old observations by levelling their edgeweights;

2. replacing large sets of sibling edges with (approxi-mately) uniformly distributed expectabilities with singleedges leading to “?” nodes;

3. removal of “?”s that are not leafs. Such nodes can occuras outcome of the previous measure.

4. keeping the EN depth constant through removal of oneold node for each new node to save space and removeobsolete structures (e.g. using the communicationhorizon H as maximum EN depth);

5. removal of edges with very low expectabilities. In casethis results in cut-off branches, these have to be con-nected with the start node subsequently.

Since these modifications are highly application-dependent,we don’t provide exact criteria for their practical applicationhere.

4. Applications and Extensions

The modelling of social structures on the basis of ex-pectation networks and communication systems allows fornovel approaches to a variety of challenging issues in mul-tiagent system technology. In the following, we review threeof these issues, namely, (i) identification of ontologies forinter-agent communication and – closely related – the find-ing of verifiable and flexible semantics for agent commu-nication languages; (ii) mirror holons as a new model forholonic theories of agency and software engineering meth-ods based on expectation-oriented modelling and analysis ofmultiagent systems; (iii) the agent-level social reasoning ar-chitecture InFFra.

4.1 Social Ontologies

In Distributed Artificial Intelligence (DAI), an ontology isa set of definitions as a means to provide a common groundin the conceptual description of a domain for communica-tion purposes. Ontologies are usually represented as graph-ical hierarchies or networks of concepts, topics or classes,and either top-down imposed on the agents or set up bottom-up by means of ontology negotiation. In a similar way, ex-pectation networks are descriptions of the social world inwhich the agents exist. But ENs do not only describe so-cial (i.e. communication) structures, but indirectly also thecommunication-external environment the message content in-forms about. Thus, communication systems can be used,in principle, for an incremental collection of ontological de-scriptions from different autonomous sources, resulting instochastically weighted, possibly conflicting, competitive andrevisable propositions about environmental objects [21]. Thecrucial difference to traditional mechanisms is that such a so-cial ontology (also called Open Ontology [21, 20, 19]) in-cludes probabilistic expectations about how a certain domain


object or event is described, constructed and used socially bymultiple autonomous knowledge sources (which do not nec-essarily have consistent views and which are not necessar-ily cooperative, reliable or trustable), i.e. in the course ofcommunication processes using communicative means likeapproval, contradiction, conflict, argumentation or specifica-tion. This opposes somewhat the traditional understanding ofontologies, where an ontology provides an a priori groundingfor communication only instead of taking into considerationthat ontologies are evolving, possibly inconsistent results ofcommunication also, and it makes this social approach appearparticularly suitable for the Semantic Web, open multiagentsystems, Peer2Peer systems and communities of practice witha highly dynamic environment, where a commonly agreed,homogenous domain perception of distributed, autonomousknowledge sources cannot be assumed. Also, it is appropri-ate whenever distributed domain descriptions are influencedby individual preferences such that a consensus cannot beachieved (think, e.g., about “politically” biased resource de-scriptions in the context of Peer2Peer systems or the SemanticWeb [22]). In the following, we’ll sketch two approaches forextracting social ontologies from expectation networks (formore information about the underlying approach called So-cial Reification see [21, 20, 19]).

4.1.1 Extraction of Speech Act Types.

The current version of FIPA-ACL [23] provides an exten-sible set of speech-act performative types with semantics de-fined in a mentalistic fashion. In our approach, we can imag-ine a special CS variant as a MAS component (e.g., a so-called multiagent system mirror [2, 1], cf. 4.2) that providesthe agents with a set of performatives without any predefinedsemantics and wait for the semantics of such “blank” perfor-matives to emerge. To become predictable, it is rational foran agent to stick to the meaning (i.e., the consequences) ofperformatives, at least to a certain extent. This meaning hasbeen previously (more or less arbitrarily) “suggested” for acertain performative by some agent performing demonstra-tive actions after uttering it.

Of course, a single agent is usually not able to define aprecise and stable public meaning for these performatives,but at least the intentional attitude associated with the re-spective performative needs to become common ground forcommunication to facilitate a non-nonsensical, non-entropicdiscourse [11, 22, 14]. A particular performative usually ap-pears at multiple nodes within the EN, with different conse-quences at each position, depending on context (especially onthe preceding path), message content and involved sender andreceiver. To build up an ontology consisting of performativetypes, we have to continually identify and combine the oc-currences of a certain performative within the current EN toobtain a general meaning for this performative (i.e., a “type”meaning). Afterwards, we can communicate this meaning toall agents using some technical facility within the multiagentsystem, like a MAS mirror or an “ACL semantics server”. Of

course, such a facility cannot impose meaning in a normativeway as the agents are still free to use or ignore public mean-ing as they like, but it can help to spread language data likea dictionary or a grammar does for natural languages. Thecriteria for the identification and extraction of performativemeaning from ENs are basically the same as the criteria weproposed in 3 for the generalization over message sequences.

4.1.2 Extraction of Domain Descriptions.

While a set of emergent speech act types constitutes a so-cial ontology for communication events, classical ontologiesprovide a description of an application domain. To obtain asocial version of this sort of ontology from an EN, two differ-ent approaches appear to be reasonable: (1) Inclusion of envi-ronment events within the EN and (2) probabilistic weightingof assertions. The former approach, which is introduced in[22, 14], treats “physical” events basically as utterances. Sim-ilar to the communicative reflection of agent actions by meansof do, a special performative happen(event)would allow ENnodes that reflect events occurring in the environment. Theseevents will be put in the EN either by a special CS which isable to perceive the agents’ common environment, or by theagents themselves as a communicative reflection of their ownperceptions. A subset of event is assumed to denote eventswith consensual semantics (think of physical laws), i.e., theagents are not free to perform an arbitrary course of actionafter such an event has occurred, whereas the remainder ofevent consists of event tags with open semantics that has tobe derived empirically from communications observation justas for “normal” utterances. If such an event appears for thefirst time, the CS does not know its meaning in terms of itsconsequences. Its meaning has thus to be derived a-posteriorifrom the communicational reflection of how the agents re-act to its occurrence. In contrast, approach (2), which weproposed for the agent-based competitive rating of web re-sources [22], exploits the propositional attitude of utterances.The idea is to interpret certain terms within LogicalExpr asdomain descriptions and to weight these descriptions accord-ing to the amount of consent/dissent (using predefined perfor-matives like Assert and Deny). The weighted propositionsare collected within a knowledge base (e.g., KB as definedbefore) and are communicated to the agents in the same wayas the emergent speech act types before. Unlike approach (1),ontologies are constructed “by description” not “by doing” inthis way. The advantage of approach (1) lies in its seamlessintegration of “physical” events into the EN, whereas (2) isprobably more easy to apply in practice.

4.2 The Mirror concept

In [1, 12], we have introduced the social system mirrorarchitecture for open MAS. The main component of thisarchitecture is a so-called social system mirror (or “mirror”for short), an intelligent system component containing a CSwhich continually observes communications, empirically


derives emergent expectation structures (represented asan ENs, which might also contain normative structures,which are given by the designer instead of being learnedfrom statistical observations) from these observations, and“reflects” these structures back to the agents. In addition tothe recording of empirical expectation structures describedin this work, the mirror is a goal-directed agent within theMAS. Its goals are to influence agent behavior by meansof system-wide propagation of social structures and normsto achieve quicker structure evolution (catalysis) and highercoherence of social structures without restricting agentautonomy, and the provision of a representation of a dynamiccommunication system for the MAS designer.

Technically, a mirror can be thought of as a knowledgebase which derives system-level expectation structures fromcommunications and makes them available as information forboth the participating agents and the designer of the multia-gent system. It can be implemented as an actual softwarecomponent (e.g. a middle agent), but it might as well beused without a physical implementation as a purely theoreti-cal concept. The mirror has three major purposes:

1. monitoring agent communication processes,

2. deriving emergent system-level expectation structuresfrom these observations, and calculating the deviationof actual agent behaviour from normative expectations,

3. making expectation structures visible for the agents andthe designer (the so-called reflection effect of the mir-ror).

The published structures are not necessarily the emergentstructures derived from the system observation - the mir-ror can be pre-structured, i.e. used to “reflect” manually de-signed (“manipulated”), non-emergent expectation structuresas well. In both cases, the agents can query the mirror verymuch like a database and actively use the otherwise latent so-cial structures which are made explicit to them through themirror as a guide for their decision making and their interac-tion behaviour.For example, agents can instantiate themselves in social pro-grams which seem to be useful to them, or refrain from acertain behaviour if the mirror tells them that it would violatea norm. If the agents make (further) use of reflected struc-tures, the structures become stronger, otherwise weaker (thedegree of these changes depending on the respective norma-tivity). Thus, the mirror reflects a model of a social system tothe agents and by means of this influences the agents – verymuch like mass media do in human society. Conversely, itcontinualy observes the actual multiagent system and adoptsthe expectation structures in its database in accordance withthe agent interactions. In doing so, the mirror never restrictsthe autonomy of the agents. Its influence is solely by meansof information, not through the exertion of control.

4.2.1 Expectation-Oriented Software Development

As it has been recognized that due to new requirementsarising from the complex and distributed nature of modernsoftware systems the modularity and flexibility provided byobject orientation is often inadequate and that there is a needfor encapsulation of robust functionality at the level of soft-ware components, agent-oriented approaches are expected tooffer interesting prospectives in this respect, because they in-troduce interaction and autonomy as the primary abstractionsthe developer deals with.

However, although interaction among autonomous agentsoffers great flexibility, it also brings with it contingencies inbehavior. In the most general case, neither peer agents nor theMAS designer can “read the mind” of an autonomous agent,let alone change it. While the usual strategy to cope with thisproblem is to restrict oneself to closed systems, this meansloosing the power of autonomous decentralized control infavour of a top-down imposition of social regulation to ensurepredictable behavior. The EXPAND method (Expectation-oriented Analysis and Design) [12] follows a different ap-proach. EXPAND is based on expectation networks as a pri-mary modelling abstraction which both system designer andagents use to manage the social level of their activities. Thisnovel abstraction level is made available to them through aspecial version of the social system mirror (4.2), i.e., a spe-cial CS, very similar to a CASE tool. For the designer, thismirror acts as an interface he uses to propagate his desiredexpectations regarding agent interaction to the agents and asa means for monitoring runtime agent activity and deviancefrom expected behavior. For agents, this mirror representsa valuable “system resource” they can use to reduce contin-gency about each other’s behavior. EXPAND also describesan evolutionary process for MAS development which con-sists of multiples cycles: the modelling of the system level,the derivation of appropriate expectation structures, the mon-itoring of expectation structure evolution and the refinementof expectation structures given the observations made in thesystem.Within the EXPAND process, a social system mirror is usedin a cyclic process of two alternating mirror operations whichare the core of our design process:

1. It makes the system-level expectations the softwaredesigner has derived from her design goals explicit andknown to the agents (i.e. it makes them the “expecta-tions of expectation” described in the previous section).

2. It monitors the system-level expectation structureswhich emerge from the communications of the runningmultiagent system.

Together, both operations allow the designer to control andinfluence if and how her design specifications are realizedand adopted by the agents. This approach borrows its un-derlying concept from the “evolutionary software engineer-


ing method”, and will now be sketched from the designers’point of view.

- Phase I: Modelling the system level

In the first phase of the process, the software designermodels the system level of the multiagent system accordingto her design goals in the form of design specifications whichfocus on “social behaviour” (i.e. desired courses of agent in-teraction) and “social functionality” (i.e. functionality whichis achieved as a “product” of agent interaction, such as coop-erative problem solving) in the widest sense (we don’t takeinto account “second-order” design goals like high executionspeed or low memory consumption). For this task, the usualspecification methods and formalisms can be used, e.g. thespecification of desired environment states, constraints, socialplans etc. In addition or as a replacement, the specificationcan be done in terms of system-level expectation structures,like social programs.

- Phase II: Deriving appropriate expectation structures

In the second phase, the designer models and derivessystem-level expectation structures from the design specifi-cations and stores them in the social system mirror. If thedesign specifications from phase I are not already expecta-tion structures (e.g. they might be given as rules of the form“Agent X must never do Y”), they have to be transformed ap-propriately. While social behaviour specifications are expec-tation structures per se, social functionalities (for instance:“Agents in the system must work out a solution for problemX together”) possibly need to be transformed, most likely intosocial programs. Sometimes a full equivalent transformationwill not be feasible. In this case, the designer models expec-tation structures which cover as much design requirements aspossible.

System-level specifications can be modelled as adaptableor normative expectations. The former can be used for theestablishing of hints for the agents which are able to adaptduring the structure evolution, the latter for the transforma-tion of constraints and other “hard” design requirements intoexpectations9.

Figure 2 shows the spectrum of system-level specificationsand expectation structures that result from this phase of theanalysis and design process.

- Phase III: Monitoring structure evolution

After the designer has finished the expectation modelling,she makes them visible for the agents via the social system

9It should be kept in mind that a norm derived from a constraint doesnot force the agents to behave conforming to the rule, since it is “only” anexpectation. In some cases, where expectations seem too “soft” a modellingtool, direct programming of agent goals and behaviour might be necessary.This is certainly the case for agents that are built by the same designer (whichis not the default case in open systems), and that would be underspecified iftheir reasoning capabilities are not defined more rigidly.

Figure 2. System-level specification

Figure 3. Evolution of expectation structures


mirror and puts the multiagent system into operation (if it isnot already running). In the third phase of the design andanalysis process, it is up to the designer to observe and anal-yse the evolution of expectation structures which becomesvisible to her through the mirror (Figure 3). In particular,she has to pay attention to the relationship of the continu-ously adapted system-level expectation structures and her de-sign specifications from phase I, which means that she anal-yses the expectation structures with regard to the fulfilmentof norms established by the designer and the achievement ofthe desired social functionality. Because the mirror is only in-tended to show expectation structures, it could be necessary tosupport the mirror with a software for the (semi-)automatical“re-translation” of expectation structures into more abstractdesign specifications like social goals.As long as the expectations structures develop in a positiveway (i.e. they match the design goals) or no emergent struc-tures can be identified that deserve being made explicit to im-prove system performance, the designer does not intervene.Otherwise she proceeds with phase IV.

- Phase IV: Refinement of expectation structures

In the last phase, the designer uses her knowledge aboutthe positive or negative emergent properties of the multia-gent system to improve the system-level expectation struc-tures. Usually, this is achieved by removing “bad” expecta-tion structures from the mirror database, and, if necessary,the introduction of new expectation structures as describedat phases I and II. In addition, expectation structures whichhave proved to be useful can be actively supported by e.g. in-creasing their expectation strength and/or their normativity.The process proceeds with phase III until all design goals areachieved or no further improvement seems probable.

For lack of space, we have to refer the interested reader to[12] for further details.

4.2.2 Mirror Holons: Multi-Stage Observation, Reflec-tion and Enactment of Communication Structures

While a social system mirror only models a single com-munication system, and, except for the propagation of expec-tations, does not take action itself, the successor architectureHoloMAS [2] is able to model multiple communication sys-tems at the same time through multiple mirror holons in orderto model large, heterogenous systems. In addition, a mirrorholon can take action himself by means of the execution of so-cial programs which are generated from emergent expectationstructures. “Ordinary agents” (and other mirror holons) canoptionally be involved in this execution process as effectors,which realize holon commands within their physical or vir-tual application domain (unless they deny the respective com-mand). In any case they can influence the social programswithin a mirror holon through the irritation of expectationstructures by means of communication. A mirror holon thusrepresents and (at least to some extent) replaces the function-ality of the ordinary agents that contribute to the emergence

of the respective expectation structures, but it does not disre-gard the autonomy of his adjoint actors. Another differencebetween mirror holons and traditional agent holons is that amirror holon does not represent or contain groups of agents,but instead a certain functionality which is identified in formof regularities in the observed communications. This func-tionality is extracted and continually adopted from dynamicexpectation structures regarding criteria like consistency, co-herence and stability, corresponding to the criteria sociolog-ical systems theory ascribes to social programs [13]. Mirrorholons pave the way for applications in which agent auton-omy should not (or cannot) be restricted on the one hand,while reliable, time-critical system behavior is desired. Theycan also be used as representatives for entire communicationsystems (e.g., virtual organizations) that behave smoothly to-wards third parties whenever the communication system itselflacks coherence due to, for example, inner conflicts.

4.3 Social reasoning with InFFrA

The Interaction Frame and Framing Architecture InFFrA[18] is a social reasoning architecture in which so-calledinteraction frames are used to represent patterns of socialinteraction and strategically employed by socially intelli-gent agents to guide their interaction and communication be-haviour. This is achieved by agents deriving models of framesfrom observation of encounters and applying the most appro-priate patterns in future interactions (this process is calledframing). The concepts of frame and framing are basedon Erving Goffman’s micro-social analyses of everyday life[16].

In the conceptual (abstract) architecture, a frame is a datastructure that contains information about

• the possible courses of interaction (so-called trajecto-ries) characteristic for a particular frame,

• roles and relationships between the parties involved inan interaction of this class,

• contexts within which the interaction may take place,and

• beliefs, i.e. epistemic states of the interacting parties.

In computational terms, the trajectory model is usually a rep-resentation of a set of admissible message and action se-quences, while the latter three elements can be collapsed intoa single set of logical constraints which then have to be veri-fied using the agent’s internal belief state (usually representedby the contents of a knowledge base).

InFFrA uses the following data structures for reasoningwith frames:

• the active frame, the unique frame currently activated todescribe the expected course of events,

• the perceived frame, an interpretation of the currentlyobserved state of affairs,


deviate

adjustmentenactmentaction

perception interpretation and matching

repositoryframe

framing decision

perceived frame activated frame difference model

trial instantiate

comply

assessment

Figure 4. Overview of the framing process

• the difference model that contains the differences bet-ween perceived frame and active frame,

• the trial frame, used when alternatives to the currentframe are sought for,

• and the frame repository, in which the agent locallystores its frame knowledge.

Using these data structures, the InFFrA framing cycle (asshown in a simplified fashion in figure 4) consists of thefolowing reasoning steps:

1. Interpretation & Matching: Update the perceived frameand compare it with the active frame.

2. Assessment: Assess the usability of the active frame interms of

(i) adequacy (compliance with the conditions of theactive frame),

(ii) validity (the degree to which the active frame’s tra-jectory matches the perceived encounter) and

(iii) desirability (depending on whether the implica-tions of the frame correspond to the agent’s privategoals).

3. Framing decision: If the active frame seems appropriate,continue with 5. Else, proceed with 4 to find suitablealternatives.

4. Adjustment/Re-framing: Search the frame repository forbetter frames. “Mock-activate” them as trial frames iter-atively and go back to 1; if no suitable frame is found,end the encounter.

5. Enactment: Derive action decisions by applying the ac-tive frame.

From a CS perspective, InFFrA is nothing but an agent-centric interpretation of our concepts. Instead of defininga general observer of communication, InFFrA exclusivelydeals with agent observers, and rather than observing generalcommunication processes, InFFrA agents only observe “face-to-face” interaction processes (mostly those they are person-ally involved in).

In other words, interaction frames in InFFrA are micro-models of communicative expectations that encode knowl-edge about communication processes from the standpoint ofan agent observer. They contain information about the surfacestructure of conversations together with logical conditions inthe same way as general ENs but as a collection of “manage-able chunks” of dialogue traces rather than a huge network ofglobal correlations. This allows for handling expectations ina computationally tractable fashion and with this the conceptof frames makes the CS approach accessible for the design ofsocial reasoning methods.

What makes InFFrA an interesting extension of the gen-eral CS framework is the fact that agents actually have tostrategically decide which utterances to generate in accor-dance with their current model of the CS to achieve theirgoals. In [11] we have suggested entropy-based methodsfor reconciling the utility-based preferences of InFFrA agentswith long-term considerations about the effect of their deci-sions on the overall CS in decision-theoretic terms. There, theinteresting question was how agents can achieve a trade-offbetween their current pursuit for high utility and the modifi-cations to the CS that will result from their current decision.

In [17], we have shown how InFFrA frames can be for-mally converted to general expectation networks. Of course,some problems occur when attempting to transform generalexpectation networks to interaction frames, since the full ex-pressiveness of expectation networks is not available in theformal model of InFFrA for reasons of practicability.

Based on the empirical semantics approach, a formalmodel of InFFrA has been developed for a particular, concreteinstance of the abstract architecture [24]. This model calledm2InFFrA is based on viewing frames as policy abstractions inthe sense of Markov Decision Processes (MDPs) [25].

More specifically, in m2InFFrA, a frame describes a set oftwo-party, discrete, turn-taking interaction encounters whichcan be thought of as conversations between two agents. Thetrajectory is given in the form of a sequence of message pat-terns that defines the surface structure of the encounters de-scribed by the frame, while a list of substitutions capturesthe values of variables in the trajectory in previously expe-rienced interactions. Each substitution also corresponds to aset of logical conditions that were required for and/or pre-cipitated by execution of the trajectory in the respective en-counter. Finally, trajectory occurrence and substitution oc-currence counters record the frequency with which the framehas occurred in the past.

In terms of MDP theory, frames can be seen as “macro”-actions can be invoked as MDP decisions and then executeduntil certain conditions apply (typically, until they are consid-ered undesirable according to some heuristics, until their con-text conditions are no more fulfilled, or until the other agenthas uttered a message that does not match the trajectory ofthe frame).

This allows us to combine the principles of InFFrA with theoptions framework [26] for hierarchical reinforcement learn-


frame level

action level

Frame

context

trajectories

beliefs

roles & relationships

+1

−1

0.705 0.655 0.611 0.388

0.762 0.611

0.9180.812 0.868

0.534

F2

F8

F1

F4

F6 F7 F9

F3

F10F5

+1

−1

0.455 0.686 0.874

0.512

0.377

0.112

0.245 0.621

0.766

framing

framing decisions + long−term payoffs = framing utility

in−frame action decisions + immediate payoffs = action utility

Figure 5. Frame-based hierarchical view ofcommunication MDPs.

ing (RL; for an introduction, see e.g. [27]). This frameworkis based on augmenting the sets of admissible “primitive” ac-tions by sets of so-called “options”, where an option is a tripleconsisting of an input set of states, a policy (that is admissi-ble once a state in the input set is entered), and a terminationcondition that determines when an option will be exited anda new one has to be selected.

Combining the options framework with m2InFFrA, we ob-tain a two-level ocial reasoning and learning view as shown infigure 5. According to this view, what we obtain is a two-levelMDP:

• At the frame level, the agent chooses a frame as a com-munication policy that may be used over an extendedperiod of time, depending on whether it can be success-fully completed. We employ Q-learning [28] to learn along-term “framing” utility function that enables us toderive optimal strategies for frame selection from ex-perience. The states of this “upper” MDP are abstractrepresentations of the goal of a conversation.

• At the action level, we have to determine which con-crete instance of a frame to select so as to optimise theoutcome of a conversation. Remembering that framescontain message patterns that may allow for additionalchoices (e.g. regarding which argument to use in anargumentation dialogue), we use adversarial search tomaximise expected utility considering the other’s poten-tial reactions.

This not only allows for combining the theory of empiricalcommunication semantics and communication systems withthe decision-theoretic principles of MDP theory, it also nicelyillustrates that we can apply hierarchical methods to cope

with the complexity of the usually huge spectrum of possiblecommunicative behaviour. Finally, and maybe most impor-tantly, it allows for the construction of agents that are ableto adapt to a particular communication system and to use itaccording to their own needs.

In future work, we are going to investigate how CS thathave been observed by global entities can be used by agentsin the system to improve their interaction behaviour.

5. Conclusion

This paper presented communication systems as a unifiedmodel for socially intelligent systems based on recording andtransforming communicative expectations. We presented for-malisms for describing expectations in terms of expectationnetworks, the formal semantics of these networks, and a gen-eral framework for transforming them with incoming obser-vation. Then, a number of important applications of CS werediscussed, some of which have already been addressed by ourpast research, while others are currently being worked on.

While a lot of work still lies ahead, we strongly believethat, by virtue of their general character, CS have thepotential of becoming a unified model for speaking aboutmethods and applications relevant to the improvement ofmultiagent systems using sociological theories [29]. Also,we hope that they can contribute to bringing key insights ofthis new research direction to the attention of the mainstreamDAI audience, as they put emphasis on certain aspects ofMAS that are often neglected in traditional approaches.

AcknowledgementsThis work has partially been supported by DeutscheForschungsgemeinschaft under contracts no. BR 609/11-2and MA 759/4-2.

6. References

[1] K. F. Lorentzen and M. Nickles. Ordnung aus Cha-os – Prolegomena zu einer Luhmann’schen Modellie-rung deentropisierender Strukturbildung in Multiagen-tensystemen. In T. Kron, editor, Luhmann modelliert.Ansatze zur Simulation von Kommunikationssystemen.Leske & Budrich, 2001.

[2] Matthias Nickles and Gerhard Weiss. Multiagent Sys-tems without Agents – Mirror-Holons for the Compila-tion and Enactment of Communication Structures. InK. Fischer and M. Florian, editors, Socionics: Its Con-tributions to the Scalability of Complex Social Systems,LNCS. Springer-Verlag, Berlin, Germany, 2004. Toappear.

[3] P. R. Cohen and H. J. Levesque. Performatives in a Ra-tionally Based Speech Act Theory. In Proceedings ofthe 28th Annual Meeting of the Association for Compu-tational Linguistics, pages 79–88, 1990.

[4] P. R. Cohen and H. J. Levesque. Communicativeactions for artificial agents. In Proceedings of the


First International Conference on Multi-Agent Systems(ICMAS-95), pages 65–72, 1995.

[5] Y. Labrou and T. Finin. Semantics and conversationsfor an agent communication language. In Proceedingsof the 15th International Joint Conference on ArtificialIntelligence (IJCAI-97), pages 584–591, 1997.

[6] M. P. Singh. A semantics for speech acts. Annals ofMathematics and Artificial Intelligence, 8(1–2):47–71,1993.

[7] Frank Guerin and Jeremy Pitt. Denotational Semanticsfor Agent Communication Languages. In Proceedingsof the Fifth International Conference on AutonomousAgents (Agents-01), pages 497–504. ACM Press, 2001.

[8] J. Pitt and A. Mamdani. A protocol-based semanticsfor an agent communication language. In Proceedingsof the 16th International Joint Conference on ArtificialIntelligence (IJCAI-99), 1999.

[9] M.P. Singh. A social semantics for agent communica-tion languages. In Proceedings of the IJCAI Workshopon Agent Communication Languages, 2000.

[10] M. Nickles and G. Weiss. Empirical Semantics ofAgent Communication in Open Systems. In Proceed-ings of the Second International Workshop on Chal-lenges in Open Agent Environments at AAMAS-2003,2003.

[11] M. Rovatsos, M. Nickles, and G. Weiß. Interactionis Meaning: A New Model for Communication inOpen Systems. In J. S. Rosenschein, T. Sandholm,M. Wooldridge, and M. Yokoo, editors, Proceedingsof the Second International Joint Conference on Au-tonomous Agents and Multiagent Systems (AAMAS-03), Melbourne, Australia, 2003.

[12] W. Brauer, M. Nickles, M. Rovatsos, G. Weiß, and K. F.Lorentzen. Expectation-Oriented Analysis and Design.In Proceedings of the 2nd Workshop on Agent-OrientedSoftware Engineering (AOSE-2001) at the AutonomousAgents 2001 Conference, volume 2222 of LNAI, Mon-treal, Canada, May 29 2001. Springer-Verlag, Berlin.

[13] N. Luhmann. Social Systems. Stanford UniversityPress, Palo Alto, CA, 1995. translated by J. Bednarz,Jr. and D. Baecker.

[14] M. Nickles, M. Rovatsos, and G. Weiss. Empirical-Rational Semantics of Agent Communication. In Pro-ceedings of the Third International Joint Conference onAutonomous Agents and Multiagent Systems (AAMAS-04), New York, NY, 2004.

[15] G. H. Mead. Mind, Self, and Society. University ofChicago Press, Chicago, IL, 1934.

[16] E. Goffman. Frame Analysis: An Essay on the Or-ganisation of Experience. Harper and Row, New York,NY, 1974. Reprinted 1990 by Northeastern UniversityPress.

[17] M. Nickles and M. Rovatsos. Communication Sys-tems: A Unified Model of Socially Intelligent Systems.

In K. Fischer and M. Florian, editors, Socionics: ItsContributions to the Scalability of Complex Social Sys-tems, LNCS. Springer-Verlag, Berlin, Germany, 2004.To appear.

[18] M. Rovatsos, G. Weiß, and M. Wolf. An Approach tothe Analysis and Design of Multiagent Systems basedon Interaction Frames. In Maria Gini, Toru Ishida,Cristiano Castelfranchi, and W. Lewis Johnson, edi-tors, Proceedings of the First International Joint Con-ference on Autonomous Agents and Multiagent Systems(AAMAS-02), Bologna, Italy, 2002. ACM Press.

[19] T. Froehner, M. Nickles, and G. Weiß. Open Ontolo-gies – The Need for Modeling Heterogeneous Knowl-edge. In Proceedings of The 2004 International Con-ference on Information and Knowledge Engineering(IKE 2004), 2004.

[20] T. Froehner, M. Nickles, and G. Weiß. Towards Mod-eling the Social Layer of Emergent Knowledge UsingOpen Ontologies. In Proceedings of The ECAI 2004Workshop on Agent-Mediated Knowledge Management(AMKM-04), Valencia, Spain, 2004.

[21] M. Nickles and T. Froehner. Social Reification forthe Semantic Web. Research Report FKI-24x-04,AI/Cognition Group, Department of Informatics, Tech-nical University Munich, 2004.

[22] M. Nickles and G. Weiß. A Framework for the So-cial Description of Resources in Open Environments.In Proceedings of the 7th International Workshop onCooperative Information Agents (CIA-2003), volume2782 of Lecture Notes in Computer Science, Berlin,Germany, 2003. Springer-Verlag.

[23] FIPA. FIPA (Foundation for Intelligent Agents),http://www.fipa.org, 1999.

[24] F. Fischer. Frame-Based Learning and Generalisationfor Multiagent Communication. Diploma Thesis, De-partment of Informatics, Technical University Munich,Munich, Germany, December 2003.

[25] M. L. Puterman. Markov Decision Problems. John Wi-ley & Sons, New York City, NY, 1994.

[26] Andrew Barto and Sridhar Mahadevan. Recent Ad-vances in Hierarchical Reinforcement Learning. Spe-cial Issue on Reinforcement Learning, Discrete EventSystems, 13:41–77, 2003.

[27] R.S. Sutton and A.G. Barto. Reinforcement Learning.An Introduction. MIT Press/A Bradford Book, Cam-bridge, MA, 1998.

[28] C.J.C.H. Watkins and P. Dayan. Q-learning. MachineLearning, 8:279–292, 1992.

[29] Th. Malsch. Naming the Unnamable: Socionics orthe Sociological Turn of/to Distributed Artificial Intel-ligence. Autonomous Agents and Multi-Agent Systems,4(3):155–186, 2001.


Matthias Nickles studied Computer Sci-ences (minor Psychology) at the Univer-sity of Munich, where he graduated with aDiploma degree. After graduation, he hasbeen a scientific software developer and a re-search assistant for a research project in The-oretical Linguistics, and worked as a devel-oper of web software for a commercial in-ternet access provider. Currently, he finishes

his PhD thesis on agent communication at the Departmentof Informatics, Technical University Munich, where he isa research associate and member of the Artificial Intelli-gence/Cognition group. His research interests include agentcommunication languages, multiagent systems, formal on-tologies and knowledge modeling, functional programminglanguages, and computational linguistics.

Michael Rovatsos is a research associateand PhD student at the Department of Infor-matics, Technical University Munich, Ger-many. His research interests include mod-els of interaction in multiagent systems, mul-tiagent learning, agent communication lan-guages, and agent-oriented software engi-neering. Michael received his Diploma in

Computer Science from the University of Saarbruecken in1999, where he also worked for the DFKI multiagent sys-tems group. Before coming to Munich, he worked for Know-botic Systems, a Frankfurt-based AI startup. He is currentlycompleting his PhD thesis on ”Computational InteractionFrames”.

Prof. Dr. Dr. h.c. Wilfried Brauer -Full professor of Informatics since 1971 (firstat the University of Hamburg, since 1985 atTechnical University Munich)- ordinary member of the Bavarian Academyof Sciences and of the Academia Europaea,London- former president of the European Associ-ation for Theoretical Informatics (EATCS);

former vice president of International Federation for Infor-mation Processing (IFIP)- third honorary member (after Konrad Zuse and F.L. Bauer- number 4 is G. Hotz) of Gesellschaft fur Informatik; recip-ient of the Werner Heisenberg Medal of the Alexander vonHumboldt Foundation and of the Felix Hausdorff MemorialAward from the University of Bonn. (Honorary doctoral de-gree from Hamburg)- Research and teaching in theoretical informatics and foun-dations of Artificial Intelligence- particularly interested in interdisciplinary research; closecooperation with industry

Dr. Gerhard Weiß leads the AI/CognitionGroup at the chair for Theoretical Informat-ics and Foundations of Artificial Intelligence(headed by Prof Dr W. Brauer) of Techni-cal University Munich TUM since September1997. His main interests have been in compu-tational intelligent and autonomous systemsin general, and in the foundations and appli-

cation of agent and multiagent technology in particular. Ger-hard Weiß co/authored a number of articles and co/edited tenbooks in his areas of interest. He is the editor-in-chief ofthe international book series on Multiagent Systems, Artifi-cial Societies, and Simulated Organizations, and he serves asan editorial board member of several international journals.Since 2000 he is speaker of the official German interest groupon distributed artificial intelligence within the German Infor-matics association (GI); since 2002 he is a member of thescientific board of the European Network of Excellence forComplex Systems (Exystence); from 1998 until 2001 he wasa member of the management board of the European Networkof Excellence for Agent-based Computing (Agentlink); andsince 1999 he is in the Board of Directors of the InternationalFoundation for Multiagent Systems (IFMAS).

Documents

Towards a Unied Model of Sociality in Multiagent · PDF fileTowards a Unied Model of Sociality in Multiagent Systems ... of CS-based applications are described which illustrate the