Embed Size (px)
Citation preview
University of Calgary
PRISM: University of Calgary's Digital Repository
Graduate Studies The Vault: Electronic Theses and Dissertations
2013-08-09
Multi Agent Protocol Creator (MAPC): Evolving Multi
Agent Communications Protocols
Heard, Jason Peter
Heard, J. P. (2013). Multi Agent Protocol Creator (MAPC): Evolving Multi Agent Communications
Protocols (Unpublished doctoral thesis). University of Calgary, Calgary, AB.
doi:10.11575/PRISM/27021
http://hdl.handle.net/11023/864
doctoral thesis
University of Calgary graduate students retain copyright ownership and moral rights for their
thesis. You may use this material in any way that is permitted by the Copyright Act or through
licensing that has been assigned to the document. For uses that are not allowable under
copyright legislation or licensing, you are required to seek permission.
Downloaded from PRISM: https://prism.ucalgary.ca
THE UNIVERSITY OF CALGARY
Multi Agent Protocol Creator (MAPC):
Evolving Multi Agent Communications Protocols
by
Jason Heard
A DISSERTATION
SUBMITTED TO THE FACULTY OF GRADUATE STUDIES
IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE
DEGREE OF DOCTOR OF PHILOSOPHY
DEPARTMENT OF COMPUTER SCIENCE
CALGARY, ALBERTA
August, 2013
c© Jason Heard 2013
Abstract
Multi agent systems rely on communication between agents, typically regulated by
protocols, to facilitate cooperation and coordination. To create protocols, multi agent
system designers spend an extensive amount of time designing and testing to ensure
that all requirements of the new protocol have been met. To aid the creation of multi
agent protocols, research has shifted from designing low-level protocols, where every
interaction must be outlined, to designing higher-level protocol description languages,
which allow high-level interactions to be described. Though the focus of research
has shifted, the general process of designing new multi agent protocols has remained
consistent and still requires that designers describe actions of agents at various stages
of a protocol. This dissertation describes an alternative process for creating protocols:
an automatic protocol creator that generates a protocol based on a description of the
requirements of the desired protocol.
The Multi Agent Protocol Creator (MAPC) was constructed to evolve protocols
using genetic programming. To evolve protocols, MAPC must first be given a protocol
domain model. This protocol domain model contains all of the information needed to
define the requirements of a new protocol in a declarative manner. It includes infor-
mation to generate potential protocols, try the generated protocols in an appropriate
environment and then evaluate their performance once they have been executed. Tests
show that MAPC is able to evolve protocols in several test protocol domains. Concise,
reasonable protocols, that are tailored to the inputs of the given protocol domain,
similar to those that would potentially be programmed by a human, are produced.
The use of MAPC will enable agent designers to focus on the desired communication
ii
instead of specific protocol details.
iii
Acknowledgements
It is a happy occasion for me to acknowledge the debts, which are both of an intellectual
and a personal nature, that I have incurred in completing this dissertation.
My thanks go to my supervisor, Rob, for all of his help throughout this unexpectedly
long journey. I have especially appreciated his support as I moved at the pace
allowed by my commitments to family and the practicalities of life. He has been
patient, thoughtful, and critical when necessary. His high standards, goodwill and
encouragement have been an inspiration to my professional development.
Many thanks go to my wife, Nell, for her love and support. This undertaking has
taken much longer than we hoped, and throughout she has been a loving companion,
providing equal doses of encouragement, understanding and incredulity at another
missed deadline. Specifically, I would like to thank her for her editorial help. She has
helped me find many hidden assumptions and unarticulated connections that I was
sure I had expressed on the page.
iv
I dedicate this to my son, Ben Peter Heard
and my daughter, Heidi Marie Heard.
v
Table of Contents
Abstract ii
Acknowledgements iv
Dedication v
Table of Contents vi
List of Tables ix
List of Figures x
List of Listings xi
Epigraph xii
1 Introduction 11.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Project Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 Related Works 72.1 Multi Agent Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2 Learning Agent Behaviours . . . . . . . . . . . . . . . . . . . . . . . . 102.3 Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.4 Genetic Programming . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.4.1 Primitive Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.4.2 Program Structure . . . . . . . . . . . . . . . . . . . . . . . . 182.4.3 Creating Trees . . . . . . . . . . . . . . . . . . . . . . . . . . 192.4.4 Genetic Operators on Tree-Based Genomes . . . . . . . . . . . 212.4.5 Fitness Functions in Genetic Programming . . . . . . . . . . . 242.4.6 Other Genetic Programming Parameters . . . . . . . . . . . . 252.4.7 Genetic Programming Design Decision . . . . . . . . . . . . . 27
2.5 Strongly Typed Genetic Programming . . . . . . . . . . . . . . . . . 272.6 Multi Objective Fitness Functions . . . . . . . . . . . . . . . . . . . . 282.7 Evolving Communication . . . . . . . . . . . . . . . . . . . . . . . . . 30
vi
2.8 Evolving “Communication Protocols” . . . . . . . . . . . . . . . . . . 312.9 Social Commitments and Conversation Policies . . . . . . . . . . . . 332.10 CASA’s Conversation Policies . . . . . . . . . . . . . . . . . . . . . . 332.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3 Requirements Analysis 393.1 MAPC Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.2 Input Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403.3 Protocol Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . 413.4 Genetic Programming Requirements . . . . . . . . . . . . . . . . . . 423.5 Model Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.6 Requirements Summary . . . . . . . . . . . . . . . . . . . . . . . . . 45
4 Protocol Domain Model 474.1 Informal Description . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.2 Formal Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.2.1 Protocol Domain . . . . . . . . . . . . . . . . . . . . . . . . . 514.2.2 Agents in MAPC . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.3 Textual Representation . . . . . . . . . . . . . . . . . . . . . . . . . . 554.3.1 Lisp Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 564.3.2 A Textual Representation of the Comparison Shopper Protocol
Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594.4 The Ease of Modelling Protocol Domains . . . . . . . . . . . . . . . . 654.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5 Individuals 675.1 Protocol Description . . . . . . . . . . . . . . . . . . . . . . . . . . . 675.2 Lisp Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695.3 Tree Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.3.1 Node Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . 725.3.2 Context Aware Nodes . . . . . . . . . . . . . . . . . . . . . . 735.3.3 Primitive Set . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
6 Protocol Creator 816.1 ECJ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 816.2 The Evolutionary Process . . . . . . . . . . . . . . . . . . . . . . . . 856.3 Evaluating Individuals . . . . . . . . . . . . . . . . . . . . . . . . . . 88
6.3.1 Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . 896.3.2 Testing an Individual . . . . . . . . . . . . . . . . . . . . . . . 906.3.3 Execution Score . . . . . . . . . . . . . . . . . . . . . . . . . . 92
vii
6.3.4 Fitness Function . . . . . . . . . . . . . . . . . . . . . . . . . 986.4 Verify Negotiable Value Mutation . . . . . . . . . . . . . . . . . . . . 996.5 Parameter Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1006.6 Operator Selection and Context Experiments . . . . . . . . . . . . . . 1016.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7 Test Cases and Results 1077.1 Test Protocol Domains . . . . . . . . . . . . . . . . . . . . . . . . . . 107
7.1.1 Comparison Shopper . . . . . . . . . . . . . . . . . . . . . . . 1087.1.2 Art Dealer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1087.1.3 Flea Market . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1097.1.4 Free Trade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1107.1.5 Test Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
7.2 Test Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . . 1147.2.1 Progress of Evolution . . . . . . . . . . . . . . . . . . . . . . . 1147.2.2 Most Common Acceptable Protocol . . . . . . . . . . . . . . . 1207.2.3 Questionable Protocols . . . . . . . . . . . . . . . . . . . . . . 1257.2.4 Buyer-Driven vs. Seller-Driven . . . . . . . . . . . . . . . . . . 128
7.3 Reaching 100% . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1327.4 Results Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
8 Conclusion and Future Work 1358.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
8.1.1 The Protocol Domain Model . . . . . . . . . . . . . . . . . . . 1368.1.2 The Context System . . . . . . . . . . . . . . . . . . . . . . . 1378.1.3 The Multi Agent Protocol Creator . . . . . . . . . . . . . . . 138
8.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Bibliography 142
A Nomenclature 146
B Protocol Domain Files 149B.1 Comparison Shopper Lisp File . . . . . . . . . . . . . . . . . . . . . . 149B.2 Art Dealer Lisp File . . . . . . . . . . . . . . . . . . . . . . . . . . . 152B.3 Flea Market Lisp File . . . . . . . . . . . . . . . . . . . . . . . . . . . 155B.4 Free Trade Lisp File . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
viii
List of Tables
2.1 A Trigonometric Primitive Set . . . . . . . . . . . . . . . . . . . . . . 172.2 Categorizing the Evolution of Communication . . . . . . . . . . . . . 302.3 Conversation Policies within CASA . . . . . . . . . . . . . . . . . . . 35
4.1 Evolutionary Parameters in Protocol Domain Input Files . . . . . . . 59
5.1 Types in MAPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 735.2 Contexts in MAPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . 745.3 Static Functions in MAPC . . . . . . . . . . . . . . . . . . . . . . . . 765.4 Static Terminals in MAPC . . . . . . . . . . . . . . . . . . . . . . . . 775.5 Action Functions in MAPC . . . . . . . . . . . . . . . . . . . . . . . 785.6 Dynamic Terminals in MAPC . . . . . . . . . . . . . . . . . . . . . . 80
6.1 Main ECJ Components . . . . . . . . . . . . . . . . . . . . . . . . . . 826.2 Comparison Shopper Scenario . . . . . . . . . . . . . . . . . . . . . . 906.3 Execution Data from Buyer Protocol . . . . . . . . . . . . . . . . . . 936.4 Execution Data from Null Protocol . . . . . . . . . . . . . . . . . . . 936.5 Penalties for Example Protocols . . . . . . . . . . . . . . . . . . . . . 976.6 Buyer Maximize Function for Example Protocols . . . . . . . . . . . 976.7 Protocol Length Minimize Functions for Example Protocols . . . . . 986.8 Total Fitness Values for Example Protocols . . . . . . . . . . . . . . . 996.9 Possible Operator Selection Configurations . . . . . . . . . . . . . . . 1026.10 Average Total Times . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
7.1 Test Machines’ Parameters . . . . . . . . . . . . . . . . . . . . . . . . 1137.2 Test Setup Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 1137.3 Buyer- vs. Seller-Driven Individuals (Including Duplicates) . . . . . . 1317.4 Buyer- vs. Seller-Driven Individuals (Unique Protocols) . . . . . . . . 131
ix
List of Figures
2.1 Example Genetic Algorithm Mutation . . . . . . . . . . . . . . . . . 132.2 Example Genetic Algorithm Crossover . . . . . . . . . . . . . . . . . 142.3 Theoretical Framework Components . . . . . . . . . . . . . . . . . . . 152.4 Simple Tree-Based Example . . . . . . . . . . . . . . . . . . . . . . . 192.5 Subtree Mutation in Genetic Programming . . . . . . . . . . . . . . . 222.6 Point Mutation in Genetic Programming . . . . . . . . . . . . . . . . 222.7 Crossover in Genetic Programming . . . . . . . . . . . . . . . . . . . 232.8 Evolution Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.9 CASA Performative Lattice . . . . . . . . . . . . . . . . . . . . . . . 342.10 CASA Inform Conversation . . . . . . . . . . . . . . . . . . . . . . . 362.11 CASA Request Conversation . . . . . . . . . . . . . . . . . . . . . . . 372.12 Class Diagram of Two Simple Agents . . . . . . . . . . . . . . . . . . 38
3.1 MAPC Components . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.1 Comparison Shopper Protocol Domain . . . . . . . . . . . . . . . . . 504.2 The Lisp functions used to create a textual description of a protocol
domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.1 ECJ Component Relationships . . . . . . . . . . . . . . . . . . . . . . 866.2 ECJ Evolution Process . . . . . . . . . . . . . . . . . . . . . . . . . . 886.3 Basic Algorithm for Test Agents . . . . . . . . . . . . . . . . . . . . . 916.4 Parameter Configuration Experimental Results . . . . . . . . . . . . . 1046.5 Results with Small Population Size (100) . . . . . . . . . . . . . . . . 105
7.1 Comparison Shopper Results by Generation . . . . . . . . . . . . . . 1167.2 Art Dealer Results by Generation . . . . . . . . . . . . . . . . . . . . 1177.3 Flea Market Results by Generation . . . . . . . . . . . . . . . . . . . 1187.4 Free Trade Results by Generation . . . . . . . . . . . . . . . . . . . . 1197.5 Most Common Comparison Shopper, Flea Market and Free Trade Protocol1227.6 Most Common Art Dealer Protocol . . . . . . . . . . . . . . . . . . . 1247.7 Questionable Comparison Shopper Protocol . . . . . . . . . . . . . . 1267.8 Questionable Art Dealer Protocol . . . . . . . . . . . . . . . . . . . . 1277.9 Questionable Flea Market Protocol . . . . . . . . . . . . . . . . . . . 1297.10 Questionable Free Trade Protocol . . . . . . . . . . . . . . . . . . . . 1307.11 Free Trade Results for Long Run (100 Generations) . . . . . . . . . . 133
x
List of Listings
4.1 comparison-shopper.lisp . . . . . . . . . . . . . . . . . . . . . . . . . 604.2 Transactor Role Type . . . . . . . . . . . . . . . . . . . . . . . . . . . 604.3 Seller Role Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624.4 Buyer Role Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644.5 Agent Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654.6 Evolutionary Parameters . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.1 The Null Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705.2 The Buyer Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
7.1 Art Dealer Buyer Role Type Variables . . . . . . . . . . . . . . . . . 1097.2 Free Trade Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 1117.3 Free Trade Negotiable Values . . . . . . . . . . . . . . . . . . . . . . 1127.4 Most Common Comparison Shopper and Flea Market Individual . . . 1227.6 Most Common Free Trade Individual (14%) . . . . . . . . . . . . . . 1237.5 Most Common Seller-Based Protocol . . . . . . . . . . . . . . . . . . 1247.7 Questionable Comparison Shopper Individual . . . . . . . . . . . . . 1267.8 Questionable Art Dealer Individual . . . . . . . . . . . . . . . . . . . 1277.9 Questionable Flea Market Individual . . . . . . . . . . . . . . . . . . 1297.10 Questionable Free Trade Individual . . . . . . . . . . . . . . . . . . . 130
B.1 comparison-shopper.lisp . . . . . . . . . . . . . . . . . . . . . . . 149B.2 art-dealer.lisp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152B.3 flea-market.lisp . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155B.4 free-trade.lisp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
xi
You can’t trust code that you did not totally create yourself.
Ken Thompson
Evolution is a tinkerer.
Francois Jacob
xii
Chapter 1
Introduction
This dissertation describes research leading towards the creation of the Multi Agent
Protocol Creator (MAPC), a system designed to facilitate the automatic creation of
multi agent communications protocols. This work is composed of two parts. First, it
outlines a protocol domain model which allows those in need of a protocol to describe
the environments in which the protocol must function, along with the desired outcomes
of the protocol. Second, it describes the protocol creator itself (MAPC) that generates
communications protocols using genetic programming for protocol domains that have
been modelled. Section 1.1 is a statement and brief overview of the research problem
that is addressed. Section 1.2 states the aim of this dissertation. Section 1.3 outlines
the objectives that were met in order to achieve the aim. Section 1.4 outlines the
remaining chapters of this thesis. Finally, Section 1.5 summarizes the dissertation.
1.1 Problem Statement
Much research has been done on communication protocols in multi agent systems.1
Much of this research has focussed on methods to create new protocols that meet
certain goals. To date, the best method to create a protocol for some protocol domain
is to describe exactly how the protocol should be completed (Yolum and Singh, 2001),
giving the participating agents a script (or set of possible scripts) to follow. It would
1Multi agent systems are systems composed of multiple, possibly-independent, agents. In a multiagent system, a communications protocol defines one commonly accepted means to communicate.
1
2
be beneficial to be able to somehow derive a communication protocol from some
description of a system and the desired outcomes. This would allow agent system
designers to focus on higher level design work and would also allow autonomous
systems to generate new protocols as the need arises.
Further complicating the issue tackled in this dissertation is the fact that there
does not yet exist a method to model required communications protocols. This is not
a trivial problem, as there are a number of components that needed to be described
so as to allow for the creation of broad protocols designed to work in a wide variety of
situations. This dissertation addresses these gaps.
1.2 Project Aim
To address the gap in current research, the aim of this dissertation is as follows:
This work enables the automatic generation of multi agent communications
protocols. First, a model for describing a protocol domain and the commu-
nicational requirements of that protocol domain was created. This model
includes details about the various agent inhabitants, roles taken by those
inhabitants and the abilities and requirements of the roles and individuals
within the protocol domain. This completed, a basic system was created
which can automatically generate a communication protocol for a modelled
protocol domain. Put together, these contributions allow both human
designers to formally specify the requirements of future communications
protocols, and automated systems to generate those needed protocols.
3
Much work has been done in the area of genetic programming (Jacob, 2001; Koza,
1992, 1994; Poli et al., 2008).2 This work was done with the goal of automatic
programming of computers. These techniques involve placing variations of a program
into multiple environments with the goal of finding the program that works well in
all, or most of, the environments. The algorithms are similar to those of genetic
algorithms, which in turn are meant to use techniques mimicking evolution in nature.
Protocols are similar to programs. They describe the communicational behaviours
of the agents in a system. Due to this similarity and the breadth of work done
in genetic programming, genetic programming is used in MAPC. While generating
protocols, the various protocol variations “run” in multi agent system environments
(the protocol domain). MAPC then evaluates how well those protocols accomplish the
goals that they were set out to perform.
In order to use genetic programming to create multi agent protocols, there must be
a suitable description of what those protocols must do. This requires that there be a
well-defined environment (or set of environments) in which the agents are placed. This
will be called the protocol domain. In addition, there needed to be a way to test each
protocol, to aid the genetic algorithm. In order to do this, some method of evaluation
must be available to determine the suitability of each protocol. A protocol domain
model was be developed which includes methods to describe both the environment in
which the protocols should execute, as well as the requirements of the new protocol.
This model is able to express a variety of protocol domains and requirements to be
generally useful.
2Genetic algorithms and genetic programming are discussed in further detail in Section 2.3, p. 12and Section 2.4, p. 15, respectively.
4
To perform the executions and testing, as well as the overall genetic algorithm
approach, MAPC builds upon a pre-made multi agent system as well as an operational
evolutionary framework. The multi agent system is extendable and easy to set up
and tear down from within the evolutionary framework. The evolutionary framework
supports genetic algorithms and is extendable to allow it to execute, evaluate and
perform genetic operators on the potential protocols.
1.3 Objectives
To achieve the aim stated above, the following objectives were met.
O1: Research related works. An analysis of related works was completed. The
analysis discusses previous research in multi agent systems, communications
protocols, social commitments, evolving communication in artificial life and
genetic programming. The focus was on both the gap in the current literature in
the area of formally describing the requirements of a communications protocol
and the area of evolving entirely new communications protocols.
O2: Analyze the requirements of MAPC. A requirements analysis was performed
to determine the requirements of MAPC, including the protocol domain model
which acts as input. This required outlining both the theoretical information
needed in a protocol domain model, the methodology used to record that
information, and the practical programming requirements of this undertaking.
O3: Design a protocol domain model. A model was created to represent any
multi agent protocol domain. This model is expressive without being too complex.
Once outlined, a method was created to record all of the information in the
5
model. This model representation is fundamentally simpler to use than similar
languages for describing communications protocols.
O4: Design an output format. The format of the protocols created by MAPC
was developed. This protocol format is adaptable to the variations possible in
the protocol domain model. This output from MAPC is general, so that the
protocol’s intentions can be determined from the output. In addition, this output
is able to drive a multi agent system so that the protocol may be evaluated.
O5: Design and implement MAPC. MAPC was designed and implemented. This
was done to meet the requirements outlined in O2.
O6: Test MAPC. MAPC, which was created to fulfill O5 was tested using several
different protocol domains. These protocol domains were designed to show
that MAPC reacts to its inputs and is capable of handling specific and general
protocol domains.
1.4 Thesis Outline
This dissertation is laid out as follows. Chapter 2 presents an extensive survey of
the current literature relevant to the work that was undertaken. Chapter 3 gives a
requirements analysis of MAPC, including the protocol domain model which is used
as the input to MAPC. Chapter 4 describes the model used to define various protocol
domains. In addition, a method for representing the model is given. Chapter 5
provides an overview of the protocols that are created by MAPC. Chapter 6 describes
the design and implementation details of MAPC. Chapter 7 presents models of several
test protocol domains and discusses the results of the execution of MAPC with those
6
protocol domain models as inputs. Finally, Chapter 8 offers concluding remarks,
summarizes the contributions of this dissertation, and offers suggestions on directions
for future work.
1.5 Summary
Much work has been done to create multi agent protocols. This work has proceeded by
creating a protocol and then testing the protocol to see if it meets certain requirements.
This is difficult because of the large number of possible scenarios that a protocol must
operate. In addition, it is hard to test whether various sub-goals of a protocol are met,
such as the desire to favour the buyer or the seller, depending on the protocol domain.
This dissertation outlines the components of MAPC, a multi agent protocol creator,
illustrate how MAPC can be used to evolve protocols, and provide an analysis of the
resulting protocols obtained from MAPC in several test protocol domains.
Chapter 2
Related Works
While there has been extensive research into the evolution of communication and
some forms of language in artificial life scenarios, the current research is limited to
fairly primitive communication. Research into the evolution of high level protocols
for software agents has not yet been undertaken. This gap in the current research is
outlined and a detailed description of how this dissertation will address this gap is
described. This fulfils Objective O1 (Research related works, p. 4).
The following sections provide a detailed overview of the current research with a
focus on topics and background information necessary for understanding the field and
the gap which will be addressed by my research. To this end, an overview of agents
and multi agent systems is provided as background information. More background is
provided by describing the genetic algorithm. Background on genetic programming is
provided, as this will be used in future sections. The evolution of communication in
artificial life is surveyed, with the intention of differentiating this from the proposed
research. A short description of superficially related (but ultimately different) work
on evolving multi agent protocols is outlined. Finally, conversation policies and the
social commitments that they rely upon are detailed as a method for describing a
multi agent protocol.
Section 2.1 defines multi agent systems as they are used in this dissertation. Section
2.2 surveys previous research into learning new agent behaviours. Section 2.3 describes
the theoretical background of genetic algorithms. Genetic programming, which is a
7
8
specific form of genetic algorithms, is described in Section 2.4. Section 2.5 describes
a specific form of genetic programming using a strong typing system. Section 2.6
outlines some considerations of multiobjective fitness functions in genetic algorithms.
Previous work on evolving communication in multi agent systems is described in
Section 2.7. Section 2.8 describes several prior works which state their purpose as
“evolving protocols,” but which are very different from this work. Section 2.9 describes
social commitments, a descriptive form used in many multi agent systems. Section
2.10 describes the conversation policies used in CASA (Collaborative Agent System
Architecture), a multi agent system used in this work. Finally, Section 2.11 summarizes
the place of this work in the literature.
2.1 Multi Agent Systems
The term agent has been defined in various ways by various researchers over the past
20 years (see Denzinger and Hamdan, 2004; Huhns and Singh, 1997; Wooldridge, 1999).
Among the various definitions, it is possible to pick out the key important ideas of
action, autonomy, communication, and environment. A generic definition of agency
that highlights these themes could therefore be: “any entity that perceives and acts
autonomously within some environment to achieve a particular purpose; this entity is
usually able to communicate” (Heard, 2005, p. 11). A bank teller is an example of a
non-computer-based agent. The teller perceives and acts within the environment of
the bank in order to fulfil the specific purpose of meeting an account holder’s banking
needs.
9
More formally, an agent can be defined as the quadruple:
Ag = Sit, Act,Dat, fAg
fAg = Sit×Dat→ Act
where Sit is the set of all possible perceived situations in which the agent may find
itself, Act is the set of all actions that can be performed by the agent, Dat is the
set of all possible combinations for the agent’s internal data, and fAg is the function
that determines the agent’s action given the current situation and internal data state
(Denzinger and Kidney, 2006). While the agent’s actions may appear to be a result
of only the current situation (f ′Ag = Sit→ Act), this is only true if the agent merely
reacts and does not act based on its internal data.1
A multi agent system is a system in which agents work together and communicate
with each other. To be more specific, a MAS is “a collection of possibly independent
agents that execute in some common environment,” (Heard, 2005, p. 11). Formally,
an entire multi agent system can be defined as the pair:
MAS = A,Env
where A is the set of agents in the system and Env is the set of possible environments
of the system (Denzinger and Kidney, 2006). The difference between the environments
of the MAS (Env) and the situations of an agent (Sit) is one of perception. Each
possible environment is fully described: the elements of Env describe the entire
1Furthermore, f ′Ag may not be a function since the same situation s ∈ Sit may lead to act a ∈ Actfor one internal data state and another act a′ ∈ Act for another internal data state.
10
environment in which the agents in A find themselves, even those components of the
environment that the agent is unaware of or cannot know. For example, an agent
cannot necessarily determine if it is in a situation that contains an agent with an
excessive amount of money, or no money at all. In addition, an agent can not know
(with complete certainty) the attitudes of agents around it. Both of these are a part
of the enviornment, but not necessarily part of the situation.
2.2 Learning Agent Behaviours
The goal of the proposed work is to create a system that can evolve a communication
protocol for a multi agent system. Stated another way, the proposed work is designed
to learn communicative agent behaviours. Therefore, it is helpful to look at previous
research in which the goal was to learn agent behaviours. In this subsection, I will
discuss previous work that has used situation-action pairs and evolutionary algorithms
to learn agent behaviours.
Panait and Luke give a detailed survey of the area of agent learning (2005). In
it, they categorize various methods that have been used to allow agents to learn
behaviours to accomplish some goal or goals. Overall, Panait and Luke describe two
main approaches. They are team learning and concurrent learning. In team learning,
one learning technique is used to find the behaviours of all of the agents, while in
concurrent learning, multiple learning techniques are used. Each learning technique is
focused on one aspect of the problem. The work undertaken in this dissertation is
team learning, since a single search for a protocol will be done, using only one learning
method (genetic programming).
Within team learning, Panait and Luke again distinguish between two major
11
categories. These are homogeneous team learning and heterogeneous team learning. In
homogeneous learning, one behaviour is used by all agents, while in heterogeneous
learning multiple behaviours are evolved simultaneously. Since the goal of this
dissertation is the creation of a protocol, and that protocol will be shared by all agents,
homogeneous learning will be taking place.
Panait and Luke also discuss agent learning and communication (2005). In their
discussion, the use of communication is seen as another tool that may be used as
agents work towards some goal, but not as the direct output of any work.
Denzinger and Kordt describe a learning multi agent system which is used to
demonstrate the on-line learning of cooperative behaviours in a variant of the pursuit
game (Denzinger and Kordt, 2000). This system is named OLEMAS (On-Line
Evolution of Multi-Agent Systems). In this system, the behaviours are represented as
situation-action-pairs, and learning is done based on models of other agents’ behaviours
modelled in the same manner and based on observations. The learning is done on-line
by executing a learn action which causes the agent to pause and evolve its own
behaviour (situation-action-pairs) using its knowledge of other agents’ behaviours and
multiple simulations. This is similar to the proposed work in that it learns cooperative
behaviours. It is different in its behaviour representation and its use of on-line learning.
In another paper, Denzinger, Loose, Gate, and Buchanan describe an evolutionary
system using sequences of actions (some of which are parameterized) which are applied
to a computer game with the hopes of finding short sequences which lead to a specified
event in a computer game (Denzinger et al., 2005). This work is related in that it
describes learning behaviours. While the behaviour is that of a single agent working
independently, the evolution is done using two populations: one population where
12
individuals represent particular combinations of parameters for a parameterized action,
and another population where individuals are a sequence of simple actions following a
parameterized action. Because of the two distinct populations, this is an example of
co-evolution and is therefore relevant to the proposed work.
2.3 Genetic Algorithms
A genetic algorithm is an algorithm that can be used to find an optimal solution
to a problem by using a process similar to evolution and natural selection (Banzhaf
et al., 1998; Mitchell, 1996). A genetic algorithm works towards an optimal solution
by performing on the current generation the steps of evaluation, selection and the
creation of the next generation. In each generation, there are many individuals. Each
individual is one possible solution to the problem. These individuals are each evaluated
based on a fitness function, which describes (usually numerically) how optimal the
individual is as a solution to the problem. Selection is based on these evaluations:
fitter individuals have a higher likelihood of being selected for the next generation.
Selected individuals are used to create the next generation. A selected individual is
either moved into the next generation unchanged (reproduction), mutated and placed
in the next generation, or paired with another selected individual for crossover.2 These
terms are taken from evolutionary biology and defined similarly in computer science.
Over many generations, a genetic algorithm will work towards a highly fit individual,
or a highly optimal solution to a problem.
Mutation and crossover are used in genetic algorithms to create new individuals
2Highly fit individuals will likely be selected multiple times. For each selection, they may undergocrossover with a different individual or mutate in a different way.
13
ADCDBCAC
ADCABCAC
Figure 2.1: Example Genetic Algorithm Mutation
from old individuals. Mutation occurs when a single individual is modified (usually
randomly) to generate a new individual. Consider a situation in which the individuals
are sequences of letters. A simple mutation might change a random letter to a new
letter. This form of mutation is illustrated in Figure 2.1 with a different colour used
for each letter for emphasis.
Crossover is designed to be similar to biological sexual reproduction in which
attributes from two parents are selected and combined to form a new individual. To
perform crossover, first two individuals must be chosen from the current generation.
The individuals are then each broken into two pieces (usually in a random manner),
and recombined in such a way as to produce a complete new individual. Using the
same letters as described above, a crossover operation might select a position randomly,
taking all the letters above the position from one parent and all the letters below the
position from the other parent. This form of crossover is illustrated in Figure 2.2.
Theoretical Framework
Holland gathered many of the concepts of genetic algorithms into a formal theory of
adaptation (Holland, 1975). He began by modelling what he called an adaptive plan
14
DABAC
ADCDDBCA
CAD
ADC
DDBCA
Figure 2.2: Example Genetic Algorithm Crossover
(τ), which describes how some system might adapt to a given environment (E).3 His
model employs the idea of discrete time steps (usually called generations), where t
is the current time step, t + 1 is the next time step, and t − 1 is the previous time
step. At each point in time (t) the adaptive plan has a new state (A(t)). This state
includes all of the information retained at that time step.4 In addition, the state
represents how the system “acts” in the environment. At each time step (t), the
environment produces a signal (I(t)) in response to the state (A(t)).5 The adaptive
plan τ then selects the next state A(t+ 1).6 In this way, the state should be adapting
to the environment over time (assuming a reasonably constrained environment). The
components of the theoretical framework are illustrated in Figure 2.3.
3More specifically, E is used to represent the set of all possible environments the adaptive plan mayencounter, while the adaptive plan only has access to one possible environment at a time (E ∈ E).
4For a genetic algorithm, the state is described by the current set of individuals.5The signal from the environment is usually an indication of the payoff from the environment
(µE) for the current state (I(t) = µE(A(t))). This payoff can be thought of as a fitness function foreach of the individuals.
6The adaptive plan actually selects an operator (ω ∈ Ω) which provides a probability distribution(P) over A, which is used to select the next state.
15
Adaptive Plan (𝜏)
Environments (ℰ)
𝒜(t)𝒜(t – 1) 𝒜(t + 1)
...
...
µE
E
µE'
E'
𝐼(t)
time →
𝜔
Figure 2.3: Theoretical Framework Components
2.4 Genetic Programming
Genetic programming was initially introduced by Koza as a method for automatically
programming a computer (1992). In the last 20 years, Koza has been joined by many
other researchers working to expand the field of genetic programming (Jacob, 2001;
Koza, 1994; Poli et al., 2008). Genetic programming uses algorithms similar to genetic
algorithms to construct computer programs. Programmers can thus define the goal
of the program, and the genetic programming system will write a program to reach
that goal. Genetic programming will be used in the process of automatically creating
communications protocols in this work.
16
For a basic understanding of genetic programming, several components must be
understood, including the primitive set of program components, the tree-based genome
used in many genetic programming systems and the mutation and crossover operations
as they apply to tree-based genomes. In addition, the evaluation process and parameter
selection for a particular problem is crucial to the success of a genetic programming
attempt. Each of these areas are discussed below.
2.4.1 Primitive Sets
Before a program can be created, the building blocks for that program must be defined.
In genetic programming, these building blocks are in what is known as the primitive
set. The primitive set contains all of the possible actions and calculations that a
program may perform, inputs that the program will receive and other fixed values
that may be needed during the program’s execution. The items of the primitive set
are nodes. The two primary types of nodes are functions and terminals. Functions
are nodes that represent actions and calculations that require some input. These
nodes have some required number of inputs that must be provided for the function
to operate. Terminals, on the other hand, take no inputs and provide either a fixed
result (representing constant values in the program) or represent one of the inputs to
the program.
Often primitive sets function on numerical values and include various mathematical
functions, numeric inputs and constant values. An example primitive set that could
be used for a trigonometric set is shown in Table 2.1. The primitive set includes the
basic arithmetic operators (with a modified division operator that is discussed below),
three basic trigonometric functions, the input value, π and a set of constant values
17
denoted by R. These constant values are represented by a single terminal, but this
terminal may take multiple values. This is called an ephemeral random constant. The
value is fixed for each instance in each individual, taking on one possible content value
and retaining that value as long as it remains in that individual. In other individuals,
however, the ephemeral random constant takes on other random values, and it may be
changed during the application of genetic operators. This ensures that many values
are available to be used in potential programs, but allows one individual to always
produce the same values for each execution.
Table 2.1: A Trigonometric Primitive SetBinary Functions Unary Functions Terminals+ (addition) sin A (input)- (subtraction) cos π× (multiplication) tan R% (protected division)
When constructing a primitive set, it is helpful if the set has closure (Koza, 1992).
Closure in a primitive set means that the value of any terminal or resulting from any
function may be used as any of the inputs for any of the functions. This allows the
construction of programs in any combination of the functions and terminals with the
only restriction being that each function receives the correct number of inputs. To
allow this, the functions must all work on the same type.7 Typically this is the set of
real numbers. In addition, some functions may need to be modified to allow any real
input. For example, division is undefined when the denominator is zero. Typically a
protected form of division is used (with the symbol %) whereby division by zero yields
a value of one.8
7This restriction can be modified, as discussed in Section 2.5.8The value of one was historically chosen so that a value of one was easy to produce by the system
without needing a separate terminal with the value one (Poli et al., 2008).
18
Another quality desired of the primitive set is completeness (Jacob, 2001). A
primitive set is complete if it has all of the components necessary to construct a
program that meets the requirement of the current situation. This means that it is
often helpful to provide simple nodes that can be combined to provide useful functions.
In addition, it is helpful to have more nodes than necessary, allowing the genetic
programming system to select the nodes that will be used for the final solutions.
2.4.2 Program Structure
A significant difference between genetic programming and traditional programming is
the format of the program. Genetic programming works best when the program format
is well suited to the crossover and mutation operators used during the execution of the
genetic algorithm. Koza originally introduced a tree-based format that accommodates
these needs (Koza, 1992).9
Tree-based genomes make use of a tree structure to represent a program. They
were originally used to mimic the parse trees that are used to represent programs in
compilers. Each of the entries in the primitive set may be a node in the program tree.
Functions, such as the addition function, take the resultant value of their children
as their inputs and result in a value. Terminals, such as the input value X and the
numeric value 2, act as the leaves in the tree. A simple example with two functions and
three terminals is shown in Figure 2.4. The simple example represents the function
(A− 5)× 2.
The advantage of tree-based genomes is that no program containing syntax errors
can be represented because each node must have the correct number of children. This
9More recently, linear, graph-based, finite automata and grammar-based genomes have beenshown to be useful for some types of problems (Banzhaf et al., 1998; Jacob, 2001; Poli et al., 2008).
19
-
x
2
Function
Terminals
A 5
Figure 2.4: Simple Tree-Based Example
is a result of the closure of the primitive sets. This means that genetic operators on
these trees need only ensure that the correct number of children exist for each parent,
and not that the correct syntax is followed. In addition, unless global memory is built
into the functions and terminals, each node in the tree only has access to the result(s)
of its children, if any. This makes the flow of information from the leaves to the root
fairly straightforward. This can aid in understanding how a genetic program executes.
2.4.3 Creating Trees
In order to begin the process of genetic programming, some initial population of
potential programs must be generated. In addition, during the process of creating
new individuals for future generations, portions of trees (known as subtrees) must
sometimes be made. In both of these cases, a similar process is followed (Banzhaf
et al., 1998; Poli et al., 2008). The procedure is as follows:
1. Choose a random node from the primitive set. This node will be the root node.
2. Until all functions have as many children as they require:
20
(a) Choose a function missing a required child.
(b) Choose a random node from the primitive set.
(c) Set the function’s child to the chosen node.
The simplest variation on this basic process is known as the grow method of tree
creation. In this method, a fixed limit is specified as to how many levels deep the
final tree may be. This limit is known as the depth of the tree. When using the grow
method, the selection of a new node may be restricted. If a new child will be at the
specified maximum depth, only terminals are considered when selecting the node.
Another limitation that is sometimes made is to prevent terminals from being chosen
before the depth limit. This is called the full method, and it results in trees that have
functions at all levels except the lowest level (at the specified depth). Both of these
methods may be used for both initial population creation as well as for the creation
of subtrees as needed when performing mutations as described below.
The structure of trees constructed using the grow and full methods described
above tend to be repetitive and favour specific shapes. In addition, many of the
individuals will have the exact depth given by the restriction. To provide a range of
structures for the initial population, the ramped half-and-half method may be used.
This method uses a variety of depths (the ramping) and uses the grow and full method
on alternating individuals (the half-and-half), thus providing a variety of individuals.
This process is typically limited to the initial population creation at the beginning of
genetic programming.
21
2.4.4 Genetic Operators on Tree-Based Genomes
To create each of the non-initial generations in a genetic programming system, processes
are used to create and modify individuals. These processes are called genetic operators.
In genetic programming, as in genetic algorithms, there are three main types of
operators. They are mutation, crossover and reproduction. This section will describe
these operations. As with basic genetic algorithms, a mutation is a genetic operator
that produces new individuals from a single individual while a crossover is a genetic
operator that produces new individuals from multiple individuals. In addition, the
reproduction operator, which merely copies an individual from the last generation
into the next, is available.
Mutation and crossover take slightly different forms when applied to tree structures.
These operations typically act on subtrees of individuals. A subtree is a portion of a
tree defined by a single node that includes that node and all of its descendants.
Mutation in genetic programming typically takes two main forms. The first form
is known as subtree mutation and is performed by replacing a random subtree with a
newly generated subtree. The second form of mutation is known as point mutation
and is performed by replacing a random node with a newly selected node with the
same arity (that is, the new node has the same number of inputs as the replaced
node, if any). The process of subtree mutation is illustrated in Figure 2.5. In the
figure the subtree rooted with an addition function is selected for replacement. A new
subtree is generated using one of the methods described above for creating trees. In
the figure, this subtree is rooted with a multiplication node. The selected subtree
from the individual is then replaced, resulting in the mutated individual. Thus the
original expression 4− (A+ 1) is mutated into 4− (2× 3).
22
x
2 3
44 +
A 1
– –x
2 3
Individual New Subtree Mutated IndividualSelectedSubtree =+-
Figure 2.5: Subtree Mutation in Genetic Programming
4 +
A 1
–
%
Individual New Function Mutated IndividualSelectedFunction =+-
4 %
A 1
–
Figure 2.6: Point Mutation in Genetic Programming
The process of point mutation is illustrated in Figure 2.6. In the figure the addition
function is selected for replacement. The protected division operator is selected as the
replacement and the node is replaced. This mutates the expression from 4− (A+ 1)
to 4− (A% 1).
Crossover with trees is typically the swapping of subtrees between two individuals.
This process can make use of one or both of the individuals that are produced by
this process. The process is illustrated in Figure 2.7. In the process, portions of
two individuals are selected. In the first case, individual 1 is used except for the
selected subtree, which is replaced by the subtree from individual 2. In the other
case, individual 2 is used except for the selected subtree, which is replaced with the
23
x
2 3
–4 +
A 1
–
Individual 1 Individual 2'sSelected Subtree Crossover IndividualSelected
Subtree =+-
A 4
x
x
2 3
4
–
–
A 4
+
A 1
– 4x
2 3
x
Individual 2 Individual 1'sSelected Subtree Crossover IndividualSelected
Subtree =+-
A 4
–
+
A 1
x
Figure 2.7: Crossover in Genetic Programming
unused subtree from individual 1. The result of this crossover is two individuals, both
composed of portions of their two parents.
Reproduction in genetic programming is used to pass individuals from one genera-
tion to the next without changes. This allows fit individuals to be retained when they
may have been lost due to a mutation or crossover that reduced their fitness.
In some situations, it is beneficial to create customized operators that perform
some operation unique to the problem area. These custom operators can be highly
beneficial when applied well.
The choice of which operators to use and in what ratio is not always clear in
genetic programming. Traditionally, Koza used only crossover and reproduction at
rates of 90% and 10% respectively. Some research has shown that mutation-only and
crossover-only genetic programming can work, depending on the problem space (Luke
24
and Spector, 1997). Poli et al. state that typically genetic programming involving
smaller populations relies more on mutation than crossover (2008).
2.4.5 Fitness Functions in Genetic Programming
In order to perform genetic programming, a fitness function must be created. As
with genetic algorithms in general, the fitness function gives a numerical value to the
effectiveness of a particular program. As in traditional genetic algorithms, the program
must be tested in some environment. In genetic programming, the environment is a
collection of fitness cases (Poli et al., 2008). For some genetic programming problems,
a fitness case is a set of input values. For other problems, a fitness case is a virtual
environment in which the program will operate. Once the fitness cases have been
chosen, each program is tested with each fitness case. The fitness function is then
calculated based on the output of the program or the resulting actions taken by the
program in each case. Traditionally, the fitness value in genetic programming gives
a value of zero to a perfect individual and a score greater than zero for non-perfect
individuals (Banzhaf et al., 1998). The lower the score, the more fit the program.
Often fitness functions in genetic programming have multiple objectives. This
can lead to conflicts when deciding on a method for comparing two fitness values.
For example, If there are two individuals to evaluate and one meets objective A, but
not objective B, while another meets objective B, but not objective A, while a third
individual is mediocre at both objectives, how do you compare these individuals? A
few of the techniques used for multi objective fitness functions are described in Section
2.6.
25
2.4.6 Other Genetic Programming Parameters
Aside from the components described above, several key parameters must be defined
in order to fully specify a genetic programming run. These parameters include
the population size, maximum individual program size, selection scenario, and the
termination criterion. Each of these parameters will be described in this section.
The population size is the number of individuals in each generation. This is used
to determine the number of individuals to initially generate as well as how many
individuals must be generated for future generations. Typically, a value of 500 or 1000
is recommended as a reasonable starting value (Banzhaf et al., 1998; Poli et al., 2008).
Research suggests that this value may need to be adjusted, however, if these default
values do not yield the desired results. Experiments have been shown to work with
populations as small as 50 individuals for some problems, while other problems have
used very large population sizes (over 100,000).
The maximum individual program size determines how large each individual pro-
gram can be. The size is measured differently for each type of program representation.
For tree-based programs, the maximum depth of the trees is used.
The selection scenario describes how the overall process of evolution will proceed.
There are two main scenarios that can be used. These are the genetic algorithms
(GA) and the evolution strategies (ES) scenarios (Banzhaf et al., 1998; Jacob, 2001).
The genetic algorithms scenario uses the fitness values to select the parents that will
be used for the next generation. This is sometimes known as mating selection. The
evolution strategies scenario creates a set of offspring (λ) from the parents (µ). The
set of offspring is necessarily larger than the set of parents (|λ| > |µ|), and the fitnesses
of the individuals in this larger pool are used to reduce the set of offspring to the set
26
Create Initial Population
Create Offspring
Evaluate Offspring
Select Populationfrom Offspring
Create Initial Population
Evaluate Population
Create Offspring
Select Parentsfrom Population
Evolution Strategies Style Genetic Algorithms Style
Figure 2.8: Evolution Strategies
of parents for the next generation. This is referred to as the (µ, λ) strategy, indicating
the two population sizes.10 This is sometimes known as overproduction selection. The
two processes are shown side-by-side in Figure 2.8.
The termination criterion for a genetic programming system is typically based on a
maximum number of generations, or some function of the current fitness of the system
or both. Typically, a generation maximum is used. Traditionally, researchers have
found that little useful adaptation occurs after 50 generations (Koza, 1992; Poli et al.,
2008). Other researchers have found that results may improve beyond generation 50
and even as high as generation 1000 (Banzhaf et al., 1998).
10In evolutionary strategies it is also possible to include the parents of the last generation in thepool of possible individuals for the next generation. When the parents are included, then the (µ+ λ)strategy is in use.
27
2.4.7 Genetic Programming Design Decision
In order to use genetic programming to automatically program a computer, several
preparatory steps are necessary on the part of the programmer. First, the primitive
set must be defined. The fitness function must also be defined. Second, the genetic
representation must be selected. In this work, a tree-based genome has been selected.
Third, the method for generating the initial population must be selected. Fourth,
the type of operators to be used during the creation of the next generation must be
selected. Finally, the values of the remaining parameters must be chosen.
2.5 Strongly Typed Genetic Programming
As discussed above, primitive sets typically must have closure. The value returned
from any node must be accepted by any other node. This places a large restriction on
the possible inputs for nodes, and ignores the idea of widely divergent types used in
many modern programming languages. To address this concern, Montana developed
the concept of strongly typed genetic programming (Montana, 1994). In this system,
nodes each have resulting types. In addition, each parameter for a function has a
required type. With these types defined for all nodes, a tree is valid only if the children
of all functions result in the correct type. This is in addition to the usual requirement
that all functions have the correct number of inputs.
When creating trees and performing genetic operators in individuals that contain
strongly-typed nodes, the restrictions of the nodes must be taken into account. During
tree creation, if a function requires a specific type of child, or parameter, then the
chosen child node must result in a type which matches the required type for the parent
function.
28
During mutation, when a function is to be replaced with point mutation, the new
function must not only match in terms of number of children, but also in types of
the children and the resulting value. If subtree mutation is performed, the newly
generated subtree must result in the same type as the subtree it is replacing.
Crossover requires modifying the process to allow for multiple attempts. After
selecting two individuals, a random destination location in the first individual is
selected, as well as a random source subtree from the second individual. After these
have been chosen, they are checked to see if their types match. If not, the selection
process repeats until a match is found or a specified number of attempts have been
made.
2.6 Multi Objective Fitness Functions
When using genetic algorithms, certain problems will have multiple objectives (Deb,
2011). These objectives may be complimentary, but often they will work at odds
with one another. For example, a meeting scheduling system might try to use as few
rooms as possible, only schedule meetings during the peak working time (10:00 a.m.
to 2:00 p.m.), and ensure that employees on vacation are never scheduled for meetings.
Each of these objectives may oppose another. Working only during the peak working
time means that it is likely that more rooms are used, and the opposite is true as
well. When designing a fitness function for a problem with multiple objectives, some
comparison method must be used which can compare individuals on all objectives
while handling situations where individuals have ranked well under one objective but
fared poorly under other objectives. This section will discuss several methods used to
handle multi-objective situations.
29
The first method used in a multi-objective genetic algorithms is an aggregate scalar
fitness function (Poli et al., 2008). When an aggregate scalar fitness function is used,
each of the objectives is given a value and then used in a sum to calculate the overall
fitness of an individual. Often, each of the objective’s values is also multiplied by
some factor that indicates its importance or balances the values. When aggregating
the objectives, a dynamic fitness function can be used. In dynamic fitness functions,
the fitness function changes as the evolution process continues. A dynamic fitness
function can be used to “lead” the evolutionary system towards a goal when known
steps towards a solution need to be taken. An alternative to dynamic fitness functions
is a staged fitness function. A staged fitness function is static (it does not change),
but has drastically different multipliers for the objectives such that a failure on one
objective indicates a much more or less severe problem than a failure on another
objective.
The primary alternative to aggregate fitness functions is to somehow keep all of
the objectives separate and compare individuals on all fronts simultaneously. This is
a very difficult problem to which a large body of research has been devoted. One tool
used in this area is the measure of Pareto optimality (or Pareto dominance) (Deb,
2011; Poli et al., 2008). The central concept is that, for a problem with n objectives,
the individuals can be distributed on an n-dimensional space. An individual can be
said to dominate another individual if it is better or the same as that individual for
every objective. When this occurs, the individual has Pareto dominance over the other
individual. The set of individuals that are not dominated by any other solution form
the n−1-dimensional shape which is termed the Pareto front (or non-dominated front).
The goal of multi-objective genetic algorithms is to move the Pareto front forward.
30
nonsituated situated
unstructuredEncode / Simulated Food /
Decode Games Predator Calls
structuredComplicated Encode / Object Descriptions /
Decode Games Command Responses
Table 2.2: Categorizing the Evolution of Communication
Once evolution has been completed, the final solution must be chosen. Typically this
would be done by a human, giving them the picture of the tradeoffs and allowing them
to make a more informed decision than they would have without the Pareto front.
2.7 Evolving Communication
In the field of artificial life the evolution of communication has been heavily researched
(Wagner et al., 2003). While the current research relates strongly to the aim of
the proposed research, it is different in several respects. In their survey of evolving
communication in artificial life, Wagner et al. placed the research in this area into
four categories: agents are either situated or nonsituated, and communication is either
structured or unstructured.
Nonsituated, unstructured communication research is conducted mainly using
encode/decode games. In these games, an agent must communicate some meaning
to another agent. More precise communications yield a higher (or better, in this
case) fitness value for the agents involved. Nonsituated, structured communication
research is similar to nonsituated, unstructured communication research in that most
of the research uses some form of game to determine if agents can communicate a
desired concept. The difference is that the goal of the latter research is to attempt to
31
evolve a structured form of communication. Batali performed an experiment where
the result was that the utterance of one agent was separated into two parts that
could be parsed separately (Batali, 1998). Situated, unstructured communication
places agents within some environment where communication with other agents is
helpful. These environments were usually simple ecosystems, where agents must “eat,”
avoid “predators,” or both. These settings lead to simple food or predator calls.
Other research involved coordination between agents with goals other than simulated
biological needs (Wagner et al., 2003). Finally, situated, structured communication
research deals with how structured communication could evolve. This generally
meant similar situations and communications as those found in situated, unstructured
research, but with the system set up so that structured communications were favoured.
The research in this dissertation is situated as described above, and would fall
under the categorization of structured communications. Most of the structured
communications surveyed by Wagner et al., however, dealt with single utterances, not
the sequencing of multiple utterances. This is the main difference between this work
and the research currently found in communication in artificial life. This dissertation
will examine the evolution of protocols between agents within the framework of some
protocol domain, as described by the aim.
2.8 Evolving “Communication Protocols”
This subsection describes two papers that deal with protocol evolution. On the surface,
these papers have the same aim as the proposed research. However, they are both
quite different from the proposed research.
Khasteh, Shouraki, and Lesani describe what they call communication protocol
32
evolution (Khasteh et al., 2006). However, in their paper, they define protocol loosely
to mean the selection of an appropriate word for describing a concept. In this way,
their system is performing a situated, unstructured encode/decode game (according
to the system of classification described by Wagner et al.). This is not really the
evolution of a protocol, but just the evolution of a shared vocabulary.
Clark and Jacob present a novel method for the creation of security protocols
(Clark and Jacob, 2001). They make use of the BAN logic named after its original
authors Burrows, Abadi, and Needham (Burrows et al., 1989). BAN logic was designed
to reason about protocol abstractions. BAN logic appears to be based on or borrows
strongly from the BDI (beliefs, desires and intentions) model of agents.11 Within BAN
logic, it is only possible for agents to utter what they believe to be true. Therefore,
each utterance is a non-empty subset of the current agent’s beliefs. Clark and Jacob
actually reduce this to a smaller set based on the simplicity of each belief. A protocol
is then defined as a sequence of utterances, where each utterance is defined by a sender,
a receiver, and a selection (by indices) of beliefs from the sender’s possible beliefs.
This would expand nicely to cover BDI protocols. It is not well suited, however, to
a non-BDI protocol. This is because their method requires a relatively small set of
possible utterances for each agent. In social commitment-based systems, the set of
possible utterances is too large to utilize this representation.
11As such, it has the same limitations as BDI: it requires agents to be truthful (see Heard, 2005).Clark and Jacob specifically state that their investigation assumes truthfulness from the participants.
33
2.9 Social Commitments and Conversation Policies
The protocol generator will make use of social commitment-based conversation policies
to outline the structure of the communications between agents in the protocol domains.
Social commitments is a term used to describe the commitments between agents in
a multi-agent system (Castelfranchi, 1995; Singh, 1996). Formally, a social commitment
is a tuple including a debtor (d), a creditor (c) and an action (a) (Flores and Kremer,
2001, 2004). The commitment states that d is committed to c to perform a.
Social commitments can be used to define a communication protocol, but to
do this, the conditions that bring about a social commitment need to be defined.
Conversation policies define these conditions (Flores and Kremer, 2001; Pasquier et al.,
2004). Conversation policies are rules that indicate when to adopt social commitments
based on the conversational tokens (these include propose, accept, reject, counter, and
inform) contained in (or implied by) the various conversational acts. A commitment
is dissolved when the debtor of that commitment has been released from the action
portion of the commitment by some conversation policy. It is possible for system
designers to add new policies to their system. This allows domain-specific policies to
be put into place when they would aid in understanding the expectations of agents
participating in that system.
2.10 CASA’s Conversation Policies
CASA is a multi agent framework that will be used for the protocol generator. It uses
conversation policies to function. These conversation policies define the communication
protocols used by agents within the CASA framework.
34
success failure
agreerefusenot-understoodpropose accept-proposalreject-proposal
request reply-request reply-proposal
inform ack
replypetition
Figure 2.9: CASA Performative Lattice
CASA uses the performative and act field in each message to determine which
conversation policy applies to a message. The performative describes the type of
speech act, while the act describes the context. To request that another agent add
two integers together, an agent might send a message with a performative of “request”
and an act of “add integer.” The other agent could then respond by agreeing to the
request using the performative “agree” or declining the request using the performative
“refuse.” In the case of misunderstanding or a scrambled message being received, the
agent could utilize one of a few less-used performatives, e.g. “not-understood.” In
each case, the response would have the same act, since that is what the agents are
discussing.
The performatives are arranged in a subsumption lattice as shown in Figure 2.9.
In the diagram, the arrows indicate an “is a” relationship. So, the arrow from reply
to inform and ack indicates that a reply is also an inform and an ack. This means
that a message that has a performative of reply triggers the reply, inform and ack
conversation policies.
35
Performative Actionaccept-proposal Delete the associated perform commitment.agree Create a perform and propose-discharge commitment.inform Create an appropriate private consider -descendant commit-
ment.propose Delete the associated propose commitment.
Create a reply-proposal commitment.reply-request Delete the associated reply-request commitment.reply-proposal Delete the associated reply-proposal commitment.request Create a reply-request commitment.
Table 2.3: Conversation Policies within CASA (Subset described and used in thisproposal)
A subset of the conversation policies that will be used in the protocol generator
is given in Table 2.3. Those policies cover both the inform conversation and the
request conversation. In operation, the policies will create and fulfill commitments.
The inform conversation is shown in Figure 2.10 and the request conversation is shown
in Figure 2.11. In both of these figures, the green arrows indicate messages passed
between the agents. The text above them indicates the performative of that message.
Above the performative are the supertypes of that performative that generate or
delete commitments.12 The text outside the black bars indicates the name of a created
commitment. The pink arrows and commitments indicate private commitments
that are not shared. These commitments are created by the agent to aid in its
internal processing, and are not socially required. The type of private commitment
created depends upon the type indicated by the orange arrow attached to the tail
of the pink arrow. The black solid arrows and black commitments indicate shared
commitments. These commitments are fulfilled by the performative from which the
12To simplify the diagrams, intermediate performatives that do not generate or delete commitmentsare not shown. For example, the performative reply is a supertype of reply-request and a subtype ofinform, but is not shown between reply-request and inform for the second message in Figure 2.11.
36
Alice (client) Bob (server)inform/act accept(act)
Figure 2.10: CASA Inform Conversation
dashed black arrows originate. Finally, the dark red arrows indicate a dependancy
between commitments. For example, the reply-request commitment cannot be fulfilled
before the consider private commitment is fulfilled.
The pink private commitments are important, because they provide a template
that needs to be filled in to create a new type of request for some act. To create a new
request with an act of “run”, a server agent (Bob) must implement perform run().13
The client agent (Alice) must implement release run(). In addition, Alice may
implement consider run() and/or conclude run(). If she does not implement
either of these functions, they will fall back on default functions, considerDefault
() and concludeDefault() respectively. The client agent (Bob) may implement
verify run(). Again, if he does not, the default function verifyDefault() is
executed. Figure 2.12 gives a class diagram illustrating the distribution of these
functions.
2.11 Summary
Extensive research has been conducted in the fields of genetic algorithms and multi
agent communication. Despite this, no work has been done to attempt to evolve a
13While this is not a private commitment, it is the only non-private shared social commitmentthat does not perform the same action for all request conversations.
37
Alice (client) Bob (server)
request
inform consider
reply-request
agreerefusenot-understood
reply-request
inform
perform
verify
propose-discharge
success failure
propose-dischargereply-proposal
informrelease
accept-proposalreject-proposal
reply-proposal
inform conclude
Figure 2.11: CASA Request Conversation
38
considerDefault ()verifyDefault ()performDefault ()releaseDefault ()concludeDefault ()
Agent
consider_run ()perform_run ()conclude_run ()
Alice (server)verify_run ()release_run ()
Bob (client)
Figure 2.12: Class Diagram of Two Simple Agents
high-level multi agent communications protocol. This dissertation fills this gap by
describing MAPC, a system capable of evolving protocols. To enable the evolution of
protocols, a model was created to define a protocol domain in which a protocol must
work, as well as the requirements of the protocol within that protocol domain. This
dissertation then uses this model as input to a genetic programming system whose
purpose is to create a high-level protocol.
Chapter 3
Requirements Analysis
In order to fulfil the aim of this dissertation (see Section 1.2, p. 2), this chapter
will outline the requirements for MAPC. This will fulfil Objective O2 (Analyze the
requirements of MAPC, p. 4). Section 3.1 outlines the high-level requirements of
MAPC. The requirements of the input and the output are given in Section 3.2 and
Section 3.3, respectively. Section 3.4 describes the requirements that must be met to
allow the use of genetic programming in MAPC. Section 3.5 states the requirements
for the model. Finally, Section 3.6 summarizes how the collective requirements, when
met, support the fulfillment of the aim of the dissertation.
3.1 MAPC Requirements
MAPC is designed to generate a protocol from a description of the requirements of
that protocol. As with any software, MAPC’s input must be well defined. Therefore,
the first requirement of MAPC is that. This input must define everything necessary to
generate and evaluate potential protocols. MAPC will produce a protocol as output.
In order for this output to be useful, it must meet certain requirements. Thus, the
second requirement is therefore that there is a well defined output format for MAPC.
Finally, MAPC will use genetic programming to produce a protocol from its input. So,
the third requirement is that the requirements for genetic programming must be met.
C1: MAPC must have a clearly defined input format. In order to automati-
cally generate a protocol, some information about the desired protocol must be
39
40
given. This information must describe what the protocol should do, so the input
should specify the requirements of the protocol domain. In addition, in order
to evaluate potential protocols, the input must describe the protocol domain in
which the protocol must operate.
C2: MAPC must have a clearly defined output format for the generated
protocols. MAPC should generate a protocol that is general enough that it may
be used in a variety of multi agent systems. In addition, this protocol should,
when executed in a manner described in the input, meet all of the requirements
outlined in the same input.
C3: The requirements for genetic programming must be met. MAPC will
use genetic programming to generate protocols from the input. Therefore, the
system must be capable of representing the protocols in a satisfactory manner to
be evolved, generate new protocols, evaluate potential protocols, and generate
new protocols from old protocols using genetic operators.
3.2 Input Requirements
In order to generate a protocol, MAPC must be given some initial input. The following
are requirements of the input for MAPC. These follow from Requirement C1 (MAPC
must have a clearly defined input format).
The input of MAPC must describe the protocol domain in which the protocol
is intended to operate. This requires defining a model that can hold all of the
necessary information about the protocol domain. This model is the first input
requirement. To act as input for MAPC, the input must have a suitable electronic
41
representation. In addition, since the model must be crafted by a human designer, the
representation for the model must be clear and relatively easy to write or manipulate.
The second requirement of the input is that it has a clear, concise and machine-readable
representation. In addition to the above requirements, in order to make MAPC useful
as a design tool, it must be generally easy to create an input file for MAPC. This is
the third and final requirement for the input.
I1: The input for MAPC must completely model the protocol domain.
MAPC will use the input to generate a new protocol. Therefore, the input
must model all relevant aspects of the protocol domain in which the protocol
must operate. This model must define the complete environment of the protocol
domain as well as the requirements of the final output of the system.
I2: The input must use a clear, concise, machine-readable representation.
MAPC will be software, and therefore must accept its input in a form that can
be parsed by a computer. In addition, this input may be created or modified by
a human designer, so it must also be understandable by humans.
I3: The input files should be easy to write. In order for MAPC to be generally
useful and used by many individuals, it must be simple to use. Therefore, it
should be easier to create a model of a protocol domain than to create a protocol
that satisfies the goals and requirements of the model.
3.3 Protocol Requirements
The output of MAPC must also meet certain requirements in order to be useful.
Following are the requirements of the output for MAPC. These follow from Requirement
42
C2 (MAPC must have a clearly defined output format for the generated protocols).
There are many different forms that protocols can take in multi agent systems
(Singh, 1999). Once MAPC has created a protocol, it should be possible to use
that protocol in any multi agent system. In order to do that, it must be possible
to evaluate and analyze the steps of the protocol. Therefore, the protocol must be
understandable by a person. This is the first requirement of the output. In addition
to being understandable by a human, it is intended that the generated protocol be
usable in a multi agent system, and so the output must therefore be machine readable.
This is the second requirement.
P1: The protocol output must be understandable by a human. The result
of MAPC should be a protocol that can be read and manipulated by a human
for potential adaptation to another multi agent system.
P2: The protocol output must be in some machine-readable format. The
result of MAPC should be a protocol that can be read into some multi agent
system and used. This requires that the protocol be designed in such a way that
it can easily be used.
3.4 Genetic Programming Requirements
The following are the requirements of the genetic programming component of MAPC
and follow from Requirement C3 (The requirements for genetic programming must be
met).
In order to create protocols, MAPC will make use of genetic programming (see
Section 2.4, p. 15). This requires determining a valid representation for potential
43
protocols. This is the first programming requirement. In order to create protocols,
MAPC must be able to generate protocols. Therefore, the second requirement is that
MAPC must be able to generate possible protocols. Finally, in order to determine
which protocol to choose amongst a population, MAPC must be able to evalute
potential protocols. This is the third requirement.
G1: The protocols must have a genetic-programming friendly representa-
tion. It is intended that genetic programming will be used to create the protocols
in MAPC. To facilitate this. there must exist a way to represent the protocols
in a way that is compatible with standard genetic operators used in genetic
programming.
G2: MAPC must be capable of generating potential protocols. In order to
produce a satisfactory protocol, the system must be able to generate potential
protocols. These protocols need to be in a suitable format, ensuring that all
generated protocols are in a valid format.
G3: MAPC must be capable of evaluating the suitability of potential pro-
tocols. In order to meet the requirements outlined in the input, potential
protocols must be evaluated. These evaluations will make use of the information
provided in the input.
3.5 Model Requirements
This section describes the requirements of the model which will be given as input to
MAPC. These requirements follow from Requirement I1 (The input for MAPC must
completely model the protocol domain).
44
This dissertation aims to create MAPC, a multi agent protocol creator. This
requires a well designed method for modelling any protocol domain in which a new
protocol must operate. The input model must meet four requirements. As with any
multi agent system, the first component will be the agents within the system. The
first requirement is therefore an ability to define the range of agents that may be
present in the protocol domain. In addition to the agents, every protocol domain will
have some resources and capabilities specific to that protocol domain. The second
requirement is that there is a suitable method for modelling protocol domain-specific
agent capabilities and resources. In each protocol domain, the data needed to describe
and keep track of the environment, including the various states of agents in that
environment, must be known. The third requirement is that the protocol domain and
agent-specific data for the protocol domain must be included in the model. In order
to outline the desired end results for a protocol domain, goals and requirements must
be given. The ability to describe the goals and requirements of a protocol domain is
the fourth requirement of the model.
M1: The model must describe the agents in the protocol domain. In a multi
agent system, most, if not all, of the protocol domain (or environment) is given
by the agents that exist in the system. Therefore, to describe the protocol
domain in which a protocol is to be created, the agents in the system must be
known. The model must include methods to outline the number and types of
agents that exist in a protocol domain.
M2: The model must describe the protocol domain-specific agent capabil-
ities. Along with the agents, a multi agent system is also defined by the actions
that can be taken by the agents within that system. The protocol domain specific
45
actions may only be performed in a certain context and may result in some
change to the system. The protocol domain model must allow the modelling of
both the prerequisites for an action and the resulting changes when an action
has been performed.
M3: The model must describe the data associated with the protocol do-
main. Within a protocol domain, certain pieces of data must be maintained.
These include certain resources held by agents that are specific to a protocol
domain. In addition the model must describe all relevant internal data for each
agent that may exist in the system,
M4: The model must outline the goals and requirements of the protocol
domain. In most multi agent systems, certain actions and outcomes are
unacceptable while other actions are necessary. The protocol domain model
must include these requirements. In addition to basic requirements, a multi
agent system exists to achieve some goal. Without a goal, there is no reason
to create the system. Goals, therefore, must also be included in the protocol
domain model. In addition to the protocol domain as a whole, there may be
goals and requirements for individual agents, so these must be modelled too.
3.6 Requirements Summary
The requirements of MAPC are summarized visually in a concept map in Figure 3.1.1
1A concept map represents knowledge in a graph (Judith G. Lambiotte and Reynolds, 1989). Thevectors (boxes) are the parts of the system (both actual and conceptually). The edges (lines) describethe relationships between the vectors.
46
MAPC
Input
Protocol Domain Model
Agents
Capabilities
Data (Environment)
Goals
Machine-Readable
Easy to Write
Output
Protocol
Understandableby Humans
Genetic Programming
Genetic Representation
Protocol Evaluations
uses
produces
uses
defines
have
describes
describes
describes
describes
must be
is
is
isis a
Well Defined is
uses
uses
uses
Test Agents
executes
Scenario
derived from
run in
has a
Figure 3.1: MAPC Components
Chapter 4
Protocol Domain Model
This chapter describes the protocol domain model in order to fulfill Objective O3
(Design a protocol domain model, p. 4). Section 4.1 describes the model informally.
This description makes use of a simple but non-trivial example to illustrate the various
components of the model. Section 4.2 gives a formal definition of the protocol domain
model in first-order logic. Section 4.3 outlines the textual representation that has been
developed to define and store protocol domain models as input for MAPC. Section
4.4 discusses the ease of writing protocol domains compared to traditional protocols.
Finally, Section 4.5 offers a summary of the model. An earlier version of this model
was presented as a poster (Heard and Kremer, 2012).
4.1 Informal Description
In MAPC, a protocol domain is modelled with three separate components: (1) a set of
possible scenarios,1 (2) a set of role types that roles in scenarios will conform to, and
(3) an outline of the digital environment or data (which is composed of information
that may be different for each execution of the protocol domain).
An example will be used to illustrate the three components of the protocol domain
model. Consider the comparison shopper protocol domain. In the comparison shopper
protocol domain, a single purchasing agent is looking to purchase a particular item
and has multiple potential suppliers to choose from. The purchasing agent is looking
1The concept of scenarios is similar to that used by Paskaradevan and Denzinger (2012).
47
48
for the lowest price but also has a maximum price that cannot be exceeded. The
sellers each may or may not have the item in question. If they do have the item, they
also have a minimum selling price, but wish to maximize that price.
To organize the description of the agents in the system, each protocol domain
has one or more role types defining the capabilities of various types of agents in the
system. The role types each describe the actions agents of that role type can perform,
the desires that agents of that role type would like to fulfill, the negotiable values
that can be used when fulfilling their desires, and the requirements of that role type.
The inclusion of role types in the protocol domain model fulfills Requirement M2
(The model must describe the protocol domain-specific agent capabilities). In the
comparison shopper example, there are two role types named buyer and seller. The
buyer is capable of giving money to another agent, the give money action. The seller
is capable of giving items to other agents, the give item action. In Figure 4.1, the
role types are represented by large clouds. Within the role types, the actions are
represented as lightning bolts. In addition to their actions, the buyer and seller role
types each have desires to receive what the other can give: the buyer has a desire for
a give item action with the buyer agent as the recipient, and the seller has a desire
for the give money action with the seller agent as the recipient. These desires are
represented within the role types as hearts. To aid in designing logical protocols, both
the buyer and the seller have a negotiable value price which the seller would prefer to
be higher and the buyer would prefer to be lower. These are represented in the figure
with bi-directional arrows. Finally, the role types each have specific requirements. In
the comparison shopper protocol domain, there are requirements that an item must
be exchanged for money and that no agent sells outside the acceptable range for that
49
agent. In the diagram, the requirements are represented by octagons (“stop signs”).
A scenario in a protocol domain is defined by a set of roles, a set of environments,
and a starting environment. Each role conforms to some role type. In Figure 4.1, the
scenarios are diamonds containing smaller clouds that are the roles. The arrows shown
for the first scenario illustrate the conforming relationship of the roles in that scenario:
two of the roles conform to the seller role type while one conforms to the buyer role
type. In addition, the small collection of cylinders represent the environments in each
particular scenario. The starting environment is shown as a cylinder with a star.
The domain information states of a protocol domain and the scenario environments
represent collections of data that completely describe a single moment in a multi
agent system. This data is composed of three types of data. It includes information
that is constant over all instances of the protocol domain, information that may be
different for different instances of the protocol domain but is constant throughout the
entire timeline of one execution of the system, and information that may change as
the system executes. In addition, data may be global to the entire protocol domain
or specific to one or more role types in the protocol domain. Requirement M3 (The
model must describe the data associated with the protocol domain) is fulfilled by
the inclusion of the environment in the model. In the comparison shopper example,
there isn’t any global data, but there is data specific to each role type. For example,
all transactor agents have a money and item count variable. In Figure 4.1, data is
represented by cylinders.
In addition, when discussing a protocol domain and the evolution of a protocol
that will satisfy a protocol domain’s requirements, the term execution will be used.
An execution of a scenario requires a given protocol or individual and produces a
50
Domain Information StatesRole Types
Scenarios
...
subset ofconforms to
...
......
sellerrole type
buyerrole type
Figure 4.1: Comparison Shopper Protocol Domain
51
sequence of domain information states from the scenario’s starting environment to the
moment when MAPC has terminated the execution due to reaching the time limit
(described in Section 4.3).
4.2 Formal Definitions
4.2.1 Protocol Domain
Formally, a protocol domain (Dom) is defined as the triple:2
Dom = (Scn, Typ,DI)
where Scn is the set of scenarios for the protocol domain, Typ is the set of role types
for the protocol domain and DI is the set of all possible domain information states.
A protocol domain is therefore defined by the possible scenarios that it contains, the
general types of agents that exist in the various scenarios, and the domain specific
information.
A scenario in a protocol domain (scnn ∈ Scn) is defined by the triple:
scnn = (Rol, Env, env0) s.t.
Env ⊂ DI ∧ env0 ∈ Env
where Rol is the specific set of roles in the scenario, Env is the set of possible
environments for this scenario and env0 is the initial environment state for this
scenario.
2For a detailed explanation of the logical symbols used to describe MAPC, please see Appendix A.
52
The set of domain information states (DI) enumerate all possible environments
that any multi agent system making use of that domain may contain. Each scenario
within the protocol domain will contain an environment (Env) that is a subset of the
domain information states.
A role type in a protocol domain (typn ∈ Typ) is defined as the 5-tuple:
typn =(Sittypn , Acttypn , Desiren, Negn, Reqn
)s.t. Sittyp ∈ DI
where Sittypn is the set of situations that agents of this role type may be in, Acttypn
is the set of actions that agent of this role type may perform, Desiren is the set of
desires for the role type, Negn is the set of negotiable values for the role type and
Reqn is the set of requirements for the role type.
An action in a role type (am ∈ Acttypn ) is defined as a function that modifies the
environment and/or data instance of an agent or agents. An action can take any
number of parameters (pi), each with a different input domain (ii). An action may
have different results in different circumstances.
am : p1 × . . .× pn ×DI → DI s.t.
p1 ∈ i1 ∧ · · · ∧ pn ∈ in
A desire in a role type (dm ∈ Desiren) is a partially defined action where am is the
desired action, p′n is either a parameter that is in the input domain of the parameter
(ii) or ⊥ to indicate that the parameter is undefined. Typically, a desired action is
53
one that is performed by another agent.
dm = am (p′1, . . . , p′n,⊥) s.t.
am ∈ Acttypn ∧ p′1 ∈ i1 ∪ ⊥ ∧ · · · ∧ p′n ∈ in ∪ ⊥
A negotiable value in a role type (nm ∈ Negn) is defined as a pair:3
nm = (valm, prefm) s.t.
valm ⊂ P Z
where valm is a set of integer values and prefm is a partial ordering of the values in
valm which indicates the preferences of an agent with this negotiable value.
A role in a scenario (roln ∈ Rol) is defined as a 3-tuple:
roln = (typn, Desirerol, Negrol) s.t.
typn ⊂ Typ ∧Desirerol ⊂ Desiren ∧Negrol ⊂ Negn
where typn is the role type that this role conforms to. Desirerol and Negrol are the
sets of desires and negotiable values that this role has (each of these sets is a subset
of the respective set of the role type).
3The symbol P is used to denote the powerset.
54
The requirements of a role type (Reqn) are given as the pair:
Reqn = (reqh, reqs)
reqh : DI → B
reqs : DI → Z
where reqh are the hard requirements and reqs are the soft requirements. The hard
requirements are defined as a function from a domain information state to a boolean
value of true for success and false for failure. The soft requirements are defined a a
function from a domain information state to an integer value indicating how well the
soft requirements were met (a lower value is better).
4.2.2 Agents in MAPC
Agents in MAPC can be defined in terms of a protocol domain and the set of role
types that the agent conforms to.
An agent fits a role type if its set of situations contains the set of role type specific
situations, its set of actions contains the set of role type specific actions and its
data state can contain the desires and negotiable values in the role type. To aid in
describing the agent, let Datimp be the set of all non-role-type-specific data that the
agent can hold in its internal data. An agent can therefore conforms to an entire
protocol domain. In this case, an agent conforms to all of the role types in the protocol
55
domain. Formally:
conformstyp : Ag × Typ→ B
conformstyp(agn, typn) = Sittypn ⊂ Sit ∧ Acttypn ⊂ Act ∧Desiren ×Negn ×Datimp = Dat
conformsdom : Ag ×Dom→ B
conformsdom(agn, domn) = ∀typm ∈ Typ : conformstyp(agn, typm)
4.3 Textual Representation
To be used as input for MAPC, the format of the protocol domain must be such that
it can be processed by a computer. To simplify the format a text-based representation
was chosen. Because of the need for both static data and computational units (for
defining actions, etc), a Lisp data structure was chosen. The remainder of this
subsection will outline the Lisp functions that form the basis of the protocol domain
description, highlighting the changes made in the transition from the formal definitions
to a textual representation, thus fulfilling Requirement I2 (The input must use a clear,
concise, machine-readable representation).
The agents in the protocol domain will be described as a set of agent groups. Role
types are kept separate from the quantity of each agent type. This allows protocol
domains to be described in which multiple agents share one role type but have some
other differentiation. Each group has a set of one or more role types that the agents in
that group take on. The role types define the abilities of a general class of agents and
since they may be shared between agent groups they are described separately, below.
In addition, each agent group has a defined quantity. The quantity describes the
possible numbers of agents in that group. This fulfills Requirement M1 (The model
56
(protocol-domain name role type(s) agent-groups [:globals
variable(s) ] [:fitness fitness ])
(role-type name [:super-type name ] [:desires desire(s) ]
[:actions action(s) ] [:variables variable(s) ] [:negotiables
negotiable value(s) ] [:fitness fitness ])
(agent-group name role type(s) minimum maximum )
(action-desire name [:required T] [:times times desired ]
[:parameter parameter value ]*)
(action name actor name function [:parameters variable(s) ]
[:pre-conditions pre-conditions ])
(variable name type [:initial-value initial value ] [:constant
T]
(negotiable name type prefer [:minimum minimum value ]
[:maximum maximum value ]
(fitness [:post-condition Boolean function ] [:maximize
numeric function ] [:minimize numeric function ])
Figure 4.2: The Lisp functions used to create a textual description of a protocoldomain.
must describe the agents in the protocol domain). In the comparison shopper protocol
domain, there are two agent groups: one agent group with a quantity of one with role
type of buyer (to be described below), and a second agent group with a quantity from
one to eight inclusive where each agent has the buyer role type.
To enhance the capability to describe the role types, the role types can be arranged
in a type hierarchy to indicate inheritance. There is a base role type called transactor
that represents any agent with money and/or items.
4.3.1 Lisp Functions
The Lisp functions used to describe a protocol domain in a textual representation
are listed in Figure 4.2. The functions generally each represent one concept from the
formal definition of a protocol domain. For example, the seller role type is represented
57
by a Lisp function that begins:
(role-type "seller" ... )
The organization of the textual representation is not the same as the formal definitions.
First, most of the components are named to allow cross reference between portions
of the text file. The protocol domain, role types, agent groups, actions and variables
all have names. Second, the hard and soft requirements are distributed throughout
the text file. This helps to ease the description of requirements that apply only to
a specific role type or action. For the protocol domain and the role type a fitness
definition function is used. The fitness definition describes any hard requirements
under the name post-condition. The soft requirements are given as functions to
minimize or maximize for each individual in the role type or the protocol domain
in general. Finally, to ensure actions occur only when appropriate, all actions have
pre-conditions that must be met in the instant before an action occurs. A short
description of each Lisp function follows:
protocol-domain Constructs a protocol domain. This requires a name for the proto-
col domain, one or more role types and one or more agent groups. In addition,
it can optionally accept one or more global variables, a fitness description and
optional evolutionary parameters, described below.
role-type Constructs a role type. This requires a name for the role type. In addition,
it can optionally accept a super-type role type, one or more desires, one or more
actions, one or more variables and a fitness description.
agent-group Constructs an agent group. This requires a name for the agent group,
one or more role type names, the minimum number of agents in the group and
58
the maximum number of agents in the group. The role type name or names
must match the names of role types that exist in the protocol domain.
action-desire Constructs an action desire. This requires a name for the action that
is desired. In addition, the desire may be marked as required, the number of
times the specified action is desired may be indicated, and values appropriate to
the action may be specified for each parameter.
action Constructs an action. This requires a name for the action, a name for the
acting agent in the remainder of the action description and the action function
as a Lisp function. In addition, it can optionally accept one or more parameters
and a pre-condition. Each of the parameters are variables. The pre-condition
is a Boolean function that must return true for the hard requirements of the
action to be met.
variable Constructs a variable. This requires a name and a type for the variable.
In addition, it can optionally accept an initial value for the variable and an
indication that the variable is constant throughout the execution of the protocol
domain.
negotiable Constructs a negotiable value. This requires a name for the negotiable
value, a type and a preference (for example, if the type is a numeric type, the
preference could be “higher,” or “lower”) In addition, it can optionally accept a
minimum and a maximum value.
fitness Constructs a fitness description. This can optionally contain a post-condition,
a function to maximize and a function to minimize. The post-condition is a
59
Parameter Lisp Keyword Default Value
# of Evaluation Threads evaluation-threads 2# of Individuals population-size 500
Execution Time (in milliseconds) run-time 1000Scoring Time (in milliseconds) score-time 1000
# of Generations last-generation 50
Table 4.1: Evolutionary Parameters in Protocol Domain Input Files
Boolean function that must return true for the hard requirements to be met. The
functions to maximize or minimize are used to represent the soft requirements
of the protocol domain.
In addition to a description of the protocol, the input file allows for a few parameters
to be set for the evolutionary process. These parameters define the number of threads
used to evaluate protocols simultaneously, the number of individuals in each generation,
the time given to agents to execute the protocol, the time given to MAPC to score
a single execution and the number of generations before terminating the evolution.
These parameters are given as optional parameters to the protocol-domain function.
These parameters are summarized in Table 4.1
4.3.2 A Textual Representation of the Comparison Shopper Protocol Do-
main
To see the various components of the textual representation together, consider the com-
parison shopper protocol domain in Lisp form. At the highest level, comparison-shopper-
.lisp contains a single function call as shown in Listing 4.1. Within the protocol-domain
function call are only two required pieces of information. The first is a list of role
types and the second is a list of agent groups. The transactor, seller and buyer role
60
(protocol-domain "comparison shopper"
;role-types
‘(
,(role-type "transactor"
... )
,(role-type "seller"
... )
,(role-type "buyer"
... ))
; agent groups
... )
Listing 4.1: comparison-shopper.lisp
(role-type "transactor"
:variables (list
(variable "item-count" "integer"
:initial-value ’(random 5))
(variable "money" "integer"
:initial-value ’(+ 100 (random 100)))))
Listing 4.2: Transactor Role Type
types and the agent groups are each described below.
The comparison shopper protocol domain is primarily described through the
definition of role types. The comparison shopper protocol domain has three role types:
transactor, seller, and buyer (see Section 4.1). The transactor role type holds the data
common to the seller and buyer role types (that is, the amount of money and number
of items). As shown in Listing 4.2, the transactor role type acts as a container for
these two values. In order to restrict the amount of money and number of items to a
reasonable range, initial values are given for these two variables.
The seller role type definition is shown in Listing 4.3. The role type definition
61
includes a reference to the transactor super-type role type, a desire for the give-money
action with this agent as the reciever, the give-item action, three variables, a negotiable
value, and a fitness description. By indicating that the transactor role type is a super-
type of the seller role type, the seller role type inherits all of the actions, variables,
negotiable values and fitness descriptions of the transactor role type. This means that
the three variables that are described in the seller role type are in addition to the two
variables described in the transactor role type. Therefore, any agent with the seller
role type has five variables.
The give-item action within the seller role type is used by seller agents to give
an item to a buyer. The action definition includes a Lisp function used to modify
the environment (or protocol domain data instance). In the case of give-item, the
giver’s number of items is reduced, the receiver’s number of items is incremented, and
a separate counter for the number of item transactions is increased. The parameter of
the action includes only the agent who will receive the item. The pre-condition for
the action is that the giver has an item to give.
The seller has a negotiable value called price that indicates that the seller would
rather have a higher price and that the range of valid prices is from the seller’s
minimum price up to 20 (an arbitrarily chosen value).
The fitness description for the seller role type includes a post-condition to ensure
that the number of items given by a seller matches the number of times the seller
received money (to ensure one money transfer per item transfer).
The buyer role type is defined in Listing 4.4. The buyer role type is composed of
an indication that it is a subtype of the transactor role type, a desire for the give-item
action with this agent as the receiver, the give-money action, a variable definition, a
62
(role-type "seller" :super-type "transactor"
:desires (list
(action-desire "give-money" :receiver
’(this-agent)))
:actions (list
(action "give-item" "giver"
’(list (set-variable giver "item-count" (-
(get-variable giver "item-count") 1))
(set-variable giver "items-given" (+
(get-variable giver "items-given") 1))
(set-variable receiver "item-count" (+
(get-variable receiver "item-count") 1)))
:parameters (list
(variable "receiver" "agent"
:initial-value ’(random-agent :role-type
"buyer")))
:pre-conditions ’(> (get-variable giver
"item-count") 0)))
:variables (list
(variable "minimum-price" "integer"
:initial-value ’(+ 1 (random 10)))
(variable "items-given" "integer"
:initial-value 0)
(variable "money-received" "integer"
:initial-value 0))
:negotiables (list
(negotiable "price" "integer" "higher" :minimum
’(get-variable (this-agent) "minimum-price")
:maximum 20 ))
:fitness (fitness
:post-condition ’(= (get-variable (this-agent)
"items-given") (get-variable (this-agent)
"money-received"))))
Listing 4.3: Seller Role Type
63
negotiable value definition and a fitness description. As with the seller role type, the
buyer role type inherits all of the features of the transactor role type.
The desire for the give-item action in the buyer is required (indicating that the
desire must be fulfilled) and can be desired from one to three times.
The give-money action is used by buyer agents to give money to a seller. The
action defines a Lisp function that modifies the environment. The modifications
include changing the amount of money held by the giver and receiver of the money.
In addition, the counter of the number of times money was received by the seller is
increased. The give-money action has two parameters: the agent who will receive the
money and the amount of money to transfer. The pre-condition for the give-money
action requires that three conditions be met: the giver must have the amount of money
needed, the amount transferred must be at least the seller’s minimum price, and the
amount transferred must be at most the buyer’s maximum price.
The buyer has a negotiable value called price that indicates that the buyer would
rather have a lower price and that the valid range for prices from the buyer’s perspective
is from 0 (free is nice) to the buyer’s maximum price.
The fitness description for the buyer role type includes a post-condition to ensure
that the buyer has acquired the correct number of items. In addition, there is a
numeric function which sums up the total value of the money and items in the buyer’s
possession (using the buyer’s maximum price as the value of the item for the buyer).
Since the numeric function is in the maximize section of the fitness description, this
should lead MAPC to create protocols that result in a larger final value. This should
make the system prefer solutions which favour the buyer over the seller, as was
indicated as a design requirement above.
64
(role-type "buyer" :super-type "transactor"
:desires (list
(action-desire "give-item" :required T :times ’(+
1 (random 3)) :receiver ’(this-agent)))
:actions (list
(action "give-money" "giver"
’(list (set-variable giver "money" (-
(get-variable giver "money") amount))
(set-variable receiver "money" (+ (get-variable
receiver "money") amount))
(set-variable receiver "money-recieved" (+
(get-variable receiver "money-recieved") 1)))
:parameters (list
(variable "receiver" "agent"
:initial-value ’(random-agent :role-type
"seller"))
(variable "amount" "integer"
:initial-value ’(+ 1 (random 10))))
:pre-conditions ’(and
(>= (get-variable giver "money") amount))
:post-conditions ’(and
(>= amount (get-variable receiver
"minimum-price"))
(<= amount (get-variable giver
"maximum-price")))
))
:variables (list
(variable "maximum-price" "integer"
:initial-value ’(+ 10 (random 10))))
:negotiables (list
(negotiable "price" "integer" "lower" :minimum 0
:maximum ’(get-variable (this-agent)
"maximum-price")))
:fitness (fitness
:maximize ’(+ (get-variable (this-agent) "money") (*
(get-variable (this-agent) "item-count")
(get-variable (this-agent) "maximum-price"))))))
Listing 4.4: Buyer Role Type
65
(list
(agent-group "seller-group" "seller" 1 8)
(agent-group "buyer-group" "buyer" 1 1))
Listing 4.5: Agent Groups
:evaluation-threads 10
:population-size 500
:run-time 4000
:score-time 4000
:last-generation 50
Listing 4.6: Evolutionary Parameters
In addition to the role types of transactor, seller and buyer, the protocol domain
is described by the two agent groups. As shown in Listing 4.5, the groups are the
seller-group, which has the seller role type and one to eight agents, and the buyer-group,
which has the buyer role type and one agent.
Finally, the evolutionary parameters are given as optional parameters to the
protocol-domain function. The values for the comparison shopper protocol domain
are shown in Listing 4.6.
4.4 The Ease of Modelling Protocol Domains
The goal of this work was to create a means to automatically generate protocols
from some model of the protocol domain in which the protocols must operate. An
intended side-effect of this work was to ease the process of creating protocols for multi
agent system designers. This was expressed in Requirement I3 (The input files should
be easy to write). While the protocol domain files are quite large (and, as shown
in Section 5.2, p. 69, longer than the complete protocols), they include information
66
that the designer should know about the system being designed, and are primarily
declarative about how the protocol should work independent of how it will accomplish
any stated goals. Therefore, with MAPC, the process of creating a complex protocol
domain would be easier than creating a complex protocol and this requirement has
been met.
4.5 Summary
A protocol domain in MAPC is modelled as scenarios, role types, and domain in-
formation. The scenarios each define a set of roles and a set of environments along
with a starting environment. The role types define the types of agents in the protocol
domain and include the abilities, data and information about those agents. The
domain information defines the information about the protocol domain as a whole
as well as the individual agents in the system. Together, these components give a
complete picture of a general multi agent system, without constricting that system to
a limited number of specific configurations.
To facilitate the creation of MAPC, a text-based representation of the protocol
domain was created. This varies from the model in that it defines rules for creating
scenarios instead of describing every scenario individually. In addition, to allow re-use
of portions of the agents, the roles can be organized in a type hierarchy in the textual
representation. The agent groups define the number and types of agents in the protocol
domain. The requirements outline what must occur when a potential protocol is
executed.
Chapter 5
Individuals
Individuals in MAPC are potential protocols. In order to evolve protocols, a specific
format was selected. This fulfils Objective O4 (Design an output format, p. 5). This
chapter describes the protocol format in general terms, as it is read by the agents,
and as it is maintained by the evolutionary system. Section 5.1 describes the general
format of the protocols. Section 5.2 states how the protocols can be represented as Lisp
files, which is how agents read the protocols. Section 5.3 outlines how the protocols
are represented as trees in the evolutionary system. Finally, Section 5.4 summarizes
the individuals in MAPC. Throughout this chapter, the comparison shopper protocol
domain is used as an illustrative example (see Section 4.1, p. 47 for an informal
description of this example).
5.1 Protocol Description
The aim of this dissertation is to allow multi agent designers to create new protocols
automatically. It is intended that the final protocols created by MAPC could be used
in a wide variety of multi agent systems. Therefore, the output from MAPC must
be a protocol described at a high level. This will also meet Requirement P1 (The
protocol output must be understandable by a human, p. 42).
The format of the protocols is a proposal-based format. If an agent would like
another agent to perform some act, then a proposal is sent to that effect. In order
to determine if the proposal should be accepted, the recipient of the proposal would
67
68
consider the proposal and either accept or refuse the proposal.1
The individuals in MAPC are defined by two sets. These sets are the desire fulfillers
and the consider functions. The desire fulfillers are used by the agents to drive the
execution and to serve as the starting point for the protocol. They provide a function
which, once an acceptable individual is found, will start the communications necessary
to potentially bring about the fulfillment of the desire. There is one desire fulfiller for
each unique type of desire in the protocol domain model. To clarify this description,
consider the comparison shopper protocol domain. In this protocol domain, there are
two types of desires, one for the give item action and one for the give money action.
The desire fulfillers are composed of one or more proposals. A proposal asks that
another agent perform some action. In addition to the basic proposals, conditional
proposals can be sent. These proposals allow the sender to put a condition on the
proposal so that if it is accepted, the proposal is in effect once the condition is met.
In the comparison shopper protocol domain, a buyer agent might send a proposal that
a seller give the buyer an item on the condition that the buyer has given the seller
money.
The consider functions serve as decision functions to ensure that agents will only
accept the proposal for those actions that are appropriate for the protocol domain.
There is one consider function for each action defined in the protocol domain model.
In the comparison shopper example, there are two actions: the give money action and
the give item action. Therefore, in the individuals (or possible protocols), there are
two consider functions.
The consider functions make use of a special function. This function allows for
1This is similar to the propose performative described by FIPA (2002a).
69
proposals to be accepted only if they are connected to some other action. In other
words, they require that some condition be attached to the proposal. For example, a
buyer might only accept a request that he give another agent an item on the condition
that the other agent is also giving the buyer money of an acceptable amount.
In addition, the negotiable values defined in the protocol domain can be used to
describe parts of the desire fulfillers and the consider functions. In the desire fulfillers,
a negotiable value may be used to fill in a value that is needed in a proposal. For
example, the price negotiable value might be used in a proposal to give a value to the
amount of money given. In a consider function, a negotiable value may be used to
ensure that an acceptable value is in a certain part of a proposal or condition. In the
comparison shopper protocol domain, a consider function may verify that a value is
within the acceptable bounds of the price negotiable value for an agent.
5.2 Lisp Representation
The protocols generated by MAPC have two main representations. The evolutionary
system uses a tree-based format, while the test agents receive a text-based format.
The test agents receive the protocols (or individuals) via an initialization file, which
is written in Lisp. This format is easily parsed by a computer, thus meeting Re-
quirement P2 (The protocol output must be in some machine-readable format, p. 42).
There are three Lisp functions which define this structure. They are mapc-agent,
action-desire-fulfiller and consider-action. The mapc-agent function serves
as a holder for the desire fulfillers and the considerers. The action-desire-fulfiller
takes the name of the action which it will attempt to fulfill as its first parame-
ter and a function that, once executed, will lead to the action taking place. The
70
(mapc-agent
:fulfillers (list
(action-desire-fulfiller "give-item"
’(nop))
(action-desire-fulfiller "give-money"
’(nop)))
:considerers (list
(consider-action "give-money"
’(T))
(consider-action "give-item"
’(NIL))))
Listing 5.1: The Null Protocol
consider-action function takes the name of the action it will consider, and a Boolean
function which will be executed to determine if an agent will accept a proposal to
perform that action. In Listing 5.1 an example protocol is shown. This protocol will
be referred to as the null protocol. This protocol is the simplest possible individual for
the comparison shopper protocol domain.2 The desire fulfiller functions are both the
nop terminal, which does nothing, while the considerers are NIL and T, which are the
Lisp equivalent of false and true, respectively.
In order to fulfil a desire for the performance of an action, there are two action-
related Lisp functions. The first, propose-action, proposes to another agent that it
performs some action. It can optionally take another action as a condition, allowing
the creation of a proposal that, if accepted, will be conditional upon the second action’s
occurrence, similar to the FIPA “propose” performative (2002a; 2002b). The second
action-related Lisp function, simply perform-action, allows an agent to perform an
2While the null protocol is the simplest possible individual, that does not imply that it would bea successful protocol. In fact, agents using this protocol would send no messages and perform noactions. Therefore, as described in Section 6.3.3, p. 92, the execution score for any execution usingthis protocol will include a heavy penalty for the inaction.
71
action of its own accord.
To aid in the definition of the condition(s) under which a proposal should be
accepted, MAPC provides the is-connected function. The is-connected function
accepts a description of an action and returns a true value if that action is either the
proposed action or the condition on the proposed action.
See Listing 5.2 for an example individual that makes use of the propose-action
and the is-connected functions. This is the buyer protocol. The buyer protocol
represents an acceptable protocol for the comparison shopper protocol domain. In
the example, the values of the action’s parameters as well as the actor name from
the action (see Section 4.3, p. 55) may be used in both the desire fulfiller and the
considerer. For the case of the desire fulfiller, these values are selected by the agent
who is attempting to fulfil its desires. For the case of the considerers, these values are
derived from the proposal that is under consideration. For example, an agent who
wishes to fulfil a desire for the give-item action must pick the giver and the receiver.
In the comparison shopper protocol domain, the receiver is defined by the desire to
be the agent with the desire. The giver would be chosen by the agent, but may not
choose to accept the proposal. The agent receiving the proposal would have the giver
and receiver defined as those in the proposal the agent is considering.
5.3 Tree Representation
In MAPC, protocols, or individuals, are represented as collections of trees.3 Each
individual has a separate tree for each desire fulfiller and consider function. Each of
the trees is composed of functions and/or terminals. Functions and terminals can only
3See Section 2.4.2, p. 18 for a description of tree-based genomes in genetic programming.
72
(mapc-agent
:fulfillers (list
(action-desire-fulfiller "give-item"
(propose-action giver "give-item" :condition
(action-descriptor "give-money" :actor receiver
:receiver giver :amount (use-negotiable
"price")) :receiver receiver))
(action-desire-fulfiller "give-money"
’(nop)))
:considerers (list
(consider-action "give-money"
’T)
(consider-action "give-item"
’(is-connected (action-descriptor "give-money"
:amount (verify-negotiable "price"))))))
Listing 5.2: The Buyer Protocol
be children of functions when they satisfy the type requirements of their parent and
fit into the context that the parent provides. This section describes these details as
follows. First, the types used in MAPC are described. Second, the context system
created for MAPC is outlined. Third, all of the functions and terminals used in MAPC
are described. These descriptions meet Requirement G1 (The protocols must have a
genetic-programming friendly representation)
5.3.1 Node Types
ECJ, the evolutionary system chosen for MAPC, facilitates the use of strongly typed
genetic programming.4 MAPC uses these facilities and makes use of 7 types. The first
two types are desire and consider. The trees that represent the desire fulfillers in
potential protocols must have a root node that returns the desire type. Similarly,
4See Section 6.1, p. 81 for a detailed description of ECJ and Section 2.5, p. 27 for a detaileddescription of strongly typed genetic programming.
73
Type Name Description
desire The root of a desire fulfiller function.consider The root of a consider function.integer An integer value.boolean A Boolean value.code Code that does something (and returns no value).agent An agent in a running system.constraint Some form of constraint or condition.
Table 5.1: Types in MAPC
the root nodes of all consider functions must return the consider type. The integer
and boolean types are used to indicate integer and Boolean data values. The code
type is synonymous with the void type and indicates a function that returns no value.
Typically, nodes that return the code type have some side-effect such as performing
an action or changing a variable. The agent type is used to denote some agent
in a running system. Finally, the constraint type describes some constraint or
condition on a proposal either proposed or required. Typically, desire fulfillers will
use a constraint to indicate an intended condition, while consider functions will use a
constraint to indicate a required condition that is needed in order to accept a proposal.
The types are summarized in Table 5.1.
5.3.2 Context Aware Nodes
During the creation of MAPC, it was found that certain functions and terminals were
unreasonable in specific locations in a tree. For example, a subtree that is evolving
to be a desire fulfiller for a seller should not be able to attempt a give-money action,
since a seller can not perform that action. To reduce the number of invalid individuals
generated by the system, MAPC adopted the novel technique of the context system.
74
Context Name Description
desire Within a desire fulfiller function.consider Within a consider function.vague Within a potentially vague constraint.no-role Within code that will be run by all roles.role type name Within code that will be executed by agents of the given
role type.action:action name Within an area of the protocol specific to the given action.
Table 5.2: Contexts in MAPC
Context can be used to refine where a function or terminal may be used. Each
function and terminal has a list of required and forbidden contexts. The function can
only exist in a position where its required contexts are provided (by an ancestor) and
its forbidden contexts are not provided. In addition, each function can provide or
absorb contexts.
In MAPC, there are 4 contexts that are used in all protocol domains. These include
the desire, consider, vague and no-role contexts. The desire context indicates
that the nodes are within a desire fulfiller function. Functions that can only exist in
a desire fulfiller would require the desire context. The consider context indicates
that a node is within a consider function. The vague context indicates that the nodes
are within a constraint that can be vague on specific details. In this way, non-specifc
constraints can be restricted to locations in which they are acceptable. Finally, the
no-role context indicates that the nodes are in a location that could be executed by
all roles. Typically, this context is forbidden for nodes that require a specific role type.
In addition to the four static contexts, each protocol domain has additional contexts
for each role type and action in the protocol domain. The contexts are summarized in
Table 5.2.
75
5.3.3 Primitive Set
MAPC has 13 different functions and terminals (collectively called nodes), that are
used during the evolution of any protocol. These are referred to as the static nodes of
MAPC. In addition, there are several functions and terminals that are generated from
the actions, variables and negotiable values within each specific protocol domain. Each
static node is described below. The static functions and terminals are summarized in
Tables 5.3 and 5.4, respectively.
Desire Fulfiller The desire fulfiller function is the root node for all desire fulfillers.
It takes a single code node as its only child. In a working protocol, at least one
child function will bring about a desire. The desire fulfiller node provides the
desire context to its children.
Consider Action The consider action function is the root node for all consider
functions. It takes a single boolean node as its only child. In a working protocol,
this function returns true to indicate that a proposal should be accepted, and
false to indicate that the proposal should be rejected. The consider action node
provides the consider context to its children.
Code Sequence The code sequence function connects two code nodes as its children.
When a code sequence node is executed by an agent, the two children of the
code sequence node are executed in order.
Is Connected The is connected function takes a constraint node as its only child.
This constraint is always checked against the proposal to see if it meets the
given constraint. If so, the is connected function returns true. If not, it returns
76
Input Type(s) ContextFunction Name Return Value #1 #2 Provided Required
Desire Fulfiller desire code - desire -Consider Action consider boolean - consider -Code Sequence code code code - -Is Connected boolean constraint - vague consider
Table 5.3: Static Functions in MAPC
false. The is connected function requires the consider context and provides the
vague context to its children.
Requestor The requestor terminal has the agent value of the requestor of a proposal
that is being considered by a consider function. It requires the consider context.
True (T) The true terminal has the boolean value true. When it is converted to
Lisp (as described in Section 5.2), it takes on the form T.
False (NIL) The false terminal has the boolean value false. When it is converted to
Lisp (as described in Section 5.2), it takes on the form NIL.
This Agent The this agent terminal has the agent value of the agent currently
executing .
Nop The code nop (short for Null OPeration) terminal is used to indicate that no
action should be taken.
Null (constraint) The constraint null terminal is used to indicate that no con-
straint should be placed on the parent proposal. It is only acceptable when the
consider context is provided. This prevents the null constraint from being the
77
Terminal Name Return Value Context Required
Requestor agent consider
True (T) boolean -False (NIL) boolean -This Agent agent -Nop code -null (constraint) constraint consider
null (integer) integer vague
null (agent) agent vague
null (boolean) boolean vague
Table 5.4: Static Terminals in MAPC
child of an is connected function, since the is connected function requires a
specific constraint to function properly.
Null (integer, agent, and boolean) The remaining null terminals are used as chil-
dren of the is connected function’s constraint, to indicate a vague constraint.
Therefore, they all require the vague context.
In addition to the static nodes, each protocol domain has several dynamically
generated functions and terminals. For every action in a protocol domain, a do action,
propose action and action descriptor function are generated and included in the
genetic programming system which will generate potential protocols. For each of these
functions, all roles that do not contain the associated action are forbidden contexts.
The action functions are summarized in Table 5.5
Do Action A do action function performs the associated action. A do action function
returns the code type and has one child for each parameter of the action.
Propose Action A propose action function sends a proposal to another agent to
request that the recipient perform the specified action. A propose action function
78
Input Type(s)Function Name5 Return Value #1 #2 #3
Do Action code action parameter(s) - -Propose Action code action parameter(s) agent constraint
Action Descriptor constraint action parameter(s) agent -
Table 5.5: Action Functions in MAPC5In addition to those listed here, all of the roles that do not contain the related action are forbidden
contexts for all action functions.
returns the code type. A propose action function has one child for each parameter
of the action in addition to two other parameters. One extra parameter is an
agent node indicating the agent that the proposal will be sent to. The second
parameter is a constraint node indicating that if the proposal is accepted the
constraint must be met before the associated action will be performed.
Action Descriptor An action descriptor function describes one or more associated
actions. An action descriptor function returns the constraint type and has
one child for each parameter of the action. Unlike the other two action-related
functions, an action descriptor function’s parameters are meant to filter actions,
not to describe how an action is performed. These filters describe the specific
actions that meet this descriptor. For example, an action descriptor for the
give money action might have an amount parameter of 12. In this case, only
actions of the give money type that have an amount of 12 would be met by the
descriptor. In addition, there is an extra agent child that indicates the agent
that must perform the action in order for an action to be met by this descriptor.
There are several terminals generated from the action parameters, variables and
negotiable values in a protocol domain. These are described in detail below and
79
summarized in Table 5.6.
Get Variable A get variable terminal retrieves the current value of the associated
variable. It has a return type that matches that of the associated variable. The
get variable terminal is forbidden in the no-role context.
Action Parameter An action parameter terminal retrieves the value of an action
parameter. It has a return type that matches that of the associated variable. An
action parameter terminal requires the action:action name context. An action
parameter terminal has a different meaning depending on whether it is within
a desire fulfiller or a considerer. Inside a desire fulfiller, the action parameter
terminal represents the value of a parameter that an agent has assigned to an
action to more completely describe its desire. Inside a consider function, the
action parameter has the value as it was proposed by the sender of the proposal.
Note: The actor of an action is an implied parameter that is also available in
this manner. This parameter has the agent type and has a name as indicated
in the definition of the action (see Section 4.3, p. 55).
Use Negotiable A use negotiable terminal retrieves an acceptable value for the
current agent for the given negotiable value. It has the same type as the
associated negotiable value. It is forbidden in the consider context.
Verify Negotiable A verify negotiable terminal resides inside an action descriptor
in a considerer. It checks the value of the action’s parameter in which it resides
against the current agents’ acceptable values for the given negotiable value. It
requires the consider and the constraint contexts and has a return type that
is the same as the associated negotiable value’s type.
80
ContextTerminal Name Return Value Required Forbidden6
Get Variable variable type - no-role
Action Parameter action parameter type action:action name -Use Negotiable negotiable value type - consider
Verify Negotiable negotiable value type consider, constraint -
Table 5.6: Dynamic Terminals in MAPC6In addition to those listed here, all dynamic terminals have all of the roles that do not contain the
related variable, action or negotiable value as a forbidden context.
5.4 Summary
The format of the individuals in MAPC was designed to be understandable by both
humans and computers. This results in a high level description of desire fulfillers and
considerers. The desire fulfillers are composed of actions and proposals that other
agents perform actions. The considerers are used by agents to determine if a proposal
should be accepted. This has been mapped onto a tree-based representation, used
internally by the evolutionary system, and a similar Lisp-based representation that is
used to give the protocols to actual software agents.
Chapter 6
Protocol Creator
MAPC was designed to automatically create multi agent protocols. This chapter
outlines the decisions that were made during the design and implementation of MAPC
in order to fulfil Objective O5 (Design and implement MAPC, p. 5). MAPC primarily
builds upon ECJ, an open source evolutionary framework written in Java. Section 6.1
describes ECJ and how it was configured for MAPC. The process of creating protocols
follows a fixed procedure within MAPC, and this is outlined in Section 6.2. Section
6.3 describes the process that is used to evaluate individuals. Section 6.4 describes
a spacialized mutation operator that is used in MAPC to transform individuals in
the search for a protocol. Section 6.5 outlines a short experiment that was conducted
to find optimal parameter settings for MAPC. Section 6.6 describes the experiments
undertaken to find the optimal mix of operators for MAPC and evaluate the novel
context system described in Section 5.3.2, p. 73. Finally, Section 6.7 will summarize
how the pieces of MAPC come together to make a protocol.
6.1 ECJ
When MAPC executes, the process of evolving an acceptable protocol begins. This
process is preformed by ECJ. ECJ is an evolutionary framework written in Java that
is highly flexible and extendable (Luke, 2009).1 In addition to basic genetic algorithms,
ECJ provides genetic programming capabilities with a tree-based genome which are
1ECJ, while written in all capital letters by the authors, is not actually an acronym.
81
82
Base Class Name Purpose
EvolutionState Holds the evolution state and controls the evolutionaryprocess.
Initializer Creates the initial population.Population Holds the current population as a set of subpopulations.Subpopulation Holds one subpopulation as a set of individuals.Species Defines a type of individual.Individual Holds a single individual.Fitness Defines the method for determining and comparing fit-
nesses with other individuals.Evaluator Evaluates the population (usually using a problem).Problem Represents a problem which is used to evaluate individu-
als.Breeder Breeds a new population from the old population.BreedingPipeline Defines the method for breeding a new individual.Statistics Prints out information during the evolutionary process.
Table 6.1: Main ECJ Components
used in MAPC. Within ECJ, the evolutionary process is controlled by several distinct
components. Each component (which is an object) is responsible for a separate part
of the evolutionary process. Customizing ECJ is done by creating new classes or
selecting from pre-made classes. The main components, including their descriptive
base class name and their job within the evolutionary system, are shown in Table 6.1.
Each component is also described below.
The evolution state component contains all of the other components and has the
responsibility of controlling the flow of the evolutionary process. The evolution state
used in MAPC is described in Section 6.2.
Setup of the evolutionary system, including the creation of the initial population,
is done by the initializer component. MAPC makes use of the GPInitializer class,
which sets up the ECJ type system which is used to constrain the creation of new and
83
evolved individuals (see Section 5.3.1 for a description of the types used in MAPC). In
addition, the initializer asks the population component to create the initial population.
The population component holds a collection of subpopulation objects. In addition,
during initialization, the population is asked by the initializer component to create an
initial population. The base Population class is the population component used in
MAPC. This class forwards the request to create an initial population to each of its
subpopulation objects.
Each subpopulation component holds a collection of individual objects and a species
object. In MAPC, only one subpopulation is used: the base Subpopulation class.
When the subpopulation is asked to create an initial population, the species object
is used. The species object is asked repeatedly for a new individual object until the
configured number of individuals has been reached. In addition to asking the species
object for enough individuals to fill the subpopulation size, the subpopulation also
attempts to reduce the number of duplicate individuals.
The knowledge for how to create both new and bred individuals is contained in the
species component. The species component is composed of a prototypical individual, a
prototypical fitness, and a breeding pipeline. The prototypes are cloned whenever new
individual or fitness objects are required for the given species. The breeding pipeline
is used to create new individuals through the use of genetic operators. In MAPC,
the GPSpecies class is used. This class ensures that all individuals are subtypes of
GPIndividuals, and generates new individuals by cloning the prototypical individual
and asking the trees within that individual to build themselves. The creation of
new individuals (and the associated classes) makes use of the standard grow method
described in Section 2.4.3, p. 19.
84
The details about one individual are contained in an individual component. These
details include a reference to the species that created the individual, and a fitness object
which holds all of the information about any previous evaluation(s) of that individual.
MAPCIndividual is used by MAPC, which extends GPIndividual. GPIndividual
holds the specific data about the individual, which is stored in a collection of GPTree
objects. More information on these trees was given in Section 5.3.
The fitness component holds all of the information on how well an individual meets
the requirements of the protocol domain. MAPC uses a custom class to hold and
calculate fitness values called MAPCFitness. The method for scoring individuals in
MAPC is described thoroughly in Section 6.3.3.
The evaluator component is responsible for evaluating individual members of the
subpopulations. This is done by repeatedly requesting that the problem component
evaluate one individual and save the result in the fitness object of that individual. In
MAPC, the SimpleEvaluator class is used. This class evaluates each individual in
turn. In addition, this class can make use of threads, which allows separate portions
of the subpopulations to be evaluated simultaneously, thus speeding up evaluation.
The specific method for evaluating an individual is contained in the problem
component. In MAPC, a custom CASAPolicyProblem class is used. This class
evaluates an individual by running several agents that use the protocol described by
the individual. The agents are monitored and then are scored. This makes use of
CASA, which was described in Section 2.10, p. 33. This process is described in detail
in Section 6.3.
When a new generation is to be made the evolution state component requests that
the breeder component perform breeding. In MAPC, the SimpleBreeder class is used.
85
This class asks the breeding pipeline (contained in the species, which is contained in
the subpopulation) to create a new individual. In addition, the simple breeder also
allows elitism to be used.2
The creation of new individuals from the previous population is done by the
breeding pipeline component. The operators used during breeding will be described in
more detail in Section 6.6.
At almost every step (typically both before and after each step) in the evolutionary
process, the statistics component both records data to output files and displays
statistics on the screen. In MAPC, a custom MAPCStatistics class is used. It writes
the best individual for each generation to a log file (for local monitoring), sends the
best individual and basic statistics on each generation to a remote computer (to
simplify monitoring of multiple executions at once), and writes the final selected
individual at the end of the evolutionary run to a log file.
The relationships between the ECJ components, as defined above, are shown in
Figure 6.1.
6.2 The Evolutionary Process
ECJ is used to perform genetic programming for MAPC. Within ECJ, the evo-
lution state defines and controls the overall process of evolution. MAPC uses a
customized MAPCEvolutionState which is a subclass of the SimpleEvolutionState
class, and primarily acts in the same manner.3 The process of evolution for the
2Elitism in genetic algorithms allows the top x individuals from one generation to be retained,without change, in the following generation. This ensures that the best individuals from a generationwill not be lost despite the random nature of genetic operators.
3The custom subclass adds the ability to set certain parameters from within the Lisp definition ofthe protocol domain, as described in Section 4.3, p. 55. In addition, in order to allow for reproducibility
86
EvolutionState
Breeder
Initializer
Population Evaluator
Statistics
holds the
holds the holds the
holds the
holds the
Subpopulation
holds all
Individual
Species
holdsmany
has a
has a
Fitness
holds theprototypical
has a
BreedingPipeline
holds theprototypical
Problem
holds and uses the
evaluates
uses
holds theprototypical
Figure 6.1: ECJ Component Relationships
87
SimpleEvolutionState is as follows:
1. Ask the initializer to create an initial population of individuals.
2. Ask the evaluator to evaluate the population (storing the resulting scores in
each individual’s fitness).
3. If an ideal individual was found and this is the criterion for stopping or if the
maximum number of generations has been reached, skip to 6.
4. Ask the breeder to breed the next generation and replace the population with
the result.
5. Increment the generation number and then go to 2.
6. Done.
In addition to these steps, the SimpleEvolutionState class allows both pre- and
post-breeding exchanges to be performed by an exchanger before and after step 4.
By using the SimpleExchanger, which performs no actions, these steps are skipped.
There is also an ability to perform an action upon the completion of the evolution
(step 6). As with the exchanger, MAPC makes use of a finisher that performs no
action (the SimpleFinisher). The entire process (with the empty steps excluded) is
illustrated in Figure 6.2. This process is similar to the genetic algorithm selection
scenario described in Section 2.4.6, p. 25. The methods used to generate environments,
construct individuals and evaluate the fitness of those individuals are described in the
remainder of this chapter.
and to aid in debugging, the seeds of the random number generators used in ECJ needed to be logged.These changes required overriding the default setup process.
88
Create Initial Population
Evaluate Population
Breed New Population
Done
Increment Generation
Number
LastGeneration?
no
yes
Figure 6.2: ECJ Evolution Process
6.3 Evaluating Individuals
In order to evaluate individuals in an evolutionary system, a fitness function is used to
determine the fitness of each individual in some environment. In the case of MAPC,
the goal is to create a protocol that will work in any of the scenarios defined in the
protocol domain model. In order to work towards this goal, the fitness value for
an individual is based on a combined score from multiple scenarios. The scores for
the separate scenarios are called execution scores. The score for each execution is
calculated by executing a set of agents as defined by a scenario in CASA. Once the
execution of the agents concludes, the execution score of the individual is calculated
based on data stored during the execution. In each generation, a fixed number of
execution scores are collected. From these scores, an overall fitness is calculated. These
methods meet Requirement G3 (MAPC must be capable of evaluating the suitability
of potential protocols).
89
6.3.1 Environments
In order to use an evolutionary algorithm such as genetic programming, a range of
possible environments must be defined. For MAPC, these environments are called
scenarios, and they are contained in the protocol domain model. Each scenario defines
the number and type of roles as well as the initial values of each of the variables in
the system (contained in the domain information states). For a formal definition of
scenario, see Section 4.2, p. 51. While formally the protocol domain contains a set of
scenarios, for practical reasons, the protocol domain descriptions in MAPC instead
describe the details needed to generate random scenarios. This section will describe
informally the steps involved in the creation of a scenario as it is performed by MAPC.
To generate scenarios, first the collection of roles is chosen. This is done by selecting
the number of agents in each agent group from the available options. A scenario
for the comparison shopper protocol domain is constructed as an example to clarify
this description. Let the number of seller-group agents be three and the number of
buyer-group agents be one (the only option). Once the agent quantities have been
determined, the initial value of all of the variables must be selected. A scenario with
three sellers and one buyer is shown in Table 6.2. As shown in the table, initial values
have been computed for each variable for each role.4 If the protocol domain had any
global values, their initial value would also have been computed at this point. Next,
the desires for each agent are determined. The single buyer role in the table has a
desire for the give-item action to be performed twice.
Once the number of roles has been selected, and each role’s initial variable values
and desires have been determined, the scenario has been completely defined. This
4In the case of the random values, the appropriate Lisp function is called. In the case of the fixedstarting values (e.g. 0 for items-given) that value is used as is.
90
seller-groupseller #1 seller #2 seller #3
money 126 137 159items 5 1 0
minimum-price 8 9 4items-given 0 0 0
money-received 0 0 0buyer-group
buyermoney 105items 4
maximum-price 11desire give-item ×2
Table 6.2: Comparison Shopper Scenario
generated scenario can then be used to test a protocol.
6.3.2 Testing an Individual
All of the agents in a single execution make use of the same protocol, or individual.
In order to drive the agents towards using the protocols, the agents in the system
will attempt to meet their desires using the desire fulfillers outlined in the protocol
(as described previously in Section 5.2, p. 69). When a proposal reaches an agent,
that agent will make use of the appropriate considerer as defined in the individual to
determine if it accepts or rejects the proposal.
When an execution is set to begin, each agent is started in a separate thread.
Each agent is passed in (via objects in shared memory) the scenario, a role which
defines that agent’s part in the scenario, a scorer object which is given a record of
each action and message that occurs, and the location or path of the individual file in
the format described in Section 5.2. Once all of the agents are created, the agents are
91
1. Wait for start signal.
2. Repeat until stop signal received:
(a) Pick a random desire.
(b) For each parameter of the desire, including the agent who will perform theaction desired, pick a value randomly.
(c) Execute the desire fulfiller with the parameters filled in.
(d) For each proposal we’ve received:
i. Verify that we can perform the action (we are capable of performingthat action).
ii. Verify the pre-conditions of the action.
iii. If there is a condition, verify that the sending agent can perform theaction.
iv. Call the appropriate consider function for final verification.
v. If all verifications pass, accept the proposal and carry out the proposedaction.
vi. If any verification failed, refuse the proposal.
(e) For each acceptance we’ve received:
i. If there was a condition on the proposal, perform the condition.
Figure 6.3: Basic Algorithm for Test Agents
given one second before a start signal is sent (to allow them to initialize themselves).
Once the start signal is received by an agent, the agent will repeatedly attempt to
fulfill its desires. This is done by executing the appropriate desire fulfiller function as
defined in the individual. At the same time, agents will respond to proposals with
accept or refuse messages based on the return value of the appropriate considerer.
The agents run until a stop signal is received. This stop signal is sent by MAPC once
a pre-defined execution time has elapsed. The basic algorithm that each agent follows
is given in Figure 6.3.
92
As the system executes, every message and action is recorded and the final variable
values are saved to be used in the evaluation of the individual. Currently, the system
only makes use of the messages to ensure that some communication is taking place,
so only the sender and receiver of each message is retained. Before each action is
executed, the current state of the scenario is evaluated based on the pre-conditions of
the action. If this fails, the failure is recorded to aid in the evaluation. Example data
from executing the buyer protocol (given in Listing 5.2, p. 72) in the scenario outlined
in Table 6.2 is given in Table 6.3. As you can see in the latter table, the give-money
action has been performed twice, the give-item action has been performed twice,
messages have been sent between the buyer agent and seller #1, and the variables
have been changed as a result of the actions performed. No data is saved as a result
of executing the null protocol (given in Listing 5.1, p. 70), since no desire fulfillers
do anything and therefore the agents will never send any messages or perform any
actions. The data for the null protocol is shown in Table 6.4.
6.3.3 Execution Score
Once the system finishes its execution, an execution score for the individual is calculated
based on the protocol domain, the final variable values, the messages sent and the
actions performed during the execution.5 This is done using the hard requirements,
soft requirements and desires as defined in the protocol domain.
Each of the post-condition functions in each of the fitness objects within the role
types and the protocol domain as a whole are considered different hard requirements.
The pre-conditions of all of the actions are also considered hard requirements. In
5As with the execution, this process is limited to a specified amount of time, termed the scoringtime.
93
Actions Performed:Action Actor Receiver Amount Fail?give-money buyer seller #1 8 Ngive-item seller #1 buyer N/A Ngive-money buyer seller #1 8 Ngive-item seller #1 buyer N/A N
Messages Sent (duplicates removed):Sender Receiverbuyer seller #1seller #1 buyer
Variable Final Values(Only Changes from the Initial Values in Table 6.2):
Initial FinalAgent Variable Name Value Valuebuyer money 105 89buyer items 4 6seller #1 money 126 142seller #1 items 5 3seller #1 items-given 0 2seller #1 money-received 0 2
Table 6.3: Execution Data from Buyer Protocol
Actions Performed:Action Actor Receiver Amount Fail?
noneMessages Sent (duplicates removed):Sender Receiver
noneVariable Final Values(Only Changes from the Initial Values in Table 6.2):
Initial FinalAgent Variable Name Value Value
no changes
Table 6.4: Execution Data from Null Protocol
94
addition, all of the required desires are hard requirements. As described above, the
number of infractions of the pre-conditions due to performing actions in a manner not
allowed by the protocol domain are counted as the system executes. The number of
unfulfilled required desires is added to this count when the execution is finished. A
relative fitness is thus given by comparing the number of failures and unmet required
desires between the executions of two potential protocols. This is the method used to
compare the relative utility of each protocol and it is useful even when none of the
potential protocols meet all of the hard requirements.
Any result which fails a single hard requirement is less desirable than an individual
that does not fail any hard requirements. The exception to this rule is if the system
performs no actions. This is to ensure that the system does not strive for the pseudo-
perfection of procrastination. Once more than one individual has met the hard
requirements, the individuals are compared on their relative ability to meet the soft
requirements as defined by the value of the maximize and minimize functions in the
fitness objects, and the soft requirement to fulfill as many desires as possible.
Following Koza’s tradition, the individuals in MAPC strive towards a fitness of
zero (1992). The varying requirements of the protocols in MAPC require using some
method for multi objective evaluation. MAPC makes use of an aggregate scalar fitness
function that is staged.6 To aid in determining which protocols have met the hard
requirements and which have not, all hard requirements are given a penalty value of
at least 1000. Therefore, an acceptable individual is one that has a score of less than
1000.7 To calculate an execution score, the sum of any penalties, normalized minimize
6See Section 2.6, p. 28 for the description of these methods.7In fact, a separate Boolean value indicating acceptability is kept, to aid with combining fitnesses
and for the case where the combination of soft requirements pushes the total score above 1000 for anacceptable individual.
95
functions, and normalized maximize functions is taken.
score = pen+∑
minnorm +∑
maxnorm
The penalties (pen) are equal to the counts of all of the failures (action pre-condition
failures, protocol domain post-condition failures and role type post-condition failures)
and the unmet required desires times various factors indicating the relative importance
of each requirement. The action failures are each multiplied by the value 1,000. This is
the lowest penalty to encourage actions first, and then correct actions. The situation
and role type post-condition failures along with the unmet desire count are each
weighted with a value of 10,000. These pressures are all trying to enforce the rules
of the system and to get the system to accomplish some of the goals (encode in the
desires). An additional penalty of 100,000 is incurred if no actions have been taken
and no messages have been sent. Finally, a penalty of 100,000 is incurred if there
is some failure in the Lisp code while executing an individual due to an ill-formed
individual.8
pen = failact × 1, 000 + (failsit + failtyp + unmet)× 10, 000 + peninaction + penlisp
peninaction =
0 if messages are sent or at least one action occurs,
100, 000 if no messages are sent and no action occurs
penlisp =
0 if no Lisp errors occur during execution,
100, 000 if at least one Lisp error occurs during execution
8This shouldn’t happen if the context system is in use, but may happen if an invalid operation iscreated when the context system is not in use as described in Section 6.6.
96
The minimize functions are normalized so that the worst performer (highest minimize
function score minhighest) is given a score of 100, with a minimized score of zero
resulting in a fitness of zero. The maximize function is normalized and inverted so
that the best performer (highest maximize function score maxhighest) is given a score
of 0, while a maximize score of 0 is given the value 100. Both the normalized minimize
and normalized maximize functions are calculated based on the highest score in that
category in the current generation. This allows the system to adapt to a wide range
of results from both the minimize and the maximize functions.
minnorm =min
minhighest
× 100
maxnorm =
(1− max
maxhighest
)× 100
In addition to the minimize functions described in a protocol domain, there is one
extra minimize function that is implicitly included in every protocol domain. This
minimize function is equal to the number of characters in the Lisp-based representation
of the protocol. This soft requirement was introduced to provide a gentle pressure to
produce shorter protocols.
For illustrative purposes, the two execution data sets given in Table 6.3 and Table
6.4 will be scored. When determining the scaled minimize and maximize functions,
these two scores will be considered the only two scores. The two protocols were
executed in the scenario described above and outlined in Table 6.2. To determine
the scores for the two executions, the penalties, scaled minimize functions and scaled
maximize functions must be computed.
The penalties for both executions are shown in Table 6.5. For the null protocol,
97
failact failsit failsit unmet peninaction penlisp penNull Protocol 0 0 0 2 100,000 0 110,000
Buyer Protocol 0 0 0 0 0 0 0
Table 6.5: Penalties for Example Protocols
money # items max. price total worth scaled, inverted valueNull Protocol 105 4 11 149 4
Buyer Protocol 89 6 11 155 0
Table 6.6: Buyer Maximize Function for Example Protocols
since no actions are performed, it receives the inaction penalty with a value of 100,000.
In addition, it would receive a penalty of 10,000 for the two required desires that it
did not meet. This is because they are both the same kind of desire and so are only
counted once. Therefore, the total penalty for the null protocol would be 110,000.
The buyer protocol did not have any failures, met all of its desires, performed some
action and did not have Lisp errors. Therefore, it has a total penalty of zero.
In the comparison shopper protocol domain there is one maximize function given
for the buyer role type. The maximize function is scored based on the buyer’s worth.
The worth is calculated by adding the buyer’s money to the value of the items, as
judged by the buyer in the maximum price variable. For example, at the conclusion
of the buyer protocol execution, the buyer has 6 items, a maximum price of 11 and a
value of 89 for money. The net worth is therefore (6×11) + 89 = 155. The buyer has a
net worth of 149 at the conclusion of the null protocol. The scaled maximize function
for the null protocol is therefore(1− 149
155
)× 100 = 4. The scaled maximize function
for the buyer protocol is(1− 155
155
)× 100 = 0. These values are all summarized in
Table 6.6.
98
# characters (length) scaled valueNull Protocol 252 53
Buyer Protocol 475 100
Table 6.7: Protocol Length Minimize Functions for Example Protocols
There is also the continual pressure to reduce the size of a protocol based on a
minimize function with a value equal to the length of the protocol (the total number
of characters). The number of characters in the null protocol is 252 while it is 475 for
the buyer protocol. The scaled values for the protocols are then(252475
)× 100 = 53 and(
475475
)× 100 = 100, respectively. These values are summarized in Table 6.7.
Finally, the total execution score is equal to the sum of the penalties, scaled
maximize function and the scaled minimize function. This results in a score of 110,057
for the null protocol and 100 for the buyer protocol. This gives an overall score for
each protocol in each execution.
6.3.4 Fitness Function
To determine the fitness of an individual, a fixed number of scenarios are created (based
on the input file) and the protocol is tested in those scenarios, resulting in execution
scores. The fitness of an individual is the mean average of all of its execution scores.
Thus an individual is evaluated by its average performance in a variety of scenarios.
In addition, an individual is deemed acceptable only if it performed acceptably in
all of its executions. In this way it is possible that an individual can be deemed
unacceptable even with an average fitness below 1000.
Referring to the example from the previous section, if the null protocol and buyer
protocol were executed in three possible environments, they might end up with the
99
Execution #1 Execution #2 Execution #3 MeanNull Protocol 110,057 110,064 110,044 110,055
Buyer Protocol 100 100 100 100
Table 6.8: Total Fitness Values for Example Protocols
scores shown in Table 6.8. The null protocol’s overall fitness would then be the mean
average of the three scores, or 110,055. The buyer protocol, on the other hand, would
have a mean score of 100.
6.4 Verify Negotiable Value Mutation
MAPC makes use of genetic programming to create new protocols. As previously
described, MAPC uses a tree-based genome. Therefore, it makes use of the standard
genetic operators for tree-based genomes (see Section 2.4.4, p. 21 for a description).
In addition, MAPC makes use of a specialized mutation operator which is termed
the verify negotiable value mutator. This mutator helps to ensure that all negotiable
values are verified by the agents using the protocol. When an individual is chosen
for the verify negotiable value mutation, one of that individual’s consider functions
is replaced with another which makes use of the verify negotiable terminal (as
described in Section 5.3.3, p. 75). Since the verify negotiable terminal can have a
range of return types, the verify negotiable value mutation operator must dynamically
create a subtree which will replace one of the consider functions. The process is as
follows:
1. Choose a random negotiable value in the protocol domain.
2. Construct a verify negotiable terminal that matches the type of the chosen
100
negotiable value.
3. Filter all of the actions to find the actions which have at least one parameter
with the same type as the chosen negotiable value.
4. Choose a random action from those remaining.
5. Construct an action descriptor function of the same type as the chosen action.
6. Filter the parameters of the chosen action to include only those with the same
type as the chosen negotiable value.
7. Choose a parameter from the remaining parameters.
8. Place the verify negotiable terminal in the child location of the action de-
scriptor function which is designated for the chosen parameter.
9. Place a null terminal in the remaining action descriptor children locations.
10. Place the action descriptor function into a newly created is connected
function.
11. Choose a random considerer.
12. Replace the child of the randomly chosen consider function with the newly
created is connected node.
6.5 Parameter Selection
As described in Section 2.4.6, p. 25, there are a number of parameters that must
be set in order to use genetic programming. These include population size and
101
a termination criterion. Based on research, there is a general consensus that the
population size should e at least 500, preferably 1000 (Banzhaf et al., 1998; Poli et al.,
2008). Unfortunately, in MAPC, individuals are quite complex and require a lot of
memory. The evaluation of individuals accounts for a significant portion of the running
time. To keep within the memory constraints of a typical computer and to minimize
runtime, a population size of 500 was selected with the intention of exploring larger
population sizes if 500 was found to be insufficient during tests. The termination
criterion for MAPC was set at a fixed number of generations. Banzhaf et al. state
that after 50 generations, typically new results are not found (1998). For this reason,
a termination criterion of 50 generations was chosen.
6.6 Operator Selection and Context Experiments
MAPC uses the three standard genetic programming operators of crossover, subtree
mutation and reproduction (described in Section 2.4.4, p. 21). In addition, the VNV
(verify negotiable value) mutation operator is used. Both the reproduction and the
custom VNV mutation operators will be selected 10% of the time. This was chosen as
it is a fairly standard value for reproduction in the literature and the VNV mutation
was so specific, a small value seemed reasonable. To determine the optimal use of the
crossover and subtree mutation operators, a set of experiments was performed. In
addition, these experiments were used to evaluate the utility of the context system
described in Section 5.3.2, p. 73.
The experiments were meant to decide between three different combinations of
operators termed crossover-heavy, mixed, and mutation-heavy. With crossover-heavy,
crossover was the primary operator at 70% while subtree mutation was only at 10%.
102
Crossover-Heavy Mixed Mutation-Heavy
Crossover 70% 40% 10%Subtree Mutation 10% 40% 70%VNV Mutation 10% 10% 10%Reproduction 10% 10% 10%
Table 6.9: Possible Operator Selection Configurations
A mixed configuration would select both the crossover and subtree mutation 40% of
the time. Finally, a mutation-heavy configuration would select the mutation operator
70% of the time and the crossover operation only 10% of the time. These three
combinations are summarized in Table 6.9.
For each possible operator selection configuration, experimental runs were executed
in which context aware nodes were and were not used. This was done towards the
end of demonstrating the utility of the context system. Each of the six configurations
were each executed 30 times to get a general view of how each performed based on the
comparison shopper protocol domain.9 The results of these experiments were then
compared to see how quickly the various configurations were able to find an acceptable
protocol and how long the actual evolutionary process took.
The percentage of the 30 runs that had found an acceptable protocol is plotted at
each generation in Figure 6.4. These results show very clearly that the experimental
runs that use the context system performed better than those that did not use
the context system. In addition, the relative speed of the three operator selection
configurations is fairly stable. The crossover-heavy configuration is the slowest at
finding acceptable protocols, the mixed configuration is faster, and the mutation-heavy
9This experimental setup will be described in more detail in Section 7.1.5, p. 112. The comparisonshopper protocol domain is described in Section 4.3.2, p. 59.
103
Crossover-Heavy Mixed Mutation-Heavy Mean
Context Used 17:32:18 17:45:44 18:07:10 17:48:24No Context 25:05:10 21:46:28 20:15:32 22:22:23
Table 6.10: Average Total Times
configuration outpaces both. The exception to this rule is in the last 3 generations,
when the crossover-heavy configuration reaches 100%, indicating that all 30 runs found
an acceptable protocol. The mixed and mutation-heavy configurations only reach
97%, indicating that 29 out of 30 runs found an acceptable protocol.
The time taken to perform the evolution is shown in Table 6.10. Again, the
experimental runs without the context system perform significantly worse than the
runs with the context system. While the number of runs with an acceptable protocol
is lower in the context-free system, the runtime is also slower. When comparing
the operator selection configurations, the times are very similar, with a range of
approximately 30 minutes for total running times just under 18 hours when using the
context system.
As a result of these experiments, the context system was shown to both increase
the number of acceptable protocols and the speed of the system overall. Therefore, the
context system will be used throughout the remainder of the tests. In addition, the
mutation-heavy configuration was chosen to use for the remainder of the tests. This
is primarily due to the speed at which this configuration attains a high percentage
of acceptable configurations. In addition, while the crossover-heavy configuration is
quicker than the mutation-heavy configuration when used with the context system,
the time savings is not significant and is countered by the slow speed at which the
crossover-heavy configuration increases the percentage of acceptable runs.
104
01
02
030
40
50
Genera
tion
0%
20
%
40
%
60
%
80
%
10
0%
Percent of Runs with Acceptable Protocol
Muta
tion H
eavy (
Conte
xt)
Mix
ed (
Conte
xt)
Cro
ssover
Heavy (
Conte
xt)
Muta
tion H
eavy (
No C
onte
xt)
Mix
ed (
No C
onte
xt)
Cro
ssover
Heavy (
No C
onte
xt)
Fig
ure
6.4:
Par
amet
erC
onfigu
rati
onE
xp
erim
enta
lR
esult
s
105
0 10 20 30 40 50Generation
0%
20%
40%
60%
80%
100%
Perc
ent
of
Runs
wit
h A
ccepta
ble
Pro
toco
l83 %
Figure 6.5: Results with Small Population Size (100)
In an attempt to optimize the system, a smaller population size was also tried.
This resulted in lacklustre results in that only 83% of the 30 runs were able to find an
acceptable solution. The results of that run are shown in Figure 6.5.
6.7 Summary
MAPC makes use of ECJ. ECJ is a flexible, object-oriented, Java-based evolutionary
platform. This platform was used and extended for MAPC. The main process of
evolution is defined by ECJ, but MAPC makes use of custom classes which define how
the system will act in certain portions of the evolutionary process. The majority of the
design and implementation of MAPC required defining the input (protocol domain)
and output (the protocol) for the system. Having defined these, the remainder of
106
the implementation involved generating environments, termed scenarios in MAPC,
scoring protocols using a fitness value calculated from several execution scores, and
finally, in the creation of a domain-specific mutation operator. All of these parts work
together to evolve protocols.
Chapter 7
Test Cases and Results
To show the utility and viability of MAPC, four test protocol domains were designed.
For each protocol domain, MAPC was executed with the protocol domain as input,
and the creation of protocols was logged and evaluated in terms of the time it takes to
reach an acceptable protocol, and the subjective quality of the generated acceptable
protocols was also evaluated. This chapter outlines the creation and setup of the test
protocol domains. Results are then described and analyzed. This fulfills Objective O6
(Test MAPC, p. 5). Section 7.1 describes the test protocol domains as well as the test
setup. Section 7.2 reports and provides an analysis of the results of the tests. Section
7.3 discusses various methods that could be used to ensure that MAPC can produce a
solution with an even higher level of reliability. Finally, Section 7.4 summarizes the
findings of these tests in terms of the efficiency of MAPC.
7.1 Test Protocol Domains
To test MAPC, four test protocol domains were created and executed. All of the
test protocol domains are variations on a protocol domain that requires a transaction
between two agents in which money is exchanged for an item. Using similar protocol
domains was done to allow for simple comparisons between the results attained with
each protocol domain. The protocol domains were designed to test variations that
occur if the protocol domains are bent towards the benefit of the buyer or the seller
and to allow for the potential of neutral protocols. It was expected that MAPC would
107
108
produce protocols that predictably favoured the buyer or the seller in specific protocol
domains.
7.1.1 Comparison Shopper
The comparison shopper protocol domain is the example protocol domain that has
been used throughout this work. It was initially described in Section 4.1, p. 47. If
MAPC reacts appropriately to the input it is provided, then the execution of this test
protocol domain should yield a protocol that favours the buyer. There is only one
buyer and from one to eight sellers, so the buyer has a choice of where to purchase
items. In addition, there is a soft requirement that the buyer gets the best deal
possible in the form of a maximize fitness function which calculates the total value of
the buyer. No similar soft requirement exists for the sellers. The complete input file
for the comparison shopper protocol domain is given in Section B.1.
7.1.2 Art Dealer
The art dealer protocol domain is designed to be the reverse of the comparison shopper
protocol domain. If MAPC reacts appropriately to the input it is provided, then the
execution of this test protocol domain should yield a protocol that favours the seller.
In the art dealer protocol domain, there is one seller and from one to eight buyers.
This protocol domain is similar to an auction, but differs in that there is no built-in
concept of calling for bids or placing bids. In addition, there is a soft requirement that
the seller should get the highest price possible for the goods.
The details of the art dealer protocol domain are similar to the comparison shopper
protocol domain. It also includes the transactor, seller and buyer role types and the
give item and give money actions. There are four significant differences between the
109
:variables (list
(variable "maximum-price" "integer"
:initial-value ’(+ 10 (random 10)))
(variable "items-received" "integer"
:initial-value 0)
(variable "money-given" "integer"
:initial-value 0))
Listing 7.1: Art Dealer Buyer Role Type Variables
art dealer and the comparison shopper protocol domains. First, the quantities of
sellers and buyers is inverted. Second, the maximize soft requirement was moved from
the buyer to the seller. Third, in order to ensure that each buyer paid out money as
many times as an item was received, the buyer role type was given two new variables
(shown in Listing 7.1)1 and the give item and give money actions were modified to
update these new variables. Finally, the seller’s desire to get money (the desire for
the give-money action with itself as the receiver) has been marked as required while
the buyer’s desire for an item is no longer required. The complete input file for the
art dealer protocol domain is given in Section B.2.
7.1.3 Flea Market
The flea market protocol domain is meant to be a neutral environment. In other words,
it is designed to favour neither the buyer nor the seller. If MAPC reacts appropriately
to the input it is provided, then the execution of this protocol domain yields a mixture
of buyer-centric and seller-centric protocols. In the flea market protocol domain there
are one to four buyers and one to four sellers. There are no soft requirements on sellers
or buyers, so no protocols which give one or the other an advantage are encouraged.
1Buyers also have the variables listed in the transactor: item-count and money.
110
The input file which defines the flea market protocol domain is similar to both
the comparison shopper and the art dealer’s input files with the soft requirements of
each removed. Because there are potentially multiple sellers and buyers, ensuring that
there is a one to one ratio of money given and items given requires extensive tracking
of these exchanges. The complete input file for the flea market protocol domain is
given in Section B.3.
7.1.4 Free Trade
The free trade protocol domain is meant to demonstrate that MAPC can function
efficiently in much less structured protocol domains. Like the flea market protocol
domain, there is no intention in the free trade protocol domain to create a bias that
favours either the buyer or the seller. Unlike the flea market protocol domain, there is
no distinction between the buyer and the seller role types. Instead there is only one
role type, termed trader. Agents in the trader role type have all of the characteristics
of the buyer and seller role types, but were tweaked to deal with the fact that the
same agent can either buy or sell items.
In addition to combining all of the desires, actions, variables and fitness require-
ments of the seller and buyer role types from the flea market protocol domain into
the trader role type of the free trade protocol domain, two further changes were
made. First, since the same agent could buy or sell items, it was not logical to
have separate, independently generated, minimum and maximum prices for each
agent, since this could lead to an agent that would sell an item for a lower price
than it would buy an item. Instead, a single item-value variable was created. The
item-value variable, along with all of the combined variables, is shown in Listing 7.2.
111
:variables (list
(variable "item-count" "integer"
:initial-value ’(+ 0 (random 5)))
(variable "money" "integer"
:initial-value ’(+ 0 (random 100)))
(variable "item-value" "integer"
:initial-value ’(+ 1 (random 20)))
(variable "items-given" "integer"
:initial-value 0)
(variable "items-received" "integer"
:initial-value 0)
(variable "money-given" "integer"
:initial-value 0)
(variable "money-received" "integer"
:initial-value 0))
Listing 7.2: Free Trade Variables
Second, the item-value variable had to be used wherever the old minimum-price
and maximum-price variables were previously used. This entailed splitting the price
negotiable value into two separate negotiable values, sell price and buy price. The buy
price negotiable value is intended to be used when an agent is buying an item. The
agent prefers a lower price and won’t pay more than the value that the agent assigns
the item. The sell price negotiable value is the money the agent desires when it sells
an item. The sell price negotiable value indicates that a higher price is preferred with
a minimum equal to one greater than the value.2 The definitions of these negotiable
values are shown in Listing 7.3. The complete input file for the free trade protocol
domain is given in Section B.4.
2This protocol domain was initially programmed such that an agent would sell an item at theprice for which it was valued, but this led to agents selling to themselves.
112
:negotiables (list
(negotiable "sell-price" "integer" "higher" :minimum
’(+ 1 (get-variable (this-agent) "item-value"))
:maximum 20)
(negotiable "buy-price" "integer" "lower" :minimum 0
:maximum ’(get-variable (this-agent)
"item-value")))
Listing 7.3: Free Trade Negotiable Values
7.1.5 Test Setup
Because of the non-deterministic nature of genetic algorithms such as the genetic
programming used in MAPC, any testing of MAPC must be done repeatedly and
evaluated on the basis of average results. Thus each test protocol domain was executed
30 times, for a total of 120 executions. Each of the 30 executions were performed
in parallel on 30 individual machines at the same time. All machines are identically
configured. The specifications of these machines are given in Table 7.1.
The non-protocol domain parameters for MAPC were identical in all cases. For
all protocol domains, 10 threads evaluated the individual protocols simultaneously,
a population size of 500 was used, and there were 50 generations.3 In addition, 4
seconds was given for the execution time and for the scoring time.4 These parameters
are summarized in Table 7.2.
3See Section 6.5, p. 100 for a justification of these parameters.4In tests, 2 seconds was enough time for the agents to perform the necessary communication with
known-good protocols. See Section 6.3, p. 88 for a description of the execution and scoring times.
113
Parameter Value
CPU Intel Core i5 2400Clock Speed 3.1 GHz
RAM 8 GBOS Scientific Linux release 6.1
Java Virtual Machine OpenJDK 64-Bit Server (build 20.0-b12, mixed mode)Java Runtime Environment IcedTea6 1.11.3
Table 7.1: Test Machines’ Parameters
All Protocol Domains
# of Runs 30Evaluation Threads 10
Population Size 500# of Generations 50
Run Time 4 secondsScore Time 4 seconds
Table 7.2: Test Setup Parameters
114
7.2 Test Results and Analysis
For every generation in every test run several data points were logged. These data
points included the best individual (the individual with the lowest fitness value) of
that generation, the best individual’s fitness, and whether the best individual was
acceptable based on the scenarios executed that generation.5 In addition, both the
mean and the median of the fitnesses of all individuals in that generation were logged.
The time of the completion of the generation was logged as well.6
In the following sections, the results of the multiple runs of the four protocol
domains will be presented in a variety of ways. Section 7.2.1 provides an overview
of the progress of the evolution throughout 50 generations for each protocol domain.
Section 7.2.2 describes the most common acceptable individual for each of the four
protocol domains. Section 7.2.3 outlines some of the more questionable solutions
generated by MAPC. Finally, Section 7.2.4 looks at the number of buyer-driven
protocols versus the number of seller-driven protocols for each of the protocol domains
to determine if MAPC has reacted as expected to the four protocol domains.
7.2.1 Progress of Evolution
To show that MAPC continuously works towards an acceptable protocol, two plots
were generated for each protocol domain. The first is of two lines. The upper line is
the mean of the mean fitness values for each generation over all of the executions. The
lower line is the mean of the best individuals’ fitness scores for each generation over
all executions. Because a lower fitness value indicates a better protocol, both of these
5Individuals may be acceptable in a limited set of scenarios and so an acceptable individual maynot be what would generally be considered acceptable in all scenarios.
6Since the actual time for a single generation is affected by the machines on which MAPC runs,all evaluations in this chapter are based on generations, not time.
115
lines should show a general downward trend to indicate that the system is continuously
progressing towards better protocols. The second plot indicates the percentage of
executions that reached at least one acceptable individual at each generation.7 This
should show a continual increase over the generations, to indicate that as the evolution
progresses, an increasing number of executions reach acceptable solutions. The plots
for the comparison shopper, art dealer, flea market and free trade protocol domains
are given in Figures 7.1, 7.2, 7.3 and 7.4, respectively.
The figures depict a quick decrease of both the mean fitnesses and the best fitness
values of individuals in the first ten generations. Corresponding with the downward
trend of the fitness scores, the figures depict a swift upward trend in the number of
executions that reached an acceptable level of fitness, with all four charts reaching at
least 97% before generation 20.
The fact that both the comparison shopper and the free trade experiments found
an acceptable protocol in only 97% of the runs indicates that one of 30 runs was not
successful. Methods to mitigate this failure to reach 100% will be discussed further in
Section 7.3.
The art dealer protocol domain runs all found an acceptable protocol in only eight
generations. The comparison shopper and flea market protocol domains each reached
their peaks a few generations later, at generations 11 and 13, respectively. Finally, at
generation 18, the free trade protocol domain reached its peak.
7As described in Section 6.3.3, p. 92, an acceptable individual is one that meets all of the hardrequirements for a given protocol domain.
116
0 10 20 30 40 50Generation
0
5000
10000
15000
20000
25000Fi
tness
MeanBest
0 10 20 30 40 50Generation
0%
20%
40%
60%
80%
100%
Runs
wit
h A
ccepta
ble
Pro
toco
l
97%
Figure 7.1: Comparison Shopper Results by Generation
117
0 10 20 30 40 50Generation
0
5000
10000
15000
20000
25000Fi
tness
MeanBest
0 10 20 30 40 50Generation
0%
20%
40%
60%
80%
100%
Runs
wit
h A
ccepta
ble
Pro
toco
l
100%
Figure 7.2: Art Dealer Results by Generation
118
0 10 20 30 40 50Generation
0
5000
10000
15000
20000
25000Fi
tness
MeanBest
0 10 20 30 40 50Generation
0%
20%
40%
60%
80%
100%
Runs
wit
h A
ccepta
ble
Pro
toco
l
100%
Figure 7.3: Flea Market Results by Generation
119
0 10 20 30 40 50Generation
0
5000
10000
15000
20000
25000Fi
tness
MeanBest
0 10 20 30 40 50Generation
0%
20%
40%
60%
80%
100%
Runs
wit
h A
ccepta
ble
Pro
toco
l
97%
Figure 7.4: Free Trade Results by Generation
120
7.2.2 Most Common Acceptable Protocol
To analyze the quality of the protocols generated, a method was chosen to determine
the “most popular acceptable” individual for each protocol domain. This was done by
looking at the best individuals in all 50 generations in all 30 executions. Of these 1500
individuals, only those that were marked as acceptable were considered. Once this set
of individuals was collected, the number of occurrences of each unique individual was
counted and the individual that occurred the most number of times was considered
the most popular individual for that protocol domain. Despite the fact that there
were four protocol domains, only two protocols were described by the most popular
acceptable individuals in the four protocol domains. The first, a buyer-driven protocol,
was described by the most popular acceptable protocol in three of the four protocol
domains, while the second, a seller-driven protocol, was described by the most popular
acceptable protocol in the art dealer protocol domain. The protocols described by
the most popular acceptable individuals are concise and easily described. In fact, the
protocols are nearly identical to the propose communicative act in the specifications:
The proposer (the sender of the propose) informs the receiver that the
proposer will adopt the intention to perform the action once the given
precondition is met, and the receiver notifies the proposer of the receivers
intention that the proposer performs the action. (Foundation for Intelligent
Physical Agents (FIPA), 2002b).
This supports the argument that MAPC drives towards protocols similar in nature to
those designed by human programmers (or in this case, a committee).
The most popular individual produced throughout all of the runs was the most
likely to be chosen in both the comparison shopper protocol domain and the flea
121
market protocol domain, and an equivalent protocol was chosen most frequently in
the free trade protocol domain. This individual is shown in Listing 7.4. This protocol
starts when an agent with the give item desire tries to fulfil that desire. This agent is
the buyer, since the buyer is driven to purchase an item by a desire to receive an item.
Before starting the desire fulfiller, the buyer agent must fill in all of the parameters of
the give item action. These include the agent who will perform the action (the giver)
and the agent who will receive the item (the receiver). In both the comparison
shopper and the flea market protocol domains, the receiver is fixed as this-agent, or
the agent with the desire. This means that the variable receiver within the action
desire fulfiller is equivalent to the buyer. To continue, the buyer needs only to select
an agent that will give the buyer an item. The second agent is the seller (the giver is
the seller). The code of the desire fulfiller has the buyer send a proposal to the seller
agent. The proposal requests that the seller give the buyer an item if and when the
buyer gives the seller money. When the seller considers this proposal, the give money
condition is verified against the bounds of the seller agent’s prices (see Figure 7.5). In
Figure 7.5, _ represent messages while represent actions.8 As shown, the buyer
sends a proposal, which is either refused or accepted by the seller. If it is refused, the
protocol ends. If the proposal is accepted, then the condition can be fulfilled by buyer
giving the seller money. Subsequently, the give item is performed by the seller. For
the comparison shopper protocol domain, this protocol represented 61% of all of the
acceptable protocols. In the flea market protocol domain, this same protocol occurred
68% of the time.
8All of the figures representing protocols in this chapter will use the same notation.
122
(mapc-agent
:fulfillers (list
(action-desire-fulfiller "give-item"
’(propose-action giver "give-item" :condition
(action-descriptor "give-money" :actor receiver
:receiver giver :amount (use-negotiable
"price")) :receiver receiver))
(action-desire-fulfiller "give-money"
’(nop)))
:considerers (list
(consider-action "give-money"
’T)
(consider-action "give-item"
’(is-connected (action-descriptor "give-money"
:amount (verify-negotiable "price"))))))
Listing 7.4: Most Common Comparison Shopper (61%) and Flea Market (68%)Individual
buyer seller
Conditional Proposal Action: give-item Condition: give-money
Accept Proposal
Give Money
Give Item
Refuse Proposal
VerifyPrice
goodprice
badprice
pickprice
Figure 7.5: Most Common Comparison Shopper, Flea Market and Free Trade Protocol
123
(mapc-agent
:fulfillers (list
(action-desire-fulfiller "give-money"
’(nop))
(action-desire-fulfiller "give-item"
’(propose-action giver "give-money" :condition
(action-descriptor "give-item" :actor receiver
:receiver giver) :receiver receiver :amount
(use-negotiable "sell-price"))))
:considerers (list
(consider-action "give-item"
’T)
(consider-action "give-money"
’(is-connected (action-descriptor "give-money"
:amount (verify-negotiable "buy-price"))))))
Listing 7.6: Most Common Free Trade Individual (14%)
In the art dealer protocol domain, the most common acceptable protocol was a
seller-driven version of the buyer-driven protocol outlined above (see Listing 7.5 and
Figure 7.6). In this version, the seller, having a desire for the give money action,
proposes that another agent (the buyer) gives the seller money if and when the seller
gives an item to the buyer. This buyer protocol was chosen 44% of the time in the art
dealer protocol domain.
In the free trade protocol domain, the most common acceptable protocol is the
same as that found in both the comparison shopper protocol domain and the flea
market protocol domain. It is a straight-forward buyer-driven protocol and occurred
14% of the time. The only differences stem from the separation of the buyer price and
seller price negotiable values (see Listing 7.6).9
9As described in Section 7.1.4 the separation of the buyer and seller price negotiable values isnecessary because in the free trade protocol domain the agents have different preferences for price(higher or lower) depending on whether they are selling or buying an item.
124
(mapc-agent
:fulfillers (list
(action-desire-fulfiller "give-item"
’(nop))
(action-desire-fulfiller "give-money"
’(propose-action giver "give-money" :condition
(action-descriptor "give-item" :actor receiver
:receiver giver) :receiver receiver :amount
(use-negotiable "price"))))
:considerers (list
(consider-action "give-money"
’(is-connected (action-descriptor "give-money"
:amount (verify-negotiable "price"))))
(consider-action "give-item"
’T)))
Listing 7.5: Most Common Art Dealer Individual (44%)
sellerbuyer
Conditional Proposal Action: Give Money Condition: Give Item
Accept Proposal
Give Item
Give Money
Refuse Proposal
VerifyPrice
pickprice
badprice
goodprice
Figure 7.6: Most Common Art Dealer Protocol
125
7.2.3 Questionable Protocols
To illustrate the breadth of the possible protocols that can be created, one “ques-
tionable” protocol was selected from each protocol domain and their descriptions
follow. They were chosen by selecting a protocol that was only deemed acceptable
once throughout 50 generations in 30 executions of a test protocol domain. These
four questionable protocols show a variety of solutions that are obviously wrong under
the intelligent scrutiny of humans, and yet were deemed acceptable by MAPC due
to the specific scenarios in which they were evaluated. The variety of the protocols
described in this section illustrate the large number of protocols that are occasionally
acceptable. This should show the utility of MAPC since the most popular protocols
described above were of such a concise and readable nature.
One questionable protocol that evolved in the comparison shopper protocol domain
is a very trusting protocol that works by the seller asking for money and then giving
away an item. The protocol is shown in text form in Listing 7.7 and in visual form in
Figure 7.7. The protocol is acceptable if the amount for the given item that the seller
asks for is an acceptable amount for the buyer. The protocol would fail if the amount
was wrong.
In the art dealer protocol domain, one questionable protocol is a combination of
the buyer protocol and a protocol that has each seller giving away items to itself. In
Listing 7.8, under the give money desire, the seller’s desire for money, the action that
occurs is to give to the receiver (the intended receiver of the money, so the seller)
an item. Since no proposals are used, the consider for this action does not trigger.
Therefore, this questionable protocol acts the same as the popular buyer protocol
described in Section 7.2.2.
126
(mapc-agent
:fulfillers (list
(action-desire-fulfiller "give-item"
’(nop))
(action-desire-fulfiller "give-money"
’(cons (propose-action giver "give-money" :receiver
(this-agent) :amount amount) (perform-action
"give-item" :receiver giver))))
:considerers (list
(consider-action "give-money"
’T)
(consider-action "give-item"
’(is-connected (action-descriptor "give-money"
:amount (verify-negotiable "price"))))))
Listing 7.7: Questionable Comparison Shopper Individual
sellerbuyer
Proposal Action: Give Money
Accept Proposal
Give Item
Give Money
pickprice
Figure 7.7: Questionable Comparison Shopper Protocol
127
(mapc-agent
:fulfillers (list
(action-desire-fulfiller "give-item"
’(propose-action giver "give-item" :condition
(action-descriptor "give-money" :actor
(this-agent) :receiver giver :amount
(use-negotiable "price")) :receiver
(this-agent)))
(action-desire-fulfiller "give-money"
’(perform-action "give-item" :receiver receiver)))
:considerers (list
(consider-action "give-money"
’T)
(consider-action "give-item"
’(is-connected (action-descriptor "give-money"
:amount (verify-negotiable "price"))))))
Listing 7.8: Questionable Art Dealer Individual
seller
Give Item
buyer seller
Conditional Proposal Action: give-item Condition: give-money
Accept Proposal
Give Money
Give Item
Refuse Proposal
VerifyPrice
goodprice
badprice
pickprice
Buyer Protocol
AND
Figure 7.8: Questionable Art Dealer Protocol
128
Another example of a protocol that is almost the same as the popular seller
protocol was found in the flea market protocol domain. As shown in Listing 7.9, the
protocol differs in how the buyer determines if the proposal should be accepted.10
This considerer for the buyer is extra careful and ensures that the buyer verifies that
the agent that will give away money is indeed the agent considering the proposal. In
addition, the standard price requirements are checked along with the requirement that
the original proposer will be getting the money. This protocol is shown in Figure 7.9.
In the free trade protocol domain, another type of odd protocol was developed.
This protocol, shown in Listing 7.10, achieves an acceptable rating by MAPC by
having the trading agents trade with themselves (see Figure 7.10).
7.2.4 Buyer-Driven vs. Seller-Driven
Finally, of all of the acceptable protocols, the percent of buyer-driven protocols and
the percent of seller-driven protocols was calculated. For the purposes of this analysis,
a protocol is buyer-driven if there is a desire fulfiller other than nop (the null operation
which performs no action) for the give item action. A protocol is seller-driven if there
is a desire fulfiller other than nop for the give money action. Occasionally a protocol is
both buyer-driven and seller-driven. When calculating the percentage of buyer-driven
and seller-driven protocols, the calculation was made both by including duplicates
and excluding duplicates. The percentage of buyer-driven and seller-driven protocols
that were deemed acceptable, including duplicates, are given in Table 7.3. Table 7.4
illustrates the percentage of buyer- and seller-driven protocols deemed acceptable (no
duplicates). As was intended, the comparison shopper protocol domain led to a much
10It also differs in the use of this-agent where receiver was previously used, but as describedin Section 7.2.2 these two values are functionally equivalent in the desire fulfiller.
129
(mapc-agent
:fulfillers (list
(action-desire-fulfiller "give-item"
’(nop))
(action-desire-fulfiller "give-money"
’(propose-action giver "give-money" :condition
(action-descriptor "give-item" :actor
(this-agent) :receiver giver) :receiver
(this-agent) :amount (use-negotiable "price"))))
:considerers (list
(consider-action "give-money"
’(is-connected (action-descriptor "give-money"
:actor receiver :receiver requestor :amount
(verify-negotiable "price"))))
(consider-action "give-item"
’NIL)))
Listing 7.9: Questionable Flea Market Individual
sellerbuyer
Conditional Proposal Action: Give Money Condition: Give Item
Accept Proposal
Give Item
Give Money
pickprice
yes
Refuse Proposal
no
Will I be giving the money away, will the seller get the money, and is the
price correct?
Figure 7.9: Questionable Flea Market Protocol
130
(mapc-agent
:fulfillers (list
(action-desire-fulfiller "give-money"
’(propose-action receiver "give-item" :condition
(action-descriptor "give-money" :actor receiver
:receiver receiver :amount (use-negotiable
"buy-price")) :receiver (this-agent)))
(action-desire-fulfiller "give-item"
’(nop)))
:considerers (list
(consider-action "give-item"
’(is-connected (action-descriptor "give-money"
:receiver (this-agent) :amount (get-variable
(this-agent) "item-value"))))
(consider-action "give-money"
’T)))
Listing 7.10: Questionable Free Trade Individual
sellerConditional Proposal Action: Give Money Condition: Give Item
Accept Proposal
VerifyPrice
pickprice
badprice
goodprice
Refuse Proposal
Give Item
Give Money
Figure 7.10: Questionable Free Trade Protocol
131
Buyer-Driven Seller-Driven Both-Driven
Comparison Shopper 92.1% 3.8% 4.1%Art Dealer 38.9% 56.4% 4.7%
Flea Market 84.8% 9.0% 6.2%Free Trade 52.9% 31.1% 16.0%
Table 7.3: Buyer- vs. Seller-Driven Individuals (Including Duplicates)
Buyer-Driven Seller-Driven Both-Driven
Comparison Shopper 34.4% 19.3% 46.2%Art Dealer 13.0% 46.2% 40.8%
Flea Market 24.1% 25.0% 50.9%Free Trade 38.4% 34.5% 27.1%
Table 7.4: Buyer- vs. Seller-Driven Individuals (Unique Protocols)
higher number of buyer-driven protocols, while the art dealer protocol domain led
to a much higher number of seller-driven protocols. Despite this trend, protocols in
every category were deemed acceptable.
In the flea market and the free trade protocol domains, the unbiased protocol
domains, the spread of buyer-driven and seller-driven protocols was expected to be
fairly even. But the buyer-driven protocols also dominated in the flea market protocol
domain and was favoured slightly in the free trade protocol domain (see Tables 7.3
and 7.4). This unpredicted outcome can be attributed to the amount terminal. The
amount terminal may appear within the give money desire fulfiller used in seller-driven
protocols. As described above, when an agent wishes to fulfil a desire, the parameters
are each chosen randomly. This random amount is much less likely to be an acceptable
price and so makes a higher number of the seller-driven protocols fail, thus leading to
more buyer-driven protocols.
132
7.3 Reaching 100%
MAPC performed well and found acceptable protocols in 29 out of 30 runs in all of
the protocol domains within the first 20 generations. To determine if MAPC could
reach 100%, an extra experiment was executed using 100 generations. The results of
that run are shown in Figure 7.11. Unfortunately, it seems that an evolutionary dead
end happens in approximately 3% of the runs. Therefore, an optimal use of MAPC
would be to perform two runs with only 25 generations, consuming the same amount
of resources, but increasing the likelihood of finding at least one acceptable solution
to over 99%.
7.4 Results Summary
To test the capabilities of MAPC, four protocol domains were designed. These protocol
domains were intended to show that MAPC would produce different protocols in
different protocol domains and to test the types of protocol domains MAPC could
reliably handle. The comparison shopper and art dealer protocol domains were meant
to generate buyer-driven and seller-driven protocols, respectively. The flea market and
free trade protocol domains were meant to produce an equal amount of both buyer-
and seller-driven protocols, and also tested the capability of MAPC to work in both
highly structured (flea market) and less structured (free trade) protocol domains.
The results reflect that MAPC was capable of reliably generating protocols in all
of the four protocol domains, generated output that was consistent with the given
input, and, in most cases, generated concise and readable protocols. MAPC produced
acceptable protocols in a large number of the executions for all of the protocol domains.
133
0 20 40 60 80 100Generation
0
5000
10000
15000
20000
25000Fi
tness
MeanBest
0 20 40 60 80 100Generation
0%
20%
40%
60%
80%
100%
Runs
wit
h A
ccepta
ble
Pro
toco
l
97%
Figure 7.11: Free Trade Results for Long Run (100 Generations)
134
The split of buyer- and seller-driven protocols was as expected for the comparison
shopper and art dealer protocol domains. When there was an expectation of an even
distribution of buyer- and seller-driven protocols, though, there was a trend towards
more buyer-driven protocols. As described in the previous section, this can probably
be explained by the difference between the nodes available within a give money and
a give item desire fulfiller. The protocols generated by MAPC that were deemed
acceptable were primarily well-written examples in the given language. The most
common protocols were the buyer and seller protocols that result in a simple exchange
initiated by either the buyer or the seller with a built-in commitment to tie the two
exchanges (money and item) together. It is worth noting that despite the abundance
of odd and quirky protocols that could be generated by the components available to
MAPC, the trend was for concise protocols.
Chapter 8
Conclusion and Future Work
This dissertation describes the design and implementation of the multi agent protocol
creator (MAPC). Research into current methods to create protocols showed that
most work has been done in the area of creating new ways to describe protocols, and
that the automated creation of protocols was previously an unstudied area of work.
Research into the use of computer-based evolution to create communication showed
primarily research aimed at how basic communication could arise from a situation
without communication, not how communication protocols and conversations could be
evolved. To address this gap in the research, MAPC was created to allow multi agent
system designers the freedom to create new multi agent protocols by only specifying
the domains in which those protocols needed to run. To allow the automatic creation
of a communications protocol, some model for describing the requirements of the
protocol was necessary. For this reason a detailed model of a protocol domain was
created. A designer can describe a protocol domain in this model and then MAPC
will create a protocol which satisfies the requirements of that protocol domain.
This chapter summarizes the contributions of this work and examines potential
future work in this area. Section 8.1 describes in detail the specific contributions made
by this dissertation. Section 8.2 outlines several interesting directions for research
that could be explored to expand upon the initial work undertaken in the creation of
MAPC.
135
136
8.1 Contributions
The contributions of this dissertation are three-fold. First, a new method was created
to describe the requirements of a communications protocol in a fashion that facilitates
the evolution of new protocols. Second, a new method for ensuring the reasonable
placement of genetic programming nodes within a program tree independent of types
was developed to aid the evolution of new protocols. Third, a fully functional multi
agent protocol creator was designed, implemented and tested. These contributions
are described in more detail in the following sections.
8.1.1 The Protocol Domain Model
To automate the creation of communications protocols, a method for describing
the requirements of a desired protocol was required. Further, due to the fact that
evolutionary algorithms were to be used to create the new protocols, that same
input for the protocol creator needed to describe all of the details necessary to create
potential protocols, generate environments in which to test a potential protocol, and
then evaluate how well each potential protocol met the requirements of the protocol
domain in order to provide a fitness value to guide the genetic programming.
Defining the input for the protocol creator was accomplished by creating what
was termed a protocol domain model (see Chapter 4 for a complete description).
This model describes the agents, each agent’s capabilities, and the requirements of a
protocol domain. The agents and the system are described by two sets. The first is
the set of role types, which defines capabilities and internal data. The second set is
the scenarios, which defines the possible configurations of the agents in the system.
In addition to the description of the agents, the model includes a description of the
137
requirements, both required and desired. The hard requirements are described in
terms of pre- and post- conditions for all agent actions and post-conditions for each of
the agents and the system as a whole. The soft requirements were included in the
model by allowing for functions which should be minimized or maximized for either
the agents or the entire system. The complete protocol domain model describes all of
the information needed by an automatic protocol creator to create a protocol.
8.1.2 The Context System
While creating the multi agent protocol creator, it was discovered that during the
creation of potential protocols as trees, many of the functions and terminals, or nodes,
were meaningless or absurd outside of a limited number of reasonable locations for
those nodes. For instance, a function that performs a specific action available only to
one role type should not be in a location that would never be executed by that same
role type. This problem is similar to the problem originally solved by Montana when
strongly typed genetic programming was proposed (Montana, 1994).1 Typing was not
enough to remove all absurd combinations, however, since typing has no means to
restrict nodes based on the location within the tree. As work progressed on MAPC,
the concept of context for genetic programming was explored (see Section 5.3.2, p. 73
for a complete explanation).
The context of a node indicates the set of reasonable locations for that node,
complimenting the information stored in the type of the node’s inputs and outputs.
In traditional programming, an analogous situation is the use of scope by compilers.
Scope allows a programmer to limit the use of a name, such as that of a variable, to
1Typing ensures that a Boolean value is not added to a number, for example, since the add
function is designated to only accept two integers as input.
138
the locations where that variable should reasonably be used. Similarly, context does
the same for genetic programming. With the use of context, a very large number of
illogical potential protocols can be avoided, which leads to more potentially viable
protocols. This was shown to speed up the time it takes for a genetic programming
system to find an acceptable solution.
8.1.3 The Multi Agent Protocol Creator
An evolutionary system was created which takes a protocol domain model and produces
a protocol that will work in a described protocol domain, meeting all of the goals of the
various components of the protocol domain. This system was termed the Multi Agent
Protocol Creator (MAPC). MAPC uses the protocol domain model, in a Lisp-based
format, as its input. MAPC is built upon a previously created evolutionary framework,
ECJ, and makes use of a previously created multi agent system, CASA, to execute
potential protocols for evaluation purposes. In addition, MAPC also makes use of
strongly typed nodes and the context system described above to ensure that nodes
are only found in reasonable locations.
To see if MAPC worked as expected, four protocol domains were modelled. These
protocol domains each modelled a fairly basic transaction between buyers and sellers
with slight variations to see how MAPC reacted. The results of these tests show
that MAPC is able to evolve acceptable protocols for a variety of simple transaction
protocol domains. In addition, the most commonly accepted protocol in every protocol
domain was a protocol that was similar in simplicity and functionality to one that a
human designer would likely have created.
139
8.2 Future Work
Evolving communications protocols is an exciting area of research in multi agent
systems. Four significant possibilities for future work stand out: First, more complex
protocol outputs could be integrated into MAPC to facilitate more interesting protocol
creations. Second, alternate methods could be examined to enforce the constraints of
the context system. Third, a system could be designed to evaluate a multi agent system,
generate a protocol domain file, and then incorporate a new protocol automatically
into a multi agent system to provide a new tool for self-organizing systems. Fourth,
more robust protocols with error handling could be generated by introducing “bad”
agents into the evaluation of protocols to ensure that bad agents receive no benefits,
or, failing that, are caught breaking social commitments.
The protocols in MAPC only allow a single action as a condition to a commitment.
This limits the complexity of the protocols which can be generated by MAPC. There
is a large body of research into commitment-based protocols beyond those used in
MAPC (see Section 2.9, p. 33 for a selection). Many of these protocol specifications
allow for more complex connections between commitments, including conditions based
on multiple actions or even other commitments. Incorporating these more intricate
conditions into MAPC’s protocols would require testing to see how an increase in
complexity would affect the ability of MAPC to arrive at acceptable protocols. It would
also require work to try to keep the produced protocols concise and straightforward.
If these more complex conditions were incorporated a much wider range of protocols
could be produced by MAPC.
The context system is used in MAPC to constrain the resulting protocols. Another
method to constrain individuals uses grammars (Poli et al., 2008). When using
140
grammar-based constraints , the genetic representation of an individual is a sequence
of numbers which are used to “walk” down a path of the grammar tree. This makes
the individuals very small while also ensuring that all of the rules of the grammar
are applied. This could potentially be used to perform the kinds of checks that the
context system is performing in MAPC currently. A method for generating grammars
based on the description of the contexts, or an entirely new way of describing the
protocols would weed to be developed to make this work. If this was successful, it
might lead to faster evolution and therefore the ability to work with more complex
protocols.
MAPC creates multi agent protocols from a description of the protocol domain in
which they must work. The input for MAPC is designed to be both machine readable
and machine generated. Because of this, self-organizing systems might be able to use
MAPC as a tool to change the way they work.2 This would require finding areas with
non-optimal communication, creating a protocol domain description file to describe
the protocol domain, running MAPC to generate a protocol, and then incorporating
that protocol once it has been developed. This work could lead to self-organizing
systems that adapt to an even larger number of changes.
Currently, during testing of potential protocols, all agents act faithfully and are
trustworthy. This behaviour is not a reasonable assumption in an open system that
may include competing agents. To allow for the creation of more robust protocols,
MAPC could be extended to incorporate agents that are knowingly untrustworthy
to determine how the potential protocols handle these agents. Since the protocols
2Autonomic computing and self-managing architectures are an area of research that aims tocreate systems which will automatically adjust as the demands and composition of the system change(Kephart and Chess, 2003; Kramer and Magee, 2007). Self-organizing systems are one type ofself-managing system.
141
produced by MAPC are described in terms of commitments, it should be the case that
an agent that is untrustworthy should either gain no advantage over trustworthy agents,
or be identified by the system as failing to meet its commitments (and therefore be
eligible for sanctions). Making this change would require researching how to evaluate
the benefits an agent has received through dishonesty. In addition, the execution
scoring in MAPC would need to incorporate the number of broken social commitments
during an execution. This will require ensuring that the well-behaved agents can
proceed without breaking any social commitments while misbehaving agents are
caught.
Bibliography
Wolfgang Banzhaf, Peter Nordin, Robert E. Keller, and Frank D. Francone. GeneticProgramming: An Introduction. Morgan Kaufmann Publishers, Inc., San Francisco,CA, USA, 1998.
John Batali. Computational simulations of the emergence of grammar. In James R.Hurford, Michael Studdert-Kennedy, and Chris Knight, editors, Approaches to theEvolution of Language, pages 405–426. Cambridge University Press, Cambridge,UK, 1998.
Michael Burrows, Martin Abadi, and Roger Needham. A logic of authentication.Research Report 39, Systems Research Center, Palo Alto, CA, USA, February 1989.
Cristiano Castelfranchi. Commitments: From individual intentions to groups andorganizations. In Victor R. Lesser, editor, Proceedings of the First InternationalConference on Multi-Agent Systems (ICMAS-95), pages 41–48, San Francisco, CA,USA, June 1995. AAAI-Press and MIT Press.
John A. Clark and Jeremy L. Jacob. Protocols are programs too: the meta-heuristicsearch for security protocols. Information and Software Technology, 43(14):891–904,December 2001. doi: 10.1016/S0950-5849(01)00195-1.
Kalyanmoy Deb. Multi-Objective Optimization Using Evolutionary Algorithms: AnIntroduction, chapter 1, pages 3–34. Springer-Verlag, London, 2011.
Jorg Denzinger and Jasmine Hamdan. Improving modeling of other agents usingtentative stereotypes and compactification of observations. In IAT 2004, pages106–112. Intelligent Agent Technology, September 2004.
Jorg Denzinger and Jordan Kidney. Evaluating different genetic operators in thetesting for unwanted emergent behavior using evolutionary learning of behavior. InProceedings of IAT 2006, pages 23–29, 2006.
Jorg Denzinger and Michael Kordt. Evolutionary on-line learning of cooperativebehavior with situation-action-pairs. In Proceedings of the Fourth InternationalConference on Multi-Agent Systems (ICMAS-2000), pages 103–110. IEEE Press,2000.
Jorg Denzinger, Kevin Loose, Darryl Gates, and John Buchanan. Dealing withparameterized actions in behavior testing of commercial computer games. InProceedings of the IEEE 2005 Symposium on Computational Intelligence and Games(CIG-05), pages 51–58, 2005.
142
143
Roberto Flores and Rob Kremer. To commit or not to commit: Modelling agentconversations for action. Computational Intelligence, 18(2):120–173, 2001.
Roberto Flores and Rob Kremer. A principled modular approach to construct flexibleconversation protocols. In Ahmed Y. Tawfik and Scott D. Goodwin, editors,Advances in Artificial Intelligence, volume 3060 of Lecture Notes in ComputerScience, pages 1–15. Springer-Verlag, Heidelberg, Germany, 2004.
Foundation for Intelligent Physical Agents (FIPA). FIPA ACL message structurespecification. document number SC00061G, FIPA TC communication., December2002a. URL http://www.fipa.org/specs/fipa00061/SC00061G.html. VisitedJune 2013.
Foundation for Intelligent Physical Agents (FIPA). FIPA communicative act libraryspecification. document number SC00037J, FIPA TC communication., December2002b. URL http://www.fipa.org/specs/fipa00037/SC00037J.html. VisitedJune 2013.
Jason Heard. Detecting broken social commitments: An implementation and investi-gation. M.Sc. thesis, University of Calgary, Calgary, Alberta, Canada, 2005.
Jason Heard and Rob Kremer. Modelling agent protocol requirements (poster). InThe Fourth International Conference on Knowledge Engineering and OntologyDevelopment (KEOD), 2012.
John H. Holland. Adaptation in Natural and Artificial Systems: An IntroductoryAnalysis with Applications to Biology, Control, and Artificial Intelligence. TheUniversity of Michigan Press, Ann Arbor, MI, USA, 1975.
Michael N. Huhns and Munindar P. Singh. Agents and multiagent systems: Themes,approaches, and challenges. In Michael N. Huhns and Munindar P. Singh, editors,Readings in Agents, chapter 1, pages 1–27. Morgan Kaufmann Publishers, Inc., SanFrancisco, CA, USA, 1997.
Christian Jacob. Illustrating Evolutionary Computation with Mathematica. MorganKaufmann Publishers, Inc., San Francisco, CA, USA, 2001.
David R. Cross Judith G. Lambiotte, Donald F. Dansereau and Sharon B. Reynolds.Multirelational semantic maps. Educational Psychology Review, 1(4):331–367, 1989.doi: 10.1007/BF01320098.
Jeffrey O. Kephart and David M. Chess. The vision of autonomic computing. Computer,36(1):41–50, January 2003. doi: 10.1109/MC.2003.1160055.
144
Seyed Hossein Khasteh, Saeed Bagheri Shouraki, and Ramin Halavati Majid Lesani.Communication protocol evolution by natural selection. In Proceedings of theInternational Conference on Computational Inteligence for Modelling Control andAutomation (CIMCA’06), Los Alamitos, CA, USA, 2006. IEEE Computer Society.
John R. Koza. Genetic Programming: On the Programming of Computers by Meansof Natural Selection. MIT Press, Cambridge, MA, USA, 1992.
John R. Koza. Genetic Programming II: Automatic Discovery of Reusable Programs.MIT Press, Cambridge, MA, USA, 1994.
Jeff Kramer and Jeff Magee. Self-managed systems: an architectural challenge. InFOSE ’07: 2007 Future of Software Engineering, pages 259–268, Washington, DC,USA, 2007. IEEE Computer Society. doi: 10.1109/FOSE.2007.19.
Sean Luke. ECJ 19: A java-based evolutionary computation research system, 2009.URL http://cs.gmu.edu/~eclab/projects/ecj/. Visited August 2009.
Sean Luke and Lee Spector. A comparison of crossover and mutation in geneticprogramming. In Genetic Programming 1997: Proceedings of the Second AnnualConference, pages 240–248, Stanford, CA, USA, July 1997. Morgan KaufmannPublishers, Inc.
Melanie Mitchell. An Introduction for Genetic Algorithms. MIT Press, Cambridge,MA, USA, 1996.
David J. Montana. Strongly typed genetic programming. BBN Technical Report 7866,Bolt Baranek and Newman, Cambridge, MA, USA, 1994.
Liviu Panait and Sean Luke. Cooperative multi-agent learning: The state of theart. Autonomous Agents and Multi-Agent Systems, 11(3):387–434, 2005. doi:10.1007/s10458-005-2631-2.
Sanjeev Paskaradevan and Jorg Denzinger. A hybrid cooperative behavior learningmethod for a rule-based shout-ahead architecture. In Web Intelligence and IntelligentAgent Technology (WI-IAT), 2012, volume 2, pages 266–273, 2012. doi: 10.1109/WI-IAT.2012.33.
Philippe Pasquier, Roberto Flores, and Brahim Chaib-draa. Modelling flexible socialcommitments and their enforcement. In Proceedings of the Fifth InternationalWorkshop on Engineering Societies in the Agents World (ESAW04), volume 3451 ofLecture Notes in Artificial Intelligence, pages 153–165. Springer-Verlag, Heidelberg,Germany, 2004. doi: 10.1007/11423355 10.
145
Riccardo Poli, William B Langdon, Nicholas F McPhee, and John R Koza. A FieldGuide to Genetic Programming. lulu.com, March 2008.
Munindar P. Singh. A conceptual analysis of commitments in multiagent systems.Technical Report TR-96-09, Department of Computer Science, North Carolina StateUniversity, Raleigh, NC, USA, May 1996.
Munindar P. Singh. An ontology for commitments in multiagent systems: Toward aunification of normative concepts. Artificial Intelligence and Law, 7:97–113, 1999.
Kyle Wagner, James A. Reggia, Juan Uriagereka, and Gerald S. Wilkinson. Progressin the simulation of emergent communication and language. Adaptive Behavior, 11(1):37–69, March 2003.
Michael Wooldridge. Intelligent agents. In Gerhard Weiss, editor, Multiagent Systems:A Modern Approach to Distributed Artificial Intelligence, pages 27–77. MIT Press,Cambridge, MA, USA, 1999.
Pinar Yolum and Munindar P. Singh. Commitment machines. In John-Jules Meyer andMilind Tambe, editors, Proceedings of the Eighth International Workshop on AgentTheories, Architectures, and Languages (ATAL-2001), pages 245–257, Heidelberg,Germany, 2001. Springer-Verlag.
Appendix A
Nomenclature
In this work, the following symbols and terms are used.
A The set of agents in a multi agent system.
am An action in some role type.
Act The set of actions an agent may perform.
Acttypn The set of all possible actions that a role type may perform.
Ag An agent in a multi agent system.
B The set of all boolean values: B = true, false .
conforms A function indicating that the given agent conforms to the given role type
or protocol domain.
dm A desire in some role type.
Dat The set of internal states of an agent.
Datimp The set of all non role type-specific data for an agent.
Desiren The set of all possible desires that a role type may have.
DI The set of all possible domain information states for a protocol domain.
146
147
Dom A protocol domain which contains the scenarios, agents and data specific to the
domain.
Env The set of all environments in a multi agent system. When a multi agent system
is implementing a domain, this set will match the set within one scenario in the
domain.
env0 The starting environment for a particular scenario.
ii The input domain for a parameter of an action.
MAS A multi agent system.
nm A negotiable value.
Negn The set of all negotiable values that may be held by a role type.
P The powerset operator.
pi A parameter in an action.
p′i A parameter in an desire (either a correct parameter or an undefined value ⊥).
prefm A partial ordering of the values in a negotiable value that indicates the prefer-
ence of an agent, role or role type.
Reqn The set of requirements for a particular role type.
Rol The set of roles in a scenario.
Scn The set of scenarios in a protocol domain. Each scenario includes the specific
roles in that scenario as well as the environment for the scenario.
148
scnn One scenario in a protocol domain.
Sit The set of situations that some agent may percieve.
Sittypn The set of situations that some agent may percieve when fit into a specific role
type.
Typ The set of role types in a protocol domain.
valm The set of posible vaules in a negotiable value.
Z The set of all integers.
Appendix B
Protocol Domain Files
Below are the four protocol domains files that were used in Chapter 7. The format for
these files is described in Section 4.3, p. 55.
B.1 Comparison Shopper Lisp File
1 (protocol-domain
2 "comparison shopper"
3 ; role-types
4 (list
5 (role-type "transactor"
6 :variables (list
7 (variable "item-count" "integer"
8 :initial-value ’(random 5))
9 (variable "money" "integer"
10 :initial-value ’(+ 100 (random 100)))))
11 (role-type "seller" :super-type "transactor"
12 :desires (list
13 (action-desire "give-money" :receiver
’(this-agent)))
14 :actions (list
15 (action "give-item" "giver"
16 ’(list (set-variable giver "item-count" (-
(get-variable giver "item-count") 1))
17 (set-variable giver "items-given" (+
(get-variable giver "items-given") 1))
18 (set-variable receiver "item-count" (+
(get-variable receiver "item-count") 1)))
19 :parameters (list
20 (variable "receiver" "agent"
149
150
21 :initial-value ’(random-agent :role-type
"buyer")))
22 :pre-conditions ’(> (get-variable giver
"item-count") 0)))
23 :variables (list
24 (variable "minimum-price" "integer"
25 :initial-value ’(+ 1 (random 10)))
26 (variable "items-given" "integer"
27 :initial-value 0)
28 (variable "money-received" "integer"
29 :initial-value 0))
30 :negotiables (list
31 (negotiable "price" "integer" "higher" :minimum
’(get-variable (this-agent) "minimum-price")
:maximum 20 ))
32 :fitness (fitness
33 :post-condition ’(= (get-variable (this-agent)
"items-given") (get-variable (this-agent)
"money-received"))))
34 (role-type "buyer" :super-type "transactor"
35 :desires (list
36 (action-desire "give-item" :required T :times
’(+ 1 (random 3)) :receiver ’(this-agent)))
37 :actions (list
38 (action "give-money" "giver"
39 ’(list (set-variable giver "money" (-
(get-variable giver "money") amount))
40 (set-variable receiver "money" (+
(get-variable receiver "money") amount))
41 (set-variable receiver "money-recieved" (+
(get-variable receiver "money-recieved")
1)))
42 :parameters (list
43 (variable "receiver" "agent"
44 :initial-value ’(random-agent :role-type
"seller"))
45 (variable "amount" "integer"
46 :initial-value ’(+ 1 (random 10))))
47 :pre-conditions ’(and
48 (>= (get-variable giver "money") amount))
151
49 :post-conditions ’(and
50 (>= amount (get-variable receiver
"minimum-price"))
51 (<= amount (get-variable giver
"maximum-price")))
52 ))
53 :variables (list
54 (variable "maximum-price" "integer"
55 :initial-value ’(+ 10 (random 10))))
56 :negotiables (list
57 (negotiable "price" "integer" "lower" :minimum 0
:maximum ’(get-variable (this-agent)
"maximum-price")))
58 :fitness (fitness
59 :maximize ’(+ (get-variable (this-agent) "money")
(* (get-variable (this-agent) "item-count")
(get-variable (this-agent)
"maximum-price"))))))
6061 ; agent groups
62 (list
63 (agent-group "seller-group" "seller" 1 8)
64 (agent-group "buyer-group" "buyer" 1 1))
6566 :evaluation-threads 10
67 :population-size 500
68 :run-time 4000
69 :score-time 4000
70 :last-generation 50
71 )
Listing B.1: comparison-shopper.lisp
152
B.2 Art Dealer Lisp File
1 (protocol-domain
2 "art dealer"
3 ; role-types
4 (list
5 (role-type "transactor"
6 :variables (list
7 (variable "item-count" "integer"
8 :initial-value ’(+ 3 (random 5)))
9 (variable "money" "integer"
10 :initial-value ’(+ 0 (random 100)))))
11 (role-type "seller" :super-type "transactor"
12 :desires (list
13 (action-desire "give-money" :required T :times
’(+ 1 (random 3)) :receiver ’(this-agent)))
14 :actions (list
15 (action "give-item" "giver"
16 ’(list (set-variable giver "item-count" (-
(get-variable giver "item-count") 1))
17 (set-variable receiver "item-count" (+
(get-variable receiver "item-count") 1))
18 (set-variable receiver "items-received" (+
(get-variable receiver "items-received")
1)))
19 :parameters (list
20 (variable "receiver" "agent"
21 :initial-value ’(random-agent :role-type
"buyer")))
22 :pre-conditions ’(> (get-variable giver
"item-count") 0)))
23 :variables (list
24 (variable "minimum-price" "integer"
25 :initial-value ’(+ 1 (random 10))))
26 :negotiables (list
27 (negotiable "price" "integer" "higher" :minimum
’(get-variable (this-agent) "minimum-price")
:maximum 20 ))
28 :fitness (fitness
153
29 :maximize ’(+ (get-variable (this-agent) "money")
(* (get-variable (this-agent) "item-count")
(get-variable (this-agent) "minimum-price")))))
30 (role-type "buyer" :super-type "transactor"
31 :desires (list
32 (action-desire "give-item" :receiver
’(this-agent)))
33 :actions (list
34 (action "give-money" "giver"
35 ’(list (set-variable giver "money" (-
(get-variable giver "money") amount))
36 (set-variable receiver "money" (+
(get-variable receiver "money") amount))
37 (set-variable giver "money-given" (+
(get-variable giver "money-given") 1)))
38 :parameters (list
39 (variable "receiver" "agent"
40 :initial-value ’(random-agent :role-type
"seller"))
41 (variable "amount" "integer"
42 :initial-value ’(+ 1 (random 10))))
43 :pre-conditions ’(and
44 (>= (get-variable giver "money") amount))
45 :post-conditions ’(and
46 (>= amount (get-variable receiver
"minimum-price"))
47 (<= amount (get-variable giver
"maximum-price")))
48 ))
49 :variables (list
50 (variable "maximum-price" "integer"
51 :initial-value ’(+ 10 (random 10)))
52 (variable "items-received" "integer"
53 :initial-value 0)
54 (variable "money-given" "integer"
55 :initial-value 0))
56 :negotiables (list
57 (negotiable "price" "integer" "lower" :minimum 0
:maximum ’(get-variable (this-agent)
"maximum-price")))
154
58 :fitness (fitness
59 :post-condition ’(= (get-variable (this-agent)
"items-received") (get-variable (this-agent)
"money-given")))))
6061 ; agent groups
62 (list
63 (agent-group "seller-group" "seller" 1 1)
64 (agent-group "buyer-group" "buyer" 1 8))
6566 :evaluation-threads 10
67 :population-size 500
68 :run-time 4000
69 :score-time 4000
70 :last-generation 50
71 )
Listing B.2: art-dealer.lisp
155
B.3 Flea Market Lisp File
1 (protocol-domain
2 "flea market"
3 ; role-types
4 (list
5 (role-type "transactor"
6 :variables (list
7 (variable "item-count" "integer"
8 :initial-value ’(+ 0 (random 5)))
9 (variable "money" "integer"
10 :initial-value ’(+ 0 (random 100)))))
11 (role-type "seller" :super-type "transactor"
12 :desires (list
13 (action-desire "give-money" :receiver
’(this-agent)))
14 :actions (list
15 (action "give-item" "giver"
16 ’(list (set-variable giver "item-count" (-
(get-variable giver "item-count") 1))
17 (set-variable receiver "item-count" (+
(get-variable receiver "item-count") 1))
18 (set-variable giver "items-given" (+
(get-variable giver "items-given") 1))
19 (set-variable receiver "items-received" (+
(get-variable receiver "items-received")
1)))
20 :parameters (list
21 (variable "receiver" "agent"
22 :initial-value ’(random-agent :role-type
"buyer")))
23 :pre-conditions ’(> (get-variable giver
"item-count") 0)))
24 :variables (list
25 (variable "minimum-price" "integer"
26 :initial-value ’(+ 1 (random 10)))
27 (variable "items-given" "integer"
28 :initial-value 0)
29 (variable "money-received" "integer"
156
30 :initial-value 0))
31 :negotiables (list
32 (negotiable "price" "integer" "higher" :minimum
’(get-variable (this-agent) "minimum-price")
:maximum 20 ))
33 :fitness (fitness
34 :post-condition ’(= (get-variable (this-agent)
"items-given") (get-variable (this-agent)
"money-received"))))
35 (role-type "buyer" :super-type "transactor"
36 :desires (list
37 (action-desire "give-item" :receiver
’(this-agent)))
38 :actions (list
39 (action "give-money" "giver"
40 ’(list (set-variable giver "money" (-
(get-variable giver "money") amount))
41 (set-variable receiver "money" (+
(get-variable receiver "money") amount))
42 (set-variable giver "money-given" (+
(get-variable giver "money-given") 1))
43 (set-variable receiver "money-received" (+
(get-variable receiver "money-received")
1)))
44 :parameters (list
45 (variable "receiver" "agent"
46 :initial-value ’(random-agent :role-type
"seller"))
47 (variable "amount" "integer"
48 :initial-value ’(+ 1 (random 10))))
49 :pre-conditions ’(and
50 (>= (get-variable giver "money") amount))
51 :post-conditions ’(and
52 (>= amount (get-variable receiver
"minimum-price"))
53 (<= amount (get-variable giver
"maximum-price")))
54 ))
55 :variables (list
56 (variable "maximum-price" "integer"
157
57 :initial-value ’(+ 10 (random 10)))
58 (variable "items-received" "integer"
59 :initial-value 0)
60 (variable "money-given" "integer"
61 :initial-value 0))
62 :negotiables (list
63 (negotiable "price" "integer" "lower" :minimum 0
:maximum ’(get-variable (this-agent)
"maximum-price")))
64 :fitness (fitness
65 :post-condition ’(= (get-variable (this-agent)
"items-received") (get-variable (this-agent)
"money-given")))))
6667 ; agent groups
68 (list
69 (agent-group "seller-group" "seller" 1 4)
70 (agent-group "buyer-group" "buyer" 1 4))
7172 :evaluation-threads 10
73 :population-size 500
74 :run-time 4000
75 :score-time 4000
76 :last-generation 50
77 )
Listing B.3: flea-market.lisp
158
B.4 Free Trade Lisp File
1 (protocol-domain
2 "free trade"
3 ; role-types
4 (list
5 (role-type "trader"
6 :desires (list
7 (action-desire "give-money" :receiver
’(this-agent))
8 (action-desire "give-item" :receiver
’(this-agent)))
9 :actions (list
10 (action "give-item" "giver"
11 ’(list (set-variable giver "item-count" (-
(get-variable giver "item-count") 1))
12 (set-variable receiver "item-count" (+
(get-variable receiver "item-count") 1))
13 (set-variable giver "items-given" (+
(get-variable giver "items-given") 1))
14 (set-variable receiver "items-received" (+
(get-variable receiver "items-received")
1)))
15 :parameters (list
16 (variable "receiver" "agent"
17 :initial-value ’(random-agent :role-type
"trader")))
18 :pre-conditions ’(> (get-variable giver
"item-count") 0))
19 (action "give-money" "giver"
20 ’(list (set-variable giver "money" (-
(get-variable giver "money") amount))
21 (set-variable receiver "money" (+
(get-variable receiver "money") amount))
22 (set-variable giver "money-given" (+
(get-variable giver "money-given") 1))
23 (set-variable receiver "money-received" (+
(get-variable receiver "money-received")
1)))
159
24 :parameters (list
25 (variable "receiver" "agent"
26 :initial-value ’(random-agent :role-type
"trader"))
27 (variable "amount" "integer"
28 :initial-value ’(+ 1 (random 10))))
29 :pre-conditions ’(and
30 (>= (get-variable giver "money") amount))
31 :post-conditions ’(and
32 (>= amount (get-variable receiver
"item-value"))
33 (<= amount (get-variable giver
"item-value")))))
34 :variables (list
35 (variable "item-count" "integer"
36 :initial-value ’(+ 0 (random 5)))
37 (variable "money" "integer"
38 :initial-value ’(+ 0 (random 100)))
39 (variable "item-value" "integer"
40 :initial-value ’(+ 1 (random 20)))
41 (variable "items-given" "integer"
42 :initial-value 0)
43 (variable "items-received" "integer"
44 :initial-value 0)
45 (variable "money-given" "integer"
46 :initial-value 0)
47 (variable "money-received" "integer"
48 :initial-value 0))
49 :negotiables (list
50 (negotiable "sell-price" "integer" "higher"
:minimum ’(+ 1 (get-variable (this-agent)
"item-value")) :maximum 20)
51 (negotiable "buy-price" "integer" "lower"
:minimum 0 :maximum ’(get-variable
(this-agent) "item-value")))
52 :fitness (fitness
53 :post-condition ’(and
54 (= (get-variable (this-agent) "items-given")
(get-variable (this-agent)
"money-received"))
160
55 (= (get-variable (this-agent)
"items-received") (get-variable
(this-agent) "money-given"))))))
5657 ; agent groups
58 (list
59 (agent-group "trader-group" "trader" 2 6))
6061 :evaluation-threads 10
62 :population-size 500
63 :run-time 4000
64 :score-time 4000
65 :last-generation 50
66 )
Listing B.4: free-trade.lisp
