Principles of Automated Negotiation - Semantic … · Principles of Automated Negotiation SHAHEEN FATIMA Loughborough University, UK SARIT KRAUS Bar-Ilan University, Israel MICHAEL

Principles of Automated Negotiation

With an increasing number of applications in the context of multi-agent systems,automated negotiation is a rapidly growing area. Written by top researchers in the field,this state-of-the-art treatment of the subject explores key issues involved in the design ofnegotiating agents, covering strategic, heuristic, and axiomatic approaches. The authorsdiscuss the potential benefits of automated negotiation as well as the unique challengesit poses for computer scientists and for researchers in artificial intelligence. They alsoconsider possible applications and give readers a feel for the types of domain whereautomated negotiation is already being deployed.

This book is ideal for graduate students and researchers in computer science whoare interested in multi-agent systems. It will also appeal to negotiation researchers fromdisciplines such as management and business studies, psychology, and economics.

www.cambridge.org© in this web service Cambridge University Press

Cambridge University Press978-1-107-00254-8 - Principles of Automated NegotiationShaheen Fatima, Sarit Kraus and Michael WooldridgeFrontmatterMore information

http://www.cambridge.org/9781107002548

http://www.cambridge.org







Principles of Automated Negotiation

SHAHEEN FATIMALoughborough University, UK

SARIT KRAUSBar-Ilan University, Israel

MICHAEL WOOLDRIDGEUniversity of Oxford, UK






University Printing House, Cambridge CB2 8BS, United Kingdom

Cambridge University Press is part of the University of Cambridge.

It furthers the University’s mission by disseminating knowledge in the pursuit ofeducation, learning and research at the highest international levels of excellence.

www.cambridge.orgInformation on this title: www.cambridge.org/9781107002548

© Shaheen Fatima, Sarit Kraus and Michael Wooldridge 2015

This publication is in copyright. Subject to statutory exceptionand to the provisions of relevant collective licensing agreements,no reproduction of any part may take place without the written

permission of Cambridge University Press.

First published 2015

Printed in the United Kingdom by Clays, St Ives plc

A catalogue record for this publication is available from the British Library

Library of Congress Cataloguing in Publication dataFatima, Shaheen.

Principles of automated negotiation / Shaheen Fatima, Loughborough University, UK,Sarit Kraus, Bar-Ilan University, Israel, Michael Wooldridge, University of Oxford, UK.

pages cmISBN 978-1-107-00254-8 (Hardback)

1. Negotiation in business. 2. Negotiation.I. Kraus, Sarit. II. Wooldridge, Michael. III. Title.

HD58.6.F38 2014658.4′052–dc23 2014014329

ISBN 978-1-107-00254-8 Hardback

Cambridge University Press has no responsibility for the persistence or accuracyof URLs for external or third-party internet websites referred to in this publication,

and does not guarantee that any content on such websites is, or will remain,accurate or appropriate.






Shaheen:To my parents

Sarit:To my family

Michael:To Georg Gottlob, Peter Millican, and Bill Roscoe, with thanks











Contents

List of illustrations page xiPreface xiiiAcknowledgements xviSummary of key notation xvii

1 Introduction 11.1 The structure of negotiation 21.2 Parameters of automated negotiation 41.3 A strategic approach 71.4 Desiderata for automated negotiation 101.5 Advantages and disadvantages of automated negotiation 111.6 Structure of this book 131.7 Historical notes and further reading 14

2 Games in normal form 172.1 Zero-sum and non-zero-sum games 192.2 Pure and mixed strategies 202.3 Rational behaviour in strategic settings 212.4 Solution concepts 222.5 Solution properties 282.6 Social measures of utility 302.7 A first glimpse of bargaining as a game 33

3 Games in extensive form 353.1 A formal definition 363.2 Types of games and strategies 403.3 Nash equilibrium 413.4 Subgame perfect Nash equilibrium 423.5 Beliefs and sequential rationality 473.6 Weak perfect Bayesian equilibrium 50

vii






viii Contents

3.7 Sequential equilibrium 523.8 The role of information 53

4 Negotiation domains 564.1 Classifying negotiation domains 574.2 Some example negotiation domains 644.3 Historical notes and further reading 72

5 Strategic analysis of single-issue negotiation 745.1 The negotiation model 755.2 An infinite-horizon alternating offers protocol 765.3 A finite-horizon alternating offers protocol 835.4 Negotiation with imperfect information 865.5 Indivisible issue negotiation 925.6 Negotiating an issue with discrete values 935.7 More on negotiation protocols 945.8 Historical notes and further reading 95

6 Strategic analysis of multi-issue negotiation 976.1 Negotiation procedures 976.2 The negotiation model 986.3 Negotiation with perfect information 1006.4 Negotiation with imperfect information 1126.5 Dealing with indivisible issues 1146.6 Negotiating multiple issues with discrete and

continuous values 1176.7 Historical notes and further reading 119

7 The negotiation agenda 1217.1 Negotiation agenda 1227.2 Optimal agendas: package-deal procedure 1237.3 Optimal agendas: sequential procedure 1307.4 Optimal agendas: simultaneous procedure 1377.5 Historical notes and further reading 137

8 Multilateral negotiations 1398.1 Alternating offers protocol with multiple bargainers 1398.2 Auction protocols 1418.3 Contract net protocol 1458.4 Two-sided matching 1488.5 Bargaining for resource reallocation 1518.6 Historical notes and further reading 155






Contents ix

9 Heuristic approaches 1579.1 Generating counter offers 1579.2 Predicting opponent preferences and generating

counter offers 1629.3 Generating optimal agendas 1699.4 Design and evaluation of heuristic strategies 1719.5 Historical notes and further reading 174

10 Man–machine negotiations 17610.1 Agent decision making 17810.2 Modelling human negotiators 18510.3 Virtual agent negotiators 19010.4 Historical notes and further reading 191

11 Axiomatic analysis of negotiation 19211.1 Background 19211.2 Single-issue negotiation 19311.3 Multi-issue negotiation 19911.4 An alternative view of the Nash bargaining solution 20111.5 Axiomatic versus non-cooperative models of bargaining 20311.6 Historical notes and further reading 204

12 Applications 20612.1 Business process management 20612.2 Electronic commerce 20912.3 Grid computing 21012.4 Load balancing 21112.5 M-services 21312.6 Data, task, and resource allocation 21512.7 Resolving policy disputes over natural resources 21512.8 Supply chain management in logistics 21712.9 Event scheduling 21812.10 Crowdsourcing 21812.11 Assisting and training human negotiators 21912.12 Energy exchange in remote communities 22112.13 Web-based software negotiation systems 222

13 Related topics 22413.1 Social choice 22413.2 Argumentation 22913.3 Fair division 23113.4 Historical notes and further reading 232






x Contents

14 Concluding remarks 233

Appendix A Proofs 235References 246Index 267






Illustrations

2.1 The zero-sum matching pennies game page 202.2 The non-zero-sum Prisoner’s Dilemma game 202.3 A game with dominant-strategy equilibrium 242.4 A game with no dominant-strategy equilibrium 242.5 Payoff matrix for the Battle of the Sexes game 252.6 Payoff matrix for the rock–paper–scissors game 262.7 A game with an interesting ε-Nash equilibrium 282.8 Illustrating Pareto optimality 313.1 Two representations of the same game: (a) in extensive form and

(b) in normal form 373.2 Profit estimates for the market game 383.3 Extensive form of the market game 393.4 Extensive form of the chain store game 423.5 Payoff matrix for the chain store game 423.6 Reduced extensive form of the chain store game 433.7 Extensive form of the extended chain store game 443.8 Payoff matrix for the extended chain store game 453.9 Payoff matrix for the simultaneous-moves game following Player

1’s entry in the extended chain store game 453.10 Extensive form of a modified version of the chain store game 473.11 Payoff matrix for a modified version of the chain store game 473.12 Extensive form of the extended chain store game 533.13 Payoff matrix for the setting IS1 543.14 Payoff matrix for the setting IS2 554.1 The negotiation set 604.2 An encounter in the delivery domain 664.3 The data allocation domain: four servers each service requests

for their respective client community. Each client communicates

xi






xii Illustrations

with only one server. If a server is asked for a document that doesnot form part of its repository, it requests this from the relevantserver 68

5.1 The alternating offers protocol 775.2 The decay in value of a resource with initial value of 1 over 10

time steps, for values of δ between 0 and 1 815.3 The effect of deadline (n) and discount factor (δ ) on an agent’s

equilibrium share. The continuous lines denote the share for thefirst mover and the dotted lines those for the other agent 85

5.4 Extensive form of the negotiation game for Example 5.3 896.1 An illustration of the possibility of trade-offs 1018.1 The contract net protocol 1469.1 Conceder, linear, and Boulware strategies 1599.2 Information exchange between the mediator and the agents 1669.3 Training the surrogate 1709.4 Using the surrogate with the outer GA 1709.5 Designing a heuristic strategy 17311.1 An illustration of lack of monotonicity 19812.1 Adept architecture 20812.2 Trace architecture 212






Preface

We all of us have to negotiate – whether formally, as part of our jobs, or in-formally, as part of our everyday lives – and the outcomes of our negotiationshave direct and often dramatic consequences, for us and others. However, itis surely a safe bet that most of us wish we were better negotiators. There aremany reasons why we might not be as good at negotiating as we would wish.For one thing, it is often hard for us to really understand the issues at stakeand the consequences of various potential settlements, and for this reason wecan end up with outcomes that are not as good as those that we might in facthave been able to obtain. Moreover, in many cultures, negotiation is regardedas greedy or impolite, and as a consequence, some people may find it sociallyawkward or stressful to negotiate. Cultural inhibitions like these can preventus from obtaining the best outcome even when the topic of negotiation is ofgreat importance to us. Wouldn’t it be wonderful, then, if we had computersthat could effectively negotiate on our behalf . . . ? In short, the main aim of thisbook is to investigate this idea.

The idea of computers negotiating on our behalf may sound like sciencefiction, but we hope to demonstrate in this book that the theory and softwaretechnology underpinning automated negotiation is sufficiently advanced andsufficiently robust that it is entirely plausible that some form of automatednegotiation technologies could soon be deployed in large-scale real-world (i.e.,non-laboratory) applications. However, we would be failing in our duty if weleft you with the impression that you will be handing over negotiations foryour next pay rise to a computer program. Reliable practical experience withautomated negotiating systems is currently limited to a few restricted domains.As with any new technology, we should be appropriately cautious about theclaims we make for its potential value and applications. Towards the end of thebook, we give a feel for the types of domains where automated negotiation hasbeen and may be deployed.

xiii






xiv Preface

There are many reasons why it would be desirable to have computers thatcan negotiate for us. Perhaps most importantly, computers can explore system-atically and dispassionately possible outcomes that human negotiators mightoverlook, thereby reaching more efficient outcomes than might be obtained byhumans. We discuss the potential advantages and disadvantages of automatednegotiation in the main text of the book.

While the desire to ultimately have computers that can negotiate on our be-half is the primary motivation for this book, we should point out that thereare many other good reasons for studying automated negotiation. First, nego-tiation is clearly an important activity in our lives, and indeed in the globaleconomy. Any such activity is surely worthy of serious academic study. Sec-ond, as we hope to demonstrate, the mathematical formalisation and analysisof negotiation raises many interesting and deep scientific problems. And third,the automation of negotiation poses unique challenges for computer scientistsand researchers in artificial intelligence; in addressing these challenges, weextend the range of problems that are amenable to solution by computerisedtechniques.

In some more detail, the aims of this book are threefold:

1. Our first aim is to show how negotiation can be modelled and analysedmathematically. The main mathematical framework we use is game theory– the theory of interactions between self-interested economic agents. Al-though usually thought of as a branch of micro-economics, game theoryhas recently been found to be of great relevance and value in the analysisof distributed computer systems in which the participant nodes cannot beassumed to share common goals, as is the case in negotiation settings.

2. Second, as the name of the book and the discussion above indicate, we aimto consider how negotiation can be automated within computer programs.One key issue here is that while we ideally seek computer programs that candetermine optimal solutions to negotiation problems, in practice, it may bebeyond the scope of conventional computing techniques to compute suchoptimal solutions (i.e., the problem may be NP-hard, or worse). In suchcases, our task is to map out the border between computationally tractablenegotiation problems and computationally intractable ones, and to developalternative techniques (heuristics or approximations) that can be used to findnearly optimal solutions, or at least satisfactory solutions, for cases wherethey cannot be resolved using conventional techniques.

3. Third, we aim to give an indication of the kinds of applications for whichautomated negotiation solutions are being considered, the kinds of issuesthat arise when automated negotiating systems interact with people, and






Preface xv

finally, how negotiation stands in relation to other related approaches toreaching agreements, such as argumentation, social choice, and fair divi-sion.

Intended readership. The main intended audience for this book is graduatecomputer science students and researchers in computer science/artificial intel-ligence who wish to understand the main challenges and state of the art inautomated negotiation. However, we hope that the book will also be of inter-est and value to negotiation researchers from disciplines such as managementand business studies, psychology, and economics: much of the most fruitful re-search is carried out in the intersections between different research disciplines,and automated negotiation seems to us to be a wonderful and fascinating ex-emplar of such interdisciplinary research.

Prerequisites. The book assumes a knowledge of computer science that wouldbe obtained from a typical undergraduate degree in computer science, anda level of mathematical sophistication that would be consistent with such acourse. With respect to the latter, the main requirements are familiarity withthe basic structures of discrete mathematics, the notion of formal proof (and inparticular proof by induction), the basic concepts of probability, some familiar-ity with series, sequences, and their sums, and ideally some understanding ofbasic concepts in operations research and optimisation (e.g., linear program-ming and related concepts).






Acknowledgements

We thank our many co-authors for giving their permission to adapt materialfrom papers that we have written with them. We are very grateful to Ya’akov(“Kobi”) Gal, Enrico Gerding, Raz Lin, Valentin Robu, and MateuszTarkowski, who gave us extremely useful comments on drafts of the book.Jordan Summers-Young carefully proofread several drafts for us, and gave de-tailed and enormously helpful comments. Many thanks to Jeff Rosenschein foranswering questions about his work and for his advice and friendship over theyears. It goes without saying that any remaining errors are entirely the respon-sibility of the authors.

Sarit Kraus would like to acknowledge the support of the European ResearchCouncil under Advanced Grant 267523. Michael Wooldridge would like toacknowledge the support of the European Research Council under AdvancedGrant 291528.

xvi






Summary of key notation

Sets

N the natural numbers: 0, 1, 2, 3, . . .R the real numbersR+ the positive real numbers: R+ = {x ∈ R | x ≥ 0}R

m+ the set of m-element vectors of positive real numbers

[0,1] the closed interval from 0 to 1: {x ∈ R | 0 ≤ x ≤ 1}(0,1) the open interval from 0 to 1: {x ∈ R | 0 < x < 1}|S| cardinality of set S2S power set of set S

Games in normal form (Chapter 2)

P the set of players (i.e., the agent set)Si strategy set for Player i ∈ PS = S1 ×·· ·×S|P| set of strategy profiless ∈ S a strategy profileUi : S → R utility function for Player i ∈ PG = (P,(Si)i∈P,(Ui)i∈P) normal form gameMSi ∈ [0,1]|P| mixed (randomised) strategy set for Player i ∈ PW (s) social welfare of strategy profile s ∈ S

Games in extensive form (Chapter 3)

G = (X ,E,Θ,T,W,A , p,U) extensive form gameX set of nodes in extensive form gameZ set of leaf nodes in game treeT : X\Z → P∪P0 function that assigns to each non-leaf node a player or chanceP0 chance move

xvii






xviii Summary of key notation

Θ = {Θ0,Θ1, . . . ,Θ|P|} a player partition of X\Z that indicates which playerhas to make the decision at each non-leaf node

Wi Player i’s information setW = {W1,W2, . . . ,W|P|} information partition: for each Player i ∈ P, Wi is a

partition of Θi

A choice partition, that is, a set of all possible choices of different players atdifferent information sets

p probability distribution function: assigns to each x ∈ Θ0 a probability distri-bution px over the edges in x

Si strategy set for Player i ∈ PS = S1 ×·· ·×S|P| set of strategy profiless ∈ S a strategy profileUi : S → R utility function for Player i ∈ Pμ belief system

Task-oriented domains (Chapter 4)

Tsk set of all tasksc : 2Tsk → R+ cost function(T1, . . . ,T|P|) encounter in a TODD = (T1,T2) a deal (Ti ⊆ Tsk)χ the conflict deal

State-oriented domains (Chapter 4)

St set of states of the environmentPln set of joint plansc : Pln×P → R+ cost function for SODsst0 ∈ S initial state of environmentγi ⊆ S goal of Agent i(s0,γ1, . . . ,γ|P|) an encounter in a SODJ : s1 � s2 plan J ∈ J transforms s1 to s2

worsti cost of most expensive plan for iUi(J,s) utility i gets from executing plan J in state s

Worth-oriented domains (Chapter 4)

W : St ×P → R worth function

The delivery domain (Chapter 4)

(Loc,Links) undirected graph with vertices Loc and edges Linksdistance : Links → R+ distance function�0 ∈ Loc location of distribution warehouse






Summary of key notation xix

(Loc,Links,distance, �0) delivery domain instanceTi ⊆ Loc Agent i’s tasks in delivery domain

Data allocation domain (Chapter 4)

DS data setsServ serversalloc : DS → Serv allocation of data sets to serversusage : DS×Serv → R+ usage of data sets by clients of serversc(i, j) cost that would be incurred by i in serving data set to j

Production sequencing domain (Chapter 4)

Prod set of productso : Prod → N an orderOrd set of ordersPS set of production sequencessat(o) set of production sequences satisfying order oci : PS → R+ cost function for production cells

Single-issue negotiation (Chapter 5)

RPi reservation price of Agent iχ conflict dealci ∈ R+ cost incurred by Agent i for a one-time-step delayδi ∈ [0,1] discount factor for Agent i (0 ≤ δi ≤ 1)n ∈ N negotiation deadlineui : [0,1]×N→ R+ Agent i’s utility for single-issue negotiation

Multi-issue negotiation (Chapter 6) and the negotiation agenda (Chap-ter 7)

I = {1, . . . ,m} set of m issuesn ∈ N deadline for completing negotiation on all the issuesn̄ ∈ N deadline for a single stageδ ∈ [0,1] common discount factorO set of possible outcomes/dealso = (o1, . . . ,om) an outcome where o ∈ O and oi ∈ R

m+

wij ∈ R+ Player i’s weight for issue j

uij : [0,1]×N→ R+ Agent i’s utility for issue j

Ui : O×N→ R+ Agent i’s cumulative utility for all issuesEUi : O×N→ R+ Agent i’s expected cumulative utility for all issuessi(t) Agent i’s (i ∈ {a,b}) strategy for time period tuai(t) Agent a’s equilibrium utility for issue i and time period t






xx Summary of key notation

ua(t) Agent a’s cumulative equilibrium utility for time period tubi(t) Agent b’s equilibrium utility for issue i and time period tub(t) Agent b’s cumulative equilibrium utility for time period t{δ1, . . . ,δr} set of possible values for δPi probability that δ = δi

Ag ⊆ I an agenda of size gAGg set of all possible agendas of size gOAg Agent a’s optimal agenda of size gOBg Agent b’s optimal agenda of size gf j : {1, . . . ,g}→ {a,b} first mover function for issue j

Multilateral negotiations (Chapter 8)

P = {1, . . . , |P|} set of |P| agentsZ = {z1,z2, . . . ,zm} set of resourcesvi : 2Z → R valuation function for Agent ipi payment to Agent ip̄ = (1, . . . , p|P|) payment vector

Heuristic approaches for automated negotiation (Chapter 9)

m ∈ N the number of negotiation issuesx j

a→b(t) price offered by Agent a to Agent b at time tmini

j the reserve price for Agent i (i ∈ {a,b})αa

j : N×R→ [0,1] negotiation decision function for Agent atimax Agent i’s negotiation deadline (i ∈ {a,b})

kij ∈ R+ concession made by Agent i (i ∈ {a,b}) in the first time period

NA(t) set of agents negotiating with Agent a at time tZ t

i↔aτa time Agent a negotiates with a single agent|Z t

i↔a| length of negotiation between i and aM ∈ N maximum amount by which an agent can change her imitative be-

haviourR(M ) a random number in the range [0,M ]

Axiomatic models of negotiation (Chapter 11)

A set of possible agreementsD disagreement pointUi : A ∪{D}→ R+ utility function for Agent iS the bargaining set, a compact convex subset of R2

+

d ∈ R2+ utility pair for the disagreement outcome

di ∈ R+ Agent i’s utility for the disagreement outcome






Summary of key notation xxi

B set of all bargaining problems of the form (S ,d)f : B → R

2+ bargaining function

xi Agent i’s utility from an agreement






Documents

Principles of Automated Negotiation - Semantic … · Principles of Automated Negotiation SHAHEEN FATIMA Loughborough University, UK SARIT KRAUS Bar-Ilan University, Israel MICHAEL