Why We Need Evolutionary Semantics.

Luc Steels(1,2)

(1) ICREA, IBE(UPF-CSIC) - Barcelona(2) Sony Computer Science Laboratory - Paris

steels@arti.vub.ac.be

Abstract. One of the key components for achieving flexible, robust,adaptive and open-ended language-based communication between hu-mans and robots - or between robots and robots - is rich deep semantics.AI has a long tradition of work in the representation of knowledge, mostof it within the logical tradition. This tradition assumes that an au-tonomous agent is able to derive formal descriptions of the world whichcan then be the basis of logical inference and natural language under-standing or production. This paper outlines some difficulties with thislogical stance and reports alternative research on the development of an‘embodied cognitive semantics’ that is grounded in the world through arobot’s sensori-motor system and is evolutionary in the sense that theconceptual frameworks underlying language are assumed to be adaptedby agents in the course of dialogs and thus undergo constant change.

Official Reference: Steels, L. (2011) Why we need evolutionary semantics. In:Bach, J., S. Edelkamp (Eds.): KI 2011: Advances in Artificial Intelligence, 34thAnnual German Conference on AI, Berlin, Germany, October 4-7,2011. Proceed-ings. Lecture Notes in Computer Science 7006 Springer 2011. pp. 14-25.

1 Introduction

Human language like communication with robots remains today a very distantgoal. A few decades ago the problem was almost entirely on the side of robots.There were not enough physical robots to work with and the scarce robots thatwere available were unreliable, difficult to control and had only weak sensingcapabilities. Also the computing power and electronics available for sensing andmotor control had strict limitations. This situation has changed significantly thepast few years. There are now thousands of powerful robots in the world andtheir capacities in terms of embodiment, sensori-motor potential and comput-ing power, are quite sufficient for high level tasks. The intense activity aroundthe Robocup and the new developments towards standardized components forrobotics, such as ROS, are illustrative of this trend and it bodes well for futureresearch. On the other hand, research on natural language processing appearsnot ready to exploit these new robotic capabilities. After promising work withsystems like Shrdlu [19] or Shakey [9] in the early seventies, the ARPA speechunderstanding projects in the eighties [5], and the Verbmobil project in the

nineties [18], the quest for artificial systems that could understand language andproduce themselves goal-directed communication slowed down and research incomputational linguistics became dominated by statistical language processing.

There is no doubt that statistical language approach has been very successfuland is of practical use. Statistical language processing relies on a large corpus ofexample sentences (the larger the better) and on general purpose machine learn-ing algorithms. It basically attempts to develop language models that predictthe probability of a word occuring in a sentence given the previous words. Thisapproach stands in contrast to the one explored in earlier deep natural languageprocessing research which used sophisticated grammars based on linguistic the-ory and procedural semantics for the precise interpretation of meaning in termsof world models derived from sensing and actuating. Parsers try to extract richgrammatical structures of sentences before interpreting them and producers usedsophisticated planning techniques to determine what to say and then map mean-ing into words and grammatical constructions.

The main reasons why statistical language processing became more popularare as follows:

1. Human languages are unlike programming languages in the sense that sen-tences are rarely fully grammatical. Often only partial fragments are commu-nicated and errors in meaning, grammar use, word choice, or pronunciationare very common due to the speed with which utterances need to be pro-duced. Consequently parsers that rely on sentences being grammatical easilybreak down on real input. Statistical language processing handles this prob-lem by being rather shallow in terms of the syntactic structures that areextracted, sometimes even relying only on sequential structure instead ofhierarchy [3]. Often these shallow structures are enough for tasks that areneeded by search engines.

2. Grammars of human languages are extraordinarily complicated. It thereforebecame clear quite early in language processing research that it would beextremely hard to design grammars and lexicons by hand. Some form ofautomatic language learning is essential, and the most effective way to do soat the moment is to use statistical machine learning techniques.

But what if the goal is to use language for interacting with complex devicessuch as robots? Shallow parsing is not sufficient because the rich grammaticalstructures underlying sentences are there to help listeners grasp meaning. If weignore them we deprive ourselves of an important source of information. Lack ofsemantics or shallow semantics is too risky because it may lead to actions by therobot which are inappropriate or outright dangerous. Language production mustrely on careful planning of meaning and this meaning needs to be the basis ofsentence formulation as opposed to retrieving from memory sentence fragmentsthat have tended to occur in similar circumstances. Most importantly it is alsocrucial that meaning gets grounded in the context through the sensori-motorapparatus of the robot, and unless we have corpora that contain vast amountsof data on grounded interactions it is not possible to apply statistical machinelearning techniques.

speaker hearer

obj-252

obj-253

obj-249

box-1robot-2

obj-266

obj-268

obj-265

box-1

robot-1 robot-1

robot-2

Fig. 1. Example of experimental set-up in language game experiments. They alwaysinvolve two robots which share some reality. The images on the left and right show(top) what each robot sees and (bottom) the world model they derive from vision. Therobots have a cooperative goal, for example one robot tries to draw attention to anotherrobot, and they need to conceptualize reality and formulate utterances to achieve thisgoal. In this example, the robot might for example say: ”the yellow block right of you”.

This paper is part of a research effort that has deep grounded languageunderstanding and production again as the main target. We have been carryingout experiments in which humanoid robots play language games about real worldscenes that they experience through cameras and sensori-motor embodiment (seefigure 1 from experiments in spatial language as discussed in [12]).

This requires work on two fronts: language processing and semantics. Lan-guage processing requires novel solutions to reach the flexibility and open-endednessthat remains problematic for deep parsing systems. How we approach this prob-lem in our group is discussed in other papers (see e.g. [16]). Here I focus only onthe semantics side.

Work on (grounded) semantics in AI has mostly been logic-based, morespecifically within the tradition of logical empiricism, which was imported intoAI through the early influence of John McCarthy [8]. This tradition has its rootin the work of early logicians like Frege and Russell, and the research program ofthe later Vienese Circle, which included Carnap, Reichenbach, (early) Wittgen-stein and others. In the hands of AI researchers, these theoretical proposals havebeen operationalised in a brilliant way and it has lead to a wealth of applica-tions in problem solving, expert systems, semantic web, common sense, etc. Therecent Computers and Thought IJCAI address of Kowalski [6] shows that thislogical framework still forms one of the core approaches within AI.

The logical approach was originally designed as a normative framework forstreamlining rational deliberative thinking and for giving some guarantee thatinference steps are correct. It therefore focuses on the connection between propo-sitions and how truthvalues are preserved across propositions. The framework oflogic appears entirely adequate for this task. The question here is whether this

framework is adequate for acting as the core of a grounded language processingsystem.

The logic-based approach simply makes the assumption that the ’Language ofThought’ is ”a simplified and canonical form of unambiguous sentences in naturallanguage” [6], p.2. It is canonical in two ways: in terms of the concepts that arebeing used, which are considered to be universal and shared, and in terms of theway these concepts are used to construct more complex propositions.

Using a canonical representation has at first sight many advantages, becausenatural language expressions which appear equivalent in terms of meaning can bemapped to the same representation and other processes, such as visual percep-tion, can construct representations using the same form and the same conceptualcontent. However, deep issues come up concerning the question how propositionsare to be expressed, how they are grounded in reality through a sensori-motorembodiment, and how they relate to natural language.

The rest of this paper discusses these issues in more detail (section 2) andhow we should go about building AI systems to support human-like languagecommunication in the light of this criticism (section 3).

2 The Nature of Conceptualisation

The main problem with the logical stance is that it trivialises conceptualisation.Conceptualisation is the process whereby language users categorize the worldin order to talk about it. For example, for the sentence ”the box left of thetable” the speaker has categorized the two objects involved in terms of classes(”box” and ”table”), and introduced a spatial relation ”left of” between them.Apparently in this context it is clear which unique box and table are intendedbecause the article ”the” signals that they are ’definite’. There is also implicitlya perspective on the scene, because left-of is relative to the point of view of thespeaker with respect to the table and the hearer.

2.1 Conceptualization relies on cognitive operations

It is well known today that mapping concepts to reality requires a vast amountof signal processing and pattern recognition. These activities can be organizedin terms of cognitive operations which perform segmentation, feature extraction,dimensionality reduction, transformation, classification, set operations, etc. Atypical example of a cognitive operation is the application of the kind of classifiersacquired by Support Vector Machines, which are based on a representation of theexamples as points in space and hyperplanes that represent the largest separationbetween classes. An example of transformation is the computation of location ofobjects to transform a perceived scene from a viewer-centered coordinate systemto that of another agent or object in the scene, which is crucial to conceive orunderstand sentences such as ”the box left of the table from your perspective” or”the pub left of the townhall” (assuming implicitly that you stand in front of the

townhall). Set operations include grouping elements into sets, picking elementsout of sets, computing unions or intersections, etc.

Whereas in the early stages of AI these cognitive operations were thought tobe straightforward and yielding a clear outcome that then would result in theset of propositions making up a world model, it is now known that the matteris not that simple.

1. Cognitive operations can be highly complex and therefore they need to beactively invoked, for example it is not computationally feasible to performgeometric perspective transformations for every object in the scene or to cat-egorize all objects in all possible ways, or to compute all possible groupingsof elements into sets. Understanding a natural language sentence requiresan active process of carrying out cognitive operations (rather than matchingsentence meaning to a prior body of propositions) and producing a sentencerequires planning the cognitive operations that the hearer should carry out.

2. Cognitive operations seldom yield clear-cut results. There are usually sev-eral alternative solutions with a degree of fit. This is why approaches such asfuzzy semantics have been developed. Which alternative is ultimately chosenin language communication will depend on many additional factors, in par-ticular their relevance for the overall communicative goals of dialog partnersand what is already available from prior discourse.

3. If the features and categories available for conceptualization are being learned,then the outcome of a cognitive operation will depend heavily on the datathat has been seen so far, and this outcome will keep changing as learningproceeds. This means for example that an object classified in one way earlyin the learning process may become classified in a different way later, andthis implies in turn that conceptualizations of a particular scene may changeas a result of learning. This is another indication that we cannot simplyassume that there are stored canonical representations of scenes in terms ofpropositions based on static concepts.

2.2 Conceptualization is strongly context-dependent

The way reality is conceptualized for language depends partly on the conceptsand compositional combinations that speakers and listeners can handle. But itdepends also very strongly on the context in which the communication takesplace. This is for example very obvious in spatial language. If we conceptualizehow the spatial location of an object is to be communicated, we need to take intoaccount the position of speaker and listener with respect to various landmarksthat we might use. Some of the landmarks (or the object itself) may not be visibleto the listener, and a particular spatial relation may be different depending onthe point of view (e.g. ”left of” can be reversed if the listener is located oppositeof the speaker). This context-dependence makes it very difficult to present in acanonical way the facts about a particular scene.

2.3 Human Languages use Inferential Coding

Human language expressions often leave out aspects of meaning which are infact critical for proper understanding (such as the perspective in the phrase ”thebox left of the table”). This is possible because human languages (in contrast toprogramming languages) are inferential coding systems that assume intelligencefrom the part of the listener as well as a shared context which does not need tobe described [11]. This implies that the listener must be able to fill in large partsof the cognitive operations that the speaker assumes, in addition to the onesthat are explicitly evoked by the utterance itself. The meaning directly conveyedby the utterance is the tip of the iceberg of what is required to fully grasp theintention of the speaker.

Human languages are also unlike programming languages in the sense thatthey do not specify which cognitive operations the listener has to perform butjust provide the arguments for cognitive operations. For example, the speakersimply says ”box” to mean ’categorise the objects in reality in terms of whetherthey are a box and retain those that are members of this class’. Very often thesame concept can be used in many different ways, as in :

1. He slows down.

2. You should take the slow train.

3. This train is slow.

4. The slower train nevertheless arrived earlier.

5. Take the slow one.

All these expressions use the same concept ’slow’, but they implicitly invokedifferent cognitive operations. For example, in case 4, trains have to be orderedin terms of the time they take to reach their destination. In case 1 slow is usedto describe an action in terms of change in the speed with which the action isgoing on. The action itself is not classified. In case 3, the speed of the train iscompared to the speed that is normally expected.

These examples show clearly that a lot of the features and categorisationsneeded to understand an utterance are indirectly invoked by the speaker. Thelistener must not only reconstruct the meaning explicitly conveyed in the utter-ance but also fill in all the contextual details that are required to interpret thismeaning in the present context.

2.4 Conceptualization is language and culture specific

Research into cognitive linguistics of the past decade has unmistakably shownthat there are significant differences in the way that different languages andcultures conceptualize reality (see e.g. [17]). Some languages (and some speakersof a certain language) will prefer one way over another and these preferencesare engrained in the language and thus culturally transmitted. Those claimingthat there is a canonical way usually use their own language as the measureof other cultures and languages, and cannot imagine that the world can be

conceptualized in different ways. But the - by now abundant - data of culturalinfluence of language on conceptualization cannot be ignored.

Let’s take a domain which at first sight is a good candidate for being uni-versally shared, namely color. Names for the basic hues (like red, green, blue,etc.) have been most intensely studied and although there was for some timea consensus that color terms are based on universal color foci, it is now clearthat there are not only important individual differences in color naming be-tween individuals of the same language group, but also significant differences incolor boundaries between different languages [10], p. 442. These cultural differ-ences become even more important when non-basic colors are considered or morecomplex color expressions (such as ’a whiter shade of pale’). Profound culturaldifferences show up in all of the other domains whose semantics has recentlybeen studied, such as for example the domain of space. [2]

Given that concepts are learned and that there are so many alternative ap-proaches possible, it would be very odd if every individual shared the samecanonical way for internally representing reality. In fact, it could only be ex-plained by assuming that there is a strong innate bias to categorize the world.But this raises the issue where these strong biases come from, particularly for themany concepts that have only become relevant quite recently and are withoutmuch effort picked up by the majority of the population. A much more plausibleexplanation is that concepts are culturally evolving and become shared by themembers of a population through tacit agreement as a side effect of commu-nication. So our challenge as AI researchers is to work out how this might bepossible.

3 An Evolutionary Approach

AI has drawn and contributed a lot to many disciplines such as decision theory,logic, linguistics, psychology and of course computer science. However, it hasbeen less influenced by biology, and particularly by evolutionary theory. Forbiologists all aspects of living systems are considered to undergo constant change,either on a small time-scale in the form of adaptation and development, or ona longer time scale in terms of evolution. The main point of this paper is thatwe need to take a similar point of view with respect to intelligent processing ingeneral and language in particular, and that we should be borrowing a lot moreconcepts from biology for achieving artificial intelligence.

Concretely, this means in the present context that we should view languageas a complex adaptive system that undergoes cultural evolution at all levels[14]. Language users not only invent and align their sound systems, modes ofinteraction, and lexical and grammatical expressions but also the way they con-ceptualize reality. The linguistic system is not static because language usersinvent new conceptualizations and new modes of expression and reuse them infuture communications. Which of these survive in future disource is based on thecommunicative needs and environments that language users encounter and theneed to dampen cognitive effort as much as possible, in other words the selec-

tionist forces are cultural rather than biological (survival). Importantly, culturalevolution not only takes place at the level of surface forms (e.g. the use of specificwords) but also at the level of semantics. The remainder of this section brieflysketches some of the directions that we have taken to explore this point of viewfurther.

3.1 Language Games

The observations in the previous sections have shown that we cannot investigatesemantics without context and without the cooperative goals that motivate com-munication. We therefore structure the investigation in terms of language games.A language game is a turn-taking interaction between at least two autonomousagents (human-robot or robot-robot) drawn from a population. Each agent caneither be speaker or hearer which implies that they are capable of both under-standing and producing language. There is a common shared goal, for exampleexecute an action or pay attention to an object in the scene. The speaker concep-tualizes reality for achieving this goal within the present context, transforms theconceptualization into an utterance, and the hearer has to parse the utteranceand reconstruct the meaning. Non-verbal communication, for example pointinggestures, form an integral part of a language game and are often needed as acomplementary source of information particularly in language learning.

In our experiments, agents either started with scaffolded partial inventoriesfor concepts and language, inspired by human languages, and then play languagegames and expand and adjust these inventories in order to be successful in thegame. Occasionally we start from experiments where no initial inventories aregiven and agents invent concepts and expressions and coordinate them in thepopulation, clearly showing that cultural evolution can give rise to an emergentcommunication system that is adaptive. There is no need to have innate conceptsto make language possible. An example of a result is shown in figure 2 (from[1]) which shows an experiment in the emergence of a color lexicon and colorcategories without initial scaffolding.

3.2 Internal Representations

To handle the meaning that is expressed by utterances, we have designed in ourgroup a system called IRL (Incremental Recruitment Language) [14], [15], [7]which comes with an (open) library of cognitive operations that can be linked innetworks as shown in figure 3. The networks operate using a data flow principleas in constraint languages, i.e. each cognitive operation can be used in multi-ple directions and as soon as enough information is available, the operation isexecuted and results propagate in the network.

Each cognitive operation uses an inventory of conceptual building blocks(semantic entities) that can be used to perform the cognitive operation on thecontextual input. For example, a classifier will need an inventory of possibleclasses. This inventory is dynamic as it is expanded by learning processes tocope with more data or more communicative situations.

0

0.2

0.4

0.6

0.8

1

0 200 400 600 800 1000 1200 1400 1600 1800 2000 0

10

20

30

40

50

60

com

mun

icat

ive

succ

ess

lexi

con

size

/ont

olog

y si

ze/in

terp

reta

tion

varia

nce

number of interactions/agentcommunicative success

lexicon sizeontology size

interpretation variance

fidate

bamova

bamoru

fidate

bamova

bamoru

fidate

bamova

bamoru

fidate

bamova

bamoru

fidate

bamova

bamoru

vamasi

riveke

bamoru

fidate

kenafo

bamova

vamasi

riveke

bamoru

fidate

kenafo

bamova

vamasi

riveke

bamoru

fidate

kenafo

bamova

vamasi

riveke

bamoru

fidate

kenafo

bamova

vamasi

riveke

bamoru

fidate

kenafo

bamova

Fig. 2. Example experiments where a population of 5 agents bootstraps a color lexiconfrom scratch. The graph on the left shows how communicative success (meaning successin the language game) moves up to 95 %. Lexicon and ontology size (i.e. inventory ofcolor concepts) grows steadily to be adequate for the game. The interpretation variancesteadily decreases. On the right are two stages in the emergence of the color categories.The top is at the initial stage with less concepts and less coherence.

Such IRL-networks are used as the meaning of utterances and lexical andgrammatical processes translate the networks into surface forms or reconstructthe meaning through a parsing process. Often the networks are incomplete, inwhich case the planning system that is used to come up with a network inlanguage production is re-invoked by the hearer to fill in missing parts.

3.3 Diagnostics and Repairs

We have found that a meta-level architecture is very useful to organize theway in which agents cope with robustness, flexibility, continuous adaptationand evolution (see Figure 4). This architecture uses diagnostics to monitor theoutcome of ongoing object level processes and repairs to expand the inventoryof semantic entities used by cognitive operations or to re-organize networks tobetter cope with the needs of communication.

For example, one diagnostic associated with a classifier would notice thatthe classifier is unable to distinguish two objects in the scene and this couldpossibly trigger a repair action in which the inventory of classifiers is expandedwith a new classifier. Another diagnostic for classifiers would trigger when thelistener observes that an object classified in one way by the speaker is classifiedin another way by himself, and this then could trigger a repair action in the formof an adjustment of the inventory.

(select-entity ?thing-2 ?set-5 ?selector-3)

(bind selector ?selector-3 unique)(filter-by-color ?set-5 ?set-4 ?category-10)

(bind color-category ?category-10 yellow)(filter-set-class ?set-4 ?set-3 ?class-7)

(bind object-class ?class-7 block)(filter-by-spatial-category ?set-3 ?set-2 ?category-2)

(bind angular-spatial-category ?category-2 right)(geometric-transform ?set-2 ?context-54 ?robot-4)

(identify-discourse-participant ?robot-4 ?context-54 ?role-2)

(bind discourse-role ?role-2 hearer)(get-context ?context-54)

Fig. 3. Example of an IRL network. Select-entity, filter-by-spatial-category, etc. arenames of cognitive operations. The variables (indicated with a question-mark, such as?set-5 or ?selector-3) are slots that are filled in a constraint propagation process.

The planning of networks is a search process like any planning process, andstandard techniques such as chunking and storing of obtained solutions for futurereferences are used to avoid search and progressively bootstrap the system tohandle more complex conceptualizations. This progressively yields an inventoryof standardized conceptualization strategies which have also standard forms ofexpression in language and therefore become culturally shared and transmitted.

3.4 Alignment

Research on natural dialog carried out by psychologists such as Simon Garrod [4]has abundantly shown that partners in dialog align themselves at all levels, notonly phonetic, lexical and grammatical but also at the conceptual level. Theyadjust their concept inventories and conceptualization strategies on the basis ofthe outcome of a communication. For example, partners in dialog may adjustthe prototypes of some of their color concepts so that they become more similarand hence that they have a higher chance for mutual understanding (see figure5 from [1]). Alignment can easily be operationalized as part of the meta-levelarchitecture discussed in the previous subsection, and when the right alignmentoperations are used, it operates very effectively.

4 Conclusions

Handling grounded deep semantics for human-like language understanding andproduction requires a reconsideration of some of the foundations of AI, partic-ularly with respect to the logical approach which has informed much of past

Fig. 4. The routine application of constructions during parsing and production is aug-mented with meta-level processes performing diagnosis and possibly repairing problemsby extending or aligning the inventory of the speaker or the hearer.

Fig. 5. Example experiment in the emergence of color categories through languagegames within a population of 10 agents. The left shows the prototypes of all agentswith alignment and the right without alignment. Without alignment, the categoriesare scattered more or less randomly over the color space. With alignment, the colorcategories cluster around certain regions of the color space.

research. This paper emphasized that we need to view language and meaningas a complex adaptive system that is continuously undergoing change, as it isshaped and reshaped by language users in order to satisy their needs within theecological settings they are confronted with. This means that we need to takean evolutionary perspective on semantics instead of assuming that the buildingblocks of meanings and their usage in language communication is static and apriori shared.

References

1. Bleys, J. (2011) Language Strategies for Color. In: Steels, L. (ed) (2011) Experimentsin Cultural Language Evolution. John Benjamins Pub. Co., Amsterdam.

2. Evans, N., and S. Levinson (2009). The myth of language universals: Languagediversity and its importance for cognitive science. Behavioral and Brain Sciences,32(5), 429-492.

3. Frank, S. and R. Bod (2011) Insensitivity of the human sentence-processing systemto hierarchical structure Psychological Science. To appear.

4. Garrod, S. and Anderson, A. (1987) Saying What You Mean in Dialogue: A Studyin Conceptual and Semantic Coordination Cognition, 27, 181–218.

5. Klatt, D. (1990) Review of the ARPA speech understanding project In: Readingsin Speech Recognition. Morgan Kaufmann Publishers Inc. San Francisco, CA.

6. Kowalski, R. (2011) Artificial Intelligence and Human Thinking. IJCAI-2011 Com-puters and Thought Lecture. http://www.doc.ic.ac.uk/ rak/papers/IJCAI2011.pdf

7. M. Spranger, S. Pauw, Loetzsch, M. and L. Steels (2011) A Formalism for EmbodiedCognitive Semantics. In: Hild, M. and L. Steels (ed) (2011) A whole-systems approachto language grounding in robots. Springer-Verlag, Berlin.

8. McCarthy, J. (1959). Programs with Common Sense. In: Mechanisa- tion of ThoughtProcesses, Proceedings of the Symposium of the National Physics Laboratory, pages77-84. London, U.K.

9. Nilsson, N. (1984) Shakey The Robot, Technical Note 323. AI Center, SRI Interna-tional.

10. Regier, T. and P. Kay (2009) Language, thought, and color: Whorf was half right.Trends in Cognitive Sciences, 13, 439-446.

11. Sperber, D. and D. Wilson (1986) Relevance. Blackwell, Oxford.12. Spranger, M. (2011) The co-evolution of basic spatial terms and categories. In:

Steels, L. (ed) (2011) Experiments in Cultural Language Evolution. John BenjaminsPub. Co., Amsterdam.

13. Steels, L. (2000). The emergence of grammar in communicating autonomousrobotic agents. In W. Horn, editor, ECAI 2000: Proceedings of the 14th EuropeanConference on Artificial Life, pages 764-769. IOS Publishing, 2000

14. Steels, L. (2000) Language as a Complex Adaptive System. In Schoenauer, M.,(ed.) Proceedings of PPSN VI, LNCS, 1917. Springer-Verlag, Berlin. pp. 17-26.

15. Steels, L. The Recruitment Theory of Language Origins. In C. Lyon, C. Nehaniv,and A. Cangelosi, editors, The Emergence of Communication and Language, pages129-151. Springer, 2007.

16. Steels, L. and R. Van Trijp (2011) How to Make Construction Grammars Fluid andRobust. In: Steels, L. (ed) Design Patterns in Fluid Construction Grammar. JohnBenjamins Pub. Co.

17. Talmy, L. (2000) Toward a Cognitive Semantics: Concept Structuring Systems(Language, Speech, and Communication) The MIT Press, Cambridge Ma.

18. Wahlster, W. (2000) Verbmobil: Foundations of Speech-to-Speech Translation.Springer-Verlag. Berlin.

19. Winograd, T. (1972) Understanding Natural Language. Academic Press, London.