COMPUTATIONAL PSYCHOLINGUISTICS

STUDIES IN THEORETICAL PSYCHOLINGUISTICS

VOLUME 20

Managing Editors

Thomas Roeper, Dept. of Linguistics, University of Massachusetts at Amherst

Kenneth Wexler, Dept. of Brain and Cognitive Science, MIT, Cambridge, Mass.

Lyn Frazier, University of Massachusetts at Amherst

Editorial Board

Robert Berwick, Artifical Intelligence Laboratory, MIT, Cambridge, Mass.

Manfred Bierwisch, Zentralinstitut fUr SprachwissenschaJt, Akademie der Wissenschaften, Berlin

Merrill Garrett, University of Arizona, Tucson

Lila Gleitman, School of Education, University of Pennsylvania

Mary-Louise Kean, University of California at Irvine

Howard Lasnik, University of Connecticut at Storrs

John Marshall, Neuropsychology Unit, Radcliffe Infirmary, O;4ord

Daniel Osherson, MI.T., Cambridge, Mass.

Yukio Otsu, Keio University, Tokyo

Edwin Williams, Princeton University

The titles published in this series are listed at the end of this volume.

MATTHEW W. CROCKER Centre for Cognitive Science,

University of Edinburgh, Scotland, U.K.

COMPUTATIONAL PSYCHOLINGUISTICS

An Interdisciplinary Approach to the Study of Language

KLUWER ACADEMIC PUBLISHERS DORDRECHT / BOSTON / LONDON

Library of Congress Cataloging-in-Publication Data

Crocker, Matthew W. Computational psycholinguistics : an interdisciplinary appoach to

the study of language! by Matthew W. Crocker. p. cm. -- (Studies in theoretical psycholinguistics ; v. 20)

Includes bibliographical references and indexes.

1. Psycholinguistics--Data processing. I. Title. II. Series. P37.5.D37C76 1996 863--dc20

ISBN-13: 978-0-7923-3806-2 001: 10.1007/978-94-009-1600-5

e-ISBN-13: 978-94-009-1600-5

Published by Kluwer Academic Publishers, P.O. Box 17, 3300 AA Dordrecht, The Netherlands.

Kluwer Academic Publishers incorporates the publishing programmes of

D. Reidel, Martinus Nijhoff, Dr W. Junk and MTP Press.

Sold and distributed in the U.S.A. and Canada by Kluwer Academic Publishers,

101 Philip Drive, Norwell, MA 02061, U.S.A.

In all other countries, sold and distributed by Kluwer Academic Publishers Group,

P.o. Box 322, 3300 AH Dordrecht, The Netherlands.

Printed on acid-free paper

All Rights Reserved © 1996 Kluwer Academic Publishers Softcover reprint of the hardcover 1 st edition 1996

95-43997

No part of the material protected by this copyright notice may be reproduced or utilized in any form or by any means, electronic or mechanical,

including photocopying, recording or by any information storage and retrieval system, without written permission from the copyright owner.

TABLE OF CONTENTS

PREFACE ix

CHAPTER I. INTRODUCTION 1 1.1 The Study of Language 6 1.2 The Competence-Performance Distinction 8 1.3 The Universal Parser 11 1.4 A Programme of Research 12 1.5 Organisation of Chapters 15

CHAPTER II. PERSPECTIVES ON SENTENCE PROCESSING 17

11.1 Modularity in Language Processing 19 11.2 The Nature of the Empirical Evidence 24

11.2.1 Ambiguity in Language 25 11.2.2 Relative Complexity and Semantic Interaction 27 11.2.3 Constraint-Based Approaches 30 11.2.4 Evidence from ERPs 32 11.2.5 Summary 33

11.3 Extant Theories of Linguistic Performance 33 11.3.1 Computationally Based Theories 35 11.3.2 Strategy Based Theories 37 11.3.3 Grammar Based Theories 44

11.4 Conclusions 52

vi

CHAPTER III. PRINCIPLES, PARAMETERS AND REPRESENTATIONS 59

111.1 Explanation in Universal Grammar 60 111.2 The Transformational Model 62

111.2.1 X-theory and Lexical Selection 64 111.2.2 Types of Movement 69 111.2.3 Case Theory 72 111.204 Command Relations and Government 75 111.2.5 Bounding Theory 76

111.3 Representations: Types VS. Levels 79 111.3.1 A Representational Model 80 111.3.2 Phrase Structure 84 111.3.3 Chains 89 111.304 Thematic Structure 92 111.3.5 Coindexation 94

IliA Summary and Discussion 94

CHAPTER IV. A PRINCIPLE-BASED THEORY OF PERFORMANCE 103

IV.1 The Foundations of the Processing Model 105 IV.2 The Nature of Processing Complexity 108 IV.3 Modularity in the Syntactic Processor 110

IV.3.1 Incrementality in the Syntactic Processor 112 IV.3.2 The Nature of Modularity 113

IVA The Phrase Structure Module 115 IV.4.1 The Use of Lexical Information in the PS

Module 116 IVA.2 Attachment Preferences in English 117 IVA.3 Processing Head-Final Languages 120 IVAA Summary 128

IV.5 The Thematic Module 129 IV.6 The Chain Module: Recovering Antecedent-Trace

Relations 131 IV.6.1 Processing Chains in a Modular Model 133 IV.6.2 Against the use of Lexical Preference 135 IV.6.3 Processing Gaps in the 2nd Dimension 137 IV.6A Summary 144

IV.7 Summary 145

vii

CHAPTER V. A LOGICAL MODEL OF COMPUTATION 149

V.1 Principle-Based Parsing 152 V.2 A Logical Model of Performance 154

V.2.1 Parsing as Deduction 155 V.2.2 Deductive Parsing: Rules vs. Principles 157 V.2.3 Deduction in a Modular System 164

V.3 Control in the Syntactic Processor 167 V.4 Summary and Discussion 171

CHAPTER VI. THE SPECIFICATION OF MODULES 173 VI.1 The Phrase Structure Module 176

VI. l. 1 Representations and Grammatical Knowledge 176 VI.L2 The Phrase Structure Interpreter 179 VI.1.3 Discussion 182

VI.2 The Chain Module 183 VI.2.1 Representations and Grammatical Knowledge 183 VI.2.2 The Chain Interpreter 186 VI.2.3 Discussion 190

VI.3 The Thematic Module 191 VI.3.1 The Thematic Interpreter 192 VI.3.2 Discussion 196

~.4~mm~ 1~

CHAPTER VII. SUMMARY AND DISCUSSION 201 VIL1 A Summary of the Theory 203

VII.L1 The Modular Syntactic Processor 204 VII.L2 A Concentric Theory of Complexity 206

VII.2 Computational Properties of the Model 210 VII.2.1 The Role of Meta-interpretation 211 VII.2.2 Decomposition and Parallelism 212 VII.2.3 Must Representations be Explicitly

Constructed? 213 VIL3 The Innate Sentence Processor 215

VII.3.1 Acquisition in the Deductive Sentence Processor 217

VII.3.2 Against Parameterisation of the HSPM 218

CHAPTER VIII. CONCLUSIONS 223

Vlll

BIBLIOGRAPHY

INDEX OF AUTHORS

INDEX OF SUBJECTS

227

239

243

PREFACE

This book is about the human facility for sentence perception: how people process the utterances they encounter and recover the syntactic structure necessary for interpretation and comprehension. The principal aim of this book is to detail an account of the human sentence pro­cessing faculty which integrates state of the art theories and methods from linguistics, psychology, and computational linguistics. The core of the proposals made here are based upon an apparent tension in the psycholinguistic community: modularity versus interaction. Evidence both for and against the existence of a distinct, encapsulated language processor abounds.

From the interactionist camp, there are studies which indicate that all sorts of linguistic and non-linguistic information sources are brought to bear rapidly and incrementally as utterances are heard or read. In­deed, there has recently been increasing support for a non-modular view. High-resolution experimental paradigms now seem to indicate that se­mantic and contextual information are brought to bear almost instantly to influence the course of even the earliest stages of sentence processing. The view is further countenanced by the recent trend of connectionist models which, in their naive form at least, emphasise massive distribu­tion and interaction in processing. The claims to brain-like computation which have been espoused have led to deus ex machina in parts of the psycholinguistic community; the view that the interactive behaviour of networks must be directly incorporated into models of language pro­cessing has led to a proliferation of 'constraint-based' and 'competitive activation' models. When combined with the recently renewed initia­tive in statistical language modelling, we see a general shift away from

x

the rationalist approaches towards empiricist methods in the study of language, its acquisition, and its use.

The modularists, however, point to evidence that simple, inform a­tionally restricted strategies still seem to govern initial processing pat­terns, in the way that only a modular architecture would systematically predict. Here, we will argue for a particular, computationally explicit notion of modularity which explains the bulk of data from both sides of the debate. Specifically, we motivate a modular language processor, containing within it an articulated syntactic sub-system. More generally, we propose and outline the paradigm for a coarse-grained, distributed processing architecture for sentence perception; one in which modules emerge to achieve performance benefits where informationally encapsu­lated sub-tasks can be identified. Within this architecture, we argue for an over-arching Principle of Incremental Comprehension (PIC), which ensures that modules function to meet the demands of real-time, in­cremental linguistic perception. It will be shown that several syntactic processing strategies derived from the PIC obtain for a range of cross­linguistic phenomena.

The model proposed is dearly motivated within the rationalist tra­dition, by the properties of current linguistic theory, and proposals con­cerning the modularity of mind. Indeed, the empirical predictions of such a model, given its incremental and concurrent operation, are at first glance difficult to distinguish from those of its non-modular, in­cremental, interactionist counterparts. This is particularly the case in English, a head-initial language, where rich syntactic and semantic prop­erties of the verbal head can be recruited quickly to assist in parsing. The modular account does, however, predict that the initial construc­tion of representations will be performed autonomously, by the relevant encapsulated module, even though subsequent integration by the other systems may quickly force an alternative. In this regard, we will present evidence, particularly from head-final languages such as German and Dutch, that several strategies which appear to be operative during the 'first pass', have an explanation within the proposed modular model. It will be shown that these syntactic processing strategies are derived from the more general Principle of Incremental Comprehension, when interpreted within the context of the proposed architecture.

A secondary aim of this work is to contribute more generally to the emerging interdisciplinary field of Computational Psycho linguistics, and demonstrate the important role of computation in our theories of lin-

Xl

guistic performance, as in the cognitive sciences more generally. The importance of an explicit computational component to our theories of perception goes beyond that of simple 'modelling', although this is in­creasingly important as theories and their predicted behaviour become too complex to validate by hand. Tacit assumptions about the com­putational mechanisms of the brain have implicitly shaped all existing theories of human linguistic performance; such assumptions guide the arguments and intuitions concerning what is 'natural' or 'follows from' various components of a particular theory, and how principles of a the­ory should be applied. It is therefore essential that such assumptions should be made explicit: not only for thorough presentation of the theory which is otherwise often ambiguous, but also so that the computational assumptions are exposed for evaluation in their own right.

This work shares with previous efforts its consideration of the princi­ples operative in the sentence processor, and their algorithmic properties. It differs, however, in that it places particular emphasis on the under­lying architecture of the sentence processor. In particular, specific pro­posals concerning the nature and degree of modularity in the language processor are advanced. It is only once this architecture is established that we can motivate and evaluate the specific principles and algorithms operative within it.

The book should be accessible to those coming from any subset of the constituent disciplines of linguistics, psycholinguistics, and compu­tational linguistics, with the following caveats: A basic knowledge of syntactic theory is assumed throughout, with chapter III presenting the specific theory being assumed for the present work. The first half of chapter III summarises (an instance of) the syntactic theory of govern­ment and binding (GB), and can be omitted by those familiar with GB. The second half, however, presents an important reformulation of GB theory which is central to the performance theory as a whole. Also, while chapter V should be accessible to readers with a basic knowledge of parsing, knowledge of Prolog will also help. Chapter VI, explores the implementation in greater detail, requiring a good knowledge of Prolog, but the chapter is not essential to understanding the core theory and arguments being presented, and can reasonably be skimmed or omitted by those without the relevant background.

On a personal note, it remains for me to extend my gratitude to those who have contributed to this effort in one way or another. First and foremost, I would like to thank Elisabet Engdahl for her constant

XlI

encouragement, guidance, and support. I would also like to thank Ron­nie Cann, Robin Cooper, and especially Eric Wehrli for their insightful comments and suggestions for improvements to this work.

Without doubt, lowe a great debt to various colleagues within the Edinburgh academic community. In particular, special thanks must go to Ian Lewin and Martin Pickering, for many helpful discussions during the course of this research. I would also like to thank Alan Black, Holly Branigan, Chris Brew, Kathrin Cooper, Steve Finch, Suresh Manand­har, David Milward, and Mike Reape - all of whom have contributed valuable comments and discussion during my time in Edinburgh. Ad­ditionally, I must acknowledge all those members of the Department of Artificial Intelligence, the Centre for Cognitive Science, and the Human Communication Research Centre who have contributed to one of the richest environments in which to pursue the interdisciplinary study of language.

I would also like to thank those members of the community world­wide who have at various times provided valuable feedback, and influen­tial discussion: Lyn Frazier, Ted Gibson, Paul Gorrell, Mark Johnson, Paola Merlo, Don Mitchell, Ed Stabler, and Amy Weinberg.

Finally, I would like to acknowledge the financial support which has made this work possible: an Edinburgh University Studentship and Overseas Research Scheme Award, the Human Communication Research Centre, and finally an ESRC Research Fellowship. Last but not least, I owe a special thanks to the generosity and support of my family.