Understanding Sentences - Informatics 1 CG: Lecture 14 › teaching › courses › inf1-cg ›...

Understanding SentencesInformatics 1 CG: Lecture 14

Mirella Lapata

School of InformaticsUniversity of Edinburghmlap@inf.ed.ac.uk

February 11, 2016

Informatics 1 Cognitive Science Understanding Sentences 1

Reading:

Trevor Harley (2001). The Psychology of Language,Chapter 9

Recap: The Associationist View of Word Meaning

You shall know a word by the company it keeps (Firth, 1957).

Distributional hypothesis about word meaning: the contextsurrounding a given word provides information about itsmeaning.

Experimental evidence indicates that the cognitive system issensitive to distributional information.

Construction of vector spaces.

Latent Semantic Analysis.

What happens after we recognize a word? How do we puttogether words to form meaningful sentences?

Recap: The Associationist View of Word Meaning

You shall know a word by the company it keeps (Firth, 1957).

Distributional hypothesis about word meaning: the contextsurrounding a given word provides information about itsmeaning.

Experimental evidence indicates that the cognitive system issensitive to distributional information.

Construction of vector spaces.

Latent Semantic Analysis.

What happens after we recognize a word? How do we puttogether words to form meaningful sentences?

Sound to Meaning

Acoustic Signal

Word Segmentation

Lexical Access

Syntactic Parsing

Semantic Interpretation

Meaning

Propagation across levels

Parsing

How do we understand a sentence? First step is to “parse” it.

Parsing takes place unconsciously (involves finding thesubjects, verbs, objects and so on).

A parser is the mental program that analyzes sentencestructure during language comprehension.

Ultimately parsing helps us to interpret sentences duringlanguage comprehension.

1 The ghost chased the vampire.

2 The vampire chased the ghost.

3 The vampire was chased by the ghost.

Parsing

The Big Picture

⇒ look at theyellow dog

yellow

Today we will look at parsing, which turns a sequence of wordsinto a syntactic representation.

Syntactic representations make explicit how the words in asentence relate to each other.

Phrase Structure Grammar

In order to build syntactic representations, we need a grammar.The simplest type of grammar is a context-free grammar:

Phrasal categories:S: sentence, NP: noun phrase, VP: verb phrase

Lexical categories (aka parts of speech):Det: determiner, N: noun, V: verb

Phrase structure rules:

S → NP VP Det → theNP → Det N N → kittenVP → V NP N → dog

V → bit

Derivations and Syntax Trees

A derivation is the sequence of strings that results from applying asequence of grammar rules:

Derivation

S → NP VP → NP V NP → NP V Det N → NP bit Det N → NPbit Det dog → NP bit the dog → Det N bit the dog → the N bitthe dog → the kitten bit the dog

Derivation

Derivations are represented as syntax trees:

kitten

Syntactic ambiguity: a sentence can have multiple syntax trees.These correspond to different interpretations.

Human Sentence Processing

A parser takes a sentence and computes a syntax tree for it, givena grammar. This is a prerequisite for assigning an interpretation tothe sentence.

The cognitive device that performs syntactic parsing is calledhuman sentence processing mechanism (HSPM).

Parsing is incremental: the HSPM builds structures word by wordas the input arrives.

But what if more than one structure is compatible with the input:

at the current point but not later: local ambiguity;

for the input overall: global ambiguity.

Global Ambiguity

Given a grammar, strings that have more than one completesyntax tree (parse) are said to have global structural ambiguity.

Examples

1 She sat on the chair covered in dust.

2 I put the book on the table in the kitchen.

3 Lung cancer in women mushrooms.

Examples from http://www.fun-with-words.com/ambiguous_garden_path.html

Global Ambiguity

cancer

mushrooms

cancer

mushrooms

Global Ambiguity

cancer

mushrooms

cancer

mushrooms

Global Ambiguity

cancer

mushrooms

cancer

mushrooms

Local Ambiguity

When only an initial substring is structurally ambiguous, thesentence is said to have local ambiguity; once the remainder of thestring is known, only one tree remains possible.

Example

The athlete realized his potential . . .a. . . . at the competition.b. . . . could make him a world-class sprinter

Local Ambiguity: Structure 1 (VP → V NP)

athlete

realized

potential

The athlete realized his potential at the competition.

Local Ambiguity: Structure 1 (VP → V S)

athlete

realized

potential

The athlete realized his potential could make him aworld-class sprinter.

Garden Paths

This is an example of a garden path:

both structures are compatible with the input up untilpotential ; only the next word disambiguates;

however, the processor commits to a single (wrong) structureearly on, and trips up when later input is inconsistent withthat structure;

presumably, the processor now has to compute a newstructure that is consistent with the input;

garden path sentences result in longer reading times, reverseeye-movements, lower comprehension accuracies, etc.;

some garden paths are so strong that the parser fails torecover from them.

Garden Paths

More examples of garden paths

1 I convinced her children are noisy.

2 Until the police arrest the drug dealers control the street.

3 The old man the boat.

4 Fat people eat accumulates .

5 The cotton clothing is usually made of grows in Mississippi.

6 The prime number few.

Garden Paths

1 I convinced her that children are noisy.

Garden Paths

2 Until the police make the arrest , the drug dealers control thestreet.

Garden Paths

3 The old people man the boat.

Garden Paths

4 The Fat that people eat accumulates in their bodies.

Garden Paths

5 The cotton that clothing is usually made of grows inMississippi.

Garden Paths

6 The prime people number few.

Two Theories of Human Parsing

1 What mechanism is used to construct interpretations?2 What information is used to determine preferred

structure?

Garden Path Theory

Parsing is autonomousand takes place in twostages; during stage 1only syntacticinformation is taken intoaccount; stage 2 takesadditional informationsources into account ifsingle analysis turns outto be incorrect.

Constraint-based Theory

Parsing is interactiveand takes place in onestage. Processor usesmultiple sources ofinformation at once,structure mostsupported by constraintsis active, plausiblealternatives also remainactive.

The Garden Path Theory (Frazier, 1987)

telecsope

Which attachment do people initially prefer?

First Strategy: Minimal Attachment

telescope

Adopt structure containing fewest number of nodes

telescope

Second Strategy: Late Closure

The reporter

the plane

crashed

last night

Add incoming material to clause/phrase currently processed.

Second Strategy: Late Closure

The reporter

the plane

crashed

last night

Add incoming material to clause/phrase currently processed.

The Competition Integration Model (McRae et al., 1998)

Diverse constraints (linguistic and conceptual) are brought to bearsimultaneously in ambiguity resolution.

Model assumes all possible analyses are constructed

Constraints provide probabilistic support for analyses

There are multiple processing cycles given each input

On each cycle, evidence in support of the syntacticalternatives is computed.

Competition ends when the activation of one alternativereaches a threshold.

Processing time is assumed to be a linear function of theduration of competition.

The Competition Integration Model (McRae et al., 1998)

The crook arrested by the detective was guilty of taking bribes.was guilty of taking bribes

AgentRating

Other Roles

Past Participle

SimplePast

RRSupport

MCSupport

Patient Rating

AgentRating

P(RR) P(MC)

RRSupport

MCSupport

ReducedRelative

MainClause

Thematic fitof initial NP

Thematic fitof agent NP

Main clause bias

Main verb bias

Verb tense/voice

Parafovealby-bias

Summary

Sentence processing (parsing) is the task of assigning astructure to a string of words;

Human sentence processing is incremental (word by word);

It can encounter global vs local ambiguity.

Garden paths derive from local ambiguities that are hard toresolve; they lead to longer processing times

Theories of sentence processing: serial vs parallel, autonomousvs. interactive.

Understanding Sentences - Informatics 1 CG: Lecture 14 › teaching › courses › inf1-cg ›...

Documents

Neural Summarization by Extracting Sentences and … Summarization by Extracting Sentences and Words Jianpeng Cheng Mirella Lapata ILCC, School of Informatics, University of Edinburgh

AachenBT - Beacon Graphics · CG Poster Bodoni CG TradeLight. CG TradeBold Extended CG TradeBold Condensed CG Triumvirate CG TriumvirateBold. CG TriumvirateBlack CG TriumvirateLight

Sentences and Non-Sentences

NN AAMMEE CCgg −Amulti-platform, multi-API C-based ...developer.download.nvidia.com/cg/Cg_2.0/2.1.0012/Cg-2.1_October... · Cg(Cg) CgTopics Cg(Cg) NN AAMMEE CCgg−Amulti-platform,

1 - PCMS. Photo · CG Al Compás Del Tango. CG El Gran Gato 2. DAVIES, GUY UK CG Newcastle Night CG Shutters & Shadows (2) CG Skeletons In The Mist ... CG Violin & Rose LATHOUWERS,

PCL 5 Printer LanguageTechnical - developers.hp.com · Agfa Division, Miles Inc. CG Bodoni, CG Century Schoolbook, CG Goudy Old Style, CG Melliza, Microstyle, CG Omega, CG Palacio,

Corporate Governance- Corporate Governance: Meaning, Historical Perspective, Theoretical basis of CG, CG Mechanism, CG System, Good CG,Land mark in the emergence of CG: CG Committees,

CHEMICAL COMPATIBILITY GUIDE - Morgan Advanced Materials€¦ · CHEMICAL COMPATIBILITY GUIDE . . Medium Temperature Concentration CG Base CG Resin CG Antimony CG Babbitt CG Copper

GRAMMAR: TOPIC SENTENCES SUPPORTING SENTENCES CONCLUDING SENTENCES

CHEMICAL COMPATIBILITY GUIDE · 2016. 7. 11. · CHEMICAL COMPATIBILITY GUIDE . . Medium Temperature Concentration CG Base CG Resin CG Antimony CG Babbitt CG Copper G Base G Resin

Sentences and Clauses Simple Sentences Compound Sentences Complex Sentences

· EU RAMET cg-14, EURAMET cg-18, PA candidate handbook (PAC02) EU RAMET cg-8, EURAMET cg-13, EURAMET cg-20 PA candidate handbook (PAC03) EURAMET cg-7, EURAMET cg-ll, EURAMETcg-15;

CG-1800 CG-1000 Owner's Manual - TASCAM · 2016-01-24 · 6 TASCAM CG-1800/CG-1000 1 – Introduction Thank you very much for purchasing a TASCAM CG-1800/ CG-1000 Master Clock Generator

CG 24ECP S CG 24ECP CG 27ECP SL CG 24ECP S CG 24ECMP AS …

Introduction to Vocabulary Russell Hamm Informatics Consultant Apelon, Inc. Co-chair HL7 Vocabulary WG Adapted from Ted Klein, CG Chute MD DrPH, Stan M

Grass Trimmer/Brush Cutter CG 22EASSL /CG 22EASS CG ... · Model CG 22EAS(SL)/CG 22EAS(S) CG 22EAD(SL)/CG 22EAB(SL) CG 24EAS(SL)/CG 24EAS(S) CG 27EAS(SL)/CG 27EAS(S) Handling instructions

Cleft Sentences - Cleft Sentences

Biomedical Informatics; CG Chute - … · Biomedical Informatics; CG Chute © Mayo Clinic College of Medicine 201 2 That Higgs Boson 7 •600 institutions •10,000 scientists •800

Sentence Structure Simple Sentences Sentence Fragments Compound Sentences Complex Sentences Compound Complex Sentences

Revising Sentences Language Network Padded and Empty Sentences Refining Padded Sentences Improving Empty Sentences Practice and Apply Revising Sentences