Natural Language Processing for

Human-Computer Interaction

Hae-Chang Rim

Korea University

2

Contents

• Introduction

• Conversational Natural Language Interface

• Language Understanding Components

• Dialog Management Models

• Conclusion

Introduction

1. What is Natural Language Processing (NLP)?

2. Two motivations for NLP

3. Research fields of NLP

4. NLP and HCI

4

Introduction

• What is Natural Language Processing (NLP)? – This is a difficult question to answer since “there are almost as many de

finitions as there are researchers studying it” (Obermeier, 1988)

The branch of information science that deals with natural language information

The formulation and investigation of computationally effective mechanisms for communication through natural language

A subfield of artificial intelligence and linguistics for making computers "understand" statements written in human languages

• Two motivations for NLP (Allen, 1994)– The scientific or linguistic motivation is to understand the nature of lan

guage through the tools provided by computer science – The technological motivation is to improve communication between hu

mans and machines

5

Introduction

• Research fields of NLP

Morphological Analysis Syntactic Analysis Semantic Analysis

Sentence Level

Discourse Structure Analysis

Speech Act Recognition Reference Resolution

Discourse Level

Dialog Management Planning & Reasoning Language Generation

Dialog Level

Information Retrieval Question Answering Conversational Agent

Application Level

And many other things …

6

Introduction

• NLP and Human Computer Interaction (HCI)– Goals of HCI (Bill 98)

• Developing systems which match or augment the physical, perceptual, and cognitive capabilities of users

• Investigating a way to ensure the user-friendliness and robustness of interactive computer systems

– NLP for HCI • Since natural language is the most effortless and effective way of

communication in human-human interaction – either spoken or typewritten, it may effectively complement other available modalities

• Sometimes, natural language may even be the only applicable modality:

– When driving car, carrying a baggage …

• NLP offers mechanisms for incorporating natural language knowledge and modalities into user interfaces (Bill 98)

Conversational Natural Language Interface

1. What is conversational NL interface system?

2. Limitation of current NLP techniques

3. Typical architecture of dialog system

8

Conversational NL interface

• What is conversational natural language interface system (i.e. Dialog system)?– Systems providing an interface that permits interaction through

natural language between the user and a computer-based application

Hi, I’d like to fly to Seattle Tuesday morning.

Ok. Let’s see, I have a United flights ..

That’s OK

Will you return to Pittsburgh from …

9

Conversational NL interface

• Some limitations of current NLP techniques – Full natural language understanding by machine may not be

realized in the near future• Difficulty of resolving ambiguity of natural language

• Lack of resources …

• Practical Dialog Hypothesis (Allen et al. 01)– Because of the limitation, current dialog systems have usually

been developed in a specific domain and for a specific task under the practical dialog hypothesis

– Hypothesis:• “The conversational competence required for practical dialogues,

while still complex, is significantly simpler to achieve than general human conversational competence”

10

Conversational NL interface

• Typical Architecture of Dialog System

Language UnderstandingComponents

Language UnderstandingComponents

Dialog ManagementComponent

Dialog ManagementComponent

Natural Language

Generation

Natural Language

Generation

QA AgentsQA Agents

Task AgentsTask Agents

Other AgentsOther Agents

SpeechRecognition

SpeechRecognition

11

Conversational NL interface

• Two important issues related to NLP in building a dialog system

How to understand user’s utterance?

How to manage dialog between user and system?

Language UnderstandingComponents

Language UnderstandingComponents

Dialog ManagementComponent

Dialog ManagementComponent

Language Understanding Components

1. Overview of language understanding process

2. Morphological analysis

3. Part of speech (POS) tagging

3. Syntactic parsing

4. Semantic analysis

5. Discourse analysis

13

Language Understanding Components

• The aim of language understanding components– Analyze user’s utterance with discourse context and transform it

to semantic structure which the machine can understand

– Korean language understanding process

POS Tagging

Syntactic Analysis

Semantic Analysis

Discourse Analysis

Disambiguate morphological ambiguities

Finding a syntactic structure of an input sentence

Finding a semantic structure without using discourse context

Resolving remained ambiguity of semantic analysis with discourse context information

Morphological AnalysisFinding all possible morphological structure of a word (or Eojeol)

14

Morphological Analysis

• Problem domain– Finding out

• Potential parts-of-speech for a given word (for English)

• Or morphological parses for a given Eojeol (for Korean)

– Morphological analyzer should produce all the grammatically possible interpretations for a given word (or Eojeol)

– Example of morphological analysis in Korean• When a sentence “ 나는 학교에 간다 (na-neun hag-gyo-e gan da)” is g

iven, the result of morphological analysis is:

15

Morphological Analysis

• Difficulties of Korean morphological analysis– Korean is

• A highly agglutinative languages – An Eojeol is composed of one or more combined morphemes

• Very productive– The number of Eojeols appeared in real texts is almost infinite

• A morphologically complex language– Korean words (or Eojeols) are formed through compounding and deriv

ation

– Also morphological changes are frequently observed » “ 날 /Nal/verb”+” 는 /Neun/connective_ending” “ 나는 /NaNeun”

• Hard to find the boundary of an unknown word– In English, words (spacing units) which are not found in a dictionary a

re unknown words

– In Korean, only subparts of them or themselves are unknown morphemes

16

POS Tagging

• Part of Speech (POS) tagging is – A task to assign a proper POS tag to each linguistic unit such as

word (in English), or morpheme (in Korean) for a given sentence

• An input of POS tagger is a result of morphological analysis, and an output is a correct sequence of morpheme-POS pairs

– Hidden Markov Model (HMM) based POS Tagging• Most popular and well-performed approach

– Regard POS tags of morphemes in a given sentence as hidden states and find the most probable path in a lattice

17

Syntactic Parsing

• Goal – Find out a syntactic structure with a specific grammar for a give

n sentence• Example of parsed sentence with the phrasal structure grammar

– “ 누나는 예쁜 꽃을 좋아한다 . (Nu-Na-Neun ye-Ppeun Kkoch-eul Coh-a-Han-Ta.)”

Nu-NaNC

NeunJX

Ye-PpeuPA

nEM

KkochNC

eulJC

Coh-a-Han-TaPV+EF

NP ADJP NP

NP

VP

VP

.SS.

S

18

Syntactic Parsing

• Parsing can be defined as – A problem that maps any input sentence to an appropriate

syntactic tree structure (Chung, 04)

– Why is the parsing so difficult? • Because of the structural ambiguity of natural language!

• Several characteristics of Korean make the parsing more difficult– Relatively free-word order, constituent ellipsis …

보았다어제 유진이 쇼를 보았다

NP

VP

S

어제 유진이 쇼를 보았다

VP

VP

S

19

Syntactic Parsing

• Examples of statistical parsing with simple PCFG model– “Astronomers saw stars with ears”

S NP VP 1.0

VP V NP 0.7

VP VP PP 0.3

PP P NP 1.0

P with 1.0

V saw 1.0

NP NP PP 0.4

NP astronomers 0.1

NP ears 0.18

NP saw 0.04

NP stars 0.18

NP telescope 0.1

TreeBank

P(t2) = 0.0006804

P(t1) = 0.0009072

t1 t2>

20

Semantic Analysis

• Semantic analysis is – The process whereby meaning representations are composed and

assigned to a user’s utterance

How can I go to KoreaUniversity? What does

it mean?

21

Semantic Analysis

• Example of semantic analysis

22

Semantic Analysis

• Shallow semantic analysis for a dialog system– Under the practical dialog hypothesis, we can simplify the

semantic analyzing process:• Restricting domain of a dialog system reduce the ambiguity

– In the pay-bill domain, the word `bank’ may not be used as the meaning of a dike

• Also, if we restrict a task of a dialog system, simple methods such as concept-spotting can be enough to capture user’s intention

Question Focus: ProgramChannel: MBCBegin_time: 18:00

“6 시에 MBC 에서 뭐 하니 ?”

Analyzed by a concept spotting method

23

Discourse Analysis

• Reference resolution– The omitted words (or phrases) and the pronominal references

are complemented by the use of common sense and discourse information

• Speech Act Identification – Speech Act: The communicative intention represented by each

utterance

– A dialog system should have the ability to • identify other participants’ speech act, predict next possible speech

acts, and generate own utterance suitable for the speech act

Statement non opinion: I'm a customer since November.

Statement opinion: I think it's great.

U: I would like to open a fixed deposit account.S: For what amount?U: Make it for 8000 dollars.

How to manage dialog between user and system?

1. Overview of dialog management

2. FST based Approach

3. Frame based Approach

4. Other Approaches

25

Dialog Management

• Dialog management model (or component) – Controlling the flow of the dialog between the system and the

user, including the coordination of other components of the system

• Dialog management model must solve two problems: – Keep track of the overall interaction with steady progress

towards task completion • The system must have some idea of the task completion ratio

• More importantly, the system must have some idea of what is yet to be done,

– Robustly handle deviations from the nominal progression towards problem solution

26

Dialog Management

• One of core issues of dialog management– System-initiative:

• system always has control, user only responds to system questions

– User-initiative: • user always has control, system passively answers user questions

– Mixed-initiative: • control switches between system and user

• Classification of dialog management strategies (Allen et al. 01)– Finite state (or graph)-based strategy– Frame-based strategy– Plan-based strategy – Agent-based strategy

27

Dialog Management

• Finite-state based dialog control– Simplest dialog control method

– Usually, system-initiative

– Dialogue consists of a sequence of predetermined steps or states• The dialog flow is specified as a set of dialogue states with

transitions denoting various alternative paths through the dialog graph

• Most commercially available spoken dialog system use this form of dialogue management strategy

– Example task: Long distance dialing by voice, Tele-banking system

– Does not require sophisticated NLP techniques, but works only for simple tasks

28

Dialog Management

Example of finite-state based dialog management: “Pay a bill”

29

Dialog Management

• Example illustrating some limitations of finite-state based dialog system

The over-informative answer cannot be accepted

I already answered for that question!

30

Dialog Management

• Frame-based dialog management– More flexible approach

– Mixed Initiative using fixed rules

– Dialog management problem is regarded as form filing : • The form specifies all relevant information (slots) for an action

– Dialog management consist of • Monitoring the form for completion

• Extract relevant elements from user utterance

• Asking question to user using empty slots as a trigger

31

Dialog Management

• Example of Frame-based Dialogue Control

Frame:Send a message

Importance of Slots

Filling a slot by a user response

Triggering a system response by an empty slot

32

Dialog Management

• More complex dialog management approaches– Plan (Task) Based Model: The dialogue involves interactively

constructing a plan (e.g. kitchen design consultant).

– Agent Based Model: Involves planning and also executing and monitoring operations in a dynamically changing world (e.g. emergency rescue coordination).

– Generally require deep semantic analysis for user utterances, rich knowledge resources, and elaborate inference/reasoning methods

33

Dialog Management

• Summary of dialog management approaches

Features/Dialog control strategies

Finite state-based Frame-based Plan/Agent-based

Input Single words or phrases Natural language with concept spotting

Unrestricted natural language

Verification Explicit confirmation Explicit and implicit confirmation

Grounding

Dialogue model Information state represented implicitly in dialog states

Dialog control represented explicitly with state diagram

Explicit representation of information states

Dialog control represented with control algorithm

Model of system’s intention, goals, and beliefs

Dialog history, context

Dialogue phenomena

User answers question User asks question, simple clarifications by system

Dynamically generated topic structure, collaborative negotiation subdialgues

Different modalities (e.g., planned and actual world

Example Task Long-distance dialing Getting train arrival and departure information

Kitchen design consultant,

Disaster relief management

34

Conclusion

• Conversational NL interface are showing promise as a new modality for HCI, because – Natural language is most familiar way of communication in

human-human interaction

– It also can provide “effortless and effective” way of communication in a human-computer interaction

• However, there are still serious obstacles to be overcome– Improving performances of NLP analysis components such as

POS tagging, parsing, so on

– Ensure domain portability of a dialog interface-based system

– …