Assessing Language Pragmatics: Who, What and How Peter A. de Villiers Jill G. de Villiers Smith College *supported by NIH grant N01-DC-8-2104 to the University

Assessing Language Pragmatics: Who, What and How

Peter A. de Villiers

Jill G. de Villiers

Smith College

*supported by NIH grant N01-DC-8-2104to the University of Massachusetts and Smith College

*webpage: www.umass.edu/aae

Pragmatics

• The functional use of language between speaker(s) and listener(s) for effective communication.

• Syntax and semantics are always intertwined with their functional use in communication so cannot be totally separated from pragmatics skills in language acquisition.

• Looking at the development or impairment of “communicative competence” (Hymes, 1972).

• We will focus on pragmatic language (not gesture or other non-verbal communication) between the ages of 3 and 9.

Who needs to be assessed?

• Some children show differential impairment of the functional and social-interactional components of language– Children with autistic spectrum disorders– Children with Asperger syndrome– Children with specific Pragmatic Language Impairment

(Bishop, 1998)

• But most children with substantial language impairment will also show pragmatic problems– Especially if they have both receptive and expressive

language impairment (Craig & Evans, 1993)

What to assess?

Five Components of Pragmatics

1. Conversational skills in social interaction

2. Speech Acts -- doing things with words and utterances

3. Reference and Presuppositionstaking into account the perspective or knowledge state of the listener

4. Extended discourse genres:e.g., Narrative, Exposition

5. Styles or “registers”of speaking

1. Conversational Skills

• Turn taking– Sensitivity to non-linguistic,

paralinguistic, and linguistic cues• Topics

– Initiation– Maintenance– Elaboration– Ending– Change

Grice’s Conversational Maxims - Cooperative Principles

• Sincerity– Assumption that speaker means what she says

• Relevance– Keeping on topic

• Quantity– Providing enough but not too much redundant

information for the listener

2. Speech Acts

• Doing things with words and utterances– Informing or reporting– Requesting information– Requesting action (indirect questions)– Commanding or demanding– Denying statements– Rejecting or prohibiting actions– Greetings

3. Referential Specificity, Continuity, and Presuppositions

• Assessing what your listener knows or needs to know– Shared information from the referential context

or world knowledge– Shared information already introduced into the

discourse

• Coherence and Cohesion in discourse– Linking utterances together thematically,

temporally and referentially

Conversational Breakdowns and Repairs

• Detecting communication breakdown and its source

• Contingent queries and requests for clarification

• Spontaneous and prompted repairs– Repetition– Further specification– Elaboration– Confirmation

4. Extended Discourse Genres - Narratives

• Coherence in narratives– Plot and episode structure– Story grammars

• Cohesion in narratives– Reference

• Introducing, maintaining, and specifying characters

– Linking events• Causal links• Temporal links

• Mental states and the “Landscape of Consciousness” (Bruner, 1986)

5. Speaking with Style - Speech Registers

• Tailoring one’s speech to the audience and social situation e.g. politeness, baby talk etc.– Age of listener– Social status of listener– Formality of the situation– Cultural expectations

How to Assess

Some existing procedures

How to assess1. Conversational Skills

a) Language sampling– Closer to authentic communication– But needs to be structured to get some features– Time consuming and labor intensive– Often lack measures of reliability, validity, and norms

b) Conversational Analysis– (Craig & Evans, 1993)

c) Pragmatic Profiles – (Prutting & Kirchner, 1987).

d) Checklists by teachers, clinicians, or parents– Children’s Communication Checklist (Bishop, 1998).

b) Conversational Analysis

• Craig & Evans (1993)– Analyzed children’s adjacent responses to adult

utterances as contingent vs noncontingent– Cohesion ties with prior adult utterance as complete vs

incomplete.– Complete ties of various sorts (e.g., lexical, referential

(e.g. pronouns), ellipsis, connectives)

• SLI children with receptive as well as expressive problems were:– Lacking in strategies for conversational interruption and

access– Made fewer connective ties to previous adult utterance– Made more incomplete ties to adult’s utterances

c) Pragmatic Profile (Prutting & Kirchner, 1987)

• Age 5 and up.• Based on 15 minute spontaneous, unstructured conversation sample

with a familiar partner.• 30 pragmatic features rated as appropriate (facilitate or neutral with

respect to communication) or inappropriate (get in the way of communicative exchange) [or no opportunity to observe]

• 18 Verbal aspects of communication:– Speech acts (pairs and variety)– Topic (selection, introduction, maintenance, change)– Turn taking (initiation, response, repair, adjacency, contingency,

quantity)– Lexical selection (specificity/accuracy, cohesion)– Stylistic variation

• 5 Paralinguistic aspects (intelligibility and prosodics)• 7 Nonverbal aspects (e.g, physical proximity, gestures, eyegaze)

Pragmatic Profile (Prutting & Kirchner, 1987)

• Compared 42 typically developing with 42 language disordered and 42 articulation disordered children between the ages of 7 and 10.

• Language disordered children were most likely to be flagged as inappropriate in turn-taking repair, turn-taking quantity, specificity/accuracy of lexical selection, and cohesion ties to the prior utterance.

• Articulation disordered children did not show these problems but were more likely to be flagged for paralinguistic aspects.

d)Children’s Conversational Checklist (Bishop,1998)

• 70 items scored as “does not apply” (0), “applies somewhat” (1), and “definitely applies” (2).

• Some items scored as positive (+), most as negative (-). • Items fall into 9 categories:

– A. Speech intelligibility and fluency– B. Syntax– C. Inappropriate initiation– D. Coherence– E. Stereotyped conversation– F. Use of conversational context– G. Conversational rapport– H. Social relationships– I. Interests

• So 5 categories involve Pragmatic Language (38 items)

Children’s Conversational Checklist

Some example items:• C. Inappropriate initiation

• Talks too much

• Keeps telling people things they know already

• D. Coherence• Has difficulty telling a story, or describing what he

has done, in an orderly sequence of events

• Uses terms like “he” and “it” without making clear what he is talking about

Children’s Conversational Checklist (cont)

• E. Stereotyped conversation• Will suddenly change the topic of conversation• Has favorite phrases and sentences which he will use

a great deal, sometimes inappropriately

• F. Use of conversational context• Tends to be over literal (e.g., “watch your hands”)• May say things which are tactless or socially

inappropriate

• G. Conversational rapport• Ignores conversational overtures from others• Seldom or never looks at the person he is talking to

Children’s Conversational Checklist

• Fairly good clinician inter-rater reliability and internal consistency -- in the .80 range.

• Pragmatic composite score (the 38 items) discriminates significantly between autistic, pragmatic language impaired (without autism), other SLI, and typically developing children.

• Low correlations between teacher/clinician ratings and parent ratings (+.48 on pragmatic composite score)

Summary of Conversational Assessment

• The pragmatic problems of autistic spectrum and pragmatic language impaired children seem to lie fundamentally in the Gricean conversational principles that depend on mutuality of communication (and theory of mind) -- turn taking, topic and referential continuity and relevance, assessing listeners state of knowledge or ignorance etc.

• The pragmatic problems of SLI children without any social-cognitive impairments seem to lie at the interface of syntax, semantics and pragmatics. Limitations in syntactic and lexical understanding and expression impair the linguistic aspects of conversational coherence and cohesion, referential specificity, and repair strategies.

How to Assess 2) Speech Acts

a) Language Sample Analysis– Structuring the interaction to get the child to produce different

speech acts -- especially requests (Roth & Spekman, 1984)– e.g., where child has to ask a “shy puppet” to do things– e.g., situations where the child has to solve a problem:

• Pencils with broken points, puzzle with a piece missing, paints without brushes etc.

b) Controlled Elicited Production Procedures– Question Asking (DELV-CR, 2003; de Villiers, 2004)– asking the right Wh-question to find out specific information

– Communicative Role Taking (DELV-CR; de Villiers, 2004; TOPL, 1992)

– understanding the communicative perspective of others and knowing what speech acts they are producing.

Key Features of Elicited Production Materials and Procedures

• They provide referential support and pragmatic motivation for the language forms and functions to be produced, so they increase the likelihood that those forms and functions will be sampled in the assessment.

• They constrain the range of appropriate utterances, so they are more easily scored than a more open-ended language sample.

• They retain a considerable degree of communicative naturalness in the elicitation procedure rather than resorting to modeling and imitation to elicit these forms.

The Subject Sample

• Data will be shown from 1014 four to nine year olds, most of them from working class backgrounds and from all regions of the USA (taken from the DELV-CR field testing).

• There were 217 four-year-olds, 266 five-year-olds, 300 six-year-olds, 56 seven-year-olds, 101 eight-year-olds and 74 nine-year-olds.

• Approximately 60% of the children were characterized by the testing clinicians as speakers of African American English (AAE), the other 40% as speakers of Mainstream American English (MAE).

• Approximately 1/3rd of the children at each age and in each dialect group were identified by the participating clinicians and schools as having a specific language impairment and were receiving language services.

Question Asking

• The child is shown a picture with something missing from it.

• They have to ask the right question to find out what the event is about.

• The missing elements of the pictures include objects, people, locations, tools, and causes of emotions -- so what, who, where, how, and why questions are motivated.

• Different levels of prompting are given for each trial if the child does not spontaneously ask an appropriate question -- varying from the semantic domain of the question to ask, to the specific wh-word to begin the question with.

• If the child asks an appropriate question they are shown the complete picture.

The girl is painting something. Ask me the right question and I’ll show you the answer.

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture.

c. The Psychological Corporation

What?




5 year old TypicalAAE 5 year old DISAAEwho WHAT IS THE NURSE FEEDING? NRwhere WHERE DID SHE GO SWIMMING? SHE MAKING A POOL.why WHAT IS THE GIRL MAD ABOUT? WHAT THE GIRLhow HOW IS THE GIRL FIXIN' THAT? SHE'S FIXING HIS BIKEwhat WHAT IS THE WOMAN EATING? WHAT SOME MEATwho WHO IS RIDING THE BIKE? WHAT A BOYwhere WHERE IS THAT BOY GOING? THE BOY IS RUNNING TO THE ICE CREAMwhy WHAT HAPPENED? WHAT?who eats what WHAT IS THEY EATING? NR

6 year old TypicalAAE 6 year old DISAAEwho WHO IS THE NURSE FEEDING? WHO IS THAT FEEDING HIM?where WHERE DID THE GIRL SWIM? SHE JUMPED IN THE WATER.why WHAT IS THE GIRL MAD FOR? SHE MAD AT THE TABLE.how WHAT IS THE GIRL FIXING? SHE IS FIXIN THE TOY.what WHAT IS THE GIRL EATING? WHO'S EATIN?who WHO IS RIDING THE BIKE? A BOY RIDIN ON THE BIKE.where WHERE IS THE BOY RUNNING? WHO'S RUNNING?why WHY IS THE BOY CRYING? HE DROPPED HIS ICE CREAM.who eats what WHAT ARE THE PEOPLE EATING? WHO'S EATIN?

8 year-old TypicalAAE 8 year old DISAAEwho WHO IS THE NURSE FEEDING? WHO IS SHE FEEDING?where WHERE DID THE GIRL GO SWIMMING? WHAT SOMETHING SHE SWIM IN?why WHY IS THE GIRL MAD? WHO IS SHE MAD AT?how HOW IS THE GIRL FIXING THE TOY? WHAT'S SHE HOLDING ON HER HAND?what WHAT IS THE WOMAN EATING? WHAT HER MOM EATING FROM HER TWO FINGERS?who WHO IS RIDING THE BIKE? SOMETHING RIDING A BICYCLE.where WHAT IS THE BOY RUNNING TO? WHERE IS HIS HOUSE?why WHY IS THE BOY CRYING? WAS HE CRYING?who eats what WHO IS EATING WHAT FOOD? HOW WAS THEY WAS EATING?

Wh-Question production in MAE and AAE speaking children following all prompts.

Wh-Question Production

0

1

2

3

4

5

6

7

8

9

4 5 6 7 8 9

Age

Average Score /9

AAEMAE

Wh-Question production in typically developing and language impaired children following all prompts.

Wh-Question Production

0

1

2

3

4

5

6

7

8

9

4 5 6 7 8 9

Age

Average Score /9

IMPAIREDTYPICAL

Appropriate why-question production in MAE and AAE speaking children following semantic prompt.

"Why crying?" (Semantic Prompt)

0

0.2

0.4

0.6

0.8

1

4 5 6 7 8 9

Age

AAE

MAE

Appropriate why-question production in typically-developing and language-impaired children following semantic prompt.

"Why crying?" (Semantic Prompt)

0

0.2

0.4

0.6

0.8

1

4 5 6 7 8 9

Age

Impaired

Typical

Communicative Role Taking and Understanding Speech Acts

• Children not only need to produce different kind of speech acts at appropriate times (e.g., asking for information, requesting action, rejecting or denying, prohibiting etc.); they also need to understand the circumstances and force of those utterances in other people.

• The children were shown pictures in which a person was communicating to another about some object or event that was clearly depicted. They were asked what the characters were telling (reporting an observed event), asking, or saying (prohibiting an action), depending on the scenario.

Picture context for “asking” (requesting an object or action).




Picture context for “asking” (requesting an object or action).




Communicative Role Taking in two AAE speaking four-year-olds.

TYPICAL AAE

TELL HIS BIG SISTER FELLED OFF HER BIKEASK CAN I PLAY BASEBALL?ASK CAN I HAVE A PI ECE OF CAKE?SAY YOU DON'T FEED THE DOG. THAT'S HIS OWN FOOD

IMPAIRED AAE

TELL HER BLEEDIN'ASK HIM CARRY SOMETHINGASK HER SAY LOOK AT THE CAKESAY HE FEEDIN THE DOG

Development of appropriate speech act production in a communicative role taking context

(MAE versus AAE speaking children)

Communicative Role Taking

0

1

2

3

4

4 5 6 7 8 9

Age

Average Score /4

AAEMAE

Development of appropriate speech act production in a communicative role taking context

(typically developing versus language impaired children).

Communicative Role Taking

0

1

2

3

4

4 5 6 7 8 9

Age

Average Score /4

IMPAIREDTYPICAL

How to Assess? 3) Referential Communication and

Specification

• Telling my listener(s) who and what I am referring to.

• Several linguistic devices in English serve to identify one object or person out of a possible set.

• These include adjectives that specify a distinctive property of the object or person, prepositional phrases that specify their location, and relative clauses that refer to either a distinctive property, location, or action.

• Eliciting these in a referential communication task – (P. de Villiers, 1988)

Reference Specification -- Testing Procedure

• Examiner and child play a referential communication game.

• The child sees a picture that the examiner cannot see. S/he has to describe an event that is happening in part of the picture so that the examiner can pick out the person or object involved in the event from a set of similar people or objects.

• In some trials there is a distinctive property that distinguishes the referent. In others it is the location or action of the referent that must be mentioned.

• In this way the linguistic form that must be produced increases in complexity from adjectives to prepositional phrases and finally to relative clauses.

Here are two horses.


Tell me what is happening in the red box. I need to know which horse it is.


Here are two policemen.


Tell me what is happening in the red box. I need to know which policeman it is.


Here are two boys.


Tell me what is happening in the red box. I need to know which boy it is.


Development of reference specification in MAE and AAE speaking children. Production of form and function following

all prompts.

Reference Specification

0

1

2

3

4

5

6

4 5 6 7 8 9

Age

AAE

MAE

Development of reference specification in typically-developing and language-impaired children. Spontaneous

production of form and function before any prompts.


0

1

2

3

4

5

6

4 5 6 7 8 9

Age

Impaired

Typical

Development of reference specification in typically-developing and language-impaired children. Production of

form and function following all prompts.


0

1

2

3

4

5

6

4 5 6 7 8 9

Age

Impaired

Typical

How to Assess? 4) Narration

• Narrative Elicitation Techniques:– Personal narratives -- anecdotes

– Story retelling

– Wordless picture books (e.g. the Frog Story)

– Shorter picture sequences

– Fantasy, creative stories

• Scoring Coherence• Scoring Cohesion• Mental States of the Characters -- the “landscape

of consciousness” (Bruner, 1986).

Scoring Narrative Coherence• Story Grammars (e.g., Stein & Glenn, 1979; Johnston, 1982)

– Setting information• Conventional (“once upon a time”)• Characters, place and time• Purpose and motivation

– Episode structure• Initiating events• Actions and behavioral reactions• Outcomes/results

– Internal reactions, plans and goals• Emotions

– simple (“happy”, “sad”, “angry”, “afraid”)– complex (“worried”, “jealous”, “surprised”, “guilty”)

• Desires– From wanting objects and actions to wanting events

• Cognitions– Thoughts, beliefs and states of knowledge

Scoring Narrative Cohesion

• Referential Cohesion– Introducing characters

– Maintaining reference to characters (e.g. article and pronoun use)

– Contrasting characters (e.g., names, adjectives, relative clauses)

• Causal Cohesion• Temporal Cohesion

– Sequencers (e.g., then, next)

– Foregrounding and backgrounding temporal clauses (e.g., while, when, after)

Ideal Properties of Picture Sequence Materials

• Two similar same sex protagonists so the child narrator has to introduce them and maintain reference to them while keeping them contrasted for the listener.

• A series of events that have to be explicitly related to each other by means of temporal and causal links.

• Explicit depiction and implicit involvement of the mental states of the characters so the child is motivated to make reference to the landscape of consciousness of the story.

The Candy Stealing Story

The Balloon Popping Story

Narrative Assessments

• Comprehensive analysis of personal narratives (e.g., Bliss, McCabe & Miranda, 1998) or other elicited narratives (e.g., Hedberg & Westby, 1993).

• Dynamic assessment of narrative skills based on picture books (Miller, Gillam, & Pena, 2003)

• Cohesion and theory of mind in picture sequence narratives (DELV-CR, 2004; P. de Villiers, 2004)

A short, wordless picture-sequence narrative to elicit reference specification, temporal cohesion, and mental state

references (DELV-CR, 2004)




Narrative Samples 1

• I want my train. I’m gonna hide the train from him. I’m gonna play out of the toy box. I’m gonna find that train. Bring that train. (C: 4;2)

• He was looking for the choo choo train because the other boy was playin’. And then… and then he said, “I want that choo choo train back”, and umm… he put it in his toy box. And then he came back to find it and he looked under the bed and it wasn’t there. (SC: 4;9)

Narrative Samples 2

• The big boy came into the little boy’s room and took away the little boy’s train. Then he hid it under the boy’s bed where he couldn’t get it. Then the little boy… when he left… he got out his train and put it in the toy box while the big boy was eating. Then the big boy thought about the train and he went under the bed to go see it but it wasn’t there.

(A: 6;4)

• The little brother was trying to get his toy from the big brother. And the big brother hiding his toy under the bed. When he is eating his sandwich, the little boy go and get it and put it inside of his toy box. When his big brother walk in, he think about the train and he look under his bed for it. (J: 6;3)

Development of reference contrast in narratives (contrasting the two main characters) in typically developing MAE and

AAE speaking children.

Reference Contrast in Spoken Narrative

0

0.2

0.4

0.6

0.8

1

4.5 5.5 6.5 8 10 12

Age

Proportion of Group

AAEMAE

Development of reference contrast in narratives (contrasting the two main characters) in typically developing and

language impaired children.

Reference Contrast in Spoken Narrative

0

0.2

0.4

0.6

0.8

1

4.5 5.5 6.5 8 10 12

Age

Proportion of the Group

ImpairedTypical

Development of the expression of temporal links between events in the narratives of typically developing MAE and

AAE speaking children.

Temporal Links in Spoken Narrative

0

0.5

1

1.5

2

4.5 5.5 6.5 8 10 12

Age

Average Score /2

AAEMAE

Development of the expression of temporal links between events in the narratives of typically developing and language

impaired children.

Temporal Links in Spoken Narrative

0

0.5

1

1.5

2

4.5 5.5 6.5 8 10 12

Age

Average Score /2

ImpairedTypical

Following their spontaneous narrative the children were asked two follow-up questions to probe for their theory of mind understanding:

-- Tell me again what is happening in this picture (picture 5)--The big boy is looking for the train under the bed.

Why is he looking there?




Development of mental state references to describe the “thought balloon” picture (typically developing MAE versus

AAE speaking children)

Mental State Descriptions of Picture 5

0

0.5

1

1.5

2

4.5 5.5 6.5 8 10 12

Age

Average Score /2

AAEMAE

Development of mental state references to describe the “thought balloon” picture (typically developing versus

language impaired children).

Mental State Descriptions of Picture 5

0

0.5

1

1.5

2

4.5 5.5 6.5 8 10 12

Age

Average Score /2

ImpairedTypical

Development of “theory of mind” explanations for the character’s mistaken action in the picture narrative (typically

developing MAE versus AAE speaking children).

Mental State Explanations of Action

0

0.5

1

1.5

2

4.5 5.5 6.5 8 10 12

Age

Average Score /2

AAEMAE

Development of “theory of mind” explanations for the character’s mistaken action in the picture narrative (typically

developing versus language impaired children).

Mental State Explanations of Action

0

0.5

1

1.5

2

4.5 5.5 6.5 8 10 12

Age

Average Score /2

ImpairedTypical

Development of combined narrative skills in MAE and AAE speaking children aged 4 to 12.

Spoken Narrative Score

0

1

2

3

4

5

6

7

4.5 5.5 6.5 8 10 12

Age

Average Score /7

AAEMAE

Overall narrative scores in typically developing and language impaired children aged 4 though 12.

Spoken Narrative Score

0

1

2

3

4

5

6

7

4.5 5.5 6.5 8 10 12

Age

Average Score /7

ImpairedTypical

Conclusion

• Assessment of language pragmatics is both important and doable (though not yet available all in one place).

• A core tool-kit would include:– A checklist of key pragmatic features that are both diagnostic of

disorder and informative about targets for intervention (and that can be reliably filled out by a professional who has observed the child in conversation with familiar adults and peers).

– A conversational protocol to elicit a range of conversational skills from a child in interaction with a familiar adult, and focused categories of analysis for that conversational sample (preferably without having to transcribe it).

– A set of controlled elicited production and comprehension procedures to assess mastery of important component communication skills (with adequate reliability, concept validity, and normative information).

References for Pragmatic Assessment

• General:• Adams, C. (2002). Practitioner review: The assessment of language pragmatics.

Journal of Child Psychology and Psychiatry, 43, 973-987.• Creaghead, N. (1984). Strategies for evaluating and targeting pragmatic

behaviors in young children. Seminars in Speech and Language, 5, 241-252.• Roth, F. & Spekman, N, (1984). Assessing the pragmatic abilities of children:

Part 1. Organizational framework and assessment parameters. Journal of Speech and Hearing Disorders, 49, 2-11.

• Roth, F. & Spekman, N, (1984). Assessing the pragmatic abilities of children: Part 2. Guidelines, considerations, and specific evaluation procedures. Journal of Speech and Hearing Disorders, 49, 12-17.

• Conversational Analysis:• Craig, H. & Evans, J. (1993). Pragmatics and SLI: within-group variations in

discourse behaviors. Journal of Speech and Hearing Research, 36, 777-789.• Ervin-Tripp, S. (2000). Studying conversation: How to get natural peer

interaction. In L. Menn & N.B. Ratner (Eds.), Methods for studying language production. Mahwah, NJ: Erlbaum.


• Checklists and Profiles:• Bishop, D. (1998). Development of the Children’s Communication Checklist (CCC): A

method for assessing qualitative aspects of communication impairment in children. Journal of Child Psychology and Psychiatry, 39, 879-891.

• Bishop, D. & Baird, G. (2001). Parent and teacher report of pragmatic aspects of communication: Use of the Children’s Communication Checklist in a clinical setting. Developmental Medicine and Child Neurology, 43, 809-818.

• Prutting, C. & Kirchner, D. (1987). A clinical appraisal of the pragmatic aspects of language. Journal of Speech and Language Disorders, 52, 105-119.

• Standardized Tests of Pragmatics:• Adams, C., Cook, R., Crutchley, A., Hesketh, A., & Reeves, D. (2001). Assessment of

Comprehension and Expression 6-11 (ACE 6-11). Windsor, UK: NFER-Nelson.• de Villiers, P. A. (2004). Assessing pragmatic skills in elicited production. Seminars in

Speech and Language, 25, 57-72.• Seymour, H., Roeper, T., & de Villiers, J. (2004). The Diagnostic Evaluation of

Language Variation (DELV). The Psychological Corporation.• Phelps-Terasaki, D. & Phelps-Gunn, T. (1992). Test of Pragmatic Language. The

Psychological Corporation.• Lloyd, P., Peers, I., & Foster, C. (2001). The Listening Skills Test. The Psychological

Corporation.


• Narrative:

• Bliss, L., McCabe, A., & Miranda, A.E. (1998). Narrative assessment profile: Discourse analysis for school-age children. Journal of Communication Disorders, 31, 347-363.

• Hedberg, N. & Westby, C. (1993). Analyzing story-telling skills: Theory to practice. Tucson, AZ: Communication Skill Builders.

• Miller, L., Gillam, R., & Pena, E. (2003). Dynamic assessment and intervention: Improving children’s narrative abilities. AGS Publishing.

• Seymour, H., Roeper, T., & de Villiers, J. (2004). The Diagnostic Evaluation of Language Variation (DELV). The Psychological Corporation.

Documents

Assessing Language Pragmatics: Who, What and How Peter A. de Villiers Jill G. de Villiers Smith College *supported by NIH grant N01-DC-8-2104 to the University