Http:// 1 of 25 Primatech: Models and Future Directions GVU Brownbag 9/30/1999 by Jim Davies, Tim Keenan,

http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 1 of 25

Primatech: Models and Future Directions

GVU Brownbag 9/30/1999

by Jim Davies, Tim Keenan,

Wasinee Rungsarityotin &

Irfan Essa{jimmyd, tk, wasinee, irfan}

@cc.gatech.edu


What is Primatech?

• Chantek: An orangutan who can sign

• Will be on display at Zoo Atlanta; visitors can interact with him

• He will be overloaded. Primatech simulates having a conversation with him

Schematic


Perception module

• 150 Signs to recognize (Starner et al. recognized 40)

• Controlled environment (any background, etc.)

• Novice users (few signs used)

• Supporting speech data


Graphics module

• Memory of sign animations

• Home positions?

• “Essence” with on the fly transformations

• Remember every transition? (22500)


Chantek and trainer Lyn Miles


Using a gorilla model


An animated sign for “like”

QuickTime™ and aVideo decompressor

are needed to see this picture.


Ape language research

• Language trained apes know 140 signs, deaf children of 3 know 132

• Chantek– Dr. Lyn Miles– 150 signs (strict criterion)– Anthropological approach


Three cognitive models

1: Simple activation with Wordnet

2: Express words that correspond to desire satisfaction

3: Sentence verification


Model 1: Simple activation with Wordnet

• ACT-R uses spreading activation

• Relevance links only (from WordNet)

• distinguish words, signs, concepts

• Retrieve anything, let spreading activation from goal do the work

Model 1: Representation• Instead of:

(banana ISA fruit color yellow)

(yellow ISA color)

• I used:

(cup-concept ISA concept word cup-word sign cup-sign r1 drink-concept)

(cup-sign ISA sign meaning cup-concept)

(cup-word ISA sign meaning cup-concept)

What do people ask? (Collected by Tara Stoinski)

1. What do you like to eat? (5 people asked)

2. What do you like to do? (3)

3. Are you bored? (3)

4. What is it like to have fur? (1)

5. Do you like the animals that you live with? (1)

6. What is it like to live in a cage? (1)

7. Do you mind people staring at you all the time? (1)

8. Do you have a favorite keeper? (1)

9. Are you happy? (7)

10. What do you want that you don't have? (4)

11. Do you want to go back to the wild? (2)

12. Tell me about evolution (1)


Model 1: Results

1. What do you like to eat?

response: food-eat-sign

3. Are you bored?

response: you-sign

5. Do you like the animals that you live with?

response: you-sign

9. Are you happy?

response: you-sign

11. Do you want to go back to the wild?

response: go-sign


Why so bad?

• definitions too dictionary-like– banana n 1: any of several tropical and subtropical treelike herbs of the genus Musa

having a terminal crown of large entire leaves and usually bearing hanging clusters of elongated fruits

– What about being yellow? Eating it? Deliciousness?

• Only 20% of words linked• “You” “eat” and “go” were the only words in

questions that were in memory.


Model 2: Express words that correspond to desire satisfaction

• ACT-R has a goal structure but no way to express desires that are always around but no specifically acted upon all the time.

• Desires: food, drink, sex, freedom, and stimulation-- represented as memory chunks

• Words were linked to desires


Model 2: Relationships

(screwdriver-word-meaning ISA relation2 relation meaning-relation first screwdriver-word second screwdriver-concept)

(dog-isa-animal ISA relation2 relation subclass-relation first dog-concept second animal-concept)

Model 2: Input Representation (goal-1-1 ISA goal-talk sentence-id 1 word what-word next-word goal-1-2)

(goal-1-2 ISA goal-talk sentence-id 1 word do-word next-word goal-1-3)

(goal-1-3 ISA goal-talk sentence-id 1 word you-word next-word goal-1-4)

(goal-1-4 ISA goal-talk sentence-id 1 word like-word next-word unknown)


Model 2: Results

• Often responds appropriately when offered something desired

• Linguistically still impoverished


Model 3: Sentence verification

• Implemented a simple grammar• Part of speech relation added• Walked through input sentence, putting words

in sentence template• Checks the memory for a fact that has the

same subject, verb, and object• If found, express verb


Model 3: Results

• Could verify relations found in the memory (Apple is a fruit, Chantek likes meat, Banana is yellow, etc.)

• Combination of Models 2 and 3 might account for much of his conversation


Cognitive Model contributions

• Input sentence representation

• Desire representation in ACT-R

• New relation representation/ relations can become activated themselves

• First computer modeling of ape language


Future directions

• Perception: Apply machine learning to signs, installation design

• Graphics: Collision detection, closer modeling of the ape’s actual motion

• Cognitive Model: Modeling transcript data, integration of models 2 and 3


Thanks

• Graphics, Visualization & Usability seed grant 98-99

• Tara Stoinski for question collection

• Victor Zordan for graphics help

• Lyn Miles for Chantek

More information

We are looking for collaborators!

http://www.cc.gatech.edu/gvu/perception/

projects/primatech/

Email me at [email protected]

Documents

Http:// 1 of 25 Primatech: Models and Future Directions GVU Brownbag 9/30/1999 by Jim Davies, Tim Keenan,