Embed Size (px)
DESCRIPTION
I gave this presentation at the Cognitive Aspects of Computational Language Acquisition workshop in the European Association of Computational Linguistics conference in 2009. It presents a freely available open source project written in Java for studying the computational modelling of child language acquisition. It supports plugging in different learning algorithms and evaluating them all within the same environment and with a standardised set of metrics.
Citation preview
Kris Jack
A Collaborative Tool for theComputational Modelling ofChild Language Acquisition
The LanguageAcquisition
Toolkit
CEA LIST, Laboratoire d'ingéneriede la connaissance multimédia multilingue,FontenayauxRoses, France
Kris Jack
IntroductionBackgroundLATDiscussionFutureConclusion
Introduction
What is the Language Acquisition Toolkit (LAT)?
It's a program that helps you to model language acquisition. You can:
● create a miniature world in which your language learner lives;
● create comprehension and production tests for your learner; and
● judge your learner's development over time.
You provide the learner, the LAT provides the environment.
Kris Jack
Background
IntroductionBackgroundLATDiscussionFutureConclusion
What motivated the creation of the LAT?
● Complicated problem numerous computational →models proposed.
● Useful to compare them but difficult due to different:
● assumptions;● training;● tests; and● goals.
What is there to gain?● Time saving benefits.● Standardisation.● Holistic models of longitudinal development.
Kris Jack
LAT
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
Analysing
Training
comprehension
Testing
+
production
Language Learner
DevelopmentalData
www.langac.com
Kris Jack
LAT
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
Analysing
Training
comprehension
Testing
+
production
Language Learner
DevelopmentalData
www.langac.com
Let's see one possible world...
Kris Jack
LAT: Training (1/2)
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
Your world's framework:
● miniature language acquisition paradigm (Feldman et al., 1990) – associate images with descriptions
● world of geometric objects in which actions occur and are described (action + description = event)
● objects (colours, shapes, sizes, 2D positions)● actions (moving, growing, shrinking, appearing,
disappearing, pushing another object and pulling another object, etc.)
● descriptions (defined by you)
321
1 2 3
321
1 2 3
Kris Jack
LAT: Training (2/2)
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
321
1 2 3
321
1 2 3
O1: square
O1: blue
O1: x2
O1: y2
O2: circle
O2: yellow
O2: x3
O2: y3
a ye llow cir cle to the u pper right of the blue square
You must teach your language to the LAT
Before action (t=1) After action (t=2)
O1: square
O1: blue
O1: x2
O1: y2
Kris Jack
LAT: Training (2/2)
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
a ye llow cir cle to the u pper right of the blue square
You must teach your language to the LAT
321
1 2 3
321
1 2 3
O1: square
O1: blue
O1: x2
O1: y2
O2: circle
O2: yellow
O2: x3
O2: y3
Before action (t=1) After action (t=2)
O1: square
O1: blue
O1: x2
O1: y2
Kris Jack
LAT: Training (2/2)
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
a ye llow cir cle to the u pper right of the blue square
You must teach your language to the LAT
321
1 2 3
321
1 2 3
O1: square
O1: blue
O1: x2
O1: y2
O2: circle
O2: yellow
O2: x3
O2: y3
Before action (t=1) After action (t=2)
O1: square
O1: blue
O1: x2
O1: y2
Kris Jack
LAT: Training (2/2)
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
a ye llow cir cle to the u pper right of the blue square
You must teach your language to the LAT
321
1 2 3
321
1 2 3
O1: square
O1: blue
O1: x2
O1: y2
O2: circle
O2: yellow
O2: x3
O2: y3
Before action (t=1) After action (t=2)
O1: square
O1: blue
O1: x2
O1: y2
Kris Jack
LAT: Training (2/2)
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
a ye llow cir cle to the u pper right of the blue square
You must teach your language to the LAT
321
1 2 3
321
1 2 3
O1: square
O1: blue
O1: x2
O1: y2
O2: circle
O2: yellow
O2: x3
O2: y3
Before action (t=1) After action (t=2)
O1: square
O1: blue
O1: x2
O1: y2
Kris Jack
LAT: Training (2/2)
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
a ye llow cir cle to the u pper right of the blue square
You must teach your language to the LAT
321
1 2 3
321
1 2 3
O1: square
O1: blue
O1: x2
O1: y2
O2: circle
O2: yellow
O2: x3
O2: y3
Before action (t=1) After action (t=2)
O1: square
O1: blue
O1: x2
O1: y2
det1
adj1
n1
prep
det2
adj2
n2
Kris Jack
LAT: Training (2/2)
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
a ye llow cir cle to the u pper right of the blue square
You must teach your language to the LAT
321
1 2 3
321
1 2 3
O1: square
O1: blue
O1: x2
O1: y2
O2: circle
O2: yellow
O2: x3
O2: y3
Before action (t=1) After action (t=2)
O1: square
O1: blue
O1: x2
O1: y2
det1
adj1
n1
prep
det2
adj2
n2
adj1 = {“ye llow”, “blue”, “red”, “green”, “pink”...}
Kris Jack
LAT: Testing
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
Analysing
Training
comprehension
Testing
+
production
Language Learner
DevelopmentalData
www.langac.com
Kris Jack
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
LAT: Testing (1/2)
What do you want to test?
● Comprehension
● description in → meaning(s) out● Production
● meaning in → description(s) out
Teach the LAT:● what to test● what results are correct
Kris Jack
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
LAT: Testing (2/2)
Define testing pairs (description + meaning)
321
1 2 3
321
1 2 3
O1: square
O1: blue
O1: x2
O1: y2
O2: circle
O2: yellow
O2: x3
O2: y3
a ye llow cir cle to the u pper right of the blue square
Before action (t=1) After action (t=2)
O1: square
O1: blue
O1: x2
O1: y2
I want to test:● all actions
Kris Jack
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
LAT: Testing (2/2)
Define testing pairs (description + meaning)
321
1 2 3
321
1 2 3
O1: square
O1: blue
O1: x2
O1: y2
O2: circle
O2: yellow
O2: x3
O2: y3
a ye llow cir cle to the u pper right of the blue square
Before action (t=1) After action (t=2)
O1: square
O1: blue
O1: x2
O1: y2
I want to test:● all actions ● all shapes
● all colours
i.e. give the description square, it could be any object at any time
Kris Jack
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
LAT: Testing (2/2)
Define testing pairs (description + meaning)
321
1 2 3
321
1 2 3
O1: square
O1: blue
O1: x2
O1: y2
O2: circle
O2: yellow
O2: x3
O2: y3
a ye llow cir cle to the u pper right of the blue square
Before action (t=1) After action (t=2)
O1: square
O1: blue
O1: x2
O1: y2
I want to test:● all actions ● all shapes
● all colours● all NPs
Kris Jack
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
LAT: Testing (2/2)
Define testing pairs (description + meaning)
321
1 2 3
321
1 2 3
O1: square
O1: blue
O1: x2
O1: y2
O2: circle
O2: yellow
O2: x3
O2: y3
a ye llow cir cle to the u pper right of the blue square
Before action (t=1) After action (t=2)
O1: square
O1: blue
O1: x2
O1: y2
I want to test:● all actions ● all shapes
● all colours● all NPs ● all NP1s
● all NP2s
Kris Jack
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
LAT: Testing (2/2)
Define testing pairs (description + meaning)
321
1 2 3
321
1 2 3
O1: square
O1: blue
O1: x2
O1: y2
O2: circle
O2: yellow
O2: x3
O2: y3
a ye llow cir cle to the u pper right of the blue square
Before action (t=1) After action (t=2)
O1: square
O1: blue
O1: x2
O1: y2
Kris Jack
LAT: Analysing
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
Analysing
Training
comprehension
Testing
+
production
Language Learner
DevelopmentalData
www.langac.com
This is your job!
Kris Jack
LAT: Analysing
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
Analysing
Training
comprehension
Testing
+
production
Language Learner
DevelopmentalData
www.langac.com
Kris Jack
LAT: Analysing (1/2)
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
What do you want to analyse?
Kris Jack
LAT: Analysing (2/2)
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
How are the results presented?
● Roundbased (results from a single round)
● Trialbased (results from all rounds across a single trial)
● Modelbased (results across all trials)
Can take stages into account
6 12 18 24 30 360 42 48Time (months)
54
Prelinguistic
HolophrasticEarly
Multiword
Late Multiword PostAbstract
Abstract
Reflexivevocalisations,cooing,vocal play,babbling
Atomiclexicalunits
Multipleunits
Multiple unitswith syntacticmarkings (e.g. word order)
Multiple units with phrasal category membership
Kris Jack
LAT: Analysing
IntroductionBackgroundLAT● Training● Testing● AnalysingDiscussionFutureConclusion
Analysing
Training
comprehension
Testing
+
production
Language Learner
DevelopmentalData
www.langac.com
Kris Jack
Discussion
IntroductionBackgroundLATDiscussionFutureConclusion
What does the LAT give us?
● Time saving benefits.● Standardisation (let's play on fairly):
● training; ● testing; and● analysing.
● Analyses for longitudinal development.
Kris Jack
Future
IntroductionBackgroundLATDiscussionFutureConclusion
What does the crystal ball show us?
● More complex miniature language
● More languages
● A richer language representation (e.g. tones)
● More realistic symbolic meanings
● Accessibility for nonprogrammers
● Can we integrate paradigms that do not use a miniature language?
● Wish list (www.langac.com)
It's collaborative so you can make these possible
Kris Jack
Conclusion
IntroductionBackgroundLATDiscussionFutureConclusion
Key points
● The LAT allows you to:
● create training data;● create testing data; and● analyse your learning model's development
(comprehension and production).● Standardisation
● let's speak the same language● Work in progress● Collaboration
● I'll show you mine if you show me yours
Kris Jack
Thank you
IntroductionBackgroundLATDiscussionFutureConclusion
Analysing
Training
comprehension
Testing
+
production
Language Learner
DevelopmentalData
www.langac.com
Kris Jack
Results (1/2)
0 15 30 45 60 75 90 105 120 135 1500.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Stages in Production
HoloEarlyLateAbstractEnd
Number of Events Observed
% o
f Req
uire
men
ts M
et Holophrastic
Early M
ultiword
Late Multiw
ord
Abstract
Postabstract
6 12 18 24 30 360 42 48Time (months)
54
IntroductionBackgroundLATDiscussionFutureConclusion
