Embed Size (px)
Citation preview
Machine Learning in Conceptual Spaces
Two Learning Processes
Lucas Bechberger
https://www.lucas-bechberger.de
Machine Learning in Conceptual Spaces / Lucas Bechberger 2
Conceptual Spaces
Symbolic Layer
Subsymbolic Layer [0.42; -1.337, ...]
∀ x :apple( x)⇒ red(x ) Formal Logics
Sensor Values,Machine Learning
?Conceptual LayerGeometric Representation
Machine Learning in Conceptual Spaces / Lucas Bechberger 3
My PhD Project / Outline
Symbolic Layer
Subsymbolic Layer
Conceptual Layer
Manually defineregions
Manually definedimensions
3.) Learning Concepts
2.) Learning Dimensions
1.) Mathematical Formalization
Machine Learning in Conceptual Spaces / Lucas Bechberger 4
My PhD Project / Outline
Symbolic Layer
Subsymbolic Layer
Conceptual Layer
Manually defineregions
Manually definedimensions
3.) Learning Concepts
2.) Learning Dimensions
1.) Mathematical Formalization
Machine Learning in Conceptual Spaces / Lucas Bechberger 5
Learning Dimensions
There are (at least) three approaches: Handcrafting Multidimensional Scaling Artificial Neural Networks
Bonus: A Hybrid Approach
Machine Learning in Conceptual Spaces / Lucas Bechberger 6
Learning Dimensions
There are (at least) three approaches: Handcrafting Multidimensional Scaling Artificial Neural Networks
Bonus: A Hybrid Approach
Machine Learning in Conceptual Spaces / Lucas Bechberger 7
Learning Dimensions: MDS
Psychological grounding
Dealing with unseen inputs
1) Psychological experiment
2) Average across participants
3) Multidimensional Scaling
space
# dimensionsmatrix
similarity judgments
Machine Learning in Conceptual Spaces / Lucas Bechberger 8
Learning Dimensions
There are (at least) three approaches: Handcrafting Multidimensional Scaling Artificial Neural Networks
Bonus: A Hybrid Approach
Machine Learning in Conceptual Spaces / Lucas Bechberger 9
Learning Dimensions: ANNs
Autoencoder (e.g., β-VAE): compress and reconstruct input
Hidden neurons = dimensions in our conceptual space
output
hidden representation
input24 75 02 53
42 91
22 76 03 50
Higgins, I.; Matthey, L.; Pal, A.; Burgess, C.; Glorot, X.; Botvinick, M.; Mohamed, S. & Lerchner, A. β-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework, ICLR 2017
Dealing with unseen inputs
Psychological grounding
Machine Learning in Conceptual Spaces / Lucas Bechberger 10
Learning Dimensions: ANNs
Centered, unrotated rectangles Differing only with respect to width and height
Use InfoGAN to learn interpretable dimensions
Chen, X.; Duan, Y.; Houthooft, R.; Schulman, J.; Sutskever, I. & Abbeel, P. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets Advances in Neural Information Processing Systems, 2016
Machine Learning in Conceptual Spaces / Lucas Bechberger 11
Learning Dimensions
There are (at least) three approaches: Handcrafting Multidimensional Scaling Artificial Neural Networks
Bonus: A Hybrid Approach
Machine Learning in Conceptual Spaces / Lucas Bechberger 12
Learning Dimensions: Hybrid
Psychological Experiment
MDS
AN
N
dog
cat
...
Psychological grounding
Dealing with unseen inputs
Bechberger, L. & Kypridemou, E. Mapping Images to Psychological Similarity Spaces Using Neural Networks. AIC 2018
Machine Learning in Conceptual Spaces / Lucas Bechberger 13
My PhD Project / Outline
Symbolic Layer
Subsymbolic Layer
Conceptual Layer
Manually defineregions
Manually definedimensions
3.) Learning Concepts
2.) Learning Dimensions
1.) Mathematical Formalization
Machine Learning in Conceptual Spaces / Lucas Bechberger 14
Learning Concepts
Machine Learning Engineer
Cognitive ScienceResearcher
Give me a big data set of labeled examples!
I’ll train a neural networkfor a bunch of epochs
to find a nicedecision boundary.
It’s just a standard ML problem!
Wait a second, that’s cognitively implausible!
In real life,we have moreunlabeled than
labeledexamples.
Plus:Humans
don’t learnvia batch
processing.
That’s too complicated for now.
Machine Learning in Conceptual Spaces / Lucas Bechberger 15
Learning Concepts: LTN
Fuzzy Logic Degree of membership between 0 and 1
One can generalize logical operators: apple AND red = min(apple, red)
We can express rules over these fuzzy sets
apple: 1.0red: 0.9round: 0.7banana: 0.0
Symbolic
Subsymbolic
Conceptual
Machine Learning in Conceptual Spaces / Lucas Bechberger 16
Learning Concepts: LTN
Use neural networks to learn membership functions
Constraints: Labels Rules
Tune NN weights such that all constraints are fulfilled
apple red sweet
0.99 0.75 0.31
Apple AND red IMPLIES sweet: 0.31
Machine Learning in Conceptual Spaces / Lucas Bechberger 17
Learning Concepts: LTN
Conceptual space of movies from Derrac and Schockaert Extracted conceptual space from movie reviews 15.000 data points, labeled with one or more of 23 genres
Use LTN to learn genres in that space Compare to kNN with respect to classification performance Compare to simple counting with respect to rule extraction
Long run: align LTN with conceptual spaces theory Convexity, domain structure, ...
Joaquín Derrac and Steven Schockaert. Inducing semantic relations from conceptual spaces: a data-driven approach to commonsense reasoning, Artificial Intelligence, vol. 228, pages 66-94, 2015
Machine Learning in Conceptual Spaces / Lucas Bechberger 18
My PhD Project / Outline
Symbolic Layer
Subsymbolic Layer
Conceptual Layer
Manually defineregions
Manually definedimensions
3.) Learning Concepts
2.) Learning Dimensions
1.) Mathematical Formalization
Thank you for your attention!
Questions? Comments? Discussions?
https://www.lucas-bechberger.de
@LucasBechberger
