Pat LangleyPat LangleyInstitute for the Study of Learning and ExpertiseInstitute for the Study of Learning and Expertise

Palo Alto, CaliforniaPalo Alto, California

andand

Center for the Study of Language and InformationCenter for the Study of Language and InformationStanford University, Stanford, CaliforniaStanford University, Stanford, California

http://cll.stanford.edu/~langleyhttp://cll.stanford.edu/~langleylangley@csli.stanford.edulangley@csli.stanford.edu

Machine Learning for Cognitive SystemsMachine Learning for Cognitive Systems

The views contained in these slides are the author’s and do not represent official policies, either The views contained in these slides are the author’s and do not represent official policies, either Expressed or implied, of the Defense Advanced Research Projects Agency or the DoD. Expressed or implied, of the Defense Advanced Research Projects Agency or the DoD.

Expanding our Computational HorizonsExpanding our Computational Horizons

these successes are prime examples of these successes are prime examples of niche AIniche AI, which, which

develops techniques that are increasingly powerful develops techniques that are increasingly powerful

but that apply to an ever narrower classes of problems.but that apply to an ever narrower classes of problems.

The field of machine learning has many success stories, but: The field of machine learning has many success stories, but:

supports the construction of supports the construction of generalgeneral intelligent systems; intelligent systems;

aspires to the same learning abilities as appear in humans.aspires to the same learning abilities as appear in humans.

Instead, we need a new vision for machine learning technology that: Instead, we need a new vision for machine learning technology that:

This would produce a broader research agenda that would take the field This would produce a broader research agenda that would take the field into unexplored regions.into unexplored regions.

niche AI

cognitivesystems

generalitygenerality

pow

erpo

wer

Challenge 1: Rapid LearningChallenge 1: Rapid Learning

methods for learning classifiers from thousands of cases;methods for learning classifiers from thousands of cases;

methods that converge on optimal controllers in the limit.methods that converge on optimal controllers in the limit.

Current learning research focuses on asymptotic behavior: Current learning research focuses on asymptotic behavior:

learn reasonable behavior from relatively few cases;learn reasonable behavior from relatively few cases;

take advantage of knowledge to speed the learning process. take advantage of knowledge to speed the learning process.

In contrast, humans are typically able to: In contrast, humans are typically able to:

We need more work on learning from few cases in the presence of We need more work on learning from few cases in the presence of background knowledge. background knowledge.

experienceexperience

perf

orm

ance

perf

orm

ance

Challenge 2: Cumulative LearningChallenge 2: Cumulative Learning

take no advantage of what has been learned before; take no advantage of what has been learned before;

provide no benefits for what is learned afterwards.provide no benefits for what is learned afterwards.

Current learning research focuses on isolated induction tasks that: Current learning research focuses on isolated induction tasks that:

incremental acquisition of knowledge over time thatincremental acquisition of knowledge over time that

builds on knowledge acquired during earlier episodes.builds on knowledge acquired during earlier episodes.

In contrast, much human learning involves: In contrast, much human learning involves:

We need much more research on such cumulative learning. We need much more research on such cumulative learning.

initial knowledge extended knowledge

Challenge 3: Varied LearningChallenge 3: Varied Learning

Current learning research emphasizes tasks like classification and reactive Current learning research emphasizes tasks like classification and reactive control, whereas humans learn: control, whereas humans learn:

grammars for understanding natural language;grammars for understanding natural language;

heuristics for reasoning and problem solving; heuristics for reasoning and problem solving;

scripts and procedures for routine behavior;scripts and procedures for routine behavior;

cognitive maps for localization and navigation; cognitive maps for localization and navigation;

models that explain the behavior of artifacts. models that explain the behavior of artifacts.

We need more work on learning such varied knowledge structures. We need more work on learning such varied knowledge structures.

current focus ofcurrent focus ofmachine learningmachine learning

human learninghuman learningabilitiesabilities

Challenge 4: Compositional LearningChallenge 4: Compositional Learning

involve one-step decisions for classification or regression; involve one-step decisions for classification or regression;

utilize simple reactive control for acting in the world.utilize simple reactive control for acting in the world.

Current learning research focuses on performance tasks that: Current learning research focuses on performance tasks that:

the acquisition of modular knowledge the acquisition of modular knowledge elementselements that that

can be can be composed dynamicallycomposed dynamically by multi-step reasoning. by multi-step reasoning.

But many other varieties of learning instead involve: But many other varieties of learning instead involve:

We should give more attention to learning such compositional knowledge. We should give more attention to learning such compositional knowledge.

knowledge knowledge reasoningreasoning

Challenge 5: Evaluating Embedded LearningChallenge 5: Evaluating Embedded Learning

favor work on minor refinements of existing component algorithms; favor work on minor refinements of existing component algorithms;

encourage mindless “bake offs” that provide little understanding.encourage mindless “bake offs” that provide little understanding.

Current evaluation emphasizes static data sets for isolated tasks that: Current evaluation emphasizes static data sets for isolated tasks that:

a a setset of challenging environments that exercise learning of challenging environments that exercise learning andand reasoning, reasoning,

that include performance tasks of that include performance tasks of gradedgraded complexity and difficulty, and complexity and difficulty, and

that have real-world relevance but allow systematic experimental control.that have real-world relevance but allow systematic experimental control.

To support the evaluation of To support the evaluation of embeddedembedded learning systems, we need: learning systems, we need:

battle managementbattle management in-city drivingin-city driving air reconnaissanceair reconnaissance

End of PresentationEnd of Presentation