Knowledge Representation in the Age of Deep Learning, Watson, and the Semantic Web

Tetherless World Constellation, RPI

KR in the age of Deep Learning,

Watson,and the Semantic Web

Jim HendlerTetherless World Professor of Computer, Web and Cognitive Sciences

Director, Institute for Data Exploration and Applications

Rensselaer Polytechnic Institutehttp://www.cs.rpi.edu/~hendler

@jahendler (twitter)Major talks at: http://www.slideshare.net/jahendler


But first, Why the Moose?

This moose gave a keynote with Tim Berners-Lee.

This moose gave a keynote with Peter Norvig.


Talk derives in large part from working on forthcoming book

(More info at Springer booth)

(Thanks Alice!)


Outline

• Several important AI technologies have moved through “knees in the curve” bringing much of the attention to AI again– Deep Learning (& ML in general)– Watson (& “cognitive computing”)– Semantic Web (& the knowledge graph)

• But what about KR– What it is, why it still matters

• And how can these come together– Which comes with a lot of important challenges


A) Deep Learning

“phase transition” in capabilities of neural networks w/machine power


Trained on lots of categorized images

Imagenet: Duck Imagenet: Cat


Impressive results

Increasingly powerful techniques have yielded incredible results in the past few years


B) Watson


The Watson DeepQA Pipeline


Watson is based on ”Associative knowledge”

© IBM, used with permission.


Impressive Results

Watson showed the power of “associative knowledge”


C) Semantic Web


From Semantic Web to the Knowledge Graph


Based on a large “knowledge graph” mined fromextracted and learned data


Many intermediate steps

(P. Norvig, WWW 2016, 4/16)


Impressive results

Google finds embedded metadata on >30% of its crawl – Guha, 2015

Google “knowledge vault” reported to have over 1.6 billion “facts” (links)


Summary: AI has done some way cool stuff

Summary (simplifying tremendously)• Deep Learning: neural learning from data with high

quality, but imperfect results• Watson: Associative learning from data with high

quality but imperfect results• Semantic Web/Knowledge Graph: Graph links

formation from extraction, clustering and learning

As much as many of us “GOFAI” folks wish it, this stuff cannot be ignoredbut, there are still problems…


Why did knowledge graph need “”Human Judgments”?

Association ≠ Correctness


Quick quiz

Who did this moose give invited talks with?

A) Stuart Russell & Vint CerfB) A deer and a keynoteC) IJCAI-16 and Alces AlcesD) Tim Berners-Lee and Peter Norvig


Associational learning cannotexplain learning by “symbolic communication”

Who did this moose give invited talks with?A) Stuart Russell & Vint Cerf (highly associated with target answer)

B) A deer and a keynote (word embedding similarity to question)

C) IJCAI-16 and Alces Alces (perceptually linked)

D) Tim Berners-Lee and Peter Norvig (Correct answer is something most of you learned today, 1-shot, via being told)


GOFAI: Knowledge Representation?

• A knowledge representation (KR) is most fundamentally a surrogate, a substitute for the thing itself, used to enable an entity to determine consequences by thinking rather than acting, i.e., by reasoning about the world rather than taking action in it.

• It is a set of ontological commitments, i.e., an answer to the question: In what terms should I think about the world?

• It is a fragmentary theory of intelligent reasoning, expressed in terms of three components: (i) the representation's fundamental conception of intelligent reasoning; (ii) the set of inferences the representation sanctions; and (iii) the set of inferences it recommends.

• It is a medium for pragmatically efficient computation, i.e., the computational environment in which thinking is accomplished. One contribution to this pragmatic efficiency is supplied by the guidance a representation provides for organizing information so as to facilitate making the recommended inferences.

• It is a medium of human expression, i.e., a language in which we say things about the world.

R. Davis, H. Shrobe, P. Szolovits (1993)


KR: Human Expression

Cute kid story: first two words


Telling cats from ducks doesn’t need KR

!


“Saying things about the world” does

"If I was telling it to a kid, I'd probably say something like 'the cat has fur and four legs and goes meow, the duck is a bird and it swims and goes quack’. "

How would you explain the difference between a duck and a cat to a child?

Woof


KR: Surrogate knowledge?

Which could you sit in?What is most likely to bite what?Which one is most likely to become a computer scientist someday?…


“Surrogate” knowledge

Which could you sit in? What is most likely to bite what?Which one is most likely to become a computer scientist someday? How would they go about doing it?


KR: Recommended vs. Possible inference

Which one would you save if the house was on fire?


Recommended vs. Possible inference

Which one would you save if the house was on fire?Would you use a robot baby-sitter without knowing which of the three possibilities it would choose?


KR systems in AI need grounded symbols

• Logic- and rule- based systems– Ground in “model theory” with a notion of truth

and falsity• Probabilistic Reasoning

– P(A|B) requires A, B map to “meaningful” concepts, P to be a “real” probability

• Constraint Satisfaction, etc– Finding an interpretation satisfying a set of

boolean (T,F) constraints(Note: Yes, I am simplifying, blurring distinctions, ignoring much cutting edge work… happy to discuss later)


The challenge

• If we want to implement KR systems on top of neural and associative learners we have an issue– The numbers coming out of Deep Learning

and Associative graphs are not probabilities– They don’t necessarily ground in human-

meaningful symbols• ”sub-symbolic” learning …• Association by clustering …• Errorful extraction …


The challenges

• Can we avoid throwing out the reasoning baby with the grounding bathwater?– We still need planning systems– We still want to be able to define the rules

that a system should follow– We want to be able to interact with and

understand these systems• Even if computers don’t need to be symbolic

communicators, WE DO!!!


Not just “theory” the applications drivingmuch modern AI require new grounding ideas

Guruduth Banavar, w/permission)


Starting Place: Rethinking grounding

– Formal Explanation vs. post hoc justification• Eg. Even if we cannot use a formal

decomposition to explain the reasoning, can we produce a justification that explains it

– Reasoning systems that “know” some of their axioms may be simply wrong• Eg.F1 of .9 doesn’t mean answers are 90%

correct, it is (simplifying) more like 9 out of 10 answers are right, the others aren’t.

– Nailing context …


Human-Aware AI

• Context is key– AI systems still perform best in well-

defined contexts (or trained situations, or where their document set is complete, etc.)

– Humans are good at recognizing context and deciding when extraneous factors don’t make sense• Extreme example: Stanislav Yevgrafovich

Petrov (the man who saved the world)


Why this REALLY matters

• Humanity faces huges challenges– eg. Our knowledge of cancer genomics

is being outpaced by mutations as cancer continues to spread

– eg. Our neighborhoods degrade as wealth disparity grows

– eg. Our climate warms as we argue about the causes without changing behaviors


Attacking these problems require the best minds we have working together: Human and AI!

The existential threat is not AI, it’s not utilizing the AI we have correctly


Summary of talk (minus moose)

• Modern AI is making some huge strides– Eg. DL, Associative Learning, Knowledge Graphs,

…• But the need for KR has not gone away

– Eg. Surrogacy, Recommended Inference, Human communication

• The integration challenge will require goring some sacred cows– Grounding, explanation, context ….

• But we need to do it.


Questions?

Technology

Knowledge Representation in the Age of Deep Learning, Watson, and the Semantic Web