60

An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

  • Upload
    others

  • View
    6

  • Download
    0

Embed Size (px)

Citation preview

Page 1: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and
Page 2: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and
Page 3: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

An Introduction to Artificial Intelligence, �Machine Learning, and Neural networks

ATA58 Carola F. Berger

Page 4: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Outline�  What is Artificial Intelligence (AI)?

  What does it do?

  How does it work?

  Will there be a robot apocalypse?

  References and Further Reading

Carola F. Berger, AI and Neural Nets, ATA58 2

Page 5: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What is AI?�  What is “intelligence”?

Carola F. Berger, AI and Neural Nets, ATA58 3

Page 6: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What is AI?�  What is “intelligence”?

Merriam-Webster: “the ability to learn or understand or to deal with new or trying situations”

Carola F. Berger, AI and Neural Nets, ATA58 3

Page 7: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What is AI?�  What is “intelligence”?

Carola F. Berger, AI and Neural Nets, ATA58 4

turingarchive.org

Page 8: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What is AI?�  What is “intelligence”?

Carola F. Berger, AI and Neural Nets, ATA58 5

Page 9: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What is AI?�  What is “machine learning”?

Carola F. Berger, AI and Neural Nets, ATA58 6

Page 10: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What is AI?�  What is “machine learning”?

“‘Machine learning’ is a fancy way of saying ‘finding patterns in data.’”

Kirti Vashee

Carola F. Berger, AI and Neural Nets, ATA58 6

Page 11: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What is AI?�  What is “deep learning”?

Carola F. Berger, AI and Neural Nets, ATA58 7

Hagmann P, Cammoun L, Gigandet X, Meuli R, Honey CJ, Wedeen VJ, Sporns O (2008) �Mapping the structural core of human cerebral cortex. PLoS Biology Vol. 6, No. 7, e159.

Page 12: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What is AI?�  What is “deep learning”?

Artificial neural nets are not a new idea: W. McCulloch, W. Pitts, 1943 D. O. Hebb, 1949 B. G. Farley, W. A. Clark, 1954 …

Carola F. Berger, AI and Neural Nets, ATA58 8

Page 13: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What is AI?�

Carola F. Berger, AI and Neural Nets, ATA58 9 Adapted from: S. Jurvetson, https://www.flickr.com/photos/jurvetson/31409423572

/

Page 14: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What does AI do?�

Carola F. Berger, AI and Neural Nets, ATA58 10

Page 15: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What does AI do?�  Play games and win

Carola F. Berger, AI and Neural Nets, ATA58 11

Page 16: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What does AI do?�  Automated classification   Object recognition   Recommender systems

Carola F. Berger, AI and Neural Nets, ATA58 12

Page 17: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What does AI do?�  Predictive typing

  Text-to-speech, speech-to-text   (Neural) machine translation

Carola F. Berger, AI and Neural Nets, ATA58 13

Page 18: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What does AI do?�  Financial trading

  Legal assistance

Carola F. Berger, AI and Neural Nets, ATA58 14

Page 19: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What does AI do?�  Self-driving cars

Carola F. Berger, AI and Neural Nets, ATA58 15

Page 20: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What does AI do?�  Chat and social

media bots

Carola F. Berger, AI and Neural Nets, ATA58 16

Page 21: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What does AI do?�  Design inspirational posters

Carola F. Berger, AI and Neural Nets, ATA58 17

Page 22: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What does AI do?�  Name rescued guinea pigs

J. Shane, http://lewisandquark.tumblr.com

Carola F. Berger, AI and Neural Nets, ATA58 18

Page 23: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

What does AI do?�  Supervised Unsupervised

learning learning

Carola F. Berger, AI and Neural Nets, ATA58 19

Page 24: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

How does it work?�

Carola F. Berger, AI and Neural Nets, ATA58 20

Page 25: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

How does it work?�  Neuron:

Carola F. Berger, AI and Neural Nets, ATA58 21

Bruce Blaus, https://commons.wikimedia.org/wiki/File:Blausen_0657_MultipolarNeuron.png

Page 26: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

How does it work?�  Unit in artificial neural net:

Carola F. Berger, AI and Neural Nets, ATA58 22

Page 27: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

How does it work?�  Unit in artificial neural net:

Carola F. Berger, AI and Neural Nets, ATA58 22

Page 28: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

How does it work?�  Neural network:

Carola F. Berger, AI and Neural Nets, ATA58 23

Hagmann P, Cammoun L, Gigandet X, Meuli R, Honey CJ, Wedeen VJ, Sporns O (2008) �Mapping the structural core of human cerebral cortex. PLoS Biology Vol. 6, No. 7, e159.

Page 29: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

How does it work?�  Neural network:

Carola F. Berger, AI and Neural Nets, ATA58 24 Adapted from: Cburnett, https://commons.wikimedia.org/wiki/File:Artificial_neural_network.svg

Page 30: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

How does it work?�  Neural network – training:

Carola F. Berger, AI and Neural Nets, ATA58 25 Adapted from: Cburnett, https://commons.wikimedia.org/wiki/File:Artificial_neural_network.svg

Adapt weights �(“arrows”) according to difference between desired output and �actual output, e.g. by backpropagation

Feed in �training data

Page 31: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�

Carola F. Berger, AI and Neural Nets, ATA58 26

Page 32: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Neural net to recognize hand-written digits

Carola F. Berger, AI and Neural Nets, ATA58 27

Page 33: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Neural net to recognize hand-written digits

Carola F. Berger, AI and Neural Nets, ATA58 28

Page 34: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Sample input (20x20 pixels)

Carola F. Berger, AI and Neural Nets, ATA58 29

Page 35: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Sample input (20x20 pixels)

Carola F. Berger, AI and Neural Nets, ATA58 29

Page 36: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Weights to hidden units

Carola F. Berger, AI and Neural Nets, ATA58 30

Page 37: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Weights to hidden units – “feature” extraction

Carola F. Berger, AI and Neural Nets, ATA58 31

Page 38: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Weights to hidden units

Carola F. Berger, AI and Neural Nets, ATA58 30

Page 39: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Weights to hidden units

Carola F. Berger, AI and Neural Nets, ATA58 30

Page 40: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Hidden units to output

Carola F. Berger, AI and Neural Nets, ATA58 31

Page 41: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Hidden units to output

Carola F. Berger, AI and Neural Nets, ATA58 31

Page 42: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Hidden units to output

Carola F. Berger, AI and Neural Nets, ATA58 31

Page 43: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�

Carola F. Berger, AI and Neural Nets, ATA58 32

Input Internal convolution Hidden Output

Internal conv.

2

Page 44: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�

Carola F. Berger, AI and Neural Nets, ATA58 32

Input Internal convolution Hidden Output

Internal conv.

2

Page 45: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Wrong!!!

Carola F. Berger, AI and Neural Nets, ATA58 32

Input Internal convolution Hidden Output

Internal conv.

2

Page 46: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�What happens with unknowns? Klingon 6 [jav]

Carola F. Berger, AI and Neural Nets, ATA58 33

Input Internal convolution Hidden Output

Page 47: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Autopsy of a Neural Net�Klingon 6 [jav]

Carola F. Berger, AI and Neural Nets, ATA58 33

Input Internal convolution Hidden Output

Internal conv.

2

Page 48: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Neural Nets - Recap�ü  Training = extraction of

“features” (=patterns) from training data

Carola F. Berger, AI and Neural Nets, ATA58 34

Page 49: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Neural Nets - Recap�ü  Training = extraction of

“features” (=patterns) from training data ü  The more hidden layers and hidden units, the

more parameters (possible overfitting!)

Carola F. Berger, AI and Neural Nets, ATA58 34

Page 50: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Neural Nets - Recap�ü  Training = extraction of

“features” (=patterns) from training data ü  The more hidden layers and hidden units, the

more parameters (possible overfitting!) ü  Beware: Garbage in -> worse garbage out!

Carola F. Berger, AI and Neural Nets, ATA58 34

Page 51: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Neural Nets - Recap�ü  Training = extraction of

“features” (=patterns) from training data ü  The more hidden layers and hidden units, the

more parameters (possible overfitting!) ü  Beware: Garbage in -> worse garbage out! ü  ANNs work well for pattern recognition

after training, including “context”

Carola F. Berger, AI and Neural Nets, ATA58 34

Page 52: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Neural Nets - Recap�ü  Training = extraction of

“features” (=patterns) from training data ü  The more hidden layers and hidden units, the

more parameters (possible overfitting!) ü  Beware: Garbage in -> worse garbage out! ü  ANNs work well for pattern recognition

after training, including “context” ü  Completely unpredictable when confronted

with new, hitherto unknown data

Carola F. Berger, AI and Neural Nets, ATA58 34

Page 53: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Neural Nets - Recap� Recall: Definition of “intelligence” according to Merriam-Webster: “the ability to learn or understand or to deal with new or trying situations”

Carola F. Berger, AI and Neural Nets, ATA58 35

Page 54: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Is the Robot Apocalypse near?�

Carola F. Berger, AI and Neural Nets, ATA58 36

Page 55: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Is the Robot Apocalypse near?�

Carola F. Berger, AI and Neural Nets, ATA58 37

https://motherboard.vice.com/en_us/article/jpdvjg/�the-real-threat-is-machine-incompetence-not-intelligence

Page 56: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Is the Robot Apocalypse near?�

Carola F. Berger, AI and Neural Nets, ATA58 38

Page 57: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Will We be Replaced by Robots?�Survey among 352 AI researchers:

Carola F. Berger, AI and Neural Nets, ATA58 39

K. Grace et al., When Will AI Exceed Human Performance? Evidence from AI Experts, �https://arxiv.org/abs/1705.08807

Page 58: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Will We be Replaced by Robots?�Survey among 352 AI researchers:

Carola F. Berger, AI and Neural Nets, ATA58 39

K. Grace et al., When Will AI Exceed Human Performance? Evidence from AI Experts, �https://arxiv.org/abs/1705.08807

Page 59: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

Will We be Replaced by Robots?�Survey among 352 AI researchers:

Carola F. Berger, AI and Neural Nets, ATA58 40

K. Grace et al., When Will AI Exceed Human Performance? Evidence from AI Experts, �https://arxiv.org/abs/1705.08807

Page 60: An Introduction to Artificial Intelligence, Machine …...Neural Nets - Recap ü Training = extraction of “features” (=patterns) from training data ü The more hidden layers and

References & Further Reading�  Slides at: http://www.CFBtranslations.com

  A. Turing, Computing machinery and intelligence, MIND: A Quarterly Review of Psychology and Philosophy, Vol. LIX, No.236, Oct. 1950, http://turingarchive.org/browse.php/B/19

  A. Ng, Machine Learning, Coursera, https://www.coursera.org/learn/machine-learning

  S. Russell, P. Norvig, Artificial Intelligence: A Modern Approach, 3rd Ed., Prentice Hall, 2009.

Carola F. Berger, AI and Neural Nets, ATA58 41