Welcome to BS6207

2021 Lee Hwee Kuan

1

2

An Innovative Way to Learn and Teach Deep Learning ✤ We all learn together - sharing sessions

✤ Students are expected to learn most techniques online by themselves

✤ What is the use of this course? ๏ This course provides concepts and understandings that cannot be easily

self-learned ๏ This course provides the real life experience for doing deep learning ๏ This course teaches you how to think in different ways

✤ Always bring your laptop, we do coding in class

✤ If you want to get my attention, call me by my first name “Hwee Kuan”, you can call me “Dr Lee” or “Prof Lee” if you want me to ignore you

3

Sharing sessions

If we categorize people into those with technical skills and those who can explain their ideas, there are 4 combinations.1.Those who have no skills & cannot explain their ideas2.Those who have skills & cannot explain their ideas3.Those who have no skills & can explain their ideas4.Those who have skills & can explain their ideas

4

Sharing sessions

1.Those who have no skill & cannot explain their ideas These people may be beggars

2.Those who have skills & cannot explain their ideas These people are good servants

3.Those who have no skill & can explain their ideas These people are the bosses

4.Those who have skills & can explain their ideas These people are the masters

5

Sharing sessions

There are 8 registered students.

We aim to have one in four students to present something in front of the class.

Presentations will be on: 1.Assignment solutions 2.Project solutions

Some presentation skill coaching will be provided by the instructors

6

I teach the whole package

✤ Communication skills✤ Teamwork✤ Decision making and scientific methods✤ Technical skills

How our world is going to change?

7

How our world is going to change?

Old New

Driving Work from home

Manufacturing Shop from home

Clerical work Self healthcare

Retail Self education

Mass education AI work

Maids & cleaners

8

How our world is going to change?

http://www.mrbrown.com/blog/2011/03/maid-for-the-army-2.html

Boston Dynamics 2017Storm 2011

9

What’s Deep Learning for?

10

Deep Learning related work in Robotics

https://www.youtube.com/watch?v=EP_NCB3KkiY https://www.youtube.com/watch?v=rVlhMGQgDkY https://www.youtube.com/watch?v=M8YjvHYbZ9w

Check out:Boston Dynamics

11

Deep Learning related work in Financial Technologies (Fintech)

“. . .fintech is on the verge of a truly revolutionary moment with the integration of artificial intelligence and deep learning into financial services.

http://fintechnews.sg/8840/fintech/deep-learning-finance-summit-comes-singapore-discuss-emerging-trends-opportunities/

Using machine learning for insurance pricing optimization

https://cloud.google.com/blog/big-data/2017/03/using-machine-learning-for-insurance-pricing-optimization

Much trading is done by machines especially high frequency trading

12

Deep Learning related work in Retail

Amazon Gono cashier, no queues shopping

https://venturebeat.com/2016/12/05/amazon-launches-amazon-go-a-brick-and-mortar-grocery-store-that-does-away-with-checkouts/

check this out!https://www.amazon.com/b?

node=1600858901113

“Legal Robot uses machine learning techniques like deep learning to understand legal language, . . . “

https://www.legalrobot.com/

Deep Learning related work in Legal

“. . . the UK-based news resource, LegalFutures, predicts that technologies automating the work of associates, herald the collapse of law in less than 15 years

A recent study by McKinsey & Co estimates that 23% of lawyer time is automatable. Similar research by Frank Levy at MIT and Dana Remus at University of North

Carolina School of Law concludes that just 13% of lawyer time can be performed by computers. . .”

https://blogs.thomsonreuters.com/answerson/artificial-intelligence-legal-practice/

14

Deep Learning related work in Manufacturing

Foxconn replaces '60,000 factory workers with robots'

http://www.bbc.com/news/technology-36376966

15https://www.youtube.com/watch?v=SNy6fEuPWbc

Deep Learning related work in Farming

Farmers are getting help from researchers and scientists who have turned the keen eye of AI toward agriculture, using deep learning

applications to not only predict crop outputs but also to monitor water levels around the world and help detect crop diseases before one

spreads.

https://dataskeptic.com/blog/news/2017/deep-learning-is-driving-the-new-agriculture-revolution

16

Deep Learning related work in Biomedical

Deep Learning Drops Error Rate for Breast Cancer Diagnoses by 85% https://blogs.nvidia.com/blog/2016/09/19/deep-learning-breast-cancer-diagnosis/

Google researchers trained an algorithm to recognize a common form of eye disease as well as many experts can.

https://www.technologyreview.com/s/602958/an-ai-ophthalmologist-shows-how-machine-learning-may-transform-medicine/

Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis

G. Litjens et al Scientific Reports 6, 26286 (2016)

Dermatologist-level classification of skin cancer with deep neural networksA. Esteva Nature 542, 115-118 (2017)

17

The Next Era: Deep Learning in Pharmaceutical Research.

S. Ekins

Pharm Res. 2016 Nov;33(11):2594-603. doi: 10.1007/s11095-016-2029-7. Epub 2016 Sep 6.

Deep Learning related work in Pharmaceutical

18

Many other Deep Learning related work

Smart City Transportation

Aviation Telecommunications

Construction Education

. . . think of anything XXX, then search for

“deep learning XXX”

19

Historical notes on

neural networks and deep learning

20

1943Warren McCulloch (neurophysiologist), Walter Pitts (mathematician) Mathematical model of the brain McCulloch, Warren; Walter Pitts (1943). "A Logical Calculus of Ideas Immanent in Nervous Activity". Bulletin of Mathematical Biophysics. 5 (4): 115–133.

1949 Donald O. Hebb Strengthening of connection between neurons Hebb, D. O. (1949). The Organization of Behavior: A Neuropsychological Theory. New York: Wiley and Sons.

1959Bernard Widrow, Marcian Hoff Single layer and multilayer neural nets. ADALINE and MADALINE An adaptive "ADALINE" neuron using chemical "memistors"

1970Seppo Linnainmaa While gradient descend algorithm dates back much earlier, Seppo contributed to the modern idea of back propagation Linnainmaa, Seppo (1970). The representation of the cumulative rounding error of an algorithm as a Taylor expansion of the local rounding errors. Master's Thesis (in Finnish), Univ. Helsinki, 6-7.

1989George Cybenko Universal approximation theorem, sigmoid function Cybenko, G. (1989) "Approximations by superpositions of sigmoidal functions", Mathematics of Control, Signals, and Systems, 2 (4), 303-314

1991 Kurt Hornik Universal approximation theorem, more general function Kurt Hornik (1991) "Approximation Capabilities of Multilayer Feedforward Networks", Neural Networks, 4(2), 251–257

https://cs.stanford.edu/people/eroberts/courses/soco/projects/neural-networks/History/history1.html

Historical notes

21

‘Contemporary’ history of neural nets

1974 Paul Werbos, Backpropagation

1980 Kunihiko Fukushima, Neocogitron which inspired Convolutional Neural Networks

1985 Hilton & Sejnowski, Boltzmann Machine

1986 Paul Smolensky, Harmonium, later known as Restricted Boltzmann Machine Michael I. Jordan Recurrent Neural Network

1990 Yann LeCun, LeNet - convolutional neural net

2006 G. Hinton, Deep Belieft Net, layer wise pretraining

2009 Salakhutdinov & Hinton, Deep Boltzmann Machines

2012 N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdin, Dropout

22

‘Contemporary’ history of neural nets

2014 Ian Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative Adversarial Networks

2015 Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, Deep Residual Network

2015 Nicolas Papernot et al, Adversarial Deep Learning

2016 Shaoqing Ren et al, Region Proposal Network

2017 David Silver et al, Alpha-Go Zero

2018 SMA Eslami et al, Generative Query Network

23

https://www.youtube.com/watch?v=RBJFngN33Qo

24

Generative Query Networks

23684184 x 4729472 =Our world versus computer world

112013685070848

25

Our world versus computer world

Life Science

Healthcare

Relationship

26

What makes Deep Leaning so good?

27

can this thing think?

28

can they think?

29

can they think?

30

31

http://img.webmd.com/dtmcms/live/webmd/consumer_assets/site_images/articles/image_article_collections/anatomy_pages/

brain2.jpg?resize=646px:*&output-quality=100

can it think?

32

can this thing think?

w1

w2

w3 a = w1 x1 + w2 x2 + w3 x3

y = a if a > 0, else y = 0x3

33

With millions of connections, it started to “think”

34

How do we get from molecules (proteins) to celland then to life?

proteins

35

How do we get from molecules (proteins) to celland then to life?

http://learn.genetics.utah.edu/content/stemcells/quickref/somaticstemcells.jpg

proteins

36

https://www.pinterest.com/sattele/termite-mound/ http://img.ev.mu/images/reportages/188/520x342/09.jpg

Life exhibits complexity

37

How extreme can life become?How extreme can emergent behaviour become?

very extreme indeed

some life forms on earth evolved the ability to control the

behaviour of another life form!

38

Ophiocordyceps sinensis冬⾍草

S$10-50 per piece

39

http://microbewiki.kenyon.edu/index.php/Ophiocordyceps_sinensis

40

The foraging carpenter ant walks through an area of rainforest floor infested with microscopic spores dropped by a mature fungus. The spore excretes an enzyme that eats through the ant’s exterior shell.

After two days, the ant leaves its tree colony and climbs down to a spot where humidity and temperature are optimal for the fungus to grow. The ant crawls onto a stem or the underside of a leaf and bites into its main middle vein so it won’t fall. Then it dies.

The fungus consumes the ant’s internal organs, using its shell as a protective casing. The fungus’ main stem, called a stroma, erupts from the back of the ant’s head and grows

The mature fungus releases spores from its stroma. The spores fall to the ground creating a 10-square-feet “killing zone” which will attack new ants.

https://microbewiki.kenyon.edu/index.php/Ophiocordyceps_unilateralis

41

42

https://www.youtube.com/watch?v=XuKjBIBBAL8

Cordyceps

How extreme can machines become?

are we able to make a machine that is as complex as the most

primitive life known to us?

43

How to learn Deep Learning

44

45

If you only know how to drive, you can go far

If you have money and get a sport car, you can go far very fast

You don’t have to know how to build a car

46

“Deep Learning” is free and they go fast

using them is as easy as driving a car

47

But there is a problem!!

…even if you have the data!

48

To go to great places, you need to know where to go

No point going fast but go in circles you need to know the route

You need to know if your car is ok or is breaking

The computer always give you

an outputis it correct?

49

Different levels of understanding Deep Learning

There are those who do not know what they are doing. Their computational results are unreasonable

There are those who know how to get some good results but cannot explain them

There are those who understand what is going on with their experiments. Able to explain their results

There are those who can combine different methods to create new things in Deep Learning

There are those who can fundamentally change Deep Learning research

50

Very basic

lets get everyone on the same level sorry if this seems too simple to some of you

51

input output

fuzzy incomplete

input

fuzzy incomplete

output

only precise input

011000111001100. . .

only precise output

010

Machine

Human

Human

52

53

Lets play a game. . .

54

i like this picture

i like this picture

i like this picturei like this picture

i don’t like this picture

i don’t like this picture

i like this picture

i don’t like this picture i don’t like this picture

i like this picture

i don’t like this picture

like or dislike?(a)

like or dislike?(b)

like or dislike?(c)

like or dislike?(d)

Testing set

54

55

Like

Dislike

Training set

like or dislike?(a)

like or dislike?(b)

like or dislike?(c)

like or dislike?(d)

Testing set

56

Like

Dislike

Training set

Answer: Like(a)

Answer: Dislike(b)

Answer: Dislike(c)

Answer: Like(d)

Testing set

57

Age Prostate Specific Antigen(a blood test reading) Got Prostate Cancer

59 4.9 ng/mL Yes72 3.9 ng/mL Yes45 6.0 ng/mL Yes47 3.2 ng/mL No39 3.9 ng/mL No89 3.5 ng/mL Yes61 5.5 ng/mL Yes62 2.1 ng/mL No49 3.4 ng/mL No95 3.1 ng/mL Yes67 4.3 ng/mL Yes49 3.8 ng/mL ?58 4.3 ng/mL ?88 4.1 ng/mL ?31 2.1 ng/mL ?

Case of prostate cancer diagnosis

58

Age Prostate Specific Antigen(a blood test reading) Got Prostate Cancer

45 6.0 ng/mL Yes61 5.5 ng/mL Yes59 4.9 ng/mL Yes67 4.3 ng/mL Yes72 3.9 ng/mL Yes89 3.5 ng/mL Yes95 3.1 ng/mL Yes

39 3.9 ng/mL No49 3.4 ng/mL No47 3.2 ng/mL No62 2.1 ng/mL No49 3.8 ng/mL ?58 4.3 ng/mL ?88 4.1 ng/mL ?31 2.1 ng/mL ?

59

Age Prostate Specific Antigen(a blood test reading) Got Prostate Cancer

45 6.0 ng/mL Yes61 5.5 ng/mL Yes59 4.9 ng/mL Yes67 4.3 ng/mL Yes72 3.9 ng/mL Yes89 3.5 ng/mL Yes95 3.1 ng/mL Yes

39 3.9 ng/mL No49 3.4 ng/mL No47 3.2 ng/mL No62 2.1 ng/mL No49 3.8 ng/mL No?58 4.3 ng/mL borderline?88 4.1 ng/mL Yes?31 2.1 ng/mL No?

prostate specific antigen (ng/mL)

age

65

45

85

4.0 4.53.53.0

Prostate cancer prediction

prostate specific antigen (ng/mL)

age

65

45

85

4.0 4.53.53.0

Prostate cancer prediction

Prostate cancerHealthy

prostate specific antigen (ng/mL)

age

65

45

85

4.0 4.53.53.0

Prostate cancer prediction

Prostate cancerHealthy

prostate specific antigen (ng/mL)

age

65

45

85

4.0 4.53.53.0

Prostate cancer prediction

Prostate cancerHealthy

?

prostate specific antigen (ng/mL)

age

65

45

85

4.0 4.53.53.0

Prostate cancer prediction

Prostate cancerHealthy ?

?

prostate specific antigen (ng/mL)

age

65

45

85

4.0 4.53.53.0

Prostate cancer prediction

Prostate cancerHealthy

Decision Boundaries

Supervised learning framework input space image,

blood pressure, text,

speech

input instance (x1,x2,. . .)

66

black / white labels on

input

Supervised learning framework

67

input: a point

in input space

output: black

or white

Machine

input space image,

blood pressure, text,

speech

Supervised learning framework

68

The decision boundary

69

The decision boundary

70

The decision boundary

?

?

?

?

71

The decision boundary

72

Animation @ playground search for

“playground tensorflow"

introduction

73

74

Different functionalities of applet

questionx1

x2

o

75

answer

76

questionx1

x2

o

77

answer

78

question

79

answer

80

question

81

answer

82

answer

83

question

84

answer

85

answer

86

answer

87

answer - with sigmoid

88

question

89

Administrative Issues

90

91

An Innovative Way to Learn and Teach Deep Learning ✤ We all learn together

✤ Students are expected to learn most techniques online by themselves

✤ Always bring your laptop, we do coding in class - a part of the time for lectures, the rest of the time for experiential learning

Communications mattersUse whatsapp for communications. This is the fastest way to get response.

For private matters you can privately WhatsApp me.

Get course materials from: https://web.bii.a-star.edu.sg/~leehk/index.html

I need one class monitor, who volunteer?

How do we want to do lecture? 2 hours on Fridays and 1 hr office 12-1pm on Wednesdays

Class monitor will set up a WhatsApp group for the whole class, including me in the group

92

General grading and expectationsThere are those who do not know what they are doing. Their results are unreasonable

There are those who know how to get some good results but cannot explain them.

There are those who understand what is going on with their experiments. Able to explain their results.

There are those who know enough to combine different methods and be creative in using Deep Learning

There are those who break the frontiers of Deep Learning research.

F/E

D/C

B

A

93

Assignments (10% * 5 = 50%)• For each assignment • It has multiple sub-tasks, including coding tasks • You need to submit the answers, code and execution results. Everything

in git hub, and one copy in NTULearn. • All assignment report must have: • Algorithm description in plain text, supplemented by equations if

needed (3-4 marks) • Code walk through, part-by-part of the code need to be explained

(2-3 marks) • Plots and results (1-2 marks) • Explanation and interpretation of results (3-4 marks)

94

Final Project (50%)• Projects will be

• Submission deadline: pre-submission around week 7, final submission around week 10, to be confirmed

• Report deadline: around week 11, to be confirmed

• Assessment is based on the report (40%) and presentation (10%) • All students will present their work in class

• One student per project submission. You are allowed discuss project solutions among students.

• No copying of source code, code everything yourself!

95

Final Project (50%)

Project details will be announced next week.

96

Report format: Strictly limited to 6 pages (figures and text font size >10) + 1 page reference, any longer report will be rejected

Report template (40%)

• Problem definition

• Highlights (new algorithms, insights from the experiments)

• Dataset pre-processing description

• Training and testing procedure

• Experimental study

• (clarity, model understanding, and highlights are important for the assessment)

• Presentation -> score (10%)

Start your project early 97

*Do not* miss the deadline

30% grade deductions for within 1 week late submissions60% grade deductions for within 2 weeks late submissions

no grade given for beyond 2 weeks late submissions

98

To be fair to everyone, request for late submission without grade deductions for emergency cases are to be done before week 9, otherwise request will not be entertained