Artificial Intelligence CIS 342

Artificial Intelligence

CIS 342

The College of Saint RoseDavid Goldschmidt, Ph.D.

Machine learning involves adaptive mechanisms that enable computers to:– Learn from experience– Learn by example– Learn by analogy

Learning capabilities improve the performanceof intelligent systems over time

Machine Learning

How do brains work?– How do human brains differ from that

of other animals?

Can we base models ofartificial intelligence onthe structure and innerworkings of the brain?

The Brain

The human brain consists of:– Approximately 10 billion neurons – …and 60 trillion connections

The brain is a highly complex, nonlinear,parallel information-processing system– By firing neurons simultaneously, the brain

performs faster than the fastest computers in existence today

The Brain

Soma Soma

Synapse

Dendrites

Synapse

Dendrites

Building blocks of the human brain:

The Brain

An individual neuron has a very simple structure– Cell body is called a soma– Small connective fibers are called dendrites– Single long fibers are called axons

An army of such elements constitutes tremendous processing power

The Brain

An artificial neural network consists of a numberof very simple processors called neurons

– Neurons are connectedby weighted links

– The links pass signals fromone neuron to another basedon predefined thresholds

Artificial Neural Networks

An individual neuron (McCulloch & Pitts, 1943):– Computes the weighted sum of the input

signals – Compares the result with a threshold value,

– If the net input is less than the threshold,

the neuron output is –1 (or 0)– Otherwise, the neuron becomes activated

and its output is +1

Artificial Neural Networks

Neuron Y

InputSignals

OutputSignals

Weights

X = x1w1 + x2w2 + ... + xnwn

threshold

Individual neurons adhere to an activation function, which determines whether they propagate their signal (i.e. activate) or not:

Sign Function

Activation Functions

iiiwxX

Step function Sign function

Sigmoid function

Linear function

XYstep

XYsign X

sigmoid

1XYlinear

hard limit functions

The step, sign, and sigmoid activation functionsare also often called hard limit functions

We use such functions indecision-making neural networks– Support classification and

other pattern recognition tasks

Write functions or methods for theactivation functions on the previous slide

Can an individual neuron learn?– In 1958, Frank Rosenblatt introduced a

training algorithm that provided thefirst procedure for training asingle-node neural network

– Rosenblatt’s perceptron model consistsof a single neuron with adjustablesynaptic weights, followed by a hard limiter

Perceptrons

Threshold

Inputs

Output

HardLimiter

LinearCombiner

X = x1w1 + x2w2

Y = Ystep

Write code for a single two-input neuron – (see below)

Set w1, w2, and Θ through trial and errorto obtain a logical AND of inputs x1 and x2

A perceptron:– Classifies inputs x1, x2, ..., xn

into one of two distinctclasses A1 and A2

– Forms a linearly separablefunction defined by:

Perceptrons

Class A2

Class A1

x1w1 +x2w2 =0

(a) Two-inputperceptron. (b) Three-inputperceptron.

x3x1w1 +x2w2 +x3w3 =0

Perceptron with threeinputs x1, x2, and x3 classifies its inputsinto two distinctsets A1 and A2

Perceptrons

Class A2

Class A1

x1w1 +x2w2 =0

(a) Two-inputperceptron. (b) Three-inputperceptron.

x3x1w1 +x2w2 +x3w3 =0

How does a perceptron learn?– A perceptron has initial (often random)

weights typically in the range [-0.5, 0.5]– Apply an established training dataset – Calculate the error as

expected output minus actual output:

error e = Yexpected – Yactual

– Adjust the weights to reduce the error

Perceptrons

How do we adjust a perceptron’sweights to produce Yexpected?– If e is positive, we need to increase Yactual

(and vice versa)

– Use this formula:, where

α is the learning rate (between 0 and 1) e is the calculated error

Perceptrons

wi = wi + Δwi Δwi = α x xi x e

Train a perceptron to recognize logical AND

Perceptron Example – AND

Use threshold Θ = 0.2 andlearning rate α = 0.1

Inputs

EpochDesiredoutputYd

Initialweights

0.30.30.30.2

0.1 0.1 0.1 0.1

Actualoutput

Finalweights

w1 w20.30.30.20.3

0.1 0.1 0.10.0

0.30.30.30.2

0.30.30.20.2

0.00.00.00.0

0.20.20.20.1

0.00.00.00.0

0.20.20.10.2

0.00.00.00.1

0.20.20.20.1

0.10.10.10.1

0.20.20.10.1

0.10.10.10.1

Threshold: =0.2;learningrate: =0.1

Inputs

Initialweights

0.30.30.30.2

0.1 0.1 0.1 0.1

Actualoutput

Finalweights

w1 w20.30.30.20.3

0.1 0.1 0.10.0

0.30.30.30.2

0.30.30.20.2

0.00.00.00.0

0.20.20.20.1

0.00.00.00.0

0.20.20.10.2

0.00.00.00.1

0.20.20.20.1

0.10.10.10.1

0.20.20.10.1

0.10.10.10.1

Inputs

Initialweights

0.30.30.30.2

0.1 0.1 0.1 0.1

Actualoutput

Finalweights

w1 w20.30.30.20.3

0.1 0.1 0.10.0

0.30.30.30.2

0.30.30.20.2

0.00.00.00.0

0.20.20.20.1

0.00.00.00.0

0.20.20.10.2

0.00.00.00.1

0.20.20.20.1

0.10.10.10.1

0.20.20.10.1

0.10.10.10.1

Inputs

Initialweights

0.30.30.30.2

0.1 0.1 0.1 0.1

Actualoutput

Finalweights

w1 w20.30.30.20.3

0.1 0.1 0.10.0

0.30.30.30.2

0.30.30.20.2

0.00.00.00.0

0.20.20.20.1

0.00.00.00.0

0.20.20.10.2

0.00.00.00.1

0.20.20.20.1

0.10.10.10.1

0.20.20.10.1

0.10.10.10.1

Train a perceptron to recognize logical AND

Inputs

Initialweights

0.30.30.30.2

0.1 0.1 0.1 0.1

Actualoutput

Finalweights

w1 w20.30.30.20.3

0.1 0.1 0.10.0

0.30.30.30.2

0.30.30.20.2

0.00.00.00.0

0.20.20.20.1

0.00.00.00.0

0.20.20.10.2

0.00.00.00.1

0.20.20.20.1

0.10.10.10.1

0.20.20.10.1

0.10.10.10.1

Inputs

Initialweights

0.30.30.30.2

0.1 0.1 0.1 0.1

Actualoutput

Finalweights

w1 w20.30.30.20.3

0.1 0.1 0.10.0

0.30.30.30.2

0.30.30.20.2

0.00.00.00.0

0.20.20.20.1

0.00.00.00.0

0.20.20.10.2

0.00.00.00.1

0.20.20.20.1

0.10.10.10.1

0.20.20.10.1

0.10.10.10.1

Repeat until convergence– i.e. final weights do not change and no

Two-dimensional plotof logical AND operation:

A single perceptron canbe trained to recognizeany linear separable function – Can we train a perceptron to

recognize logical OR?– How about logical exclusive-OR (i.e. XOR)?

(b) OR (x1 x2)

(c) Exclusive-OR(x1 x2)

Two-dimensional plots of logical OR and XOR:

Perceptron – OR and XOR

(b) OR (x1 x2)

(c) Exclusive-OR(x1 x2)

Modify your code to:– Calculate the error at each step– Modify weights, if necessary

i.e. if error is non-zero

– Loop until all error values are zero for a full epoch

Modify your code to learn to recognize the logical OR operation– Try to recognize the XOR

operation....

Perceptron Coding Exercise

InputLayer OutputLayer

MiddleLayer

Multilayer neural networks consist of:– An input layer of source neurons– One or more hidden layers of

computational neurons– An output layer of more

computational neurons

Input signals are propagated in alayer-by-layer feedforward manner

Multilayer Neural Networks

InputLayer OutputLayer

MiddleLayer

Inputlayer

Firsthiddenlayer

Secondhiddenlayer

Outputlayer

Inputlayer

Outputlayer

Inputsignals

Error signals

Hiddenlayer

XINPUT = x1 XH = x1w11 + x2w21 + ... + xiwi1 + ... + xnwn1

XOUTPUT = yH1w11 + yH2w21 + ... + yHjwj1 + ... + yHmwm1

Inputlayer

Outputlayer

Hiddenlayer

Three-layer network:

Commercial-quality neural networks often incorporate 4 or more layers– Each layer consists of

about 10-1000 individual neurons

Experimental and research-based neural networks often use 5 or 6 (or more) layers– Overall, millions of individual neurons may

be used

A back-propagation neural network is a multilayer neural network that propagates error backwards through the network as it learns– Weights are modified based on the

calculated error

– Training is complete when the error isbelow a specified threshold

e.g. less than 0.001

Back-Propagation NNs

Inputlayer

Outputlayer

Inputsignals

Error signals

Hiddenlayer

Inputlayer

Outputlayer

Hiddenlayer

Write code for the three-layer neural network below

Use the sigmoid activation function; andapply Θ by connecting fixed input -1 to weight Θ

0 50 100 150 200

Sum-Squared Network Error for 224 Epochs

Start withrandom weights– Repeat until

the sum of thesquared errorsis below 0.001

– Depending oninitial weights,final convergedresults may vary

After 224 epochs (896 individual iterations),the neural network has been trained successfully:

Inputs

Desiredoutput

0.0155

Actualoutput

Sum ofsquarederrors

0.98490.98490.0175

0.0010

+0.5 2.0

No longer limited to linearly separable functions

Another solution:

– Isolate neuron 3, then neuron 4....

Combine linearly separable functions of neurons 3 and 4:

x1 +x2 – 1.5 =0 x1 +x2 – 0.5 =0

Handwriting recognition

Using Neural Networks

Inputlayer

Firsthiddenlayer

Secondhiddenlayer

Outputlayer

0100 => 4

0101 => 50110 => 60111 => 7 etc.

Advantages of neural networks:– Given a training dataset, neural networks

learn– Powerful classification and pattern

matching applications

Drawbacks of neural networks:– Solution is a “black box”– Computationally intensive

Using Neural Networks

Artificial Intelligence CIS 342

Documents

Determination of cis,cis- and cis,trans Muconic Acid from ... · 9/4/2019 · Determination of cis,cis- and cis,trans-Muconic Acid from Biological Conversion Laboratory Analytical

Full page photo2.wlimg.com/product_images/bc-full/dir_16/476101/...50 Nm to 12000 MANUAL OVERRIDE ... 342 342 342 342 342 342 z 1 go 190 1 go 1 go 1 go 1 go Mounting pCo F05, F07 F07,

RED DEAD REVOLVER Artificial Intelligence Critique By Mitchell C. Dodes CIS 588

70-342 - Pass4Sure : Premier IT Certification Training Portal · * Microsoft 70-342 brain dump free content featuring the real 70-342 test questions. Microsoft 70-342 certification

physics 342

Computing & Information Sciences Kansas State University Lecture 20 of 42 CIS 530 / 730 Artificial Intelligence Lecture 20 of 42 Introduction to Classical

IJCNN 2005 Tutorial - Biologically Plausible Artificial ...ewh.ieee.org/cmte/cis/mtsc/ieeecis/Joao_luis_Garcia_Rosa.pdfBiologically Plausible Artificial Neural Networks João Luís

Computing & Information Sciences Kansas State University Lecture 18 of 42 CIS 530 / 730 Artificial Intelligence Lecture 18 of 42 Knowledge Representation

Dragon Magazine 342

Computing & Information Sciences Kansas State University Lecture 32 of 42 CIS 530 / 730 Artificial Intelligence Lecture 32 of 42 Machine Learning: Basic

CIS 678 Artificial Intelligence problems deduction, reasoning knowledge representation planning learning natural language processing motion and manipulation

UNIVERSITY OF CYPRUS - ucy.ac.cy · PDF filecen cen cis cis cis cis cis cis cis cis cis cis cis cis cis cis cis cis 101 102 600 600 600 600 600 600 600 600 600 600 600 600 600 600

CIS 595 Bioinformatics Lecture 2 Based on the book chapter: Hunter, L., Molecular Biology for Computer Scientists. Artificial Intelligence for Molecular

-Queens | 342 referencesliacs.leidenuniv.nl/~kosterswa/nqueens/nqueens_feb2009.pdf · n-Queens | 342 references This paper currently (November 20, 2018) contains 342 references (originally

Naïve Bayes for Text Classification: Spam Detection CIS 391 – Introduction to Artificial Intelligence (adapted from slides by Massimo Poesio which were

Introduction to Python II CIS 391: Artificial Intelligence Fall, 2008

Computing & Information Sciences Kansas State University Lecture 38 of 42 CIS 530 / 730 Artificial Intelligence Lecture 38 of 42 Natural Language Processing,

Introduction to Python III CIS 391: Artificial Intelligence Fall, 2008

ISU Agron 342

Artificial Intelligence CIS 342