-
Data Mining and Neural Networks
Danny LeungCS157B, Spring 2006Professor Sin-Min Lee
-
Artificial Intelligence for Data MiningNeural networks are
useful for data mining and decision-support applications.
People are good at generalizing from experience.
Computers excel at following explicit instructions over and
over.
Neural networks bridge this gap by modeling, on a computer, the
neural behavior of human brains.
-
Neural Network Characteristics
Neural networks are useful for pattern recognition or data
classification, through a learning process.
Neural networks simulate biological systems, where learning
involves adjustments to the synaptic connections between
neurons
-
Anatomy of a Neural Network
Neural Networks map a set of input-nodes to a set of
output-nodes
Number of inputs/outputs is variable
The Network itself is composed of an arbitrary number of nodes
with an arbitrary topology
Neural Network
Input 0
Input 1
Input n
...
Output 0
Output 1
Output m
...
-
Biological BackgroundA neuron: many-inputs / one-output unit
Output can be excited or not excited
Incoming signals from other neurons determine if the neuron
shall excite ("fire")
Output subject to attenuation in the synapses, which are
junction parts of the neuron
-
Basics of a NodeA node is an element which performs a
function
y = fH((wixi) + Wb)ConnectionNode
fH(x)
Input 0
Input 1
Input n
...
W0
W1
Wn
+
Output
+
...
Wb
-
A Simple PreceptronBinary logic application
fH(x) [linear threshold]
Wi = random(-1,1)
Y = u(W0X0 + W1X1 + Wb)
fH(x)
Input 0
Input 1
W0
W1
+
Output
Wb
-
Preceptron TrainingIts a single-unit network
Adjust weights based on a how well the current weights match an
objective
Perceptron Learning Rule
Wi = * (D-Y).Ii
= Learning RateD = Desired Output
-
Neural Network LearningFrom experience: examples / training
data
Strength of connection between the neurons is stored as a
weight-value for the specific connection
Learning the solution to a problem = changing the connection
weights
-
Neural Network LearningContinuous Learning Process
Evaluate output
Adapt weights
Take new inputs
Learning causes stable state of the weights
-
Learning PerformanceSupervisedNeed to be trained ahead of time
with lots of data
Unsupervised networks adapt to the inputApplications in
Clustering and reducing dimensionalityLearning may be very slowNo
help from the outsideNo training data, no information available on
the desired outputLearning by doingUsed to pick out structure in
the input:ClusteringCompression
-
Topologies Back-Propogated NetworksInputs are put through a
Hidden Layer before the output layer
All nodes connected between layers
Input 0
Input 1
Input n
...
Output 0
Output 1
Output o
...
O0
O1
Oo
H0
H1
Hm
...
...
Hidden Layer
-
BP Network Supervised Training
Desired output of the training examples
Error = difference between actual & desired output
Change weight relative to error size
Calculate output layer error , then propagate back to previous
layer
Hidden weights updated
Improved performance
-
Neural Network Topology Characteristics
Set of inputs
Set of hidden nodes
Set of outputs
Increasing nodes makes network more difficult to train
-
Applications of Neural NetworksPrediction weather, stocks,
disease
Classification financial risk assessment, image processing
Data association Text Recognition (OCR)
Data conceptualization Customer purchasing habits
Filtering Normalizing telephone signals (static)
-
OverviewAdvantagesAdapt to unknown situationsRobustness: fault
tolerance due to network redundancyAutonomous learning and
generalization
DisadvantagesNot exactLarge complexity of the network
structure
-
Referenced WorkIntro to Neural Networks - Computer Vision
Applications and Training Techniques. Doug Gray.
www.soe.ucsc.edu/~taoswap/ GroupMeeting/NN_Doug_2004_12_1.ppt
Introduction to Artificial Neural Networks. Nicolas Galoppo von
Borries. www.cs.unc.edu/~nico/courses/
comp290-58/nn-presentation/ann-intro.ppt
