Section 2: On-line Learning

Section 2: On-line Learning

Mini-course on ANN and BN, The Multidisciplinary Brain Research center, Bar-Ilan University, May 2004

Based on slides from Michael Biehl’s summer course

קורס מרוכז תחומי הרב ברשתות המרכז לחקר המוח אוניברסיטת בר־ אילן תשס״ד

Section 2.1: The Perceptron



The Perceptron

Input:

Adaptive Weights JJ

Output: SMini-course on ANN and BN, The Multidisciplinary Brain Research center, Bar-Ilan University, May 2004


Perceptron: binary output

Implements a linearly separable classification of inputs

Milestones:Perceptron convergence theorem, Rosenblatt

(1958)Capacity, winder (1963) Cover(1965)Statistical Physics of perceptron weights,

Gardner (1988)

How does this device learn?


W


Learning a linearly separable rule from reliable examples

Unknown rule: ST()=sign(BB) =±1

Defines the correct classification.

Parameterized through a teacher perceptron with weights BBRN, (BBBB=1)

Only available information: example data

D= { , ST()=sign(BB) for =1…P }



Learning a linearly… (Cont.)

Training: finding the student weights JJ– J J parameterizes a hypothesis SS()=sign(JJ) – Supervised learning is based on the student

performance with respect to the training data DD– Binary error measure

T(JJ)= [S

S(),ST()]

T(JJ)=0 if S

S()ST()

T(WW)=1 if SS()=S

T()



Off-line learning

Guided by the minimization of a cost function H(JJ), e.g., the training error

H(JJ) tT(JJ)

Equilibrium statistical mechanics treatment:– Energy H of N degrees of freedm– Ensemble of systems is in thermal equilibrium at formal

temperature– Disorder avg. over random examples (replicas) assumes

distribution over the inputs– Macroscopic description, order parameters– Typical properties of large sustems, P= N



On-line training

Single presentation of uncorrelated (new) {,S

T()} Update of student weights:

Learning dynamics in discrete time



On-line training - Statistical Physics approach

Consider sequence of independent, random Thermodynamic limit Disorder average over latest example self-

averaging properties Continuous time limit



Generalization

Performance of the student (after training) with respect to arbitrary, new input

In practice: empirical mean of mean error measure over a set of test inputs

In the theoretical analysis: average over the (assumed) probability density of inputs

Generalization error:



Generalization (cont.)

The simplest model distribution:

Isotropic density P(), uncorrelated with B B and JJ

Consider vectors of independent identically distributed (iid) components jj with



Geometric argument

Projection of data into (BB, JJ)-plane yields isotropic density of inputs


BBJJ

ST()=SS()

g=/

For |BB|=1


Overlap Parameters

Sufficient to quantify the success of learning

R=BBW W Q=JJJ J

Random guessing R=0, g=1/2

Perfect generalization , g=0



Derivation for large N

Given BB, JJ, and uncorrelated random input i=0, i j =ij, consider student/teacher fields that are sums of (many) independent random quantities:

x=JJ=∑iJiI

y=BB=∑iBii



Central Limit Theorem

Joint density of (x,y) is for N→∞, a two dimensional Gaussian, fully specified by the first and the second moments

x=∑iJii=0 y=∑iBii=0

x2 = ∑ijJiJjij = ∑iJi2 = Q

y2 = ∑ijBiBjij = ∑iBi2 = 1

xy = ∑ijJiBjij = ∑iJiBi = R



Central Limit Theorem (Cont.)

Details of the input are irrelevant.

Some possible examples: binary, i1, with equal prob. Uniform, Gaussian.



Generalization Error

The isotropic distribution is also assumed to describe the statistics of the example data inputs



Exercise: Derive the generalization error as a function of R,Q use Mathematical notes

Assumptions about the data

No spatial correlatins No distinguished directions in the input space No temporal correlations No correlations with the rule Single presentation without repeatitionsConsequences: Average over data can be performed step by step Actual choice of B B is irrelevant, it is not necessary to

averaged over the teacher



Hebbian learning (revisited) Hebb 1949

Off-line interpretation Vallet 1989

Choice of student weights given D={,ST}=1

P

JJ(P) = ∑ST/N

Equivalent On-line interpretation

Dynamics upon single presentation of examples

JJ() = JJ(-1) + ST/N



Hebb: on-line

From microscopic to macroscopic: recursions for overlaps



Exercise: Derive the update equations of R,Q

Hebb: on-line (Cont.)

Average over the latest example …

The random input, enters only through the fields

The random input and JJ(-1), BB are statistically independent

The Central Limit Theorems applies and obtains the joint density






Exercise: Derive the update equations of R,Q as a function of use Mathematical notes


Continuous time limit, N→∞, = /N, d=1/N



Initial conditions - tabula rasa R(0)=Q(0)=0

What are the mean values after training with N examples???

[See matlab code]

Hebb: on-line mean values

Self average properties of A(JJ):– The width of the distribution vanishes– The observation of a value of A different from its mean occurs with

vanishing probability The order parameters, Q and R, are self averaging for infinite N



Learning Curve: dependent of the order parameters



The normalized overlap between the two vectors, BB, J J provides the angle between the vectors two vectors

Learning Curve: dependent of the order parameters



Asymptotic expansion [draw w. matlab]



Modified Hebbian learning

The training algorithm is defined by a modulation function f

JJ() = JJ(-1) +f(…) ST/N

Restriction: f may depend on available quantities: f(JJ(-1),,S

T)



Questions:

Is the perceptron algorithm Rosenblatt 1959, that learns only when there is a mistake performs better than the Hebb algorithm?

What training algorithm will provide the best learning/ the fastest asymptotic decrease?

Is it possible to achieve an asymptotic behavior, on-line?



Documents

Section 2: On-line Learning