TutorialOnNeuralModelingSystems_2

7/28/2019 TutorialOnNeuralModelingSystems_2

1/10

Sinauer Associates Inc. Publishers

Sunderland, Massachusetts U.S.A.

Tutorial on

Neural Systems Modeling

Ta J. Aata

Sinauer Associates, I nc. This material cannot be copied, reproduced, manufactured

or disseminated in any form wi thout express wri tten permission from the publisher.


2/10

CHapter 1

Vcos, Mics, nd Bsic Nul Comuions 1

CHapter 2

rcun Conncions nd Siml Nul Cicuis 27

CHapter 3

Fowd nd rcun Ll Inhibiion 65

CHapter 4

Coviion Lning nd auo-associiv Mmoy 97

CHapter 5

Unsuvisd Lning nd Disibud rsnions 135

CHapter 6

Suvisd Lning nd Non-Uniom rsnions 171

CHapter 7

rinocmn Lning nd associiv Condiioning 213

CHapter 8

Inomion tnsmission nd Unsuvisd Lning 251

CHapter 9

pobbiliy esimion nd Suvisd Lning 299

CHapter 10

tim Sis Lning nd Nonlin Signl pocssing 341CHapter 11

tmol-Dinc Lning nd rwd pdicion 387

CHapter 12

pdicoCoco Modls nd pobbilisic Innc 423

CHapter 13

Simuld evoluion nd h Gnic algoihm 481

CHapter 14

Fuu Dicions in Nul Sysms Modling 515

Brif C




3/10

C

1.1 Nul Sysms, Nul Nwoks, ndBin Funcion 2

1.2 Using MatLaB: th Mi Lbooypogmming envionmn 3

1.3 Imiing h Soluion by Guss o hBusywok poblm 4

1.4 Oion nd Hbiuion o h Gill-Wihdwl r o Aplysia 7

MatLaB BOx 1.1 T t t a at atat t Aplysiag-taa 12

1.5 th Dynmics o Singl Nul Uni wihposiiv Fdbck 13

MatLaB BOx 1.2 T t at t t a g t tv

ak 15

MatH BOx 1.1 GeomeTric decAy o The discreTepulse response 16

1.6 Nul Nwoks: Nul Sysms wihMulil Inconncd Unis 18

MatH BOx 1.2 VecTor And mATrixmulTiplicATion: Across The row And down

The column 19

MatLaB BOx 1.3 T t at t t a a tk t t t a

t tt t tat a a t 23

Exercises 24

References 25

CHapter 1 Vcos, Mics, nd Bsic Nul Comuions 1




4/10

viii Contents

3.1 Simuling edg Dcion in h ely Visul

Sysm o Limulus 69

MatLaB BOx 3.1 T t ak a tvtat aatg t v a tvt

f 72

MatH BOx 3.1 iniTe (discreTe) dierenceApproximATions T0 conTinuous deriVATiVes 74

MatH BOx 3.2 The mArr ilTer 79

MatH BOx 3.3 one- And Two-dimensionAl GAboruncTions 80

3.2 Simuling Cn/Suound rciv FildsUsing h Dinc o Gussins 81

MatLaB BOx 3.2 T t ak a Gaa

tvt f 81

3.3 Simuling aciviy Bubbls nd Sbl pnFomion 85

MatH BOx 3.4 summATion noTATion 86

MatLaB BOx 3.3 T t t t -tak-a tk tat a t, ata t ata tat t a a t 87

3.4 Sing Signls om Nois nd Modlingtg Slcion in h Suio Colliculus 89

Exercises 94

References 95

CHapter 3 Fowd nd rcun Ll Inhibiion 65

MatH BOx 2.1 The GeomeTric decAy is The discreTe-Time AnAloG o The exponenTiAl decAy 29

MatH BOx 2.2 discreTe ApproximATion o AconTinuous dierenTiAl equATion model o

A sinGle neuron 30

2.1 th Dynmics o two Nul Unis wih Fdbckin Sis 30

MatLaB BOx 2.1 T t t a avgt t , a t t, tat -

t ag t t t t tv ak

tv (t t t a ak tgat) 32

MatH BOx 2.3 discreTe ApproximATion o A sysTemo Two coupled, conTinuous dierenTiAl equATions

ThAT model Two inTerconnecTed neurons 34

MatH BOx 2.4 eiGenmode AnAlysis o discreTe,lineAr dynAmic sysTems 35

2.2 Signl pocssing in h Vsibulo-Ocul r(VOr) 38

2.3 th plll-phwy Modl o Vlociy Sogin h pim VOr 40

MatLaB BOx 2.2 T t t t aa-ata a tv-ak vt tag,a t gatv-ak vt akag 42

2.4 th posiiv-Fdbck Modl o Vlociy Sog

in h pim VOr 44

2.5 th Ngiv-Fdbck Modl o VlociyLkg in h pigon VOr 46

2.6 Oculomoo Nul Ingion vi rcioclInhibiion 48

MatLaB BOx 2.3 T t t t t-t t tgat t t t

2.7 Simuling h Insc-Fligh Cnl pnGno 53

MatLaB BOx 2.4 T t t a a v w t t-gt ta att

gat 56

Exercises 60

References 61

CHapter 2 rcun Conncions nd Siml Nul Cicuis 27




5/10

Contents ix

CHapter 4 Coviion Lning nd auo-associiv Mmoy 97

MatH BOx 5.1 VecTor norms, normAlizATion, Andinner producTs reVisiTed 141

5.1 Lning hough Comiion o Sciliz oScifc Inus 142

MatLaB BOx 5.1 T t t t K-gag a (som) agt 143

5.2 tining Fw Ouu Nuons o rsn MnyInu pns 145

MatLaB BOx 5.2 T t a t va somtt t t t att a ak-a-

t ag 146

5.3 Simuling h Fomion o Bin Ms using

Cooiv Mchnisms 148

5.4 Modling h Fomion o tonooic Ms in haudioy Sysm 153

MatLaB BOx 5.3 T t t K somagt t at t at a tta 156

5.5 Simuling h Dvlomn o OinionSlciviy in Visul Co 158

5.6 Modling possibl Mulisnsoy M in hSuio Colliculus 164

Exercises 167

References 168

CHapter 5 Unsuvisd Lning nd Disibud rsnions 135

MatH BOx 4.1 coVAriATion And iTs relATionshipwiTh coVAriAnce 100

4.1 th Fou Hbbin Lning ruls o NulNwoks 101

MatLaB BOx 4.1 T t t t tt t ak t gt at t

at-aatv tk g t h, t-at,

-at, a hf (vaat) 104

MatH BOx 4.2 The ouTer producT o TwoVecTors 106

MatH BOx 4.3 mATrix mulTiplicATion: Across Therow And down The column 108

4.2 Simuling Mmoy rcll Using rcunauo-associo Nwoks 109

MatH BOx 4.4 liApunoV uncTions 112

MatLaB BOx 4.2 T t t at t at-aatv tk. T ttat vt at a t t 113

4.3 rclling Disinc Mmois Using NgivConncions in auo-associos 116

4.4 Synchonous vsus asynchonous Uding inrcun auo-associos 118

MatLaB BOx 4.3 T t t aat t at-aatv tk. T tat

t, at a, at a t

t 120

4.5 Gcul Dgdion nd SimuldFoging 121

4.6 Simuling Sog nd rcll o Squnc opns 124

MatLaB BOx 4.4 T t ak at hft gt at a tat-aatv a tk 124

4.7 Hbbin Lning, rcun auo-associion, ndModls o Hiocmus 126

Exercises 131

References 133




6/10

x Contents

6.1 Using h Clssic Hbb rul o Ln SimlLbld Lin rsons 173

MatLaB BOx 6.1 T t ta t-a tk a t t aat att g t h 175

6.2 Lning Siml Coningncy Using hCoviion rul 178

6.3 Using h Dl rul o Ln ComlConingncy 180

MatH BOx 6.1 The GrAdienT o VecTorcAlculus 181

MatH BOx 6.2 deriVATion o The delTA ruleleArninG AlGoriThm 183

MatH BOx 6.3 dierenTiATinG The squAshinGuncTion 184

MatLaB BOx 6.2 T t ta t-a tk ga t t aat att g t ta

186

6.4 Lning Innuonl rsnions usingBck-pogion 189

MatH BOx 6.4 mulTilAyered neTworKs o lineArprocessinG uniTs 190

MatH BOx 6.5 deriVATion o The bAcK-propAGATionleArninG AlGoriThm 191

MatLaB BOx 6.3 T t ta t-a tk ga t t aat att g ak-

agat 194

6.5 Simuling Csohic rociv Inncin Lning 196

6.6 Simuling h Dvlomn o Non-UniomDisibud rsnions 198

6.7 Modling Non-Uniom Disibud rsn-

ions in h Vsibul Nucli 201

Exercises 209

References 210

CHapter 6 Suvisd Lning nd Non-Uniom rsnions 171

7.1 Lning h Lbld-Lin tsk vi pubiono On Wigh tim 216

MatLaB

BOx 7.1 T t ta t-a tk ga t (t tt t) t aatatt g tat t tat t gat

at gt at a t 220

7.2 pubing all Wighs Simulnously nd hImonc o Sucu 221

MatH BOx 7.1 The browniAn moTionAlGoriThm 222

MatLaB BOx 7.2 T t ta t-atk ga t t aat att g

tat t tat t gat at a gtta 223

7.3 plusibl Wigh Modifcion using pubivrinocmn Lning 227

MatLaB BOx 7.3 T t ta t-a tk ga t t aat att tg agt ta 228

7.4 rinocmn Lning nd Non-UniomDisibud rsnions 230

MatLaB

BOx 7.4 T t ta t-a tk ga t t v a tt tatg aa, tatv t ag 232

7.5 rinocmn in Schm Modl o avoidncCondiioning 234

MatLaB BOx 7.5 T t at avatg a t ag 239

7.6 eloion nd eloiion in Modl oavoidnc Condiioning 242

MatLaB BOx 7.6 T t at avatg a t ag t atat

ajtt at 244

Exercises 247

References 248

CHapter 7 rinocmn Lning nd associiv Condiioning 213




7/10

Contents xi

8.1 Som Bsic Concs in Inomion thoy 253

8.2 Msuing Inomion tnsmission hough Nul Nwok 257

MatLaB BOx 8.1 T t t tttta at a a tk t t

tat, a t t a t tt, a

tt t 258

MatLaB BOx 8.2 T t t t t att t a t ttt ta at

a a tk 261

8.3 Mimizing Inomion tnsmission in NulNwok 265

MatH BOx 8.1 deriVATion o The bellseJnowsKiinomAx AlGoriThm or A neTworK wiTh one inpuT

uniT And one ouTpuT uniT 266

MatLaB BOx 8.3 T t ta a 2--2, atk g t bsjk a agt, a

f t ttt ta at 268

8.4 Inomion tnsmission nd ComiivLning in Nul Nwoks 276

MatLaB BOx 8.4 T t ta a 2--2, atk g ttv, v ag, a f

t ttt ta at 277

8.5 Inomion tnsmission in Sl-Ognizd MNwoks 279

MatLaB BOx 8.5 T t ta a -gaga tk a f ata a, g

tat tt, g tag 281

MatH BOx 8.2 rATe-disTorTion Theory 284

8.6 Inomion tnsmission in Sochsic NulNwoks 290

MatLaB BOx 8.6 T t t ata

a a tat tk a t t t t a ata a tt t 293

Exercises 296

References 298

CHapter 8 Inomion tnsmission nd Unsuvisd Lning 251

9.1 Imlmning Siml Clssif s th-Lyd Nul Nwok 302

MatLaB BOx 9.1 T t ta a t-a tk ga t g ak-agat t a fag t t gt 303

MatH BOx 9.1 uniVAriATe, mulTiVAriATe, AndbiVAriATe GAussiAn disTribuTions 305

9.2 pdicing rin s n evydy eml opobbilisic Innc 308

MatH BOx 9.2 The deriVATion o bAyes rule 309

9.3 Imlmning Siml Clssif Using Bysrul 311

MatLaB BOx 9.2 T t t t tat a t tta f a

g ba 313

9.4 Modling Nul rsonss o Snsoy Inu spobbilisic Innc 315

MatLaB BOx 9.3 T t t t tat a tagt gv t at

(.., va) 317

MatLaB BOx 9.4 T t ta a g ga tg t ta t tat t tagt at

gv t at (.., va) 319

MatH BOx 9.3 solVinG or The inpuT And biAsweiGhTs o A siGmoidAl uniT ThAT compuTes A

posTerior probAbiliTy 321

9.5 Modling Mulisnsoy Collicul Nuons spobbiliy esimos 323

MatLaB BOx 9.5 T t t t tat a tagt gv t t at

(.., va a at) 328

MatLaB BOx 9.6 T t ta a g ga tg t ta t tat t tagt at

gv t t at (.., va aat) 330

Exercises 338

References 339

CHapter 9 pobbiliy esimion nd Suvisd Lning 299




8/10

xii Contents

10.1 tining Conncion Wighs in rcunNul Nwoks 344

MatH BOx 10.1 deriVATion o reAl-Time recurrenTbAcK-propAGATion 345

10.2 tining two-Uni Nwok o Simul hOculomoo Nul Ingo 348

MatLaB BOx 10.1 T t t ak-agat t ta a t tk t t a

tt t t at a a ak tgat 351

10.3 Vlociy Sog in h Vsibulo-Oculr 354

10.4 tining Nwok o Lin Unis o poducVlociy Sog 357

MatLaB BOx 10.2 T t t ak-

agat t ta a a t tk t vt tag 360

10.5 tining Nwoks o Nonlin Unis o poducVlociy Sog 363

MatLaB BOx 10.3 T t t ak-agat t ta a a t tk t

vt tag (Tag va t) 364

10.6 tining rcun Nul Nwok o SimulSho-tm Mmoy 374

MatLaB BOx 10.4 T t t ak-agat t ta a a t tk tat t-t (Tag va

t) 377

MatLaB BOx 10.5 T t tt t at ta tk t at t-t 378

Exercises 383

References 384

CHapter 10 tim Sis Lning nd Nonlin Signl pocssing 341

MatLaB BOx 11.1 T t t a tatg 391

MatH BOx 11.1 indinG sTATe VAlues by solVinG AseT o simulTAneous lineAr equATions 394

11.1 Lning S Vlus Using Iiv Dynmicpogmming 395

MatLaB BOx 11.2 T t at tat vatat t tat g g tatv

a gag 396

11.2 Lning S Vlus Using Ls MnSqus 399

MatLaB BOx 11.3 T t at tat vatat t tat g g at-a-

a ag 401

11.3 Lning S Vlus Using h Mhod otmol Dincs 403

MatLaB BOx 11.4 T t at tat vatat t tat g g ta-

ag 405

11.4 Simuling Domin Nuon rsonss Usingtmol-Dinc Lning 408

MatLaB BOx 11.5 T t at t a a g ta-

ag 414

11.5 tmol-Dinc Lning s Fom oSuvisd Lning 416

Exercises 419

References 420

CHapter 11 tmol-Dinc Lning nd rwd pdicion 387




9/10

Contents xiii

MatH BOx 12.1 The KAlmAn ilTer 425

MatLaB BOx 12.1 T t t a gavag 426

12.1 Modling Visul Sysm Dicion SlciviyUsing asymmic Inhibiion 428

MatLaB BOx 12.2 T t at ttvt t va t 431

12.2 Modling Visul pocssing s Boom-U/to-Down pobbilisic Innc 434

MatLaB BOx 12.3 T t at tt-/t- g t va t g t jt

tt 442

MatH BOx 12.2 belie propAGATion 446

MatLaB BOx 12.4 T t at tt-/t- g t va t g at 447

12.3 a pdicoCoco Modl o pdicivtcking by Midbin Nuons 450

MatH BOx 12.3 decision-TheoreTic AGenT

desiGn 454

MatLaB BOx 12.5 T t t t tagt-takgtt 462

MatLaB BOx 12.6 T t t a tt at t t

aaga 463

12.4 tining Sigmoidl Uni o Simul tjcoypdicion by Nuons 469

MatLaB BOx 12.7 T t t ta t taa g ga t t ak t at t

t aaga 472

Exercises 475

References 477

CHapter 12 pdicoCoco Modls nd pobbilisicInnc 423

13.1 Simuling Gns nd Gnic Oos 487

13.2 eloing Siml eml o SimuldGnic evoluion 488

MatH BOx 13.1 The schemA Theorem or TheGeneTic AlGoriThm 489

MatH BOx 13.2 indinG The minimA (mAximA) ouncTions 490

MatLaB BOx 13.1 T t t gt agtt a g t f t a t 491

13.3 evolving h Sizs o Nul Nwoks oImov Lning 495

MatLaB BOx 13.2 T t t gt agtt t t ag vvg t

t t-a, a tk ta

g ak-agat 499

MatLaB BOx 13.3 T t ta a t-atk ga t t aat att g

ak-agat 500

13.4 evolving Oiml Lning ruls o auo-associiv Mmois 502

MatLaB BOx 13.4 T t t gt agtt t t hf 504

13.5 evolving Connciviy pofls o aciviy-BubblNul Nwoks 506

MatLaB BOx 13.5 T t t gt agtt t t tvt f t atvt-

tk 506

Exercises 511

References 513

CHapter 13 Simuld evoluion nd h Gnic algoihm 481




10/10

xiv Contents

14.1 Nuoinomics nd Molcul Nwoks 516

14.2 enhncd Lning in Nul Nwoks wihSm Synss 522

14.3 Combining Comlmny Nwok pdigmso Mmoy Fomion 526

14.4 Sm Synss nd Comlmny ruls inCbll Lning 531

14.5 a Finl Wod 539

References 541

CHapter 14 Fuu Dicions in Nul Sysms Modling 515

Sinauer Associates, I nc. Thismaterial cannot be copied, reproduced, manufactured

Documents

TutorialOnNeuralModelingSystems_2