102
Perception & Attention Perception is effortless but its underlying mechanisms are incredibly sophisticated. Biology of the visual system Representations in primary visual cortex and Hebbian learning Object recognition Attention: Interactions between systems involved in object recognition and spatial processing

Perception & Attention - Brown Universityski.clps.brown.edu/cogsim/cogsim.10percept.pdf · 2019. 9. 1. · Perception & Attention Perception is effortless but its underlying mechanisms

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

  • Percep

    tion

    &A

    ttentio

    n

    Percep

    tion

    iseffo

    rtlessb

    ut

    itsu

    nd

    erlyin

    gm

    echan

    isms

    arein

    credib

    ly

    sop

    histicated

    .

    •B

    iolo

    gy

    of

    the

    visu

    alsy

    stem

    •R

    epresen

    tation

    sin

    prim

    aryv

    isual

    cortex

    and

    Heb

    bian

    learnin

    g

    •O

    bject

    recog

    nitio

    n

    •A

    ttentio

    n:

    Interactio

    ns

    betw

    eensy

    stems

    inv

    olv

    edin

    ob

    jectreco

    gn

    ition

    and

    spatial

    pro

    cessing

  • Percep

    tion

    &A

    ttentio

    n

  • Percep

    tion

    &A

    ttentio

    n

    So

    me

    mo

    tivatin

    gq

    uestio

    ns:

    1.W

    hy

    do

    esp

    rimary

    visu

    alco

    rtexen

    cod

    eo

    riented

    bars

    of

    ligh

    t?

  • Percep

    tion

    &A

    ttentio

    n

    So

    me

    mo

    tivatin

    gq

    uestio

    ns:

    1.W

    hy

    do

    esp

    rimary

    visu

    alco

    rtexen

    cod

    eo

    riented

    bars

    of

    ligh

    t?

    2.W

    hy

    isv

    isual

    system

    split

    into

    wh

    at/w

    here

    path

    way

    s?

  • Percep

    tion

    &A

    ttentio

    n

    So

    me

    mo

    tivatin

    gq

    uestio

    ns:

    1.W

    hy

    do

    esp

    rimary

    visu

    alco

    rtexen

    cod

    eo

    riented

    bars

    of

    ligh

    t?

    2.W

    hy

    isv

    isual

    system

    split

    into

    wh

    at/w

    here

    path

    way

    s?

    3.W

    hy

    do

    esp

    arietald

    amag

    ecau

    seatten

    tion

    pro

    blem

    s(n

    eglect)?

  • Percep

    tion

    &A

    ttentio

    n

    So

    me

    mo

    tivatin

    gq

    uestio

    ns:

    1.W

    hy

    do

    esp

    rimary

    visu

    alco

    rtexen

    cod

    eo

    riented

    bars

    of

    ligh

    t?

    2.W

    hy

    isv

    isual

    system

    split

    into

    wh

    at/w

    here

    path

    way

    s?

    3.W

    hy

    do

    esp

    arietald

    amag

    ecau

    seatten

    tion

    pro

    blem

    s(n

    eglect)?

    4.H

    ow

    do

    we

    recog

    nize

    ob

    jects(acro

    sslo

    cation

    s,sizes,rotatio

    ns

    with

    wild

    lyd

    ifferent

    retinal

    imag

    es)?

  • Ov

    erview

    of

    the

    Visu

    alS

    ystem

    Hierarch

    ieso

    fsp

    ecializedv

    isual

    path

    way

    s,starting

    inretin

    a,toL

    GN

    (thalam

    us),to

    V1

    &u

    p:

  • Tw

    oS

    treams:

    Ven

    tral“w

    hat”

    vs.

    Do

    rsal“w

    here”

    V1

    V2V

    3V

    4T

    EO

    TF

    TE

    PO

    V3A

    MT

    FS

    T

    MS

    T

    VIP

    PGP

    G

    TE

    V1

    p

    mm

    d

  • Th

    eR

    etina

    Retin

    ais

    no

    ta

    passiv

    e“cam

    era”

    Key

    prin

    ciple:

    con

    trasten

    han

    cemen

    tth

    atem

    ph

    asizesch

    ang

    eso

    ver

    space

    &

    time.

    retinal

    ou

    tpu

    tg

    ang

    lion

    cells

  • LG

    No

    fth

    eT

    halam

    us

    A“relay

    station

    ”,bu

    tso

    mu

    chm

    ore.

    •O

    rgan

    izesd

    ifferent

    typ

    eso

    fin

    form

    ation

    into

    differen

    tlay

    ers.

    •P

    erform

    sd

    yn

    amic

    pro

    cessing

    :m

    agn

    ocellu

    larm

    otio

    np

    rocessin

    gcells,

    attentio

    nal

    pro

    cessing

    .

    •O

    n-

    and

    off-cen

    terin

    form

    ation

    from

    retina

    isp

    reserved

    inL

    GN

  • Prim

    aryV

    isual

    Co

    rtex(V

    1):E

    dg

    eD

    etectors

    V1

    com

    bin

    esL

    GN

    (thalam

    us)

    min

    pu

    tsin

    too

    riented

    edg

    ed

    etectors:

    •E

    dg

    esd

    ifferin

    orien

    tation

    ,size

    (spatial

    frequ

    ency

    ),and

    po

    sition

    .

    •F

    or

    coh

    erent

    visio

    n,

    need

    tod

    etectv

    aryin

    gd

    egrees

    of

    allth

    ese.

  • Ex

    amp

    leV

    1ed

    ge

    detecto

    rHu

    bel

    &W

    ieselN

    ob

    elP

    rize

  • Prim

    aryV

    isual

    Co

    rtex(V

    1):T

    op

    og

    raph

    y

    Hy

    perco

    lum

    n:

    Fu

    llset

    of

    cod

    ing

    for

    eachp

    ositio

    n

  • Prim

    aryV

    isual

    Co

    rtex(V

    1):T

    op

    og

    raph

    y

    Hy

    perco

    lum

    n:

    Fu

    llset

    of

    cod

    ing

    for

    eachp

    ositio

    n

    Pin

    wh

    eelcan

    arisefro

    mlearn

    ing

    and

    lateralco

    nn

    ectivity

    :n

    ot

    hard

    -wired

    !

  • Rero

    utin

    go

    fV

    isual

    Info

    toA

    ud

    itory

    Co

    rtex

    •S

    harm

    a,An

    gelu

    cci&

    Su

    r(2000),N

    ature

    Rero

    uted

    fib

    ersfro

    mR

    etina→

    aud

    itory

    thalam

    us

    (MG

    N)→

    A1

    •If

    visu

    alp

    rop

    ertiesare

    learned

    ,they

    sho

    uld

    dev

    elop

    inA

    1.

  • Rero

    utin

    go

    fV

    isual

    Orien

    tation

    Mo

    du

    lesin

    A1

    Ba-d

    :O

    rientatio

    nm

    aps,d

    ark-

    hig

    hact

    for

    giv

    eno

    rientatio

    n(b

    otto

    mrig

    ht).

    C:co

    mp

    osite

    map

    of

    orien

    tation

    preferen

    ces

    D:red

    do

    ts=

    pin

    wh

    eelcen

    ters

  • Visu

    alB

    ehav

    ior

    After

    Rero

    utin

    gR

    igh

    tV

    isual

    Field

    vo

    nM

    elchn

    er,Pallas

    &S

    ur

    (2000)

  • Visu

    alA

    cuity

    After

    Rero

    utin

    g

  • Visu

    alA

    cuity

    After

    Rero

    utin

    g

    →S

    olearn

    ing

    isp

    ow

    erful,b

    ut

    sois

    evo

    lutio

    n!

  • AQ

    uestio

    n

    Wh

    atm

    akes

    visu

    alco

    rtexv

    isual

    cortex

    ?W

    hy

    do

    esit

    represen

    to

    riented

    bars

    of

    ligh

    t?

  • Prim

    aryV

    isual

    Rep

    resentatio

    ns

    Key

    idea:

    Orien

    teded

    ge

    detecto

    rscan

    dev

    elop

    from

    Heb

    bian

    correlatio

    nal

    learnin

    gb

    asedo

    nn

    atural

    visu

    alscen

    es.

  • Th

    eM

    od

    el:S

    imu

    lating

    on

    eH

    yp

    ercolu

    mn

    •N

    atural

    visu

    alscen

    esare

    prep

    rocessed

    by

    passin

    gth

    em(sep

    arately)

    thro

    ug

    hlay

    erso

    fo

    n-cen

    teran

    do

    ff-center

    inp

    uts

    •H

    idd

    enlay

    er:ed

    ge

    detecto

    rsseen

    inlay

    ers2/

    3o

    fV

    1;Lay

    er4

    (inp

    ut)

    just

    represen

    ts

    un

    orien

    tedo

    n/

    off

    inp

    uts

    like

    LG

    N(b

    ut

    canb

    em

    od

    ulated

    by

    attentio

    n)

  • Th

    eM

    od

    el:S

    imu

    lating

    on

    eH

    yp

    ercolu

    mn

    •H

    ebb

    ianlearn

    ing

    on

    ly

    •F

    FF

    Bin

    hib

    com

    petitio

    nfo

    rsp

    ecialization

    (seeC

    h4)

  • [v1rf.p

    roj]

  • Th

    eR

    eceptiv

    eF

    ields

    01

    23

    45

    67

    89

    1011

    1213

    Red

    =o

    n-cen

    ter>

    off-cen

    ter,B

    lue

    =o

    ff-center

    >o

    n-cen

    ter

  • So

    me

    differen

    ces,b

    ut

    pin

    wh

    eelsstill

    emerg

    e

  • Percep

    tion

    and

    Atten

    tion

    1.W

    hy

    do

    esp

    rimary

    visu

    alco

    rtexen

    cod

    eo

    riented

    bars

    of

    ligh

    t?

    Co

    rrelation

    allearn

    ing

    based

    on

    natu

    ralv

    isual

    scenes.

    Refl

    ectsreliab

    lep

    resence

    of

    edg

    esin

    natu

    ralim

    ages,w

    hich

    vary

    insize,

    po

    sition

    ,o

    rientatio

    nan

    dp

    olarity.

    →m

    od

    elsh

    ow

    sh

    ow

    do

    cum

    ented

    V1

    pro

    perties

    canresu

    ltfro

    m

    interactio

    ns

    betw

    eenlearn

    ing

    ,architectu

    re(co

    nn

    ectivity

    ),an

    dstru

    cture

    of

    env

    iron

    men

    t.

  • Percep

    tion

    and

    Atten

    tion

    1.W

    hy

    do

    esp

    rimary

    visu

    alco

    rtexen

    cod

    eo

    riented

    bars

    of

    ligh

    t?

    Co

    rrelation

    allearn

    ing

    based

    on

    natu

    ralv

    isual

    scenes.

    2.H

    ow

    do

    we

    recog

    nize

    ob

    jects(acro

    sslo

    cation

    s,sizes,rotatio

    ns

    with

    wild

    lyd

    ifferent

    retinal

    imag

    es)?

    3.W

    hy

    isv

    isual

    system

    split

    into

    wh

    at/w

    here

    path

    way

    s?

    4.W

    hy

    do

    esp

    arietald

    amag

    ecau

    seatten

    tion

    pro

    blem

    s(n

    eglect)?

  • Th

    eO

    bject

    Reco

    gn

    ition

    Pro

    blem

    Pro

    blem

    :R

    ecog

    nize

    ob

    jectreg

    ardless

    of:

    locatio

    n,size,ro

    tation

    .S

    ame

    Diff

    Th

    isis

    hard

    becau

    sed

    ifferent

    pattern

    sin

    same

    locatio

    ncan

    ov

    erlapa

    lot,

    wh

    ileth

    esam

    ep

    atterns

    ind

    ifferent

    locatio

    ns/

    sizes/ro

    tation

    scan

    no

    t

    ov

    erlapat

    all!

  • Grad

    ual

    Inv

    ariance

    Tran

    sform

    ation

    s(F

    uk

    ush

    ima,’80)

  • Grad

    ual

    Inv

    ariance

    Tran

    sform

    ation

    s(F

    uk

    ush

    ima,’80)

    Increasin

    grecep

    tive

    field

    sizeen

    ables:

    Co

    nju

    nctio

    no

    ffeatu

    res(to

    form

    mo

    reco

    mp

    lexo

    bjects);an

    d

    Co

    llapsin

    go

    ver

    locatio

    nin

    form

    ation

    (“spatial

    inv

    ariance”)

  • Grad

    ual

    Inv

    ariance

    Tran

    sform

    ation

    s(F

    uk

    ush

    ima,’80)

    ifd

    idsp

    atialin

    varian

    cein

    on

    efell

    swo

    op

    :b

    ind

    ing

    pro

    blem

    -can

    ’ttell

    Tfro

    mL

  • Grad

    ual

    Inv

    ariance

    Tran

    sform

    ation

    s(F

    uk

    ush

    ima,’80)

    Go

    al:U

    nits

    atth

    eto

    po

    fth

    eh

    ierarchy

    sho

    uld

    represen

    tco

    mp

    lexo

    bject

    features

    ina

    locatio

    nan

    dsize

    inv

    ariant

    fashio

    n

  • Grad

    ual

    Inv

    ariance

    Tran

    sform

    ation

    s(F

    uk

    ush

    ima,’80)

    Go

    al:U

    nits

    atth

    eto

    po

    fth

    eh

    ierarchy

    sho

    uld

    represen

    tco

    mp

    lexo

    bject

    features

    ina

    locatio

    nan

    dsize

    inv

    ariant

    fashio

    n

    (alsow

    ant

    ben

    efits

    of

    top

    -do

    wn

    amp

    lificatio

    n,

    pattern

    com

    pletio

    n,

    distrib

    uted

    reps

    etc)

  • “Co

    nv

    olu

    tion

    alN

    eural

    Netw

    ork

    s”

    •v

    eryp

    op

    ular

    for

    “deep

    learnin

    g”

    inm

    achin

    elearn

    ing

    ,Yan

    LeC

    un

    ,Hin

    ton

    etc.

  • Th

    eM

    od

    el:co

    mb

    inin

    gF

    uk

    ush

    ima

    with

    con

    vo

    lutio

    nal

    neu

    raln

    ets,bid

    irection

    alco

    nn

    ectivity

    and

    learnin

    g!

    V1

    =o

    riented

    line

    (edg

    e)d

    etectors,

    hard

    -cod

    edV

    4u

    nits

    enco

    de

    con

    jun

    ction

    so

    fV

    1ed

    ges

    across

    asu

    bset

    of

    space

    Each

    ITu

    nit

    pay

    satten

    tion

    toall

    of

    V4

    (V2

    om

    ittedh

    ere,imp

    ortan

    tfo

    rfi

    gu

    re-gro

    un

    detc)

  • Th

    eO

    bjects

    Each

    ob

    jectis

    presen

    tedat

    mu

    ltiple

    locatio

    ns,sizes

    Netw

    ork

    ’sjo

    bis

    toactiv

    ateth

    eap

    pro

    priate

    Ou

    tpu

    tu

    nit

    (0-19)fo

    reach

    ob

    ject,regard

    lesso

    flo

    cation

    and

    size

  • [ob

    jrec.pro

    j]

  • Activ

    ation

    -Based

    Recep

    tive

    Field

    s

    •H

    ow

    do

    we

    plo

    trecep

    tive

    field

    sfo

    rV

    4?

    •R

    eceivin

    gw

    eigh

    tssh

    ow

    wh

    ichV

    1u

    nits

    aV

    4u

    nit

    respo

    nd

    sto

    ,bu

    tth

    ey

    do

    n’t

    sho

    ww

    hat

    thin

    gin

    the

    wo

    rldth

    eu

    nit

    respo

    nd

    sto

    •S

    olu

    tion

    :S

    ho

    wth

    en

    etwo

    rklo

    tso

    fin

    pu

    tp

    atterns.

    •T

    hen

    ,d

    isplay

    aco

    mp

    osite

    of

    allth

    ein

    pu

    tp

    atterns

    that

    activate

    the

    un

    it.

  • V4

    Recep

    tive

    Field

    s

    •S

    om

    eV

    4u

    nits

    cod

    efo

    rlo

    cation

    -specifi

    cco

    nju

    nctio

    ns

    of

    V1

    features

    –T

    his

    will

    sho

    wu

    pas

    ash

    arprecep

    tive

    field

    for

    Imag

    ein

    pu

    t

  • V4

    Recep

    tive

    Field

    s

    •S

    om

    eV

    4u

    nits

    cod

    efo

    rsim

    ple

    features

    ina

    locatio

    nin

    varian

    tw

    ay

    –T

    his

    will

    sho

    wu

    pas

    smeary

    parallel

    lines

    inIm

    age

    inp

    ut

  • V4

    Recep

    tive

    Field

    sfo

    rO

    utp

    ut

    •C

    analso

    loo

    kat

    wh

    ichO

    utp

    ut

    un

    itsten

    dto

    get

    active

    for

    any

    giv

    enV

    4u

    nit

    –G

    enerally

    ag

    iven

    V4

    un

    itis

    associated

    with

    mu

    ltiple

    ob

    jects

  • Gen

    eralization

    •C

    anth

    en

    etwo

    rkg

    eneralize

    tou

    nseen

    view

    so

    fstu

    died

    ob

    jects?

    •In

    oth

    erw

    ord

    s:D

    oes

    trainin

    gth

    en

    etto

    recog

    nize

    aset

    of

    ob

    jectsin

    a

    size/lo

    cation

    inv

    ariant

    fashio

    nh

    elpit

    recog

    nize

    new

    ob

    jectsin

    a

    size/lo

    cation

    inv

    ariant

    fashio

    n?

    •P

    roced

    ure:

    –T

    ake

    an

    ettrain

    edo

    n18

    ob

    jects

    –T

    rainw

    ith2

    new

    ob

    jectsin

    on

    lyso

    me

    locatio

    ns/

    sizes

    –T

    estth

    en

    etw

    ithn

    on

    stud

    ied“v

    iews”

    (sizes/lo

    cation

    s)o

    fn

    ew

    ob

    jects

  • Gen

    eralization

    •C

    anth

    en

    etwo

    rkg

    eneralize

    tou

    nseen

    view

    so

    fstu

    died

    ob

    jects?y

    es

    •A

    pp

    rox

    .90%

    correct

    on

    no

    vel

    view

    sfo

    llow

    ing

    trainin

    go

    nju

    st6%

    of

    po

    ssible

    sizes/lo

    cation

    s

  • Gen

    eralization

    •C

    anth

    en

    etwo

    rkg

    eneralize

    tou

    nseen

    view

    so

    fstu

    died

    ob

    jects?y

    es

    •A

    pp

    rox

    .90%

    correct

    on

    no

    vel

    view

    sfo

    llow

    ing

    trainin

    go

    nju

    st6%

    of

    po

    ssible

    sizes/lo

    cation

    s

    Ex

    plan

    ation

    :D

    istribu

    tedrep

    resentatio

    ns

    and

    Heb

    blearn

    ing

    !

    •V

    4rep

    resents

    ob

    jectfe

    atu

    res

    ina

    locatio

    n/

    sizein

    varian

    tw

    ay

    •E

    acho

    bject

    activates

    ad

    istribu

    tedp

    atterno

    fth

    esein

    varian

    tfeatu

    re

    detecto

    rs

  • Yeah

    ,bu

    tth

    eseo

    bjects

    arereg

    ularly

    shap

    ed,straig

    ht

    lines...

    wh

    atab

    ou

    treal

    ob

    jects?

  • “Em

    er”th

    ero

    bo

    treco

    gn

    izing

    ob

    jects..

    Vid

    eoD

    emo

    :em

    erd

    emo

    1.mo

    v

  • O’R

    eilly,C

    og

    Sci

    2009

  • Big

    ger

    netw

    ork

    mo

    del,b

    idirectio

    nal

    dy

    nam

    ics

    Wy

    atteet

    al.,2012;O’R

    eillyet

    al.,2013

  • Bid

    irection

    alD

    yn

    amics

    O’R

    eillyet

    al.,2013;W

    yatte

    etal.,2012

  • AC

    hallen

    ge

  • Deep

    Pred

    ictive

    Learn

    ing

    :W

    hat-W

    here-In

    tegratio

    n

    •u

    sesp

    redictiv

    elearn

    ing

    via

    recurren

    tfeed

    back

    bu

    tn

    osu

    perv

    isedtarg

    etlab

    els!O

    ’Reilly

    etal,

    2017arX

    iv

  • Still

    missin

    g...

    Mo

    tion

    :m

    otio

    nn

    oise.m

    p4

  • Still

    missin

    g...

    Mo

    tion

    :m

    otio

    nn

    oise.m

    p4

    •N

    euro

    ns

    inarea

    MT

    very

    sensitiv

    eto

    mo

    tion

    •L

    ots

    of

    wo

    rko

    nh

    ow

    do

    wn

    streamareas

    integ

    ratem

    otio

    nsig

    nals

    across

    time

    tod

    etectco

    heren

    ce(e.g

    .S

    had

    len,N

    ewso

    me,

    etc)

    •T

    ho

    mas

    Serre

    has

    sho

    wn

    that

    mo

    tion

    sign

    alsv

    eryreliab

    lefo

    r

    discrim

    inatin

    gb

    etween

    particu

    laractio

    ns

    (egth

    row

    ing

    ab

    aseball)

  • Still

    missin

    g...

    Mo

    tion

    :m

    otio

    nn

    oise.m

    p4

    •N

    euro

    ns

    inarea

    MT

    very

    sensitiv

    eto

    mo

    tion

    •L

    ots

    of

    wo

    rko

    nh

    ow

    do

    wn

    streamareas

    integ

    ratem

    otio

    nsig

    nals

    across

    time

    tod

    etectco

    heren

    ce(e.g

    .S

    had

    len,N

    ewso

    me,

    etc)

    •T

    ho

    mas

    Serre

    has

    sho

    wn

    that

    mo

    tion

    sign

    alsv

    eryreliab

    lefo

    r

    discrim

    inatin

    gb

    etween

    particu

    laractio

    ns

    (egth

    row

    ing

    ab

    aseball)

    •S

    ho

    uld

    be

    able

    toso

    lve

    pro

    blem

    via

    bid

    irection

    alin

    flu

    ence

    of

    mo

    tion

    integ

    ration

    sign

    als,ob

    jectreco

    gn

    ition

    ,an

    dsp

    atialatten

    tion

    (nex

    t)....

  • Percep

    tion

    and

    Atten

    tion

    1.W

    hy

    do

    esp

    rimary

    visu

    alco

    rtexen

    cod

    eo

    riented

    bars

    of

    ligh

    t?

    Co

    rrelation

    allearn

    ing

    based

    on

    natu

    ralv

    isual

    scenes.

    2.H

    ow

    do

    we

    recog

    nize

    ob

    jects(acro

    sslo

    cation

    s,sizes,rotatio

    ns

    with

    wild

    lyd

    ifferent

    retinal

    imag

    es)?T

    ransfo

    rmatio

    ns:

    increasin

    gly

    com

    plex

    featural

    enco

    din

    gs,in

    creasing

    levels

    of

    spatial

    inv

    ariance;

    Distrib

    uted

    represen

    tation

    s.

    3.W

    hy

    isv

    isual

    system

    split

    into

    wh

    at/w

    here

    path

    way

    s?

    4.W

    hy

    do

    esp

    arietald

    amag

    ecau

    seatten

    tion

    pro

    blem

    s(n

    eglect)?

  • Sp

    atialA

    ttentio

    n:

    Un

    ilateralN

    eglect

    Patien

    tco

    py

    ing

    ascen

    e:

    neg

    lect.mp

    4

    Self

    po

    rtrait,co

    py

    ing

    ,lin

    eb

    isection

    tasks:

    Inall

    cases,patien

    tsw

    ithp

    arietal/tem

    po

    rallesio

    ns

    seemto

    forg

    etab

    ou

    t

    1/2

    of

    space!

    bu

    tth

    eystill

    seeit!

  • Effects

    of

    Parietal

    Lesio

    ns

    on

    Po

    sner

    Task

    40−

    60−

    80−

    100−

    120−01

    2

    Intact

    Lesioned

    Neutral

    Valid

    Invalid

    •P

    atients

    perfo

    rmn

    orm

    allyin

    the

    “neu

    tral”(n

    ocu

    e)co

    nd

    ition

    ,

    regard

    lesso

    fw

    here

    the

    target

    isp

    resented

    •P

    atients

    ben

    efit

    just

    asm

    uch

    asco

    ntro

    lsfro

    mv

    alidcu

    es

    •P

    atients

    areh

    urt

    mo

    reth

    anco

    ntro

    lsb

    yin

    valid

    cues

  • Po

    ssible

    Mo

    dels

    +

    Alert

    Interrupt

    Localize

    Disengage

    Move

    Engage

    Inhibit

    Atten

    tion

    emerg

    esfro

    mb

    idirectio

    nal

    con

    straint

    satisfaction

    &in

    hib

    itory

    com

    petitio

    n.

  • Sim

    ple

    Mo

    del

    Input

    V1

    Spat1

    Spat2

    Obj1

    Obj2

    targ

    cue

    Output

    Object 1 (C

    ue)

    Object 2 (T

    arget)

  • [attnsim

    ple.p

    roj]

  • Po

    sner

    Task

    Data

    Valid

    Inv

    alidD

    iffA

    du

    ltN

    orm

    al350

    39040

    Eld

    erlyN

    orm

    al540

    60060

    Patien

    ts640

    760120

    Eld

    erlyn

    orm

    alized(*.65)

    350390

    40P

    atients

    no

    rmalized

    (*.55)350

    41868

  • Po

    sner

    Task

    Sim

    s

    •T

    he

    mo

    del

    exp

    lains

    the

    basic

    fin

    din

    gth

    atv

    alidcu

    essp

    eedtarg

    et

    pro

    cessing

    ,w

    hile

    inv

    alidcu

    esh

    urt

    •A

    lsoex

    plain

    sfi

    nd

    ing

    that

    patien

    tsw

    ithsm

    allu

    nilateral

    parietal

    lesion

    s

    ben

    efit

    no

    rmally

    from

    valid

    cues

    inip

    silateralfi

    eldb

    ut

    are

    disp

    rop

    ortio

    nately

    hu

    rtb

    yin

    valid

    cues.

    •N

    on

    eedto

    po

    sit“d

    iseng

    age”

    mo

    du

    le!

  • Po

    sner

    Task

    Sim

    s

    •T

    he

    mo

    del

    exp

    lains

    the

    basic

    fin

    din

    gth

    atv

    alidcu

    essp

    eedtarg

    et

    pro

    cessing

    ,w

    hile

    inv

    alidcu

    esh

    urt

    •A

    lsoex

    plain

    sfi

    nd

    ing

    that

    patien

    tsw

    ithsm

    allu

    nilateral

    parietal

    lesion

    s

    ben

    efit

    no

    rmally

    from

    valid

    cues

    inip

    silateralfi

    eldb

    ut

    are

    disp

    rop

    ortio

    nately

    hu

    rtb

    yin

    valid

    cues.

    •N

    on

    eedto

    po

    sit“d

    iseng

    age”

    mo

    du

    le!

    •A

    lsoex

    plain

    sfi

    nd

    ing

    of

    ne

    gle

    ct

    of

    con

    tralateralv

    isual

    field

    afterlarg

    e,

    un

    ilateralp

    arietallesio

    ns

    wh

    enso

    me

    stimu

    lus

    isp

    resent

    inip

    silateral

    field

    (“extin

    ction

    ”)

  • Mo

    reP

    osn

    erL

    esion

    Fu

    n

    •R

    eturn

    ing

    top

    atient

    with

    leftp

    arietallesio

    n...

    •W

    hat

    hap

    pen

    sif

    cues

    arep

    resented

    inc

    on

    trala

    tera

    l(affected

    )

    hem

    ifield

    ?(“R

    everse

    Po

    sner”)

  • Mo

    reP

    osn

    erL

    esion

    Fu

    n

    Retu

    rnin

    gto

    patien

    tw

    ithleft

    parietal

    lesion

    ...

    •W

    hat

    hap

    pen

    sif

    cues

    arep

    resented

    inc

    on

    trala

    tera

    l(affected

    )

    hem

    ifield

    ?

    Pred

    iction

    s:

    •S

    maller

    ben

    efit

    for

    valid

    cues

    •P

    atients

    sho

    uld

    be

    hu

    rtless

    than

    con

    trols

    by

    inv

    alidcu

    es.

  • [attnsim

    ple.p

    roj]

  • Inh

    ibitio

    no

    fR

    eturn

    •T

    yp

    ically,targ

    etd

    etection

    isfaster

    on

    trialsw

    ithv

    alidv

    sin

    valid

    cues

    •H

    ow

    ev

    er,if

    the

    cue

    isp

    resented

    for

    alo

    ng

    ertim

    e(eg

    .500

    ms),

    perfo

    rman

    ceis

    fastero

    nin

    va

    lidv

    sv

    alidtrials

    •C

    anex

    plain

    interm

    so

    fa

    cc

    om

    mo

    da

    tion

    (neu

    ralfatig

    ue)

  • [attnsim

    ple.p

    roj]

  • Sim

    ple

    mo

    del:

    too

    simp

    le?

    •H

    asu

    niq

    ue

    on

    e-to-o

    ne

    map

    pin

    gs

    betw

    eenlo

    w-lev

    elv

    isual

    features

    and

    ob

    jectrep

    resentatio

    ns

    (no

    trealistic)

    •D

    oes

    no

    tad

    dress

    issue

    of

    spatial

    attentio

    nw

    hen

    tryin

    gto

    perceiv

    e

    mu

    ltiple

    ob

    jectssim

    ultan

    eou

    sly

  • Sim

    ple

    mo

    del:

    too

    simp

    le?

    •H

    asu

    niq

    ue

    on

    e-to-o

    ne

    map

    pin

    gs

    betw

    eenlo

    w-lev

    elv

    isual

    features

    and

    ob

    jectrep

    resentatio

    ns

    (no

    trealistic)

    •D

    oes

    no

    tad

    dress

    issue

    of

    spatial

    attentio

    nw

    hen

    tryin

    gto

    perceiv

    e

    mu

    ltiple

    ob

    jectssim

    ultan

    eou

    sly

    •“C

    om

    plex

    ”m

    od

    elco

    mb

    ines

    mo

    rerealistic

    mo

    del

    of

    ob

    jectreco

    gn

    ition

    (starting

    from

    LG

    N)

    with

    simp

    leatten

    tion

    mo

    del

    →C

    anu

    sesp

    atialatten

    tion

    torestrict

    ob

    jectp

    rocessin

    gp

    athw

    ayto

    on

    eo

    bject

    ata

    time,en

    ablin

    git

    toseq

    uen

    tiallyp

    rocess

    mu

    ltiple

    ob

    jects.

    •L

    esion

    so

    fen

    tiresp

    atialp

    athw

    aycau

    sesim

    ultan

    agn

    osia:

    inab

    ilityto

    con

    curren

    tlyreco

    gn

    izetw

    oo

    bjects

  • Co

    mp

    lexM

    od

    el

    LGN

    _On

    LGN

    _Off

    V1

    V2

    V4

    /IT Output

    Spat1

    Spat2

    Target

    Sp

    at1h

    asrecu

    rrent

    pro

    jns

    toen

    cou

    rage

    focu

    so

    no

    ne

    regio

    no

    fsp

    ace

  • Co

    mp

    lexM

    od

    el

    LGN

    _On

    LGN

    _Off

    V1

    V2

    V4

    /IT Output

    Spat1

    Spat2

    Target

    Sp

    at1h

    asrecu

    rrent

    pro

    jns

    toen

    cou

    rage

    focu

    so

    no

    ne

    regio

    no

    fsp

    ace

    Bu

    to

    nly

    mech

    anism

    for

    switch

    ing

    isacco

    mm

    od

    ation

    ...

  • Percep

    tion

    and

    Atten

    tion

    1.W

    hy

    do

    esp

    rimary

    visu

    alco

    rtexen

    cod

    eo

    riented

    bars

    of

    ligh

    t?

    Co

    rrelation

    allearn

    ing

    based

    on

    natu

    ralv

    isual

    scenes.

    2.H

    ow

    do

    we

    recog

    nize

    ob

    jects(acro

    sslo

    cation

    s,sizes,rotatio

    ns

    with

    wild

    lyd

    ifferent

    retinal

    imag

    es)?T

    ransfo

    rmatio

    ns:

    increasin

    gly

    com

    plex

    featural

    enco

    din

    gs,in

    creasing

    levels

    of

    spatial

    inv

    ariance;

    Distrib

    uted

    represen

    tation

    s.

    3.W

    hy

    isv

    isual

    system

    split

    into

    wh

    at/w

    here

    path

    way

    s?

    Tran

    sform

    ation

    s:em

    ph

    asizing

    and

    collap

    sing

    across

    differen

    tty

    pes

    of

    relevan

    td

    istinctio

    ns;atten

    tion

    4.W

    hy

    do

    esp

    arietald

    amag

    ecau

    seatten

    tion

    pro

    blem

    s(n

    eglect)?

    Atten

    tion

    asan

    emerg

    ent

    pro

    perty

    of

    com

    petitio

    n

  • Gen

    eralIssu

    esin

    Atten

    tion

    Atten

    tion

    :

    •P

    rioritizes

    pro

    cessing

    .

    •C

    oo

    rdin

    atesp

    rocessin

    gacro

    ssd

    ifferent

    areas.

    •S

    olv

    esb

    ind

    ing

    pro

    blem

    sv

    iaco

    ord

    inatio

    n.

  • Gen

    eralIssu

    esin

    Atten

    tion

    Atten

    tion

    :

    •P

    rioritizes

    pro

    cessing

    .

    •C

    oo

    rdin

    atesp

    rocessin

    gacro

    ssd

    ifferent

    areas.

    •S

    olv

    esb

    ind

    ing

    pro

    blem

    sv

    iaco

    ord

    inatio

    n.

    Bu

    tatten

    tion

    sho

    uld

    be

    mu

    chm

    ore

    flex

    ible

    than

    just

    spatial

    bias!

    Later:

    ho

    wto

    inco

    rpo

    rateg

    oals,rein

    forcem

    ent

    pro

    bab

    ility,in

    toatten

    tion

    al

    allocatio

    n