ACT-R 6.0(Cognitive Science Prosem)

Wayne D. GrayRensselaer Polytechnic Institute

CogWorks Laboratorygrayw@rpi.edu

Mike SchoellesRensselaer Polytechnic Institute

CogWorks Laboratoryschoem@rpi.edu

Tutorial Overview

Cognitive Architecture/Modeling OverviewACT-R Theory - Symbolic levelAddition,counting and letter models

ACT-R Theory Sub-symbolic levelSternberg and Building Sticks models

Production Compilation

What is a Cognitive Architecture?

Infrastructure for an intelligent system

Cognitive functions that are constant over time and across different task domains

Analogous to a building, car, or computer

Unified Theories of Cognition

Account of intelligent behavior at the system-level

Newell’s claim“You can’t play 20 questions with nature and win”

Integrated Cognitive Architecture

Cognition does not function in isolation

Interaction with perception, motor, auditory, etc. systems

Embodied cognitionRepresents a shift from• “mind as an abstract information processing system”

• Perceptual and motor are merely input and output systems

Must consider the role of the environmentOther body processes• Effects of caffeine, stress and other moderators

Motivations for a Cognitive Architecture *

1. Philosophy: Provide a unified understanding of the mind.

2. Psychology: Account for experimental data.

3. Education: Provide cognitive models for intelligent tutoring systems and other learning environments.

4. Human Computer Interaction: Evaluate artifacts and help in their design.

5. Computer Generated Forces: Provide cognitive agents to inhabit training environments and games.

6. Neuroscience: Provide a framework for interpreting data from brain imaging.

7. All of the above

Requirements for Cognitive Architectures*

1. Integration, not just of different aspects of higher level cognition but of cognition, perception, and action.

2. Systems that run in real time.

3. Robust behavior in the face of error, the unexpected, and the unknown.

4. Parameter-free predictions of behavior.

5. Learning.

Newell’s Time Scale of Human Activity (amended)

TaxonomyComputational

Cognitive Models

Connectionist

Cognitive Architectures

Home grown -- one-off codeSymbolic

Other AI

other Production System

ACT-R 6.0

HybridSymbolic only

SOAR EPIC

Model Name Acronym/ Abbreviation

Pew and Mavor (1998)

Ritter, Shadbolt et al. (2001)

Present IDA Study

Atomic Components of Thought ACT Yes No Yes

Adaptive Resonance Theory ART No No Yes

Architecture for Procedure Execution APEX No Yes Yes

Artificial Neural Networks ANNs Yes No Noa

Business Redesign Agent-Based Holistic Modeling System

Brahms No No Yes

Cognition and Affect Project CogAff No Yes Yes

COGnition As a NEtwork Of Tasks COGNET Yes No Yes

Cognitive Complexity Theory CCT No No Yes

Cognitive Objects within a Graphical EnviroNmenT COGENT No Yes Yes

Concurrent Activation-Based Production System CAPS No No Yes

Construction-Integration Theory C-I Theory No No Yes

Distributed Cognition DCOG No No Yes

Elementary Perceiver And Memorizer EPAM No Yes Nob

Executive Process/Interactive Control EPIC Yes No Yes

Human Operator Simulator HOS Yes No Yes

Belief-Desire-Intention architecture BDI No Yes Noc

Man-machine Integrated Design and Analysis System MIDAS Yes No Yes

Micro Systems Analysis of Integrated Network of Tasks

Micro SAINT Yes No Yes

MIDAS Redesign Yes No Yesd

Operator Model ARchitecture OMAR Yes No Yes

PSI PSI No Yes Yes

Situation Awareness Model for Pilot-in-the-Loop Evaluation

SAMPLE Yes No Yes

Sparse Distributed Memory SDM No Yes Nob

State, Operator, And Result Soar Yes No Yes

History of the ACT-framework*

Predecessor HAM (Anderson & Bower 1973)

Theory versions ACT-E (Anderson, 1976)ACT* (Anderson, 1978)ACT-R (Anderson, 1993)ACT-R 4.0 (Anderson &

Lebiere, 1998)ACT-R 5.0 (Anderson &

Lebiere, 2001)

ImplementationsGRAPES (Sauers & Farrell, 1982)PUPS (Anderson & Thompson,

1989)ACT-R 2.0 (Lebiere &

Kushmerick, 1993)ACT-R 3.0 (Lebiere, 1995)ACT-R 4.0 (Lebiere, 1998)ACT-R/PM (Byrne, 1998)ACT-R 5.0 (Lebiere, 2001)Windows Environment (Bothell, 2001)Macintosh Environment (Fincham, 2001)ACT-R 6.0 (Bothell, 2004??)

Other Cognitive Architectures

SoarProduction rule system• Organized in terms of operators associated with problem spaces

• Goal oriented– Sub-goaling

• Learning mechanism - Chunking

EPIC -- Executive Processes in Cognition

Parallel firing of production rules (in principle)Well developed visual and motor systemEmphasis on executive processes

MetaIssues

The divide between high-level architectures and low-level (primarily connectionist ones) is mainly a levels issueModeling in high-level architectures reflects a concern with the task structure of behavior and can be considered a form of task analysis

MetaIssues

The problems tackled by the two sets of approaches tends to reflect this differenceFrom this perspective ACT-R is a major success in “re-use” -- low level parameters are reused in most ACT-R models. The higher level production rules differ in part because they reflect the task analysis for the different tasks being modeled

MetaIssues

From this perspective the focus on reusability should focus onLow-level productions that control the interleaving of cognitive, perceptual, and action at the 1/3 sec level of analysis

Not onHigh-level productions that implement task-specific executive control

MetaIssuesLow-Level Productions Implement Interactive Routines

MetaIssuesInteractive Routines

Neurocognitive evidence increasing stresses the modular nature of human cognitionBut -- these modules are constrained by their need to work together to survive in the worldInteractive routines can be viewed as the basic level elements of embodied cognitionProvide constraints on the input/output functions for perception, attention, memory, and motor systemsNotion is that the use of cognitive, perceptual, and motor resources is optimized via the selection of one set of interactive routines over another

ACT-R Overview

Modules (buffers) Knowledge RepresentationSymbolic/Sub-symbolicPerformance/Learning

I. ACT-R Theory IV. Problem Solving and Decision Making

II. Perception and Attention Tower of Hanoi

Psychophysical Judgement Choice and Strategy Selection

Visual Search Mathematical Problem Solving

Eye Movements Spatial Reasoning and Navigation

MultiTasking Dynamic Systems

Task Switching Use and Design of Artifacts

Subitizing Game Playing

Stroop Insight and Scientific Discovery

Driving and Flying Behavior Programming

Situational Awareness and Embedded Cognition Reasoning

Graphical User Interfaces Errors

Time Perception V. Language Processing

III. Learning and Memory Parsing

List Memory Analogy and Metaphor

Interference Language Learning

Implicit Learning Sentence Memory

Skill Acquisition Lexical Processing

Cognitive Arithmetic VI. Other

Category Learning Cognitive Development

Learning by Exploration and Demonstration Individual Differences

Updating Memory and Prospective Memory Motivation, Emotion, & Cognitive Moderators

Causal Learning Cognitive Workload

Working Memory Computer Generated Forces, Video Games, and Agents

Practice and Retention fMRI

Communication, Negotiation, and Group Decision Making

Instructional Materials

User Modeling

Intelligent tutoring systems

Information Search

Tools

Comparative

ACT-R Applications559 Papers Listed on

http://act-r.psy.cmu.edu/publications/index.php

In the following areas as of 2006-01-24

Interactive Session

Load and run Addition model

Addition model exerciseIn this exercise you will load a simple model and run it to see

how a model runs. You will also get some experience with the interface.

1. Open model

Click on the "Open Model" button on the Environment Control Pane, and select the Addition model. This will open up the model so that you can see it and its parts.

You should be able to see the working memory elements in the model (window "Chunk"), the productions (Production window). There are three further windows, Chunk Type, Command, and Miscellaneous, that we will cover later.

You should briefly examine the chunk and production contents. You may note that there about 11 pieces of working memory, and just 4 rules in this system.

2. Run the model

You can run the model using the Lisp command line, but we will use the environment because it provides a recognition-based interface rather than a recall-based interface.

You should first click on "Reset"; this will reset the model and make it ready to run. You can do this to a model that has run as well, or has been stopped in the middle of a run.

You can run the model by clicking on the "Run" button. A trace of the model will appear in the (Lisp) "Listener" window. You can see how the order that rules are selected and fired, as well as when chunks are retrieved from memory by the rules.

3. Inspect the model

Click on "Declarative viewer" in the Control Pane to bring up an inspector window for the declarative memory elements. If you scroll, you can find the chunks a-j and second-goal. Pay most attention to their structure, and note that they have several parameters. These parameters are used to compute how fast they are used and if they can be retrieved. With learning and use, the activation, for example, goes up. These are covered later in this tutorial.

The Procedural viewer provides a view onto the rules.

End of First Tuesday

HomeworkAnderson, J. R., Bothell, D., Byrne, M. D., Douglas, S., Lebiere, C., & Quin, Y. (2004). An integrated theory of the mind. Psychological Review, 111(4), 1036-1060.

Change addition model to subtraction model

ACT-R 6.0 Architecture

MotorModules

Current Goal

PerceptualModules

DeclarativeMemory

Pattern MatchingAnd

Production Selection

Check

RetrieveModify

Test

Check State Schedule

Action

IdentifyObject

MoveAttention

ACT-R 6.0

Environment

ACT-R 6.0 Mapping to the Brain*

Environment

Pro

duct

ions

(Bas

al G

angl

ia)

Retrieval Buffer (VLPFC)

Matching (Striatum)

Selection (Pallidum)

Execution (Thalamus)

Goal Buffer (DLPFC)

Visual Buffer (Parietal)

Manual Buffer (Motor)

Manual Module (Motor/Cerebellum)

Visual Module (Occipital/etc)

Intentional Module (not identified)

Declarative Module (Temporal/Hippocampus)

PerformanceDeclarative Procedural

SymbolicRetrieval of

chunksApplication of

production rules

SubsymbolicNoisy activationscontrol speed and

accuracy

Noisy utilitiescontrol choice

LearningDeclarative Procedural

SymbolicEncoding environment

andcaching goals

Productioncompilation

SubsymbolicBayesianlearning

Bayesianlearning

ACT-R: Assumption Space*

Addition-FactThree Seven

Four

addend1 sum

addend2

Declarative-Procedural Distinction

Procedural Knowledge: Production Rules

for retri eving chunks to solve problems.

336

+848

4

IF the goal is to add the numbers in a column

and n1 + n2 are in the column

THEN retrieve the sum of n1 and n2.

Productions serve to coordinate the retrieval of

information from declarative memory and the enviroment

to produce transformations in the goal state.

Declarative Knowledge: Chunks

Configurations of small numbers of elements

ACT-R: Knowledge Representation*

goal buffer visual buffer retrieval buffer

Declarative Memory Syntax

(chunk-type count-order first second)

(chunk-type add arg1 arg2 sum count)

Chunk type : ((chunk-type type-name slot-name1 …(slot-namek init-val)..slot-namen)

(add-dm (a ISA count-order first 0 second 1)

(b ISA count-order first 1 second 2)

(second-goal ISA add arg1 5 arg2 2) ) ;;sum and count are nil

Chunk Instantiation: (add-dm (chunk-name isa type-name slot-name slot-value) …)

Declarative Memory

Chunks that are added explicitlyAdd-dm

Chunks merge into DM from buffersAll buffers’ chunks go to DM when cleared

Mergings are the references for BLLNot the LHS usage as in ACT-R 5

Because buffers hold copies, DM chunks can’t be changed from within a production

Previously it was a recommendation

ADDITION-FACT

AADDENDDDEND11 TTHREEHREE

AADDENDDDEND22 FFOUROUR

SSUM UM

FACT3+4(

SSEVENEVEN )

isa

Addition Fact Example

(Chunk-type addition-fact addend1 addend2 sum)

Addition Example(CLEAR-ALL)(DEFINE-MODEL addition)(CHUNK-TYPE addition-fact addend1 addend2 sum)(CHUNK-TYPE integer value)(ADD-DM (fact3+4

isa addition-fact addend1 three addend2 four sum seven) (three

isa integer value 3) (four

isa integer value 4) (seven

isa integer value 7)

ADDITION-FACT

FACT3+4ADDEND1 SUM

ADDEND2

THREE

FOUR

SEVEN

isa

isa

INTEGER

isa

VALUE VALUE

3 7

isa

Addition Fact Example

VALUE4

A Production is*

1. The greatest idea in cognitive science.

2. The least appreciated construct in cognitive science.

3. A 50 millisecond step of cognition.

4. The source of the serial bottleneck in otherwise parallel system.

5. A condition-action data structure with “variables”.

6. A formal specification of the flow of information from cortex to basal ganglia and back again.

Key Properties • • modularitymodularity• • abstractionabstraction• • goal/buffer factoringgoal/buffer factoring• • conditional asymmetryconditional asymmetry

Productions*

( p

==>

)

Specification ofBuffer Transformations

condition part

delimiter

action part

name

Specification ofBuffer Tests

Structure of productions

Productions LHSOnly four possible conditions available=buffer>

•Test the chunk in the buffer just like in 5

!eval! or !safe-eval!!bind! or !safe-bind!

•Same as in ACT-R 5•Safe- versions accepted by production compilation

?buffer>•Query the buffer or its module•Come back to queries later

Possible RHS actions

=buffer>-buffer>+buffer>!eval! and !safe-eval!!bind! and !safe-bind!!output!!stop!

RHS actions

=buffer>!eval! and !safe-eval!!bind! and !safe-bind!!output!

All the same as in ACT-R 5The safe- versions do not inhibit the production compilation mechanism

!stop!Not actually new, but does work nowGenerates a break event in the schedulerTerminates the current “run” command

RHS +buffer>+buffer> isa chunk-type {{modifier} [slot | request parameter] value}*

or

+buffer> chunk-reference

Sends a request to the moduleAlways clears the buffer implicitlyEssentially the same as ACT-R 5

Buffer queries

Replaces the *-state buffersSyntax

?buffer>{ {-} query value}+

Either true or false No bindings

Must all be true for production to match

Examples

?retrieval> ?visual> state busy - state error buffer empty buffer =check

Queries continued

Every buffer/module must respond toState •Values: busy, free, or error

Buffer •Values: full, empty, requested or unrequested

Others can be added by a module writer•Modality for the current PM modules for example

Production Syntax(P initialize-addition =goal> ISA add arg1 =num1 arg2 =num2 sum nil==> =goal> sum =num1 count 0 +retrieval> isa count-order first =num1 ?retrieval> state free)

(P increment-sum =goal> ISA add sum =sum count =count =retrieval> ISA count-order first =sum second =newsum==> =goal> sum =newsum +retrieval> isa count-order first =count)

(p got-number =goal> isa make-a-call who =person =retrieval>

isa phone-number who =person where =office ph-num =num==> !output! (The phone number for =person is =num) +goal> isa dial-number who =person ph-num =num current-digit 1)

(p dial-a-digit =goal> isa dial-number ph-num =num current-digit =digit state nil==> !bind! =d (get-digit =digit =num) +visual-location> isa visual-location value =d =goal> state dialing)

(P increment =goal> ISA count-from start =num1 - end =num1 step counting =retrieval> ISA count-order first =num1 second =num2==> =goal> start =num2 +retrieval> ISA count-order first =num2 !output! (=num1))

(p read-choose =goal> isa read-letters state verify-choose =visual> isa text value "choose" ==> =goal> state find-letter )

(P rotate-counter-clockwise =goal> ISA translate&rotate step counter-clockwise reference-y =y axis =axis form =form !bind! =y1 (+ =y 15)) =retrieval> ISA point-i > screen-y =y1 ;;no longer can do (!eval! (+ =y 15))==> !eval! (rotate-counter-clockwise =form =axis))

(P get-direction =goal> ISA make-report step find-point =retrieval> ISA point-i screen-x =x screen-y =y==> =goal> step find-direction !bind! =x1 (+ =x 15) +retrieval> ISA direction < screen-x =x1 ;;;no longer (!eval! (+ =x 15)) )

Interactive Session

Load and run Counting model

Count model

This model works much like the previous model, but prints out its count.

1. Open the model

Either quit and restart your Lisp, or else click on "Close Model".

Open the Count model by clicking on "Open Model" and then selecting the Count model.

Run the model to see its trace, and examine its rules and chunks.

2. Using the Stepper

Click on "Stepper", and a stepper window should appear.

Reset the model, and then click on the run button. This starts the stepper. You can now step through the model by clicking on the "Step" button on the Stepper.

As you step through the model, you should be able to see most of the mechanisms in ACT-R now, the productions, how they are matched, the chunks, and how they are retrieved, and the buffers (click on Buffer Viewer to see the buffers and their contents).

3. Checking on a rule that does not fire.

After you have run the model a few steps, click on the Procedural Viewer. Select a rule in the dialogue box, and see why it does not fire.

4. Edit the model

Look at the model and consider how to have it count backwards.

You can change the production rules in the Production window. After you make changes, save the model (it will automatically increment). Close the model and reopen it to try your new model.

The Modules(reprise)

CognitionMemoryVisionMotorAuditionSpeech

ACT-R 6.0 Buffers*

1. Goal Buffer (=goal, +goal)-represents where one is in the task-preserves information across production cycles

2. Retrieval Buffer (=retrieval, +retrieval)-holds information retrieval from declarative memory-seat of activation computations

3. Visual Buffers-location (=visual-location, +visual-location)-visual objects (=visual, +visual)-attention switch corresponds to buffer transformation

4. Auditory Buffers (=aural, +aural)-analogous to visual

5. Manual Buffers (=manual, +manual)-elaborate theory of manual movement include feature preparation, Fitts law, and device properties

6. Vocal Buffers (=vocal, +vocal)-analogous to manual buffers but less well developed

Cognition

Executive Control - Production SystemSerialParallel at sub-symbolic levelUtility selects production to fireUtility = benefit - cost•Benefit = probability of success * value of achieving goal

Production System Cycle

Match conditions of all rules to buffers

Those that match enter the conflict set

Conflict resolution selects a rule to fire

Action side of rule initiates changes to one or more buffers

If no production can match and no action is in progress then quit else repeat

Goal directed

Represents what you are trying to doA declarative memory element that is the focus of “internal” attention(goal-focus second-goal)

Memory Module

Activation basedFrequency and recencyContextual cues

CognitionRequests retrieval• Specifies constraints• Partial matching

Memory Parallel search of memory to match constraintsCalculates activation of matching chunksReturns most active chunk

Vision Module

ACT-R’s “eyes”Dorsal “where” systemVentral “what” system

“Where” SystemCognition

Requests “pre-attentive” visual searchSpecifies a set of constraintsAttribute/value pairs • Properties or spatial location

– e.g. color red, screen-x greater-than 150

“Where” system Returns a “location” chunk• Specifies location of an object whose features satisfy the constraints

OnsetsFeatures are held in vision module’s memory

Visual location

(P find-unattended-letter =goal> ISA read-letters state start ==> +visual-location> ISA visual-location :attended nil =goal> state find-location)

(chunk-type visual-location screen-x screen-y distance kind color value size nearest)

(P attend-letter =goal> ISA read-letters state find-location =visual-location> ISA visual-location ?visual> state free==> +visual> ISA move-attention screen-pos =visual-location =goal> state attend)

“What” System

CognitionRequests “move attention” Provides “location” chunk

“What” System Shifts visual attention to that locationEncodes object at that location• Added to Declarative Memory• Episodic representation of visual scene

Places encoding in “visual” bufferCalculates latency• EMMA

Visual Object

(P encode-letter =goal> ISA read-letters state attend =visual> ISA text value =letter==> =goal> letter =letter state respond)

(P attend-letter =goal> ISA read-letters state find-location =visual-location> ISA visual-location ?visual> state free==> +visual> ISA move-attention screen-pos =visual-location =goal> state attend)

(chunk-type visual-object screen-pos value status color height width)

Visual State

(P attend-letter =goal> ISA read-letters state find-location =visual-location> ISA visual-location ?visual> state free ==> +visual> ISA move-attention screen-pos =visual-location =goal> state attend)

Vision Module Memory

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Motor Module

ACT-R’s HandsBased on EPIC’s Manual Motor ProcessorMovement StylesPhased Processing

Motor Syntax(P respond =goal> ISA read-letters letter =letter state respond ?manual> state free==> +manual> ISA press-key key =letter =goal> state stop)

Movement Styles

Style

Punch (hand finger)

HFRT(hand finger r theta)

PeckPeck-recoilPly (hand r theta)

Hand Cursor

Movement Styles

Ply - moves a device (e.g. move-mouse) to a given locationPunch - pressing a key below finger or click-mousePeck - directed movement of finger to a location followed by keystrokePeck-recoil - same as peck but finger moves back

Phased Processing (1)

Preparation PhaseHierarchical feature preparation•Style->hand->finger

Prep time depends on•Complexity of movement•Number of features

State buffer set to prep busy

Phased Processing (2)

Initiation (fixed 50 ms)Execution• Time depends on

– Type of movement– Minimum execution time

– Distance– Fitt’s Law

Allow overlapping of preparation and execution

Audition Module

Simulated perception of audioMemory of featuresTemporal-extent - sound events

Tones, digits, and speechAttributesOnset, duration, delay, recode time

Audition Module Syntax

(defp alpha-task/listen =goal>

isa alpha-task step listen==> +aural-location> isa audio-event onset highest :attended NIL =goal> step check-feature )

(defp alpha-task/check-feature =goal>

isa alpha-task step check-feature =aural-location> isa audio-event ?aural> state free

==> +aural> isa sound event =aural-location =goal> step attend-sound)

(chunk-type audio-event onset offset pitch kind location) (chunk-type sound kind content event)

Audition Module Processing

Parallels vision systemCognition

Specifies a set of constraintsAttribute/value pairs

AuditionReturns a “location” chunk

CognitionRequests shift of auditory attention providing the “location”chunk

AuditionEncodes the sound

Device InterfaceSimulated device with which ACT-R interacts

Contains graphical objects

Typically a WindowCan be entire screen

InteractionConstructing vision system’s iconic memory (sets of features) from graphical objectsHandle mouse and keyboard actions

(defclassic button-

feature (icon-

feature) feature-slot)

(defclassic my-button-dialog-item (button-dialog-item) dialog-item-slot)

(defmethod build-features-for (self my-button-dialog-item )

(defclassic button-feature (icon-feature) feature-slot)

Visual Icon (visicon)

button-feature

+visual-location

(defmethod featinfo->loc (icon-feature)

(CHUNK-TYPE visual-location slota, slotb, slotc)

Declarative Memory

visual-location DMEs

Locs

+visual

(CHUNK-TYPE visual-object slota, slotb, slotc)

Declarative Memory

visual-objectt DMES

(defmethod feat-to-dmo ( button-feature )

linked

Visual Icon (vi...

THESE are Lisp elements

-- but are into vision -- pre

move location

(defmethod featinfo->loc (icon-featu...

Where button-feature is subclass of icon-feature

This builds a visual-location DME using the features -- puts

them into the slots of the chunk-type.

(defclassic my-button-dialog-item (button-dialog-item) dialog-item-slot)

simply a new screen object that inherits from predefined screen object (button-dialog-item)

I. (defmethod build-features-for (self my-button-dialog-item)

Makes an instance of "button-feature"

Is called by pm-proc-display

A. Visual Icon (visicon)

button-feature

THESE are Lisp elements -- but are into vision -- pre move location

1. +visual-location

a. (defmethod featinfo->loc (icon-feature)

Where button-feature is subclass of icon-feature

This builds a visual-location DME using the features -- puts them into the slots of the

chunk-type.

(1) Declarative Memory

visual-location DMEs

Locs

These are part of ACTR --- so these "locs" are DMEs

(a) +visual

i) (defmethod feat-to-dmo (button-feature)

(1) Declarative Memory

visual-objectt DMES

2. linked

(defclassic button-feature (icon-feature) feature-slot)

This inherits slots from icon-feature. Can also have new slots as indicated by "feature-slot" in the above.

(CHUNK-TYPE visual-location slota, slotb, slotc)

(CHUNK-TYPE visual-object slota, slotb, slotc)

Handy commands(dm)(sdm slot value)(sdp chunk-name)(get-chunk chunk-name) ;;returns chunk structure(get-chunk-type 'name) ;;gets type structure from type name(get-module module-name)(my-name (get-module :vision))(print-module-state (get-module :vision))(current-mp)(current-device)(current-device-interface)(buffers)(buffer-chunk buffer-name)(buffer-read 'buffer-name)(chunk-slot-value chunk-name slot-name)(pprint-a-chunk chunk-name)(sdp-fct (no-output (dm)))(sdp-fct (no-output (sdm isa ...)))(gethash :vision (act-r-modules-table *modules-lookup*))(act-r-chunk-type-name (act-r-chunk-chunk-type (get-chunk (buffer-read 'goal))))

Interactive Session

Load and run Letter model

Sub-symbolic level

Sub-symbolic learning allow the system to adapt to the statistical structure of the environment

Production Utilities are responsible for determining which productions get selected when there is a conflict.

Chunk Activations are responsible for determining which (if any chunks) get retrieved and how long it takes to retrieve them.

Chunk Activations have been simplified in ACT-R 5.0 and a major step has been taken towards the goal of parameter-free predictions by fixing a number of the parameters.

Parameters

NoiseUtility and activation

LearningActivation - frequency and recencyUtility - probability and cost

ThresholdsUtility and activation

Sub-symbolic ACT-R theory

Activation equationProduction Utility equationProduction Compilation

Chunk i

Seven

Three FourAddend1 Addend2

Sum

=Goal> isa write

relation sum arg1 Three arg2 Four

+Conditions

+Retrieval> isa addition-fact

addend1 Three addend2 Four

+Actions

Sji

Simkl

Bi

Activation*A i =Bi + Wj ⋅ Sji

j∑ + MPk ⋅Simkl +N(0,s)

k∑

Chunk Activation

Activation = Base Level + Associative + Partial Matching + Noise

Reflects the degree that a chunk will be useful in the current context based upon past experiences:

Base level - general past usefulness

Associative - relevance to current context

Partial Matching - relevance to the specific match

Noise - stochastic, non-deterministic behavior

A i =Bi + Wj ⋅ Sjij∑ + MPk ⋅Simkl +N(0,s)

k∑

Base-level Activation

The base level activation Bi of chunk Ci reflects a context-independent estimation of how likely Ci is to match a production, i.e. Bi is an estimate of the log odds that Ci will be used.

Two factors determine Bi:

• frequency of using Ci

• recency with which Ci was used

Bi is set globally using the “set-all-base-levels” functionBi is set for individual chunks using (set-base-level (get-wme 'set-dow) '(100 -100)) ;;if learning on(set-base-level (get-wme 'set-dow) 50.0) ;if learning off

Ai = Bi

baseactivati

on

activation

=

Base-level Learning

Base-Level Activation reflects the log-odds that a chunk will be needed.

The odds that a fact will be needed decays as a power function of how long it has been since it has been used.

The effects of multiple uses sum in determining the odds of being used.

Base-level Learning Equations

Base Level Learning (BLL) equation (approximation) is

Bi = ln(∑nj=1t-dj) (computationall y expensive learning version tha t is

used if :ol parameter is se t t o n)il

n = number o f presentations for chunkit j= t hetime since t hejth presentationd = decay rate, controlled by base level learni ng paramete (r :bll)

defaul t= 0.5 (s gp:bll 0.5)

Bi = log(∑t-dj) ≈ lo (g n / (1 – d)) – d*lo (g L) (optimize d learning versi ontha t is used if :ol parameter is se t t o )t

L = The lifetime of chunk I (t hetime since its creati )on

Chunk “Presentation”

CreationInitialization (add-dm)Encode from environment (+visual)New goal (+goal> isa …)

MergingWhen goal is cleared (-goal or +goal)Merged if matches a current chunk

Harvest of a retrieval=retrieval Could get multiple presentations from one +retrieval

Source Activation*

The source activations Wj reflect the amount of attention given to elements of the current goal. ACT-R assumes a fixed capacity for source activation

W= Wj reflects an individual difference parameter.

associativestrength

sourceactivation( )+ *

+ Wj * Sjij

Associative Strengths*

The association strength Sji between chunks Cj and Ci is a measure of how often Ci was needed (retrieved) when Cj was element of the goal, i.e. Sji estimates the log likelihood ratio of Cj being a source of activation if Ci was retrieved.

associativestrength

sourceactivation( )+ *

+ Wj * Sji

Sji = S - ln(FANj)FANj = # of chunks in which chunk j is the

value of a slot + 1 for chunk j being associated with itself

S = maximum associative strength- a constant (:mas)

S11

W1

W2

S12

S13

S21

S22

S23

Slot 1

Slot 2

Source1

Source 2

chunk1

chunk2

chunk3

Goal

B1

B2

B3

A1

= B1

+ W1

S11

+ W2

S21

A2

= B2

+ W1

S12

+ W2

S22

A3

= B3

+ W1

S13

+ W2

S23

Partial Matching*

• The mismatch penalty is a measure of the amount of control over memory retrieval: MP = 0 is free association; MP very large means perfect matching; intermediate values allow some mismatching in search of a memory match.

• Similarity values between desired value k specified by the production and actual value l present in the retrieved chunk. This provides generalization properties similar to those in neural networks; the similarity value is essentially equivalent to the dot-product between distributed representations.

+ MPk ⋅Simklk∑

similarityvalue

mismatchpenalty( )*+

Noise

•Noise provides the essential stochasticity of human behavior

•Noise also provides a powerful way of exploring the world

•Activation noise is composed of two noises:

•A permanent noise accounting for encoding variability

•A transient noise for moment-to-moment variation

σ

2

=

π

2

3

s

2

• Generated according to a logistic distribution characterized by parameter s. The mean the distribution is 0 and variance σ2.

Latency

• Retrieval time for a chunk is a negative exponential function of its activation:

A = activation of chunk being retrievedF = latency scale factor (set globally with the :lf parameter)

• If no chunk matches or no chunk is above retrieval threshold

Timei =F ⋅e−Ai

€

Time i = F ⋅e−t

t = retrieval threshold

Probability of Retrieval

• Probability of retrieval of a chunk follows the Boltzmann (softmax) distribution:

€

Pi =eA it

eA jt

j

∑t= 2⋅s=

6⋅σπ

•The chunk with the highest activation is retrieved provided that it reaches the retrieval threshold

•For purposes of latency and probability, the threshold can be considered as a virtual chunk

Interactive Session

Load and run Sternberg model

Making Choices: Conflict Resolution

Expected Gain = E = PG-C + noise

Probability of choosing i = e

E

i

/ t

e

E

j

/ t

j

∑

P =Successes

Successe s + Failures

P is expected probability of successG is value of goalC is expected costNoise = σ2 2 3 s2

where s is set globally by the :egs parameter

t reflects noise in evaluation and is like temperature in the Bolztman equation

Production Utility*

P & C

P =

C =

SuccessesSuccesses + Failures

EffortsSuccesses + Failures

Interactive Session

Load and run Building Sticks model

Production Compilation: The Basic Idea* (p read-stimulus =goal> isa goal step attending state test =visual> isa text value =val==> +retrieval> isa goal relation associate arg1 =val arg2 =ans =goal> relation associate arg1 =val step testing)

(p recall =goal> isa goal relation associate arg1 =val step testing =retrieval> isa goal

relation associate arg1 =val arg2 =ans==> +manual> isa press-key key =ans =goal> step waiting)

(p recall-vanilla =goal> isa goal step attending state test =visual> isa text value "vanilla==> +manual> isa press-key key "7" =goal> relation associate arg1 "vanilla" step waiting)

Production Compilation: The Principles*

1. Perceptual-Motor Buffers: Avoid compositions that will result in jamming when one tries to build two operations on the same buffer into the same production.

2. Retrieval Buffer: Except for failure tests proceduralize out and build more specific productions.

3. Goal Buffers: Complex Rules describing merging.

4. Safe Productions: Production will not produce any result that the original productions did not produce.

5. Parameter Setting: Successes = P*initial-experience*Failures = (1-P) *initial-experience*Efforts = (Successes + Efforts)(C + *cost-penalty*)

References

Introduction to ACT 5.0 Tutorial by Christian Lebiere,http://act-r.psy.cmu.edu/tutorials/

ACTR 5.0 Equations, Variables and Parameters by Jerry Ball

Base Level Activation - Bi

The base level activation Bi of a chunk Ci reflects a context-independent estimation of how likely Ci is to match a production, i.e. Bi is an estimate of the log odds that Ci will be used.

(set-base-level (get-wme 'set-dow) '(100 -100)) ;;if learning on

(set-base-level (get-wme 'set-dow) 50.0) ;if learning off

Associative LearningNot ACT-R 6.0

Sji ≈ ln[ (P i|)/j P( )]i(P i|j) = probability i is needed given j( )P i = base probability ofi

(per Christia n LeBiere if P(i|) j < (P i) then inhibition results)this equation applies in a learning context

Sji = ln(Rji)(aka Posterior Strength Equation 4.3, p. 130 – used in

learning and turne d on bysetting the :al parameter tot)

Rji = assoc*R* ji + F(Cj) Eji / asso c+ F(Cj)assoc* = estimate of we ighting of prior strengthR* ji (fan) = (1/n)/(1/m) = /m n (default prior strengths

tha t reflec tguesses as towha t t he strengths should be)F = frequenc ycount used toestimate Pe(Ni|Cj) a nd Pe(Ni)Cj = the eve ntt hat j is in a goal slot

Eji = Pe(Ni|Cj)/Pe(Ni) = probability ratioP e= Probability estimate

Ni = the eve ntt hat i is neededassoc = weighting of prior strength