Cognitive RulesThe Paradigm of Human Memory

Elizabeth F. Churchill• B.Sc. In experimental psychology

• University of Sussex

• M.S in Knowledge based systems• University of Sussex

• PhD in Cognitive Science -1988• University of Cambridge

• Conducted research on SOAR

SOAR • Cognitive architecture• Goal was to create a system with the same cognitive abilities as humans• Created by John, Laird, Allen Newell, and Paul Rosenbloom at Carnegie

Mellon university• Currently managed by John Laird’s research group at the University of

MichiganNate Derbinsky Discussing SOAR Architecture

A site for SOAR Eyes• Dr. Churchill was creating mental models in a laboratory setting• Intrigued by DR. King’s research method – best site of study was where

the activity was • Contrasted Dr. Beach’s field study with her own form of laboratory

experimentation• Came away feeling that “Cognition becomes inextricably linked to place –

physically, perceptually, culturally, and theoretically”• Believes that attention, learning, and memory are integral to today’s

design decisions

King D. Beach III• Ph.D. developmental psychology• Previously an Associate professor –

MSU• Currently Associate professor - FSU• Background in cultural anthropology• Interested in how people learn and

develop with changing forms of social organizations

Activity Theory• Philosophical framework to help understand the human developmental1

process at the individual and social level• Helps us understand Why people perform activities• Originally developed by Russian Psychologist Aleksei Leontiev• Activities as purposeful interactions between people and objects• Helps us understand the changing nature of external mnemonic cues

• Kaptelinin, 2012

Activity theory concepts

• Mediation– Human activity mediated by external and internal tools(Drink glasses)

with a particular history and culture

• Internalization/Externalization– Recollection/analysis/calculation of activity through transforming

outward activities to internal ones. • Ex: Mental calculations, simulations, reciting drink ingredients• Can help us find the best way to perform an action before doing it

– Externalization transitions internal activities to external ones

Question

Why is the activity theory important as a concept in HCI research?

Becoming a Bartender

• Experimental science conducted by Dr. Beach in order to study memory.

• Specifically, how people remember the ingredients of complex cocktails and mix them together in a busy, time sensitive environment

• Dr. Beach enrolled in a 2 week bartending course to conduct the study

• Discussed Mnemonic cues as aids to recollection

Bartending School

• 2 week course for ages 18 and over• Formal lecture demonstrations and practice• 22 to 30 students• 4 hour class – 1 hour lecture, 3 hour practice• Graduates expected to be able to mix any combination of 100

different mixed drinks rapidly and accurately & from memory

» San Francisco School of Bartending

Mnemonic Cues

• Two types of mnemonic cues discussed – Verbal mnemonic symbols– words used to aid recollection– Have an arbitrary relation to their referents

• Example: reciting recipe ingredients while making drinks

– Material mnemonic symbols – based on environment and context– Direct relationship to their referents

• Example: using visually distinctive glasses and visible ingredients

Question

What Form of Memory Cues are more helpful to you in aiding recollection?

Experimental Study• On site experiments and interviews in addition to qualitative observations• Purpose was to measure differences in speed and accuracy disparities

from novice to pro bartenders, as well as the use of memory cues• 10 novices, 10 recent grads and 2 instructors/professionals volunteered • Study was videotaped • Experiment devised around speed drills

– Mixing a combination of 4 drinks quickly and accurately– Each subject completes 6 speed drills– Each order called for at least 2 different glass shapes– Clear bar glasses were used at first, then black glasses to hide drinks– Distractions were used during second and third drills

Findings• The use of VMS sharply decreased with increasing experience• Graduates and instructors were faster and more accurate than novices

with detecting and pouring ingredients• Distractions like counting backwards increased drink errors only for

novices• MMS was helpful with making drinks for grads, not as much for pro’s• As VMS transitioned to MMS and beyond, there was a “shift in internally

driven recall to mediated recognition”1

• 1 – Churchill, 2008

Question

What method of study is more effective? Field investigations or Laboratory study?

Question

Is DR. Beach’s assessment of activity theory – of transitioning from VMS to MMS - relevant

today?

Adam M. Fass• B.S Computer Science

• State University of New York at Buffalo – 1998

• Ph.D in Computer Science• Carnegie Mellon University – 2005• Ph.D advised by Randy Pausch• Ph.D dissertation - MessyBoard: Lowering the Cost of

Communication and Making it More Enjoyable

Software Engineer at Google

Jodi Forlizzi• BFA, Illustration

– Philadelphia College of Art

• MDes, Interaction Design– Carnegie Mellon University

• Ph.D., Design in Human Computer Interaction – Carnegie Mellon University– Advisors – Sara Kiesler and Pamela J. Hinds

• Tenured Associate Professor• HCI Institute & School of design – Carnegie Mellon University

• Previously held the position of Innovator and Project Manager for E-Lab in Chicago, where she specialized in research for new product design.

Randy Pausch• Born October 23, 1960 in Baltimore, MD

– Grew up in Columbia, MD

• Died July 25, 2008 in Chesapeake, Virginia (Pancreatic Cancer)

• B.S, Computer Science– Brown University

• Ph.D., Computer Science– Carnegie Mellon University

• Time 100 most influential – May 2008• Known worldwide for the “Alice” Software Project

as well as the “Last Lecture”• The Last Lecture - Alice

“Don’t complain; just work harder.” 

MessyDesk & MessyBoard

• Two different programs designed by researchers with the initial goal of improving human memory for information that a user views or works with on a computer

Messy Motivations

• Currently, computers do not provide usable context to help the user recall relevant information

• Ex: Presenting the user with the same environment and surroundings that existed when the information was first consumed

• Researchers believed the best context to aid memory needed distinction, be meaningfully related, and be user created.

• Present the user with the chance to create context

MessyDesk• Purpose was to build a computer with context• Software program that makes use of the desktop for decoration and

information management, acts as a replacement desktop• Can drag an image from any app onto the desktop, or post from clipboard,

can also add notes• Used by researchers at Carnegie Mellon and UVA

MessyDesk Findings• Initially very hard to get anyone, even researchers to use this• Only those with multiple monitors would use this• Decorating alone is not a strong enough incentive to create memory

assisting context• Artists thought it was cool to decorate, grad students failed to see the

usefulness

Question

What could the researchers have done to increase the incentive to decorate?

MessyBoard• Created because decoration is not a strong enough incentive• Networked bulletin board used for sharing pics, files, etc.• People see the same content at the same time, changes seen in real time• Stores a complete history of all activity, can slide back in time • Believed with communication as incentive, Messyboard could enhance

memory

MessyBoard Findings• Different teams used MessyBoard differently• Projecting content on the wall affected interest and use• Was good for simple games and sharing jokes• Good for discussion and collaboration – Scheduling meetings

Messy Memory• Interviewed users to see how they used the software• Would people use content and the history slider to cue memory?• Believed shared information management along with decoration

encourages memory• Neither MessyDesk or MessyBoard were formally tested to gauge

measureable effect on memory

Question

Do we really need to think about supplementing memory when designing interfaces?

Question

Were the Messy Applications a good approach as a memory aid?

Question

Did the applications succeed in opening up new or exciting areas of future research?

Thanks!