Grand Challenge: Memories for Life

Nigel Shadbolt and Wendy Hall

The University of Southampton

And Where is Wendy Hall?

Structure

• What is a Grand Challenge?• History of the M4L GC• The phenomena of memory• The science of memory• The technology of memory• Application Opportunities• Issues

– Ethical– Funding– Collaboration

What is a Grand Challenge?

• Revolutionary shift in thinking or practice• Enthusiastic support from scientific communities• Appeals to the public imagination• A clear criterion for success or failure• Long term benefits to science, industry, society• International scope• Interdisciplinary, collaborative research• Examplars…

Grand Challenges

• Put a man on the moon within a decade

• Map the human genome

• Build a computer to beat the world chess champion

Memories for Life Grand Challenge

• Part of the UKCRC/EPSRC sponsored Grand Challenges for Computer Science Research – November 2002 Workshop

• Surfaced as a topic again during the Foresight Cognitive Systems Workshops

• Submitted a proposal to hold a planning event to Foresight – held London mid August

• Why the interest?

Memories are Compelling

• We are quite literally our memories• They are both personal and social• They exist in all modalities• Pervasive and central in all walks of life • What is your first memory?• Do you remember you first day at school?• Do you remember the first time you hurt your sibling?• Are they real or constructed?• Are they lost, diminished or overlaid?

Wendy’s Life Bits

Memories: Shared and Private

• Certain events as memories evoke time and place with exquisite clarity

The scale of evocation

• Some are global flash bulb memories others are national and many are personal

Memories and Science Fiction

The Science of Memory

• Memory – recognition of multiple systems• Working and Short Term Memory

– Multi–component; e.g auditory/verbal STM, visual/spatial LTM

• Long Term Memory– Episodic When did you last ride a bicycle?– Semantic What is a bicycle?– Procedural How do you ride a bicycle?– Recognition Is this a bicycle?– Value Phobic to bicycles?

The Psychology of Memory: Exemplar Differences

Episodic Memory:

Reference is to oneselfOrganised temporallyEvents remembered

“consciously”Susceptible to forgettingContext dependent

Semantic Memory:

Reference is with respect to general knowledge

Not organised temporally

Events are “known”Relatively permanentContext independent

Shacter’s Seven “Sins” of Memory

Transience Weakening or loss of memory over time

Absent-mindedness Breakdown between attention and memory

Blocking Failure to retrieve

Misattribution Incorrect provenance

Suggestibility Reconstruction and reinterpretation

Bias Misrepresentation of memory

Persistence Inability to suppress or remove memory

The Locus of Memory

Mechanisms for Memory: LTP

Memories and Plasticity

• Neurons send out axons to synapse with targets

• Once established targets supply neurotrophic (NT) factors

• These factors are essential to the continued survival of innervating neurons

• If a neuron receives too little factor it dies

• Target innervation and neuronal elimination are adaptive mechanisms

Model of Plasticity

Applied in silico to development of a variety of cortical maps

Assumption – time average uptake of NTF by i from x determines the number of synapses projected by i to x

T. Elliott and N.R. Shadbolt. Developmental robotics: Manifesto and application." Philosophical Transactions of the Royal Society of London, Series A. In the press.

Koala Experiments

Memories for Life: The Dix Measure

• 70 years ~ 25,550 days, 613,00 hours, 2.2x109 secs

• 100 kbits/sec for audio/video

• 27.5 terabytes for a life

• 343 80 gigabyte hard drives

• 2 years ~ 60 million seconds of data ~ 6 hard drives

• 2073 storage capacity could have doubled 47 times

• Capacities could have increased 12 orders of magnitude

• Your life on a grain of sand!

Memories for Life: The Computing Power

Memories for Life: The Hardware

• Fujitsu - .8 inch 80 gigabyte hard drive

• Video cameras

• Video transmitter

• Complete PC

• Other modalities e.g smell

Memories for Life: The Computer Science

• Database Systems

• Security

• Operating Systems and Versioning,

• Persistence of format, re-represenations

• Artificial Intelligence

• Human Computer Interaction

• Visualisation and Virtual Reality

Memories for Life and Information Overload

We are drowning in information and starving for knowledge

Infosmog: The condition of having too much information to be able to take effective action or make an informed decision

The deluge of data is overwhelming

Ontologies: Shared Conceptualisations

Automatic Annotation

• Associating meta-data with content

• Large-scale annotation of natural language texts, images, streaming media

Narrative Generation

• Using NLP, structure generating and linking techniques to build narratives

Associative Linking

• Supporting multiple context or associative indexes into content

Web and Grid Services

• Seamless access to computationally intensive services – image registration, annotation, classification…

The Perfect Storm

• Convergence of at least three disciplines– Neuroscience and psychology– Device engineering– Computer Science

• Potential to create a truly remarkable range of applications

Memories for Life: Application Contexts

• Multimedia Searching• Large Scale Experience Repositories

– (e.g Big Brother, early child interactions….)

• Continuous health record• Stories from a Life• Intelligent Mathematics Tutor• Memory support in Elder Care• Virtual Memories

Lest we forget

• The the nature and role of forgetting in natural and artificial systems

• The desirability of post-mortem memories

• The way in which these memory prostheses and the externalisation of memory will change social practice

• Memory is a social and cultural construct

Other efforts

• Xanadu – Ted Nelson

• MyLifeBits – Gordon Bell@Microsoft

• LifeLog – DARPA

Ethical Issues

• Ethical– Privacy– Trust

Future Events

• M4L London August 20th

• IAC Bristol• Follow up workshop primarily life science based

CS early 2004• Primarily CS workshop at March BCS Grand

Challenges in Computing 04 Newcastle• End of 2004 CS and Life Science International

Workshop

Summary

• This is a compelling Grand Challenge• We have enthusiastic engagement of researchers

and Learned Societies - BCS/IEE/BNA/EPS• Like to seek RCUK funding for network support• Support for large scale experience repositories• All communities stand to gain

– Common problem space – fundamental science etc– Richness/availability of data, technology and methods

will support/mediate interaction

• Ethical and social issues must be considered