1
www.sciencemag.org SCIENCE VOL 340 5 APRIL 2013 21 NEWS&ANALYSIS CREDITS: ADAPTED FROM T. HORIKAWA ET AL., SCIENCE (ADVANCED ONLINE); (PHOTO) MARK HARMEL/SCIENCE SOURCE “From the sky, I saw something like a bronze statue, a big bronze statue,” the drowsy man told neuroscientists as he lay inside an MRI machine that moments earlier had been recording his brain activity as he dozed. “The bronze statue existed on a small hill. Below the hill, there were houses, streets, and trees in an ordinary way.” This surreal recounting of a just- interrupted dream—along with nearly 500 more descriptions gathered from the man and two other research subjects in a similar fashion—is the basis for an ambitious new study reported online this week in Science by a research team at the ATR Computational Neuroscience Labo- ratories in Kyoto, Japan (http://scim. ag/THorikawa). Utilizing these dream reports, the authors have, for the first time, successfully predicted images seen in sleep based exclusively on MRI scans of brain activity. Although researchers warn that we’re still far from having a machine that can fully read our dreams, Robert Stickgold, a neuroscientist at Harvard Medical School in Boston who studies dreams, describes the work as “stunning in its detail and success.” Not long ago, the idea that you could “decode” what people were seeing, thinking, or dreaming about based on brain activity alone was Star Trek, at best,” Stickgold says. Over the past decade, however, ATR neuroscientist Yukiyasu Kamitani has been developing computer algorithms aimed at doing just that. In a 2005 paper in Nature Neuroscience, for example, he and colleagues described creating a computer program that can learn to associate the brain activity patterns recorded during functional magnetic resonance imaging (fMRI) with specific visual stimuli, in order to accurately predict at which of eight orientations of a grid a subject is looking. To determine whether a similar approach could detect what sleeping people see, Kamitani and his colleagues recently recruited three volunteers to lie in MRI machines for 3-hour sessions over the course of 10 days. The loud banging caused by the vibration of an MRI machine’s metal coils isn’t conducive to slumber, but the volunteers wore headphones and found a way to doze. As each one drifted off, the researchers mon- itored the volunteer’s brain activity with the fMRI machine, which measures blood flow in the brain and is thought to reflect neuro- nal activity. They also used an EEG machine to track the brain’s overall electrical activity. Rather than waiting more than an hour for the people to enter rapid eye movement (REM) sleep, which is marked by long, often bizarre narrative dreams and temporary paralysis, the researchers took advantage of the frequent hallucinations that occur dur- ing the onset of sleep, called stage 1. People often don’t know that they are asleep during this stage, Stickgold says. (This is the same stage in which you might try to take the remote control away from a snoring com- panion and be grumpily refused, he adds.) Once the three volunteers became accus- tomed to napping in the fMRI machines, they would fall easily into a light slumber, Kamitani says. Every 6 or 7 minutes, just as one began to drop into a deeper sleep, the researchers would see a ripple of EEG activ- ity suggesting the likely presence of hallu- cinations. The researchers woke subjects, asked them to report anything they had seen, then told them to go back to sleep. After gathering roughly 200 of these reports from each subject, Kamitani’s team extracted frequently repeated visual elements, such as “tree” or “man,” from the reports and grouped them into approximately 20 broad categories tailored to each participant. In one category, for example, “ice pick,” “key,” and “plunger,” were all grouped under the category “implement.” To train a computer program to associate broad categories of images seen in dreams with specific patterns of brain activity in the visual cortex, the researchers gathered photos on the Internet that corresponded to each category and recorded the three vol- unteers’ fMRI activity as they viewed the images. Like other machine-learning tech- nologies that can learn to identify hand- writing based on subtle differences in penmanship, the program weeded out nonvisual brain activity that occurred during sleep and soon learned to asso- ciate specific brain activity “signa- tures” with different types of images, Kamitani says. When the researchers applied the newly trained programs to a second round of dream-reporting by the same trio, coupled with fMRI monitoring, they were able to predict what the people had seen with 60% accuracy— a far higher rate than can be attributed to chance, Kamitani says. “This is probably the first real dem- onstration of the brain basis of dream content,” Stickgold says, describing the results as “incredibly robust.” One exciting possibility, he says, is that the technique might reveal not only what we remember in our dreams, but also what we forget or fail to notice. Technically, the images that the people in this study describe should not be called “dreams” at all, cautions Allan Hobson, a psychiatrist at Harvard Medical School who studies dreams. Instead, they are “hypnagogic hallucinations” with a differ- ent underlying physiology from the classic dreams that occur in REM sleep, he contends. Researchers still find REM and stage 1 sleep mysterious, maintains ATR neuro- scientist Masako Tamaki, a study co-author. “They may not be that different.” She hopes that the technique will be applied in clini- cal sleep research and to help people who experience bad dreams. At a minimum, “it’s nice to hear that what people report seeing when they’re asleep is at least somewhat accurate,” Stickgold says. “Up until this moment, there were no grounds on which to say we don’t just make up our dreams when we wake up.” –EMILY UNDERWOOD Dream catcher. Researchers collected reports of dreamed images from people awoken after sleeping in an MRI machine. fMRI scans Awake Sleep Stage 1 EEG Awakening book building car character commodity computer screen covering dwelling electronic equipment female food furniture male mercantile establishment point region representation street Awakening index 50 100 150 Verbal Report How to Build a Dream-Reading Machine NEUROSCIENCE Published by AAAS on January 18, 2014 www.sciencemag.org Downloaded from

How to Build a Dream-Reading Machine

  • Upload
    e

  • View
    238

  • Download
    3

Embed Size (px)

Citation preview

Page 1: How to Build a Dream-Reading Machine

www.sciencemag.org SCIENCE VOL 340 5 APRIL 2013 21

NEWS&ANALYSISC

RE

DIT

S: A

DA

PT

ED

FR

OM

T. H

OR

IKA

WA

ET

AL., S

CIE

NC

E (A

DV

AN

CE

D O

NLIN

E);

(P

HO

TO

) M

AR

K H

AR

ME

L/S

CIE

NC

E S

OU

RC

E

“From the sky, I saw something like a bronze

statue, a big bronze statue,” the drowsy man

told neuroscientists as he lay inside an MRI

machine that moments earlier had been

recording his brain activity as he dozed.

“The bronze statue existed on a small hill.

Below the hill, there were houses, streets,

and trees in an ordinary way.”

This surreal recounting of a just-

interrupted dream—along with nearly 500

more descriptions gathered from the man

and two other research subjects in a similar

fashion—is the basis for an ambitious

new study reported online this week in

Science by a research team at the ATR

Computational Neuroscience Labo-

ratories in Kyoto, Japan (http://scim.

ag/THorikawa). Utilizing these dream

reports, the authors have, for the fi rst

time, successfully predicted images

seen in sleep based exclusively on

MRI scans of brain activity. Although

researchers warn that we’re still far

from having a machine that can fully

read our dreams, Robert Stickgold,

a neuroscientist at Harvard Medical

School in Boston who studies dreams,

describes the work as “stunning in its

detail and success.”

Not long ago, the idea that you

could “decode” what people were

seeing, thinking, or dreaming about

based on brain activity alone was

“Star Trek, at best,” Stickgold says.

Over the past decade, however, ATR

neuroscientist Yukiyasu Kamitani has

been developing computer algorithms

aimed at doing just that. In a 2005 paper in

Nature Neuroscience, for example, he and

colleagues described creating a computer

program that can learn to associate the brain

activity patterns recorded during functional

magnetic resonance imaging (fMRI) with

specifi c visual stimuli, in order to accurately

predict at which of eight orientations of a

grid a subject is looking.

To determine whether a similar approach

could detect what sleeping people see,

Kamitani and his colleagues recently

recruited three volunteers to lie in MRI

machines for 3-hour sessions over the course

of 10 days. The loud banging caused by the

vibration of an MRI machine’s metal coils

isn’t conducive to slumber, but the volunteers

wore headphones and found a way to doze.

As each one drifted off, the researchers mon-

itored the volunteer’s brain activity with the

fMRI machine, which measures blood fl ow

in the brain and is thought to refl ect neuro-

nal activity. They also used an EEG machine

to track the brain’s overall electrical activity.

Rather than waiting more than an hour

for the people to enter rapid eye movement

(REM) sleep, which is marked by long, often

bizarre narrative dreams and temporary

paralysis, the researchers took advantage of

the frequent hallucinations that occur dur-

ing the onset of sleep, called stage 1. People

often don’t know that they are asleep during

this stage, Stickgold says. (This is the same

stage in which you might try to take the

remote control away from a snoring com-

panion and be grumpily refused, he adds.)

Once the three volunteers became accus-

tomed to napping in the fMRI machines,

they would fall easily into a light slumber,

Kamitani says. Every 6 or 7 minutes, just as

one began to drop into a deeper sleep, the

researchers would see a ripple of EEG activ-

ity suggesting the likely presence of hallu-

cinations. The researchers woke subjects,

asked them to report anything they had seen,

then told them to go back to sleep. After

gathering roughly 200 of these reports from

each subject, Kamitani’s team extracted

frequently repeated visual elements, such

as “tree” or “man,” from the reports and

grouped them into approximately 20 broad

categories tailored to each participant. In

one category, for example, “ice pick,” “key,”

and “plunger,” were all grouped under the

category “implement.”

To train a computer program to associate

broad categories of images seen in dreams

with specifi c patterns of brain activity in

the visual cortex, the researchers gathered

photos on the Internet that corresponded to

each category and recorded the three vol-

unteers’ fMRI activity as they viewed the

images. Like other machine-learning tech-

nologies that can learn to identify hand-

writing based on subtle differences in

penmanship, the program weeded out

nonvisual brain activity that occurred

during sleep and soon learned to asso-

ciate specifi c brain activity “signa-

tures” with different types of images,

Kamitani says.

When the researchers applied the

newly trained programs to a second

round of dream-reporting by the same

trio, coupled with fMRI monitoring,

they were able to predict what the

people had seen with 60% accuracy—

a far higher rate than can be attributed

to chance, Kamitani says.

“This is probably the fi rst real dem-

onstration of the brain basis of dream

content,” Stickgold says, describing

the results as “incredibly robust.” One

exciting possibility, he says, is that the

technique might reveal not only what

we remember in our dreams, but also

what we forget or fail to notice.

Technically, the images that the

people in this study describe should

not be called “dreams” at all, cautions Allan

Hobson, a psychiatrist at Harvard Medical

School who studies dreams. Instead, they are

“hypnagogic hallucinations” with a differ-

ent underlying physiology from the classic

dreams that occur in REM sleep, he contends.

Researchers still fi nd REM and stage 1

sleep mysterious, maintains ATR neuro-

scientist Masako Tamaki, a study co-author.

“They may not be that different.” She hopes

that the technique will be applied in clini-

cal sleep research and to help people who

experience bad dreams. At a minimum, “it’s

nice to hear that what people report seeing

when they’re asleep is at least somewhat

accurate,” Stickgold says. “Up until this

moment, there were no grounds on which to

say we don’t just make up our dreams when

we wake up.”

–EMILY UNDERWOOD

Dream catcher. Researchers collected reports of dreamed images

from people awoken after sleeping in an MRI machine.

fMRI

scans

Awake

Sleep Stage 1

EEG

Awakening

bookbuilding

carcharacter

commoditycomputer screen

coveringdwelling

electronic equipmentfemale

foodfurniture

malemercantile establishment

pointregion

representationstreet

Awakening index

50 100 150

Verbal

Report

How to Build a Dream-Reading Machine

N E U R O S C I E N C E

Published by AAAS

on

Janu

ary

18, 2

014

ww

w.s

cien

cem

ag.o

rgD

ownl

oade

d fr

om