Bioelectronics, Medical Imaging and Our Bodies

Week 5: Reverse engineering & neuromorphic engineering

Maryse de la Giroday6-week course

SFU Liberal Arts & Adults 55+ program

It’s all about the … (1 of 4)

• Sitting quietly in jars in a custom-built room at Yale's medical library are 550 human brains.

It’s all about the … (2 of 4)

• Harvey Cushing, who preserved the brains from 1903 to 1932 as part of his tumor registry. When Cushing died in 1939, his undergraduate alma mater Yale inherited the brains.

It’s all about the … (3 of 4)

• Cushing introduced practices that dramatically lowered the mortality rate, such as monitoring blood pressure during surgery and operating with a local anesthesic instead of ether. He was also the first to use x-rays to diagnose brain tumors.

It’s all about the … (4 of 4)

• Prior to being restored and placed in the medical library in 2010, the leaky jars holding Cushing's brain collection were locked in a basement under Yale's med student dorms.

• http://www.slate.com/blogs/atlas_obscura/2014/10/06/brains_in_jars_at_the_cushing_center_in_the_yale_medical_library.html

and reverse engineering

• What is that?

Example of reverse engineering (1 of 8)

• (http://www.pbs.org/wgbh/nova/ancient/great-cathedral-mystery.html)

Brunelleschi’s Duomo (2 of 8)

• The dome that crowns Florence’s great cathedral of Santa Maria del Fiore—the Duomo—is a towering masterpiece of Renaissance ingenuity and an enduring source of mystery. Still the largest masonry dome on earth after more than six centuries, it is taller than the Statue of Liberty and weighs as much as an average cruise ship.

Brunelleschi’s Duomo (3 of 8)

• Historians and engineers have long debated how its secretive architect, Filippo Brunelleschi, managed to keep the dome perfectly aligned and symmetrical as the sides rose and converged toward the center, 40 stories above the cathedral floor. His laborers toiled without safety nets, applying novel, untried methods. Over 4 million bricks might collapse at any moment—and we still don’t understand how Brunelleschi prevented it.

Brunelleschi’s Duomo (4 of 8)

• How did a hot-tempered , 15th century, goldsmith with no formal architectural training create the most miraculous edifice of the Renaissance?

• The cathedral was started in 1296 to showcase Florence and its status.

• The dome was intended to be the largest cupola on earth (http://ngm.nationalgeographic.com/2014/02/il-duomo/mueller-text)

Brunelleschi’s Duomo (5 of 8)

• Even by the early 15th Century , no one had figured out how to build a dome that was 150 ft. across and started 180 ft. above the ground on walls that had already been built.

• In 1418 there was a competition and Brunelleschi who won “promised to build not one but two domes, one nested inside the other, without elaborate and expensive scaffolding. Yet he refused to explain how he’d achieve this, fearing that a competitor would steal his ideas. Brunelleschi’s stubbornness led to a shouting match with the overseers, who twice had him restrained and forcibly ejected from the assembly, denouncing him as ‘a buffoon and a babbler’.”

Brunelleschi’s Duomo (6 of 8)

• He (as a boy) mastered – drawing and painting, – wood carving,– sculpture in silver and bronze, – stone setting, – niello, and – enamel work.

Brunelleschi’s Duomo (7 of 8)

• Later he studied optics and tinkered endlessly with wheels, gears, weights, and motion, building a number of ingenious clocks, including what may have been one of the first alarm clocks in history.

• And, he single-handedly worked out the rules of linear perspective.

Brunelleschi’s Duomo (8 of 8)

• Italian academic Massimo Ricci believes he has successfully reverse-engineered the dome without any contemporary tools (computers, building materials, etc.)

• Key to solving the puzzle was a flower• Ricci believes that Brunelleschi created a metal

flower and used the petals as a model for how to place the bricks (http://www.saylor.org/site/wp-content/uploads/2011/11/ARTH-206-Dome.pdf)

Reverse engineering the brain

• Why?

Brain-in-a-dish (1 of 5)

Brain-in-a-dish (2 of 5)

• Tufts researchers have created a new, modular design of bioengineered brain-like tissue. On injury, this brain-like tissue responds with biochemical and electrophysiological outcomes that mimic observations in vivo.

Brain-in-a-dish (3 of 5)

• This model offers new directions for studies of brain function, disease and injury.

• Each module combined two materials with different properties: a stiffer porous scaffold made of silk protein on which cortical neurons, derived from rats, could anchor and a softer collagen gel matrix that allowed axons (projections from the neuron that conduct impulses away from the nerve body) to penetrate and connect three-dimensionally.

Brain-in-a-dish (4 of 5)

• The silk scaffolds were assembled into concentric rings to simulate the layers of the neocortex. Each layer was dyed with food color and seeded with [rat]neurons independently.

• Tufts University researchers today [Aug. 11, 2014] announced development of the first reported complex three-dimensional model made of brain-like cortical tissue that exhibits biochemical and electrophysiological responses and can function in the laboratory for months.

Brain-in-a-dish (5 of 5)

• "There are few good options for studying the physiology of the living brain, yet this is perhaps one of the biggest areas of unmet clinical need when you consider the need for new options to understand and treat a wide range of neurological disorders associated with the brain. … ) said Kaplan [David Kaplan, Stern Family professor and chair of biomedical engineering at Tufts School of Engineering]

• http://www.eurekalert.org/pub_releases/2014-08/tu-bmf080614.php

Brain-in-a-dish (2nd view)

• Bioengineers have created three-dimensional brain-like tissue that functions like and has structural features similar to tissue in the rat brain and that can be kept alive in the lab for more than two months.

Brain-in-a-dish (2nd view)

• As a first demonstration of its potential, researchers used the brain-like tissue to study chemical and electrical changes that occur immediately following traumatic brain injury and, in a separate experiment, changes that occur in response to a drug. The tissue could provide a superior model for studying normal brain function as well as injury and disease, and could assist in the development of new treatments for brain dysfunction.

• http://www.eurekalert.org/pub_releases/2014-08/niob-bcf080714.php

Brain-in-a-dish (3rd view)

• Dr. Kaplan said his team was working on sustaining the brainlike tissue for six months — and with human neurons created from stem cells. He plans to add a model of the brain’s vascular system, so researchers can study what happens when drugs cross the blood-brain barrier.

• http://www.nytimes.com/2014/08/12/health/scientists-create-3d-model-that-mimics-brain-function.html

Radiogenetics (1 of 2)

• New way to remotely control brain cells: radio waves– Could allow researchers to rapidly activate or silence

neurons within a small area of the brain or dispersed across a larger region, including those in difficult-to-access locations

– potential for use treating patients• The project will make use of a technique called

radiogenetics that combines the use of radio waves or magnetic fields with nanoparticles to turn neurons on or off

Radiogenetics (2 of 2)

• Researcher: Sarah Stanley/Rockefeller University

• Grant for $1.26 million over three years, is one of 58 projects in the first round of BRAIN awards

• http://newswire.rockefeller.edu/2014/10/07/rockefeller-neurobiology-lab-is-awarded-first-round-brain-initiative-grant/

Controlling the brain (1 of 7)

• Trauma– McLean Hospital researchers are reporting that xenon gas, used in

humans for anesthesia and diagnostic imaging, has the potential to be a treatment for post-traumatic stress disorder (PTSD) and other memory-related disorders.

– "In our study, we found that xenon gas has the capability of reducing memories of traumatic events," said Edward G. Meloni, PhD, assistant psychologist at McLean Hospital and an assistant professor of Psychiatry at Harvard Medical School. "It's an exciting breakthrough, as this has the potential to be a new treatment for individuals suffering from PTSD."

• http://www.eurekalert.org/pub_releases/2014-08/mh-xes082214.php

Controlling the brain (2 of 7)

• "We have shown how the emotional valence of memories can be switched on the cellular level.“– By manipulating neural circuits in the brain of mice, scientists

have altered the emotional associations of specific memories. The research, led by Howard Hughes Medical Institute investigator Susumu Tonegawa at the Massachusetts Institute of Technology (MIT), reveals that the connections between the part of the brain that stores contextual information about an experience and the part of the brain that stores the emotional memory of that experience are malleable.

• http://www.eurekalert.org/pub_releases/2014-08/hhmi-rct082714.php

Controlling the brain (3 of 7)

• Injury– Treatment with xenon gas after a head injury

reduces the extent of brain damage, according to a study in mice.

– Head injury is the leading cause of death and disability in people aged under 45 in developed countries, mostly resulting from falls and road accidents. The primary injury caused by the initial mechanical force is followed by a secondary injury which develops in the hours and days afterwards.

Controlling the brain (4 of 7)– This secondary injury is largely responsible for patients'

mental and physical disabilities, but there are currently no drug treatments that can be given after the accident to stop it from occurring.

– Scientists at Imperial College London found that xenon, given within hours of the initial injury, limits brain damage and improves neurological outcomes in mice, both in the short term and long term. The findings, published in the journal Critical Care Medicine, could lead to clinical trials of xenon as a treatment for head injury in humans.

• http://www.eurekalert.org/pub_releases/2014-09/icl-xgp090914.php

Controlling the brain (5 of 7)

• … researchers at the UC Davis Center for Neuroscience and Department of Psychology have used light to erase specific memories in mice, and proved a basic theory of how different parts of the brain work together to retrieve episodic memories.

• http://news.ucdavis.edu/search/news_detail.lasso?id=11053

Controlling the brain (6 of 7)

• Remember the single dose of brain-changing antidepressant (week 4)?

• A single dose of antidepressant is enough to produce dramatic changes in the functional architecture of the human brain. Brain scans taken of people before and after an acute dose of a commonly prescribed SSRI (serotonin reuptake inhibitor) reveal changes in connectivity within three hours, say researchers who report their observations in the Cell Press journal Current Biology on September 18.

Controlling the brain (7 of 7)

• "We were not expecting the SSRI to have such a prominent effect on such a short timescale or for the resulting signal to encompass the entire brain," says Julia Sacher of the Max Planck Institute for Human Cognitive and Brain Sciences

• http://www.eurekalert.org/pub_releases/2014-09/cp-sdo091114.php

Controlling devices with the brain (1 of 12)

• Brain-controlled robotic arm• http://www.frogheart.ca/?p=6817 (May 2012)• The video shows a woman getting herself a

cup of coffee for the first time in 15 years. She’s tetraplegic (aka quadriplegic) and is participating in a research project funded by DARPA (US Defense Advanced Research Projects Agency) for developing neuroprostheses.

Controlling devices with the brain (2 of 12)

• Miguel Nicolelis & the 2014 world cup kick• I’ve been covering this story since 2011 and,

even so, was late to the party … Nicolelis has been working on a brain-controlled exoskeleton to allow paraplegics to walk again since roughly 2004 (http://www.frogheart.ca/?p=13351)

Controlling devices with the brain (3 of 12)

• Kick Video: http://www.youtube.com/watch?v=fZrvdODe1QI

• Juliano Pinto, a 29-year-old paraplegic, dressed in Brazil's home colours, walked out into the Arena de Sao Paulo stadium and performed the ceremonial first kick at the 2014 FIFA World Cup. (June 2014)

Controlling devices with the brain (4 of 12)

• His power-kick came with the help of a robotic exoskeleton suit, a new technology that is mind-controlled.

• More than 150 scientists and rehabilitation professionals from around the world collaborated on the mind-controlled robotic exoskeleton suit, also called the Walk Again Project. Led by Nicolelis, the group's goal is to eventually make wheelchairs obsolete.

• http://www.ctvnews.ca/sci-tech/cyborg-soccer-how-a-paraplegic-took-first-kick-at-the-world-cup-1.1868837

Controlling devices with the brain ( 5 of 12)

• Consortium of European research institutions and companies want to get affected patients quite literally back on their feet.

• In the EU’s [European Union] NEUWalk project, which has been awarded funding of some nine million euros, researchers are working on a new method of treatment designed to restore motor function in patients who have suffered severe injuries to their spinal cord. (video: http://www.neuwalk.eu/)

Controlling devices with the brain (6 of 12)

• In another project: the technique relies on electrically stimulating the nerve pathways in the spinal cord.

• “In the injured area, the nerve cells have been damaged to such an extent that they no longer receive usable information from the brain, so the stimulation needs to be delivered beneath that,” explains Dr. Peter Detemple

• Fraunhofer device (frogheart post & in notes)

Controlling devices with the brain (7 of 12)

Controlling devices with the brain (8 of 12)

• The implantable microelectrode sensors are flexible and wafer-thin. © Fraunhofer IMMhttp://www.fraunhofer.de/en/press/research-news/2014/may/hope-for-paraplegic-patients.html

• http://www.frogheart.ca/?p=13448

Controlling devices with the brain (9 of 12)

• For the first time, robotic prostheses controlled via implanted neuromuscular interfaces have become a clinical reality. A novel osseointegrated (bone-anchored) implant system gives patients new opportunities in their daily life and professional activities.

• In January 2013 a Swedish arm amputee was the first person in the world to receive a prosthesis with a direct connection to bone, nerves and muscles.

• http://www.eurekalert.org/pub_releases/2014-10/cuot-mpa100214.php & http://www.frogheart.ca/?p=14845

Controlling devices with the brain (10 of 12)

• Week 1: Intraosseous Transcutaneous Amputation Prosthesis (ITAP) in the “Growing into your prosthetics “ post (http://www.frogheart.ca/?p=3964)

Controlling devices with the brain (11 of 12)

• Media, pop culture, pull marketing, & product placement

• Glee http://www.youtube.com/watch?v=IaiO8a1ZY5g0

Controlling devices with the brain (12 of 12)

• Grey's Anatomy• Owen hopes that by showing Callie the

Veterans Hospital patients, she will help them via her robotic limb lab. Elsewhere, Bailey and Alex get ready to go in front of the board while Jo becomes jealous of Alex and Meredith’s friendship. (Thursday, Oct. 9, 2014)

Cyborgs

• What’s a cyborg?• Wheelchair as part of the body mentioned in

Week 1 (RoboLaw)

Alfred North Whitehead

• There is no absolute gap between living and nonliving. (paraphrasing: A Key to Whitehead's Process and Reality, paperback, Sep 15 1981 by Donald W. Sherburne [Process and reality, 1929])

Issue of undecidability

• Both dead and alive [inanimate and animate], human and inhuman [flesh and machine], monsters always threaten the security of our closed economies. Rather than confronting us from the “outside,” the monster, like the pharmakon, always shows (monstrare) as a disturbance or undecidability that already resides on the “inside.”

• The philosophy of Derrida by Mark Dooley and Liam Kavanaugh (p. 29)

Zombies anyone?

• http://www.youtube.com/watch?v=6XZ9R2TUq94

• Zombie as a set of symbols have been of particular interest for the last few years (e.g. Zombie Survival Guide, 2003, Max Brooks started Brooks’ career and led to a high point: a zombie movie, World War Z with Brad Pitt based on Brooks’ 2006 book of the same title)

Preparedness 101: Zombie PandemicUS Centers for Disease Control

Zombies, history, and pop culture (1 of 2)

• Haiti (1804 and 1825) and voodoo• Race• Writers (you can find a history of voodoo and

writers in, “Better Off Dead: The Evolution of the Zombie as Post-Human”)

• Hollywood– 1930s: White Zombie (young man turns to a witch

doctor to lure the woman he loves away from her fiancé, but instead turns her into a zombie slave)

– The White Zombie has no will of her own (1932)

Zombies, history, and pop culture (2 of 2)

• Hollywood– 1960s: Night of the Living Dead (A group of people

hide from bloodthirsty zombies in a farmhouse.)– Zombies have become cannibals who lust after

your brains (1969)• Current zombie epidemic in movies seems to

coincide with an increasing interest in the brain (exercising it, avoiding Alzheimers/senility, controlling it, figuring out how it works, etc.)

Zombie, culture, and science (1 of 3)

• The Girl With All the Gifts, a zombie novel of ideas (review by Torie Bosch for Slate0

• http://www.slate.com/articles/arts/books/2014/07/mike_carey_s_zombie_novel_the_girl_with_all_the_gifts_reviewed.html

• The book’s monsters are steered by a mutant version of the fungus Ophiocordyceps unilateralis—which you may know as the parasite behind “zombie ants.”

Zombie, culture, and science (2 of 3)

• Military excursions into zombie-infested turf begin finding normal-looking children who can speak, learn, and think, but are nevertheless infected with Ophiocordyceps. Rounded up into a lockdown boarding school, the infected kids live in individual cells, leaving only for weekly shower-and-grub sessions (and yes, those are literal grubs), and for class, where they are strapped and locked into chairs. …

• Miss Justineau and Caldwell do battle throughout the novel, with Miss Justineau’s compassion and eventual love for Melanie contrasting with Caldwell’s hunger for the girl’s brain.

Zombie, culture, and science (3 of 3)

• . (Who’s the zombie again?) During one argument, Caldwell rants:

• You should ask yourself … why you’re so keen on thinking of me as the enemy. If I make a vaccine, it might cure people like Melanie, who already have a partial immunity to Ophiocordyceps. It would certainly prevent thousands upon thousands of other children from ending up the way she has. Which weighs the most, Helen? Which will do the most good in the end? Your compassion, or my commitment to my work? Or could it be that you shout at me and disrespect me to stop yourself from having to ask questions like that?

Emergence of a new culture? (1 of 2)

• Zombie as a symbol of a new culture• Cyborgs• Robots• Androids• Transhumans (Transhumanism (abbreviated as H+ or h+)

is an international cultural and intellectual movement with an eventual goal of fundamentally transforming the human condition by developing and making widely available technologies to greatly enhance human intellectual, physical, and psychological capacities.[1]

Emergence of a new culture? (2 of 2)

• Transhumanist thinkers study the potential benefits and dangers of emerging technologies that could overcome fundamental human limitations, as well as the ethics of developing and using such technologies.[2] They speculate that human beings may eventually be able to transform themselves into beings with such greatly expanded abilities as to merit the label "posthuman".[1] (http://en.wikipedia.org/wiki/Transhumanism)

The uncanny valley (What about the heart?)

Mori’s uncanny valley (1 of 3)

• Masahiro Mori (http://www.frogheart.ca/?p=10170)

Mori’s uncanny valley (2 of 3)

• Some prosthetic hands attempt to simulate veins, muscles, tendons, finger nails, and finger prints, and their color resembles human pigmentation. So maybe the prosthetic arm has achieved a degree of human verisimilitude on par with false teeth. But this kind of prosthetic hand is too real and when we notice it is prosthetic, we have a sense of strangeness. (Masahiro Mori)

Mori’s uncanny valley (3 of 3)

• So if we shake the hand, we are surprised by the lack of soft tissue and cold temperature. In this case, there is no longer a sense of familiarity. It is uncanny. In mathematical terms, strangeness can be represented by negative familiarity, so the prosthetic hand is at the bottom of the valley. So in this case, the appearance is quite human like, but the familiarity is negative.

• This is the uncanny valley. (Masahiro Mori) (http://www.frogheart.ca/?p=3077)

Animator’s uncanny valley (1 of 3)

Animator’s uncanny valley (2 of 3)

• But then the Uncanny Valley kicks in. That curvy line changes direction, plunging downwards. This is the pit into which many characters from The Polar Express, Final Fantasy and Mars Needs Moms fall. We stop empathizing with these characters. They are unintentionally disturbing, like moving corpses. This is a big problem with realistic CGI characters: that unshakable perception that they are animated zombies. [comments: Chris Landreth {zombie emphasis mine}] http://www.frogheart.ca/?p=10170

Animator’s uncanny valley (3 of 3)

• (Canada) NFB video clip of Landreth’s Subconscious Password at: http://www.frogheart.ca/?p=10170

• … I want to be honest in showing you this world. My own Uncanny Valley. You have one too. It’s something to celebrate. (Landreth’s comment about his work)

Re-entering the Uncanny Valley

• Which is the robot and which is the philosopher

Geminoid robots (1 of 3)

• The first geminoid, HI-1, was created in 2005 by Prof. Hiroshi Ishiguro of ATR and the Tokyo-based firm, Kokoro. A geminoid is an android, designed to look exactly as its master, and is controlled through a computer system that replicates the facial movements of the operator in the robot. [sic]

• In the spring of 2010, a new geminoid was created. The new robot, Geminoid-F was a simpler version of the original HI-1, and it was also more affordable, making it reasonable to acquire one for humanistic research in Human Robot Interaction.

Geminoid robots (2 of 3)

• Geminoid|DK will be the first of its kind outside of Japan, and is intended to advance android science and philosophy, in seeking answers to fundamental questions, many of which that have also occupied the Japanese researchers. The most important questions are:

• – What is a human?– What is presence?– What is a relation?– What is identity?

• (http://www.frogheart.ca/?p=3077)

Geminoids and philosophy (3 of 3)

• Henrik Scharfe, an associate professor at Aalborg University in Denmark director of the center for Computer-Mediated Epistemology

• commissioned a Geminoid robot• to probe “emotional affordances” between

robots and humans, as well as “blended presence” (2011; http://www.frogheart.ca/?p=3077 &

• video http://www.frogheart.ca/?p=13624)

What about the heart? (1 of 3)

• [Luisa] Whitton [photographer/artist] spent several months in Japan working with Hiroshi Ishiguro, a scientist who has constructed a robotic copy of himself. Ishiguro’s research focused on whether his robotic double could somehow possess his “Sonzai-Kan,” a Japanese term that translates to the “presence” or “spirit” of a person.

What about the heart? (2 of 3)

• It’s work that blurs the line between technology, philosophy, psychology, and art, using real-world studies to examine existential issues once reserved for speculation by the likes of Philip K. Dick or Sigmund Freud.

What about the heart? (3 of 3)

• And if this sounds like a sequel to Blade Runner, it gets weirder: after Ishiguro aged, he had plastic surgery so that his face still matched that of his younger, mechanical doppelganger. (http://www.fastcodesign.com/3031125/exposure/japans-uncanny-quest-to-humanize-robots)

• http://www.frogheart.ca/?p=13624

Robots and intelligence (1 of 6)

• Excellent 2007 essay on robots and artificial intelligence by Mary King: http://robotandai.blogspot.ca/

• This research project explores the theories and work of Japanese and Western scientists in the field of robotics and AI. I [Mary King] ask what differences exist in the approach and expectations of Japanese and Western AI scientists, and I show how these variances came about.

Robots and intelligence (2 of 6)

• Because the Western media often cites Shinto as the reason for the Japanese affinity for robots, I ask what else has shaped Japan’s harmonious feelings for intelligent machines. Why is Japan eager to develop robots, and particularly humanoid ones? I also aim to discover if religion plays a role in shaping AI scientists’ research styles and perspectives. In addition, I ask how Western and Japanese scientists envision robots/AI playing a role in our lives. Finally, I enquire how the issues of roboethics and rights for robots are perceived in Japan and the West.

Robots and intelligence (3 of 6)

• The fields of robotic technology and AI are closely related and often overlap. Robotics falls under the umbrella of artificial intelligence research. Both “The New Oxford Dictionary of English” and Japan’s authoritative “Kojien” dictionary define artificial intelligence as the performance by computer systems of tasks normally requiring human intelligence. …

Robots and intelligence (4 of 6)

• Meanwhile, “The New Oxford Dictionary of English” describes a robot as “a machine (sometimes resembling a human being) that is capable of carrying out a complex series of actions automatically, especially one programmable by a computer.” The “Kojien” dictionary says a robot is a “complicated man-made automaton, an artificial person or cyborg, a machine for work or a machine that is controlled to perform automatically.”

Robots and intelligence (5 of 6)

• Both the East and the West have an ancient history of mechanical “machines,” toys and dolls that can be considered to be the forerunners of the robot. However, Leonardo da Vinci’s 1495 drawing of a mechanical knight is reputed to be the first actual plan for a humanoid robot. Stories of golem and of Frankenstein have also held sway over Western imaginings of artificial man-made beings. The word “robot,” with its connotations of beings that replace humans, derives from the Czech noun “robota,” meaning forced labour.

Robots and intelligence (6 of 6)

• First staged in 1921, many people interpreted [Czech playwright Karel Capek ‘s play Rossum’s Universal Robots] RUR as an attack on technology but Capek aimed only to question the idea of humans becoming slaves of machines. The play, however, created a vastly different impression after it opened in Tokyo in 1924. The Japanese found the idea of artificially created humans to be more intriguing than threatening. But RUR lost its intended meaning in Japan because both the title of the play and the word “robot” were translated as “jinzo ningen,” meaning artificial human, which gave the Japanese a warm feeling.

• (http://robotandai.blogspot.ca/)

Robots and learning (1 of 5)

• RoboEarth (a European Union project)• … Markus Waibel,• As part of the European project RoboEarth, I am

currently one of about 30 people working towards building an Internet for robots: a worldwide, open-source platform that allows any robot with a network connection to generate, share, and reuse data. The project is set up to deliver a proof of concept to show two things:

Robots and learning (2 of 5)

• * RoboEarth greatly speeds up robot learning and adaptation in complex tasks.

• * Robots using RoboEarth can execute tasks that were not explicitly planned for at design time.

• The vision behind RoboEarth is much larger: Allow robots to encode, exchange, and reuse knowledge to help each other accomplish complex tasks. This goes beyond merely allowing robots to communicate via the Internet, outsourcing computation to the cloud, or linked data.

Robots and learning (3 of 5)

• http://www.frogheart.ca/?p=2897 links to Waibel article and more including the RoboEarth website and a video

Robots and learning (4 of 5)

• … an Aug. 25, 2014 Cornell University news release by Bill Steele (also on EurekAlert with some editorial changes) about the US Robo Brain project immediately caught my attention,

• Robo Brain – a large-scale computational system that learns from publicly available Internet resources – is currently downloading and processing about 1 billion images, 120,000 YouTube videos, and 100 million how-to documents and appliance manuals. The information is being translated and stored in a robot-friendly format that robots will be able to draw on when they need it. (http://www.frogheart.ca/?p=14500)

Robots and learning (5 of 5)

• An enormous gap exists between human abilities and machine performance when it comes to understanding the visual world from images and videos. Humans are still way out in front. …

• In her research, Parikh is proposing to use visual abstractions or cartoons to teach machines. She works from the idea that concepts that are difficult to describe textually may be easier to illustrate. By having thousands of online crowd workers manipulate clipart images to mimic photographs, she seeks to teach a computer to understand the visual world like humans do. (Devi Parikh, Virginia Tech)

• http://www.eurekalert.org/pub_releases/2014-09/vt-ccb092614.php

Artificial intelligence/Artificial brains (1 of 2)

• Artificial Intelligence = software• Artificial brains = devices/entities• This is a distinction I (Maryse de la Giroday)

make in my presentations. To my knowledge this is not a standard distinction in the field.

Artificial intelligence/Artificial brain (2 of 2)

• Standard computing architecture: boolean logic; separation between processing and memory used for a single purpose only as compared to a multipurpose biological brain (since 1960s)

• Memristors/electrochemical atomic switches = emulate synaptic plasticity of a biological brain making possible an artificial brain that learns (multipurpose) like a biological brain (since 2000s)(Massimiliano Versace & Ben Chandler 2010 IEEE article)

AI: Eliasmith and virtual neurons (1 of 3)

• Chris Eliasmith and a team from the University of Waterloo (Ontario) announced SPAUN (Semantic Pointer Architecture Unified Network) in Nov. 2012

• http://www.frogheart.ca/?p=8594 (includes video)

AI: Eliasmith and virtual neurons (2 of 3)

• Spaun is actually a simulated brain. It contains2.5 million virtual neurons — many fewer than the 86 billion in the average human head, but enough to recognize lists of numbers, do simple arithmetic and solve reasoning problems. (excerpt from the FrogHeart post)

AI: Eliasmith and virtual neurons (3 of 3)

• … Henry Markram, who leads a different project to reconstruct the human brain called the Blue Brain, questions whether Spaun really captures human brain behavior. Because Spaun’s design ignores some important neural properties, it’s unlikely to reveal anything about the brain’s mechanics, says Markram, of the Swiss Federal Institute of Technology in Lausanne. “It is not a brain model.” (excerpt from the FrogHeart post))

Neural plasticity and the Memristor (Copyright: HP Labs Credit: R. Stanley Williams)

Memristor and circuit elements(electrical engineering)

• Capacitor• Inductor• Resistor• Memory + Resistor = Memristor (1971), the

fourth fundamental circuit element forming a non-linear relationship between electric charge and magnetic flux linkage– ‘remembers’ how much voltage is carried and for

how long = memory & learning

Memristor

• In 2011, Leon Chua (who first proposed the memristor in 1971 in work separate from Bernard Widrow who proposed a similar entity, the memistor, in the 1960s) argued for a broader definition: all 2-terminal non-volatile memory devices based on resistance switching should be considered memristors

Memristor concept is contested

• Stan Williams/HP Labs has argued (along with Chua) that MRAM, phase change memory, and RRAM, should be considered memristor technologies.

• Some researchers say biological structures such as blood & skin should also be considered memristors.

• Others say memory devices under development by HP Labs not actually memristors or memristive systems but part of a broader class of variable resistance systems & a broader definition of memristor is a scientifically unjustifiable land grab to favor the memristor patents of Hewlett-Packard. (Wikipedia Memristor essay accessed Oct.14.12)

Memristor objections

• One of the latest objections added to Wikipedia entry (http://en.wikipedia.org/wiki/Memristor): Thermodynamic considerations, however, show that such a memristor component cannot exist as a solid state device in physical reality because its behavior would be inconsistent with fundamental laws of non-equilibrium thermodynamics.[4][5] The potential applicability of the memristor concept to any physically realizable device is thus very much open to question.

Neural plasticity and the electrochemical atomic switch

• Nanoscale device with a gap bridged by a copper filament under a voltage pulse stimulation = a change in conductance which is time-dependant — a change in strength that's nearly identical to the one found in biological synaptic systems

• Mimics short-term and long-term memory• Responds to the presence of air and temperature

changes: it has the potential to perceive the environment much like the human brain

• (George Dvorsky, June 11, 2012 article for IO9)

Memristor/atomic switch and neuromorphic engineering teams

• Memristor: HP Labs, University of Michigan, & Boston University– MoNETA (Roman goddess of memory)Boston

University/HP Labs– Wei Lu/University of Michigan/HRL Laboratories

• Atomic switch: UCLA (James Gimzewski) & National Institute for Materials Science (Japan) [Neuromorphic Atomic Switch Networks, Aug. 6, 2012, PLoS] also nanoionic devices (Dec. 2012 post: http://www.frogheart.ca/?p=8802)

Brain-on-a-chip• Survey of research for repairs to human brains and for making

computers more like human brains• http://www.frogheart.ca/?p=13001• “A completely different – and revolutionary – human brain model

has been designed by researchers in Japan who introduced the concept of a new class of computer which does not use any circuit or logic gate. This artificial brain-building project differs from all others in the world. It does not use logic-gate based computing within the framework of Turing. The decision-making protocol is not a logical reduction of decision rather projection of frequency fractal operations in a real space, it is an engineering perspective of Gödel’s incompleteness theorem.” (http://www.nanowerk.com/spotlight/spotid=35084.php)

TrueNorth (IBM) brain chip architecture

• A neurosynaptic supercomputer the size of a postage stamp that runs on the energy equivalent of a hearing-aid battery, this technology could transform science, technology, business, government, and society by enabling vision, audition, and multi-sensory applications. (August 8, 2014: http://www.frogheart.ca/?p=14360)

Consciousness

• Hiroshi Ishiguro and geminoid robots• – What is a human?

– What is presence?– What is a relation?– What is identity?

• David Chalmers (philosopher; btw, Eliasmith is a computer scientist & a philosopher)

• http://blog.ted.com/2014/03/19/the-hard-problem-of-consciousness-david-chalmers-at-ted2014/

David Chalmers and consciousness

• “Right now you have a movie playing inside your head,” says philosopher David Chalmers. It’s an amazing movie, with 3D, smell, taste, touch, a sense of body, pain, hunger, emotions, memories, and a constant voice-over narrative. “At the heart of this movie is you, experiencing this, directly. This movie is your stream of consciousness, experience of the mind and the world.”

• This is one of the fundamental aspects of existence, Chalmers says: “There’s nothing we know about more directly…. but at the same time it’s the most mysterious phenomenon in the universe.” What is the difference between us and robots? Nobody knows the answers.

Coming to the end of Bioelectronics, Medical Imaging and Our Bodies

• Referring back to week 4 & the ASAP science video on myths about the brain

Teachers, teaching, and myths about the brain (1 of 4)

• New research from University of Bristol (pub. Oct. 15, 2014 in Nature Reviews Neuroscience)

• Teachers in the UK, Holland, Turkey, Greece and China were presented with seven so-called 'neuromyths' and asked whether they believe them to be true.

Teachers, teaching, and myths about the brain (2 of 4)

• Over 70 per cent of teachers in all countries wrongly believe a student is either left-brained or right-brained, peaking at 91 per cent in the UK.

• And almost all teachers (over 90 per cent in each country) feel that teaching to a student's preferred learning style - auditory, kinaesthetic or visual - is helpful, despite no convincing evidence to support this approach.

Teachers, teaching, and myths about the brain (3 of 4)

• The report blames wishfulness, anxiety and a bias towards simple explanations as typical factors that distort neuroscientific fact into neuromyth.

• The report highlights several areas where new findings from neuroscience are becoming misinterpreted by education, including brain-related ideas regarding early educational investment, adolescent brain development and learning disorders such as dyslexia and ADHD.

Teachers, teaching, and myths about the brain (4 of 4)

• http://phys.org/news/2014-10-myths-brain-hampering.html

• Neuroscience and education: myths and messages by Paul A. Howard-Jones. Nature Reviews Neuroscience (2014) doi:10.1038/nrn3817 Published online 15 October 2014 (open access with registration on nature.com)

Back to the eye (1 of 2)

• Superhuman eyesight (femtophotography)• http://www.frogheart.ca/?p=14819• Superhuman navigation (human eye, brain of a rat, &

a robot; Sept. 16, 2014, Australia)• "This is a very Frankenstein type of project," Dr

Milford said.• "It's putting two halves of a thing together because

we're taking the eyes of a human and linking them up with the brain of a rat.

• http://www.news.qut.edu.au/cgi-bin/WebObjects/News.woa/wa/goNewsPage?newsEventID=78859

Back to the eye

• http://www.the-scientist.com/?articles.view/articleNo/41052/title/The-Bionic-Eye/

• In 1755, French physician and scientist Charles Leroy discharged the static electricity from a Leyden jar—a precursor of modern-day capacitors—into a blind patient’s body using two wires, one tightened around the head just above the eyes and the other around the leg. The patient, who had been blind for three months as a result of a high fever, described the experience like a flame passing downwards in front of his eyes. This was the first time an electrical device—serving as a rudimentary prosthesis—successfully restored even a flicker of visual perception.

Eye projects around the world mentioned in The Scientist article on bionic eyes

• US• Italy• Japan• South Korea• Australia• European Union• Canada• Spain

Course summary