Audio Workgroup Neuro-inspired Speech Recognition

Audio WorkgroupAudio Workgroup

Neuro-inspired Speech RecognitionNeuro-inspired Speech Recognition

Audio WorkgroupAudio Workgroup

Localization EffortLocalization Effort

Interaural Time Difference (ITD)

Estimated from time difference between spikes of two matching channels.

Interaural Intensity Difference (IID)

Difference of spike counts between two cochleae.

Azimuth: Combination of ITD and ILD

Audio WorkgroupAudio Workgroup

Localization EffortLocalization Effort

Audio WorkgroupAudio Workgroup

Relational Network (Simple)Relational Network (Simple)

X Y

Z

MM

X

M

Y

M

Z

m

Patches of neuronsEach measureone quantityBidirectionalrelations for feedback/feedforward

Audio WorkgroupAudio Workgroup

Relational Network (example)Relational Network (example)

Input hereRelation specification

Relational feedback

RelationFeedback

Audio WorkgroupAudio Workgroup

ASR Relational NetworkASR Relational Network

Cochlea

Delay

Phone Recognizer

Phone Recognizer

Word Recognizer

A patch of neurons(one of N output)

We don’t know how to represent time

Audio WorkgroupAudio Workgroup

ASR AdvantagesASR Advantages

Not an HMM

Top-Down, Bottom-Up Hypothesis

Hallucinate

Audio WorkgroupAudio Workgroup

Silicon CochleaSilicon Cochlea

Ganglion cells

Basilar membrane

highfrequency

lowfrequency

Inner hair cells

(van Schaik, Liu, 2004)

BASILAR MEMBRANE

INNER HAIR CELLS

GANGLION CELLS

Audio WorkgroupAudio Workgroup

Silicon CochleaSilicon Cochlea

Tone raster plots

Vowel Rate Profiles

Audio WorkgroupAudio Workgroup

Learning ChipLearning Chip

Architecture

Tone Rasters?

Vowel Rasters

Learning Algorithm

Alternative LearningStatistics

LeastSquares

Audio WorkgroupAudio Workgroup

LSM RecognizerLSM Recognizer

Audio WorkgroupAudio Workgroup

Infrastruture DifficultiesInfrastruture Difficulties

RemapperReplace with Matlab

Power ?

Sharing chips?

PC replacement

Audio WorkgroupAudio Workgroup

FPAA/MoteFPAA/Mote

Audio WorkgroupAudio Workgroup

Word RecognizerWord Recognizer

Four example raster plot (silence, A_, A_ with relational, AI)

Audio WorkgroupAudio Workgroup

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Software SimulationSoftware Simulation

Audio WorkgroupAudio Workgroup

Behind the CurtainBehind the Curtain

Audio WorkgroupAudio Workgroup

Hardware OverviewHardware Overview

Cochlea

Cochlea

Remapper(in Matlab)

Learning

GiacomoGiacomo

PhonemeWord

skype

PCI-AER (for remapping)

PCI-AER (for remapping)

Audio WorkgroupAudio Workgroup