Interactive AudioManipulation Processor

Todd Carter

Daniel Ross

Stephen Tang

What is the iAMP?

It’s interactive: the iAMP is an FPGA-based processor that lets YOU perform real-time manipulation of an input audio source.

More specifically, using a joystick, the user can graphically position the location of a sound source (such as a CD player) in real-time.

Overview

Physical Setup User Interface

video output joystick input

Audio Path audio cues how iAMP localizes sound

A-D and D-A

iAMP Setup

Speakerjoystick

VGA Monitor

Speaker Speaker

Speaker

iAMP Setup

iAMP(Altera UP1)

D/A 1

D/A 2

A/DAudioSource

VGA Monitor joystick

AudioAmp

Left

Right

Left

Right

Rear

Front

Design Overview

Audio_Manipulator.vhdAdc_port.vhd

Ram_interface.vhd

Ram_port.vhd

Dac_port.vhd

Dac_port.vhd

Joystick.vhd Video.vhdUI.vhd

Dac_interface.vhd

Dac_interface.vhd

Main.vhd

iAMP User Interface

User Interface

speakerspeaker

speaker speaker

centericon

cursor

Top-Level UI Interface

videoui

vga_red

vga_green

vga_blue

vga_h_sync

vga_v_sync

cursorY

cursorX

clock

vga_red

vga_green

vga_blue

vga_h_sync

vga_v_sync

mainclock

Video Interfaceclock

video

vga_red

vga_green

vga_blue

vga_h_sync

vga_v_sync

cursorX

cursorY

ColourChooser

countXcountY

count_xysyncgen HSync

VSync

Joystick

1

6789

5 4 3 2

Female DB-9

pin # function

1 up2 down3 left4 right5 unused6 button7 unused8 ground

9 unused

iAMP Audio Path

Audio Cues

What techniques exist to ‘fool’ the ear?

Auditory mechanism isn’t capable of understanding the environment as a domain.

Headphones -- externalization of sound.

We can take advantage of these weaknesses!

Audio Cues

Ears use many cues to comprehend its surrounding.

Binaural Intensity Difference Binaural Phase Difference Cone of Confusion Outer Ear Frequency Response Reverberation

Audio Cues

Binaural Intensity Difference BID aids the individual to localize sound by the

difference in signal intensity between the two ears.

Sound appears to originate from direction of higher intensity.

10 dB (10:1) difference required to place sound 45o to one side.

Binaural Phase Difference Not often used - only effective for pure sinusoidal

waves <1500 Hz.

Audio Cues

Cone of Confusion The ear cannot directly perceive whether a

sound is coming from ahead or behind, when source is to one side of the median plane. Any sound source along the edge of the “cone of confusion” is indistinguishable from any other sound source along the edge.

Audio Cues

Left

Back

Front

Right

Up

Down

Cone of Confusion

Audio Cues

How does the ear tell the difference between sounds coming from ahead or behind?

Audio Cues

Outer Ear Frequency Response Reverberation

Sounds closer to the ear have lower intensity echo signals vs. incident intensity.

Echoes and reverberation fool the brain into thinking a sound is farther away than it really is.

More echo means more diffused sound path to ear = farther away.

Sound Model

How does iAMP create the sound “location”?

Gain differences between speaker outputs. Echo created by delayed samples kept in RAM.

Our Simplified Model Multiplying 24-bit samples takes up too much

space! Simplified linear models applied. Not as

accurate, but saves lots of space on Flex 10K20!

Audio Path

RAM

D/A 1

A/D

D/A 1

Audio_Manipulator.vhdAdc_port.vhd

Ram_interface.vhd

Ram_port.vhd

Dac_port.vhd

Dac_port.vhd

Audio Manipulation

Sequentially capture 8 samples per sample period from RAM Sample Stack

4 ‘incident’ delayed samples (initial wavefronts) 4 ‘echo’ delayed samples (reflected wavefronts)

Multiply each sample by gain (1/16 to 15/16) Provides 11.8 dB Binaural Intensity Difference (> 10 dB)

“Mix” the echo and incident samples, output to DACs for conversion out to speakers.

(Really summing…)

Audio Manipulation

Audio Controller

Register Bank Register BankRegister Bank

sample_req

data_ready

delay

sample_in

X_loc Y_loc System_clock Sample_clock

sample_clock

gain

FL Speaker

FR Speaker

RL Speaker

RR Speaker

Location Grid1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Speaker Gain vs. Cursor Location (/16)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 151 15 14 13 12 11 10 9 8 7 6 5 4 3 2 12 14 14 13 12 11 10 9 8 7 6 5 4 3 2 13 13 13 13 12 11 10 9 8 7 6 5 4 3 2 14 12 12 12 12 11 10 9 8 7 6 5 4 3 2 15 11 11 11 11 11 10 9 8 7 6 5 4 3 2 16 10 10 10 10 10 10 9 8 7 6 5 4 3 2 17 9 9 9 9 9 9 9 8 7 6 5 4 3 2 18 8 8 8 8 8 8 8 8 7 6 5 4 3 2 19 7 7 7 7 7 7 7 7 7 6 5 4 3 2 110 6 6 6 6 6 6 6 6 6 6 5 4 3 2 111 5 5 5 5 5 5 5 5 5 5 5 4 3 2 112 4 4 4 4 4 4 4 4 4 4 4 4 3 2 113 3 3 3 3 3 3 3 3 3 3 3 3 3 2 114 2 2 2 2 2 2 2 2 2 2 2 2 2 2 115 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Similar for all four speakers.

Time Delay vs. Cursor Location ()

1 2 3 4 5 6 7 8 9 10 11 12 13 14 151 0 1 2 3 4 5 6 7 8 9 10 11 12 13 142 1 2 3 4 5 6 7 8 9 10 11 12 13 14 153 2 3 4 5 6 7 8 9 10 11 12 13 14 15 164 3 4 5 6 7 8 9 10 11 12 13 14 15 16 175 4 5 6 7 8 9 10 11 12 13 14 15 16 17 186 5 6 7 8 9 10 11 12 13 14 15 16 17 18 197 6 7 8 9 10 11 12 13 14 15 16 17 18 19 208 7 8 9 10 11 12 13 14 15 16 17 18 19 20 219 8 9 10 11 12 13 14 15 16 17 18 19 20 21 2210 9 10 11 12 13 14 15 16 17 18 19 20 21 22 2311 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2412 11 12 13 14 15 16 17 18 19 20 21 22 23 24 2513 12 13 14 15 16 17 18 19 20 21 22 23 24 25 2614 13 14 15 16 17 18 19 20 21 22 23 24 25 26 2715 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

is a time constant based on the speed of sound and the distance between speakers.

Total Loudness

1 3 5 7 9

11 13 151

6

11

-2.5

-2

-1.5

-1

-0.5

0

0.5

Total Normalized Volume (dB)

X location

Y location

A/D Converter

CRYSTAL CS5360 24-Bit Stereo A/D Converter for Digital Audio

24-bit samples -- for 16 or 8 bit sampling, we take the most significant bits (Big-Endian -- the first 16 or 8 bits.)

D/A Converter

CRYSTAL CS4390 24-Bit Stereo D/A Converter for Digital Audio

24-bit samples -- for 16 or 8 bit sampling, we take the most significant bits (Big-Endian -- the first 16 or 8 bits.)

Questions?