KAIST: Korea Advance Institute of Science and Tech(4000 ... new Sound... · Sound Visualization and Manipulation 12 Sound visualization as a mapping [†] This illustration is a modified

KAIST: Korea Advance Institute of Science and Tech(4000 undergrad, 4000 grad, 600 faculty members)

NOVIC: Center for Noise and Vibration Control(7+7 faculty members, 35 M.Sc. 50 Ph.D. about 500 graduates since)

C.W. Lee: Rotor Dynamics, Golf Dynamics Younsik Park: Structural Modification and Modal Analysis

Kwang-Joon Kim: Vibration Isolation J.G. Ih: BEM, BEM holography, Active Control

Youngjin Park: Active Sound Control, HRTF Customization Jungwoo Choi: 3D sound and audio

Yang-Hann Kim

Korea Advanced Institute of Science and Technology (KAIST) Center for Noise and Vibration Control (NOVIC)

[email protected] 2013. 4. 29.

Sound Visualization & Manipulation

3 Sound Visualization and Manipulation

Introduction

We can take a picture using a camera


Sound Visualization

Can we take the picture of sound?


Sound Manipulation

Or, can we draw the picture of sound?


Sound visualization and manipulation

See what we want using microphone array

Draw what we want using loudspeaker array

Sound Visualization Sound Manipulation

Sound Visualization


Introduction

[†]

[†] SMInstruments


What is sound visualization?

Color organ by Bainbridge Bishop(1893,US)

Sound Color

[†]

[†] A souvenir of the color organ, with some suggestions in regard to the soul of the rainbow and the harmony of light, Bainbridge Bishop, The De Vinne Press, 1893.

The first attempt to visualize the sound


What is sound visualization?

Many means to express organ sound [†] Sound field by five in-phase monopoles

[†]

Sound Color


MAPPING

Sound visualization as a mapping

Measured data Acoustic image


Sound visualization as a mapping

[†] This illustration is a modified version of the figure, pp.111-112, Science with a Smile (Robert L. Weber, Institute of Physics Publishing, 1992)

The result of sound visualization depends on the selection of basis function

[†]

Measured data Acoustic image

Mapping


Category of sound visualization

Two popular visualization methods using different types of basis functions

Provides information of the source surface. (pressure, particle velocity, intensity..)

Non-parametric method

Parametric method

Sound Visualization

Acoustic Holography Beamforming

Only provides information of the source location.


I. Acoustic Holography

One-dimensional case

0( , )p x t

0 0( , ) ( ) j tp x t P x e ω−=

0xx =

1xx =

1( , ) ?p x t =

What is Measured

Propagation or Prediction

0 1( )jk x xe− −×1( , )p x t =What We

See

1 0( | ; )H x x ω×


What we see (sound field)

What is measured: (pressure & velocity on the boundary)

I. Acoustic Holography

Three-dimensional case

Propagation (or prediction)

Kirchhoff-Helmholtz integral equation

O 0r

r 0S

0n……

……

( ) ( ) ( ) ( )0

00

000

00

| ;1( ;4

|;) ;;

S

G r r fG r r

P rP r

fP rf Sf f d

nnπ∂

= − ∂ ∂

∂∫

( )0;P r f

( ; )P r f


Prediction of Mona Lisa’s image by Acoustic Holography

I. Acoustic Holography: example

Measurement plane

What we see What is measured

Propagation

Source plane


II. Beamforming

Plane wave propagation model

measured pressure

sin ( 1) sin1Tjkd jk M de eθ θ− = W

scan vector (modeled signal)

[ ]1 2 3 4T

MP P P P P=P

[ ]1 2 3 4T

MW W W W W=W

d

sθ

x

y

sθ

1PMP

1WMW

1

1 M

iM =∑

2|P |ower( ) HEθ = P W

mP

mW

θ

Scan vector W is the basis function of beamforming method.


II. Beamforming: example

Measurement plane

What we see What is measured

Propagation

Source plane

Prediction of Mona Lisa’s image by beamforming

ACOUSTIC HOLOGRAPHY: APPLICATIONS

19


Moving Frame Acoustic Holography

Magnetic levitation train

• Pressure distribution at source plane (900Hz) • Intensity plot (900Hz)


Cylindrical Acoustic Holography

King Seong-Deok bell

• Step by step measurement • Microphone array

- Number of microphone: 30 - Aperture size : 4.42 m - Microphone spacing : 0.15 m - Radius of hologram: 0.2 m

BEAMFORMING: APPLICATIONS

22


Temporal basis function

Source localization using two different basis functions

Beamforming of an impulsive signal interfered by steady sound


Time domain beamforming of impulsive source localization

Impulsive signal interfered by steady noise

K-21 Infantry Fighting Vehicle 0.8m from top of the turret Idle level (98 dB) Moving 5~20 km/h (106~112 dB))


Real time implementation using FPGA

Digital MEMS microphone + FPGA(Field Programmable Gate Array)

High Frame Rate Unique Portable Design Light Weight Simple Connection Compact Controller Low Price

Digital MEMS Microphone Array

IP Camera

sbRio-9606

Ethernet Hub

Battery

Note Book

Power Supply

Advantages:


Real time implementation using FPGA

Engine noise visualization


Summary of sound visualization

The selection of basis function leads to different results

Beamforming Holography

Measured pressure


Sound manipulation problem

measurement plane

Sound Visualization

source plane

What if we replace the microphone array with a loudspeaker array?

Sound Manipulation

We can then draw the sound picture using a loudspeaker array !

Sound Manipulation


[†]

[†] Big Bang Theory S02E09, 2008. Available partly on Youtube (http://www.youtube.com/watch?v=vTcpE9sGmH4&feature=endscreen)

Introduction


How can he make what he wants to hear at his seat?

cool!

How can we manipulate a sound field at a desired region? : Zone control of sound field

Objective of sound manipulation


What to draw on the selected zone?

Basis function depends on the impression of sound we want to draw.

Drawing a desired shape of wavefront

Radiating sound ball Focused sound ball

Drawing dot(s) in space

Sound Manipulation


I. Radiating sound ball: overview

Reproduction of the sound field from a radiating sound ball over the zone of interest


II. Focused sound ball: overview

Generation of a focused sound ball in the zone of interest


I. Radiating sound ball: mathematical expression

One-dimensional case

0( , )p x t

0 0( , ) ( ) j tp x t P x e ω−=

0xx =

1xx =

1( , )p x t

What we can control

Propagation (transfer function)

0 1( )jk x xe− −×1( , )p x t =What to

manipulate

Loudspeaker

1 0( | ; )H x x ω×


I. Radiating sound ball: mathematical expression

Three-dimensional case

( ) ( ) ( ) ( )| ; ,|, ,) ;( s

sS

ss

ss

G r r fG r r fP r f

P r fP r f dS

n n∂

= − ∂ ∂

∂ ∫

Kirchhoff-Helmholtz integral equation

What to manipulate (sound field)

What we can control (pressure & velocity on the boundary)


O

sr

rS

sn……

……

Loudspeaker

( )P r

( )sP r

Λ

VVirtual source


II. Focused sound ball: mathematical expression

Regional sound focusing: acoustic brightness and contrast control[†]

Maximizing energy ratio between acoustically bright zone and dark zone

Find maximum brightness or contrast by eigenvalue analysis

r

0V

pointsM

sH

p

=p Hs

bV dV

tV

Potential energy in Vb

Potential energy in Vd

Potential energy in Vb

Input power Bright zone Dark zone

Hsp =

: Acoustic brightness control : Acoustic contrast control [†] J. -W. Choi and Y. -H. Kim, “Generation of the acoustically bright zone within an illuminated region using multiple sources,” J. Acoust. Soc. Am., Vol. 111(4), pp.1695–1700, 2002.

Radiating Sound Ball: Applications


3D reproduction of a virtual source

Example

- Nspk = 194(Lebedev quadrature grids), Control source at 2λ, Virtual source at 1λ


KAIST multichannel loudspeaker system

Implementation

Amplifiers

DA Converters

24ch. Line array

49ch. Spherical array

PC

Wi-Fi

Smart phone or Tablet PC User interface


Realization Demo: Interface for virtual sources realization

Focused Sound Ball: Applications


Personal audio system[†]

Regional focusing: applications

A personal audio system can be used without uncomfortable earphones or headsets and bothering user’s neighbors.

[†] J-H Chang, C-H Lee, J-Y Park, and Kim Y-H, “A realization of sound focused personal audio system using acoustic contrast control,” J. Acoust. Soc. Am. 125(4), pp.2091-2097 (2009).



Frequencies of interest: 800 Hz ~ 5 kHz

Personal audio system: control results



Personal audio system with scattering effect[†]

x(m)

y(m

)

Contrast control

-0.4 0 0.4-0.4

-0.2

0

0.2

x(m)

Contrast control

-0.4 0 0.4-0.4

-0.2

0

0.2<800Hz> <2kHz> ( )ref . 0.1,0totp −Total field soln.

x(m)

Contrast control

-0.4 0 0.4-0.4

-0.2

0

0.2dB

-30

-20

-10

0

10

x(m)

y(m

)

Contrast control

-0.4 0 0.4-0.4

-0.2

0

0.2dB

-30

-20

-10

0

10

x(m)

y(m

)

Contrast control

-0.4 0 0.4-0.4

-0.2

0

0.2dB

-30

-20

-10

0

10

- Effect of listener’s head scattering: ( )ref . 0,0incp ( )ref . 0,0incp

- Control results considering scattering effect: <5kHz>

[†] J-H Chang, J-Y Park, and Kim Y-H, “Scattering effect on the sound focused personal audio system,” J. Acoust. Soc. Am. 125(5), pp.3060-3066 (2009).


Conclusion

[†] This illustration is a modified version of the figure, pp.111-112, Science with a Smile (Robert L. Weber, Institute of Physics Publishing, 1992)

We can visualize and manipulate any sound field by choosing appropriate basis function!

[†]

Thank you [email protected] soundmasters.kaist.ac.kr

Prof. J. W. Choi

Acknowledgment

Dr. M. H. Song Mr. D. H. Seo, K. W. Kim, K. H. Kim, J. M. Lee,

D. S. Kang, Ph.D. Candidate Ms. M. R. Lee, M.S. Candidate

of NOVIC KAST

SMInstruments

Appendix


Mona Lisa Beamforming


Mona Lisa Beamforming

Theory of Sound Focusing

with Four Principles (+ - × ÷)


Sound Focusing Problem

Manipulation of focused sound beam by adjusting excitation function of loudspeaker array

/ 2

/2( , ) ( )

jkRL

L

eP r dxq xR

θ−

= ∫Excitation function

(unknown)

/ 2 sin

/2( )

jkr L jk x

L

e e dxr

q x θ−

−≈ ∫

( )b θBeam pattern

q

( , )P r q

Loudspeaker array

( )q x

q

( , )P r q

( )q x( , )P r q

…

We can design the beam pattern with four principles(+ - × ÷).


Uniform Gain: ( )w x

(+) Theorem: Delay-and-Sum

Beam shaping and steering by gain weighting and time delay

( )b θ

Time delay:

( )( ) ( ) j xq x w x e ωτ=

Gain weighting

( ) xxcατ =

( )xτTime delay: ( )xτTime delay:

x

( )q x


(-) Theorem: Differential Sources

Differential sources can produce a directional radiation pattern despite limitation of the aperture size.

( )b θ

Excitation function for dipole beam pattern

( ) ( ) ( )2 2x xq x x xδ δ∆ ∆

= + − −

x( )q x

x∆ : source spacing


(x) Theorem: Composition of Beam Patterns

Design various beam shapes by multiplication of beam patterns

( )b θ

x( )q x

( )b θ

x

( )q x

x

( )q xArray of dipoles


(÷) Theorem: Optimization with Energy Ratio

Maximization of energy ratio

- Acoustic contrast control: - Acoustic brightness control:

Bright zone

Dark zone

bp

dpx

q

Dark zone

Control zone

Maximization of energy ratio between at bright zone and dark zone

Hb bHd d

β =p pp p

Maximize

Maximization of ratio between energy at bright zone and input power

Hb bHα =

p pq q

Maximize

Theory of Virtual Source Generation


Interior Virtual Source Problem in 1D Case

How can we reproduce 1D sound from a virtual source inside control zone?

1x x0

Right going Left going Virtual source

vx 2x

Control zone

( )

( )( , )v

v

jk x xv

t jk x xv

Ae for x xP x f

Ae for x x

−

− −

≤= <

Target field

: right going wave

: left going wave


Reproduction of Interior Virtual Source in 1D Case

By considering time-reversed propagation, right going wave can be generated.

1x x0

Virtual source

vx 2x

Control zone

Time-reverse of target field

( )*

, ( )( , )v

v

jk x xv

t tr jk x xv

Ae for x xP x f

Ae for x x

− −

−

≤= <What we can control

(excitation function)

(1,

)*1 1( ; )( , ) vjk

t trx xP x f H x x A xf e for x−= ≤


What to manipulate (reproduced field)



But time-reversed propagation of omni-directional virtual source generates left going wave as well.

1x x0 vx 2x

Reproduced field by left control sources

Control zone

Reproduced field by right control sources

Control zone

Left going

Right going

Since the left going wave is an artifact, it must be removed!

Virtual source

Virtual source



Time-reversed propagation of the directional virtual source can excite the left-side control source to reproduce only right going wave.

1x x0

Directional Virtual source

vx 2x

Control zone

Control sources can be selectively excited for reproducing a directional virtual source.


2D Reproduction of Interior Virtual Source

- Nspk = 64 (circular array), Control source at 2λ from center

Time-reversed radiation of the directional virtual source can selectively excite the control sources.

x [m]

y [m

]

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

Target field

Virtual Source

x [m]

y [m

]

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

Time-reversed field

Circular Control Source Array

Virtual Source

Control source selection

x [m]

y [m

]

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

Virtual Source

Reproduced field

Circular Control Source Array

Control source gain

Documents

KAIST: Korea Advance Institute of Science and Tech(4000 ... new Sound... · Sound Visualization and Manipulation 12 Sound visualization as a mapping [†] This illustration is a modified