Physics and Sound

Zhimin & Dave

Motivation

• Physical simulation

• Games

• Movies

• Special effects

What is done now?

• Foley artist

Lucasfilm Foley Artist

Vibration Pressure Wave Perception

How sound works

• Transverse waves

• Doppler effect

Why sound rendering is hard

• Very small time step

• Ideally 44.1kHz

• Small time step at lest 10-6

• Stability

Sound Rendering

• Method for synchronized soundtracks and animations

• Sound scripts

• Sound rendering pipeline

• Key framing audio, Approximate refraction, Different models for sound generation, etc.

Sound Rendering

• Sound rendering definition– The process of creating a composite sound

track from component sound objects

• Sound scripts– Control sound instantiation and

modification• User specified• From system, ex. simulation, periodic, etc.

Sound Rendering Pipeline

Vibrational Analysis

Instantiate Sound

Propagation Resample

Modeling

Rendering

Vibrational Analysis

• Generate prototype sound, characteristic of each object and interaction with other objects

• Physically Based– FEM, Natural vibration modes,

Approximation

• Friction and Turbulence• Procedural Sounds (what they used)

Instantiate Sound

• Prototype sound instantiate

• Attach to some 3D object

• Start energy of sound wave

• Use of sound scripts

Propagate

• Physical Simulation

• Beam tracing

• Frustum tracing

• Refraction

• Effects - reverberation, refraction

Resampling

• Sound prototype to sound track– Sound at time steps (key-frame audio)

• Amplitude - linear interpolation• Frequency - high order approximation or very dense

sampling

• Like texture mapping– Object space algorithm– Image space algorithm– Warning: aliasing problems similar to texture

mapping, solved similar methods

Sound Rendering Demo

Sound Rendering Pipeline

Vibrational Analysis

Instantiate Sound

Propagation Resample

Modeling

Rendering

DeformablebodySimulation

Deformable objects

• Piggy back on top of deformable body simulator

• Use audible vibrational component of simulation

• Generate sound pressure and model sound propagation

Requirements

• Temporal resolution higher then 10-5

• Dynamic deformation– FEM– mass spring

• Explicit Surface Representation• Physical Realism, if not

– sounds will reveal errors in motion– strange sound responses

Modeling surface vibration

• I will put the details of this here (commming soon)

Deformable objects Demo

Sound Rendering Pipeline

Vibrational Analysis

Instantiate Sound

Propagation Resample

Modeling

Rendering

PhysicalShapes

NOTE to Ming

• Some of the slides on Sound of Physical Shapes I am going to replace with an example using a Circular membrane, since I have pictures that it may be more clear.

The Sound of Physical Shapes

•Sounds of physical shapes are important for enhance realism

•Sounds produced by colliding objects depend on object material, shape and the location of contact.•Sounds provide listeners with clues about the object.

The framework

•Precompute characteristic frequencies

•Divide object boundary into small regions

•Determine the amplitudes of excitation for an impulse applied to a region

•Normalize energies energies, so only scaling is needed when rendering

Vibrating Shapes from Impact

•Calculating the Modes:•Vibration of the object obeys the wave equation:

•Initial conditions:

•Norm of the eigenfunctions:

2

2 2

1( ) ( , ) ( , )A x t F x t

C tμ∂

− =∂

0

0

( ,0) ( ),

( ,0)( ).

x y x

xv x

t

μμ

=∂

=∂

2 ( ) kn n

S

x d xα = Φ∫

•Calculating the Modes:•Solution to the wave equation:

• are related to eigenvalues of That is

1

( , ) ( sin( ) cos( )) ( )n n n n nn

x t a ct b ct xη ω ω∞

=

= + Φ∑nω A

2 2( ) ( ) 0n nA xω+ Φ =

Vibrating Shapes from Impact

•Calculating the Modes:•Calculating the amplitudes of the vibration modes

•Time averaged energy of the vibration •(if mass distributed uniformly)

0

0

( ) ( ),

( ) ( )

knn

n nS

knn

nS

v x xa d x

c

y x xb d x

α ω

α

Φ=

Φ=

∫

∫

2 2 2n

1

constant ( )n n nn

E a bα ω∞

=

= × +∑

Vibrating Shapes from Impact

•Initial conditions:

• ,the k-dimensional Dirac delta function•The amplitudes are:

•The energy is:

0

0

( ) 0,

( ) ( ).

y x

v x x pδ== −

( )xδ

( ),

0.

nn

n n

n

pa

c

b

α ωΦ

=

=2

1

( )constant

fnn

n n

pE

α=

Φ= ×∑

Mode Amplitudes from Impact Location

•Compute sound emitted from the vibration:•Intensity of the sound emitted:

It integrates the intensity of the vibration.

2 2( ) constant ( )n n n n

S

I E x p= Φ = ×Φ∫

Sound Sources from Vibrating Shapes

•Each mode decays with decay time:

• is the internal friction parameter, which depends on material only.

•Having emulated the external damping of object, sound-wave:

1

taniif

τπ φ

=

φ

0/ tan

1

( ) sin(2 )f

i

nt tfS

s i ii

p t e a e f tτ π φ π− −

=

= ∑

Sounds and Material Properties

•Store the pre-computed frequency spectrum

, and the excitation spectrum for a number of locations on the surface.

•Treat two sounds and as the same, if the

the difference between them ,

where is a threshold.

2n

i

cf

ωπ

= ia

1S 2S

1 2 0( , )d S S d<

0d

Sound Map

http://www.cs.ubc.ca/spider/kvdoel/app6JavaSound/demo.html

Live Demo

Sound Rendering Pipeline

Vibrational Analysis

Instantiate Sound

Propagation Resample

Modeling

Rendering

RigidbodySimulation

Sound from Rigid body simulations

• Precompute shape and frequency of deformation modes– Result, quick computation of vibrational

response at runtime

• Numerically via eigen-decomposition of system of matrices from FEM model

• Model sound from geometric description and material paarameters

Modal Analysis

• Zhimin and I were planning on ommiting the calculation of the eigenvectors from this so she can speak about it in Nikunj’s paper, as it uses the same process.

Rigid body simulation

• Normal rigid body simulation – One change, export collision forces to

sound synthesizing process.

• One thing…– Sound synthesis can reveal flaws of bad

rigid body simulator

Deforming the model

• Need to be able to generate K, C and M

• Mass spring, FEM etc.

• FEM used, because of past success, see [OSG02] for details

A Sound Generation Alg

• Details of the alg here

How to speed up

• Eval 18 use previous values e^{\omega(t+s)} = e^{\omega*t} + e^{\omega*s}

Other effects

• Cost of additional impulses is low (analytical solution)

• Convolve with Gaussian kernel– Force for soft collisions– Noise for small scale friction

Rigid body Demo