Physically Based Sound COMP259 Nikunj Raghuvanshi Overview Background FEM Simulation Modal Synthesis (FoleyAutomatic) Comparison/Conclusions Motivation Sounds could in-principle be produced automatically, just like graphics: Sound Rendering Sound Rendering has not.
Download
Report
Transcript Physically Based Sound COMP259 Nikunj Raghuvanshi Overview Background FEM Simulation Modal Synthesis (FoleyAutomatic) Comparison/Conclusions Motivation Sounds could in-principle be produced automatically, just like graphics: Sound Rendering Sound Rendering has not.
Physically Based Sound
COMP259
Nikunj Raghuvanshi
Overview
Background
FEM Simulation
Modal Synthesis (FoleyAutomatic)
Comparison/Conclusions
Motivation
Sounds could in-principle be produced
automatically, just like graphics: Sound
Rendering
Sound Rendering has not received
much research effort
Main Goal: Automatic generation of
non-music, non-dialogue sound
Sound Production Today
Movies: Foley Artists
http://www.marblehead.net/foley/index.html
Games: Anyone noticed the
huge sound directory in
Unreal Tournament?
PBS: Sound Production in Nature
Collisions/Other interactions lead to
surface vibrations
Vibrations create pressure waves in air
Pressure waves sensed by ear
Vibration
Surface Vibration
Propagation
Pressure Wave
Perception
Ear
Main Aims of PBS
Physics simulator gives contact/collision
information
Assign material properties for sound,
Wood, concrete, metal etc.
Sound simulator generates sound using
this data (in real time?)
Challenges
Sound must be produced at a minimum of
~44,000 Hz
Extremely High Temporal Resolution
(timesteps in the range of 10-6-10-8 s)
Stiffness of underlying systems (eg.
Metallic sounds. K/m~=108)
Stability may require even smaller
timesteps
Two Approaches
FEM deformable simulation
O'Brien, J. F. et. al., “Synthesizing Sounds from
Physically Based Motion.” SIGGRAPH 2001.
FoleyAutomatic (Modal Synthesis)
Kees van den Doel et. Al., “FoleyAutomatic: Physicallybased Sound Effects for Interactive Simulation and
Animation.” SIGGRAPH 2001.
Main ideas
Deformable Simulation (arguably) much more
“physically based”
Foley Automatic: Additive Synthesis
Component
Sinusoids
Sound Signal
Overview
Background
FEM Simulation
Modal Synthesis (FoleyAutomatic)
Comparison/Conclusions
Simulation Requirements
Temporal Resolution
Simulate Vibration as well as Propagation
Vibration Modeling: Deformable Model for
Objects
Propagation Modeling: Explicit Surface
Representation
Physical/Perceptual Realism
System Structure
Vibration Modelling
FEM with Tetrahedral Elements
Linear Basis Functions, green’s strain
Explicit Time Integration
Typically #nodes = 500, #elements = 1500,
dt = 10-6-10-7 s
Sound Propagation Modelling
Fluid Dynamic FEM simulation of
surrounding air? Very expensive. Instead…
Employ Huygen’s Principle: Pressure Wave
may be seen as sum of pressure wavelets
Receiver
Receiver
Pressure
Wave
Pressure
“Wavelets”
Surface Vibrations and Sound
ˆ
Pressure contribution of a patch, p z v n
Unit Normal
Velocity
v
nˆ
ds
Density of Air
z c 415Pa s / m
Acoustic Impedance of Air
Sound Propagation Speed in Air
Surface Vibrations and Sound
Approximate differential elements with
surface triangles
Apply band pass filters:
Low pass: windowed sinc filter
High pass: DC blocking filter
Result: Pressure known for all surface
triangles
Putting it all together
Pressure/Signal at Receiver
Filtered Average Pressure
Area of Triangle
Visibility Term
Receiver
~
pa x r
s(t )
cos( )
x r
nˆ
xˆ
Approximation of Beam Pattern
Distance Falloff
r
Vibration
Propagation Delay
Accumulation Buffer
Receiver Distance from Source
1
d1
d
Delay
c
Source
d2
t2= d2/c
Receiver
t=0
Sound Propagation Speed
t1= d1/c
2
Results: Capabilities
General models
Generated sounds are accurate
Stereo Sound
Doppler’s Effect
Demo
Results: Accuracy
Results: Speed
Scene
TimeStep(s)
Nodes/Elems
Time/Audio Time
Bowl
10-6
387/1081
91.3/4.01 mins
125/265
240.4/1.26 mins
539/1484
1309.7/5.31 mins
Clamped Bar 10-7
Vibraphone
10-7
(~1 day)
Timings on a 350MHz SGI Origin MIPS R12K processor
Overview
Background
FEM Simulation
Modal Synthesis (FoleyAutomatic)
Comparison/Conclusions
Features
Modal resonance model of solids
Location dependent sounds
Impact, slide, roll excitation models
Real-time, low latency
Easy integration with simulation/animation
Practical
Do not model propagation of sound from source
to receiver
Synthesis Method
Emission
SoundVibration
Samples
Force
User
Propagation
Listener
Speakers
Vibration
Surface u(x,t) of body responds to external contact force
F(x,t)
u(x,t)
F(x,t)
i
1 2
[ g ( i , x ) 2 2 ]u( x i , t ) F ( x i , t )
x
c t
Strain Functional
Speed of Sound
Under suitable boundary conditions, the solution to
the PDE is a sum of sinusoids
Emission
Sound pressure s(t) linear functional L of surface
vibration u(x,t)
s(t)
L
u(x,t)
s(t ) L[u( x , t )]
i
pi ~ z vi nˆ
Note that propagation is not modeled in above
The Modal Synthesis Model
s(t)
L
u(x,t)
F(p,t)
“The response u(x,t) of an arbitrary solid object to an
external force can be described as a weighted sum of
damped sinusoids”
Impulse response/modal model
Since L is linear, it implies at s(t) must be a sum of
damped sinusoids too
Example: A 1D string
a1
a0
1st Mode
Frequency = f0
a0e
d 0t
2nd Mode
Frequency = f1= 2*f0
sin( 2f 0t ) + a1e
d1t
ak
…Higher modes
Frequency = fk= k*f0
sin( 2f1t ) +...+ ak e
dkt
sin( 2f k t )
Main Idea: Sum contributions of all the modes
The point of impact decides the proportions in which the modes are
to be mixed: ak. Therefore, ak is a function of p, the point of impact
The frequencies and damping parameters are a property of the
object, and independent of how the object is hit
The Modal Synthesis Model
s(t)
L
u(x,t)
F(p,t)
N
s(t ) ak ( p )e d k t sin( 2f k t )
k 1
Kth mode: Gain Factor Point
Damping
of impact
Term
Impulse response,
modal model
Vibration
Frequency
Parameters measured experimentally
Force Modeling
At runtime: Find gain parameters given the location,
strength and kind of force.
Synthesize sound from previous equation.
Impact
Sliding
Rolling
Wavetable
Stochastic
Impact Forces
•Duration: hardness (T)
•Magnitude: energy transfer (w)
•Multiple micro-collisions
Example: F (t ) w (1 cos( 2t / T )), 0 t T
Sliding/Scraping
Micro-collisions lead to noisy audio-force
Sliding/Scraping
Wavetable approach
Store force parameters
Modulate amplitude with energy transfer
Modulate rate with contact speed
Synthesis Approach
Fractal noise represents roughness
Filter through reson filter
Resonance ~ contact speed
Width ~ randomness of surface
Rolling
No relative surface motion
Differences with sliding:
•Smoother: Use low
pass
•More damping
•Harder to create
•Less understood
•Essential coupling?
Rolling: Smooth Surfaces
Polyhedral objects do not lead to smooth rolling forces
Instead use smooth surfaces directly
Rolling: Contact Evolution
c(u,v)
Evolve the contact in
Reduced coordinates
q = (u,v,s,t, )
..
q
d(s,t)
.
q
q
Rolling: Contact Evolution
Piecewise parametric surfaces, loop
subdivision surfaces
Explicit integration, no stabilization
Multiple contacts and conforming contacts
are not handled
Used only when multiple contacts in close
spatio-temporal proximity
Demo
Dynamic Forces
Pebble-in-Wok Demo
Contact force
Slipping speed
Rolling speed
Impulses
…and locations
Results
0.1% CPU time per mode
Graceful degradation of quality
The bell demo is interactive
Uses a PHANToM for interaction
Authors do not report any real timings
State that “sound quality” is perceptionbased and has no metric as of now
Overview
Background
FEM Simulation
Modal Synthesis (FoleyAutomatic)
Comparison/Conclusions
Discussion
FEM: Physically Rigorous and General
Too slow for interactive applications
Doesn’t scale well
Inappropriate to apply a 30fps technique to
44000fps?
Maybe too general for the problem
domain?
Discussion
Modal model exploits the vibrational
nature
Higher Efficiency
But, not rigorously physically based
Finding the parameters requires
experimentation and “earballing”
No rigorous correlation between physical
and perceptual parameters
Discussion
For Realtime: Need for a technique to
cover the middle ground
Extracting modal parameters in general
requires solving PDEs
Not possible to do in an automated
manner
Approximate modal parameters and then
use modal synthesis?
Conclusion
PBS involves orders of magnitude smaller
temporal and spatial scales
Research is sparse, problems are dense
Main contributions of the two papers
besides vibration modeling:
FEM: Efficient modeling of sound propagation
FoleyAutomatic: Efficient, Approximate models
to handle surface properties and contact forces
References
O'Brien, J. F., Cook, P. R., Essl G., "Synthesizing Sounds from
Physically Based Motion." The proceedings of ACM
SIGGRAPH 2001, Los Angeles, California, August 11-17, pp. 529536.
Kees van den Doel, Paul G. Kry and Dinesh K. Pai,
“FoleyAutomatic: Physically-based Sound Effects for Interactive
Simulation and Animation” Computer Graphics (ACM SIGGRAPH
01 Conference Proceedings), pp. 537-544, 2001.
Acknowledgements
Some images were taken from the referred
papers and the corresponding SIGGRAPH
slides