Transcript Powerpoint PPT document - MusICA Seminars

The NESS Project
MUSICA Seminar Series
University of Edinburgh
13 February 2013
Stefan Bilbao, Paul Graham, Alan Gray, Brian Hamilton
Kostas Kavoussanakis, James Perry, Alberto Torin,
and Craig Webb
Acoustics Group
EPCC
Work supported by
the European
Research Council
1
Background
2
Background
Hwtt  D 4 w  L(w,F)

 4
EH
 F 
L(w,w)


2
L( f ,g)  xx fyy g  yy fxx g  2xy fxy g
3
•
Technology
Transfer
HPC Research
Facilities
The HPC Centre of the University of
Edinburgh
– 70 staff
– £4M turnover (almost) all from
external sources
Training
European
Projects
Visitor
Programmes
4
Case Study: FilmGrid
• Distributed digital asset management system
–
–
–
–
Accurate supervision of the progress of a film production
Secure asset transfer between entities working on the film
Complements existing film editing software (e.g. Nucoda)
Software support for current day-to-day manual workflow processes
Photo by Brad & Sabrina
5
Case Study: FilmGrid
Photo by Jeremy Keith
6
•
Academic:
– Permanent staff with degrees in various backgrounds:
• Physics, Computer Science, Mathematics, Chemistry, Biology…
– Education and Training provider:
• MSc in HPC
• Bespoke training on HPC, accelerators, multi-core computing
•
Strengths:
– Project management
– Interdisciplinary projects: small pilots, distributed programmes
– User support
•
Technical:
–
–
–
–
Software development for academia and industry
Code optimisation (serial, parallel, accelerators)
Simulation and modelling
Wide language, operating system and computer architecture expertise
http://www.epcc.ed.ac.uk/
7
NESS
•
An ERC funded project (PE6: Informatics), running at Edinburgh, 2012—2017.
•
The goal: digital sound synthesis…
…of musical sound
•
Today:
– Digital sound synthesis
and physical modeling
– Ness project: structure and activities
– Group member presentations
8
Digital Sound Synthesis
•
•
•
A longstanding attempt to move away from the use of recorded sound (sampling)
Analogous to computer graphics rendering?
At the philosophical level…yes
“Sampling”
•
“Synthesis”
At the technical level…not really!
9
Abstract Digital Sound Synthesis
•
Early synthesis methods (1950s—1960s): based on simple heuristic building blocks--efficient and easy to program…
Sinusoidal Oscillators:
•
•
•
Wavetables:
Frequency modulation synthesis: a very successful variant!
Still extremely popular!
Can be difficult to control (lots of user input), and sound quality is generally very
artificial!
FM trumpet
FM bell
10
Physical Modeling Sound Synthesis
Physical models:
– based on physical descriptions of
musical “objects”…
– can be computationally demanding…
– potentially very realistic* sound
– control parameters: few in number, and
perceptually meaningful
* realism needs a good definition, if there is
not a real-world reference!
A fair degree of hybridization abounds:
physical modeling + sampling is analogous to, say, motion
capture!
11
Methods: Lumped Mass Spring Networks
Earliest large scale attempt at physical modeling synthesis:
• networks of
masses/springs/dampers + simple
ODE solver
• basis for “Cordis” and “Cordis
Anima” systems (Cadoz and
collaborators, Grenoble, from 1979 to
present!)
• generally abstract (modular), but if
put in a regular arrangement, it is
possible to simulate distributed objects
(strings, membranes, etc.)
Cymbal
Timpani
12
Methods: Modal Synthesis
A different approach: decompose dynamics of vibrating object into modes…
• basis for “Modalys”
synthesis system (IRCAM,
1985—present)
• geared towards linear
objects (with interesting
extensions to the nonlinear
case…Volterra series, e.g.)
• a lot of offline
precomputation (modal
shapes, frequencies)
13
Methods: Digital Waveguides
Yet another approach: decompose dynamics of vibrating object into traveling waves…
• developed at CCRMA,
Stanford University, 1985—
present
• roots in early scattering-based
speech synthesis methods (Kelly
Lochbaum, 1962)
• basis for many synthesis
systems, including Yamaha VL1
(1994)
• meant for simulating
distributed linear objects in 1D
(strings, acoustic
tubes)…extremely efficient!
Guitar
14
Methods: Time-stepping Methods (FDTD, FVTD,
etc.)
The obvious approach: represent dynamics of vibrating object on a grid and integrate
using direct solvers…
• distinct roots in musical acoustics
investigations, and a few early
synthesis attempts (Ruiz, vibrating
string, 1969!)
• tools exist to handle virtually any
system…much more general than
other methods
• but…a lot of specialization work for
audio applications…
15
NESS: Target Systems
•
•
Trying to span the full range of musical acoustic systems…
difficult to approach using other physical modeling techniques…
Brass Instruments
Nonlinear Plate and
Shell Vibration
Trumpet
Gong
Cymbal
Electromechanical Instruments
16
NESS: Target Systems
Modular Synthesis
Environments
Room Acoustics
Modelling
Embeddings and
Spatialization
Snare Drum
Excerpt: Orbit, G. Delap, 2009
17
Sample rates and bandwidth
•
•
•
•
•
•
•
A basic constraint for audio: choice of sample rate Fs, and time step k = 1/Fs
Constraint 1 (necessary): Need to be able to fully render audio up to limits of
human audio perception, so:
Fs ≥ 40 000 Hz, k ≤1/40000 s
Constraint 2 (desirable): Don’t want to render audio above this range, for
efficiency reasons, so:
Fs ≈ 40 000 Hz, k ≈ 1/40000 s
Time step is small…lots of computational work to do…
Some aspects of time stepping methods need to be reconsidered in this
light!
18
Grids and Bandwidth Limitation
Suppose operation at
xa given
x sample rate…need to choose the grid carefully, for perceptual reasons:
Simple string model…
min
Increase
in grid
spacing
x  2xmin
x2
decrease
in
operation
count
decrease
in output
bandwidth
x4
x  xmin
• an additional constraint in numerical design…certain techniques (grid refinement) are dangerous in an
audio context
19
Audibility of Numerical Dispersion
Numerical Dispersion: speed of wave propagation is incorrect, numerically!
Example: thin bar, simple
explicit FD method:
Phase velocity
Exact and Numerical
Mistuning!
Exact:
44100 Hz:
Careful design necessary…methods with free parameters allow a means of
tuning the scheme…at the price of linear system solutions (hard on GPU!)
20
Numerical Instability
•
A critical concern in synthesis design for non expert users…
Linear membrane instability
•
Not too hard to deal with…
Nonlinear shock wave instability
•
Harder to deal with without
compromising audio output…
21
Modular Instability
•
•
Consider two
rudimentary
systems
Problems can appear here if the connection is not handled properly:
Stable Connection
•
Ideal
String
Mass/
spring
Unstable Connection
Difficulties are compounded for more complex systems…
22
Energy-based Stability
Numerical energy conserved to machine accuracy:
…giving a stability guarantee
Extremely useful in
debugging, and in
designing complex
modular systems:
23
Computational Costs and HPC
•
•
Audio sample rates are high: 44 100 Hz, 48 000 Hz, 92 000 Hz…
Flop rates/memory requirement scale as power of sample rate (2,3,4)…
arithmetic operations/second output, at 48 000 Hz:
106
107
108
109
1010
1011
1012
1013
1014
1015
1016
1017
Brass instruments
optimal realtime
performance on
commercially available
single core
Nonlinear plates/shells
Electromechanical
Instruments
Small embeddings
•
•
•
Small rooms
Concert Halls
Musical use/experimentation: reasonable compute time (no overnight jobs!)
Solutions: Parallel implementations (GPU, e.g.)
New algorithmic issues: parallelizability, memory management, stability in finite
precision
24
3-D Room Acoustics
(Brian Hamilton, Acoustics Group)
25
Simulating 3-D Room Acoustics
• A wave is a spatial field that changes over time
• Sound propagates as a pressure wave
• Simulating sound wave propagation:
– Pick some 3-D lattice (grid) of points
– Calculate sound pressure at each point
– Iterate in time...
• Problem to solve:
– How to do this as efficiently as possible?
– Any audible artifacts? How to minimise them?
26
Spatial Lattices
Which to choose?
27
Spatial Lattices
Many choices...
28
Spatial Lattices
• Waves should propagate uniformly in every direction
• Symmetry is key!
Stacking fruit
29
Numerical Dispersion
• Numerical dispersion → wave speed error
• Simulated waves propagate along axes of grid
Wave speed depends on grid orientation!
Example: Without Dispersion
31
Example: With Dispersion
32
Wave Speed Error
• We want the error to be isotropic (direction independent)
• Delicate cancellation of error in space and time
33
Wave Speed Error
34
Wave Speed Error
35
Audible Artifacts: Examples
• Castanet (clean)
• Castanet (dispersive)
• Guitar (clean)
• Guitar (dispersive)
36
Percussion Instruments
(Alberto Torin, Acoustics)
Membranes
Plates and Shells
Low excitation!
L
NL
High excitation!
L = Linear,
NL = Non-linear
37
Linear Plates
w = transverse displacement
 = density, H = thickness,
D = stiffness parameter
- There is no interaction
between different modes!
38
Non-linear Plates
• Add non-linear terms to previous equation
von Kármán equations for non-linear plates
F = Airy’s function, E = Young’s modulus
39
Non-linear Plates
- Energy exchange between
different modes is allowed!
- Crashes, Pitch glide effects
40
Air coupling
• Add the pressure on the plate
•
Introduce the acoustic field , that obeys the wave equation
• Add coupling
conditions
between the air
and the plate
41
Numerical schemes
• Stability and Energy conservation
• Need for a Fast algorithm
– Bottleneck of the code is the solution of a sparse linear system
(matrices involved have a few non-zero entries)
– We can use iterative solvers…
• works well for the simple plate
• needs extra work when air coupling is present
42
Example: MultiPlate3D
Roll gesture
Several strikes
43
What is a GPU?
• Graphics processing unit
– Originally designed for rendering
3D graphics fast
– Now also used for general
purpose computations (GPGPU)
• Very well suited for problems
like ours
– Especially the 3D ones
– Same simple computation
required for huge number of
points
44
Porting Process
• Port from Matlab to C
– Faster than Matlab, will run anywhere
– Easier to debug and modify than CUDA
– Good basis for CUDA port
Matlab
C
• Port from C to CUDA
– Some code (e.g. setup code) remains in
C
– Time critical main loop is ported to
CUDA
CUDA
• Optimize CUDA code
– Gain high performance (as far as
possible)
Optimized CUDA
45
Example
• 3dabc code
– Simulates a 3D box of air with
various boundary conditions
• Run times:
Version
Run time (s)
Speed up
Matlab
31
1x
C
0.441
70x
CUDA
0.0786
395x
• Matlab version not optimised
• Small simulation size - would expect larger
speed-up from C to CUDA for large size
46
Large-scale 3D virtual acoustics
(Craig Webb, Acoustics)
•
•
•
•
Computing 3D wave propagation in a virtual space.
Dynamic simulations, with full wave behavior.
Can inject dry audio to produce reverberation.
Or embed virtual instruments.
47
Computation Size
At a sample rate of 44.1kHz :
• 1 cubic metre requires 422 thousand grid points.
• 1 second of output requires 185G floating-point
operations.
• 2,000 cubic metres – 370T operations.
• A 10 second simulation – 3.7P operations, that’s
3,700,000,000,000,000.
Use multiple GPU cards to accelerate the model.
• With 4 cards, speedup over serial C code is in the
range x100 ~ x140.
• Under an hour per second of simulation, instead of
5 days in serial C.
• Requires 10Gb of memory at single precision.
75 points
1 cubic metre = 75x75x75 grid points
48
Hall Simulation: Dry Audio Input
Audio examples of 2,000 cubic metre hall
1. Dry guitar input :
Output :
2. More guitars :
3. Opera singer (anechoic) :
Output :
1. Can move sound around during runtime :
49
Embedded Instruments
Timpani Drum
The timpani drum is a good test case for 3D physical
modeling.
We use a non-linear circular membrane, attached to
a parabolic shell with fixed boundaries.
This is then placed inside the room simulations, and
we can model any number of timpani inside the space.
These can then be played together, by specifying the
timing and type of strikes on each drum.
Audio examples
One Timpani :
Two Timpani :
Three Timpani :
Four Timpani :
50
Creative Uses: Composition
•
•
A new world of sound for musicians and
composers---fully multichannel, synthetic
music environments
But---a learning curve! As for any mature
instrument design…
51
Control and Interfaces
•
NESS: audio only! Not really any attempt at building live, performable instruments, or
developing complex interfaces…
•
Two subsequent levels of work:
–
–
Figuring out useful, parsimonious ways of representing input
UI design (simple!)
52
NESS
• Thank you for your
attention
• Questions?
53