Transcript Real Time Walkthrough Auralization - the first year B.

Real Time Walkthrough
Auralization - the first year
B.-I. Dalenbäck CATT
M. Strömberg Valeo Graphics
Gothenburg Sweden
• from static to dynamic auralization
• properties and limitations
• model and receiver grid examples
• current and future options
• applications and summary
1
From static to dynamic
auralization: 1/6
• traditionally, starting around 20 years ago,
auralizations have been static: fixed listening
positions with fixed head directions.
• a single static, typically binaural, room impulse
response (FIR) is convolved with anechoic sound
• schematic representation of a static FIR (one head
direction):
direct early
late
L
time
R
2
From static to dynamic
auralization: 2/6
• a head-tracked binaural early part FIR with a static
late part, medium calculation time, fairly memory
consuming:
direct early
late
L
time
R
Many early part FIRs
corresponding to
many head directions
A single late part FIR
corresponding to one
head direction
3
From static to dynamic
auralization: 3/6
• a head-tracked full-length binaural FIR, long
calculation, very memory consuming:
direct early
late
L
time
R
Many full-length FIRs corresponding to many
head directions, still just a fixed position
4
From static to dynamic
auralization: 4/6
• the solution: full-length B-format FIRs, short
calculation, not memory consuming:
direct early
late
W
X
Y
time
Binaural
decode
L
R
Z
A single full-length B-format FIR,
rotation performed afterwards.
Each head direction
created when needed
5
From static to dynamic
auralization: 5/6
CATT-Walker™, putting it all together with many
positions, rotating and interpolating while convolving
Pre-processing
Anechoic WAV
Prediction
Echograms
Room
geometry
Postprocessing
Creates
WXYZ
FIRs
.
..
Source Mulitiple PA
sources
addition
(delay,
gain)
Real time
walkthrough
auralization
L
R
Single source
HRTFs
Headphone eq.
listener position
and rotation
6
From static to dynamic
auralization: 6/6
Real time processing details
Anechoic WAV
Real time walkthrough auralization
Rotated and
interpolated
WXYZ FIRs
WXYZ FIRs for
each receiver
position
Room
geometry
Continuous
rotation and
interpolation
W
x
hW
X
x
hX
Y
x
hY
.
..
Binaural
downmix
(convolution)
L
R
Z
x
hZ
Ambisonic
decode
Convolution
HRTFs
listener position
and rotation
Headphone eq.
Anechoic WAV
Prediction
Echograms
Room
geometry
Postprocessing
Creates
WXYZ
FIRs
.
..
Source Mulitiple PA
sources
addition
(delay,
gain)
Real time
walkthrough
auralization
L
R
Single source
HRTFs
Headphone eq.
listener position
and rotation
7
Properties and limitations
• the real time part of the process is independent of
room complexity (3 sec church “=“ 3 sec shoebox)
• the prediction and post-processing methods are
exactly the same as for static auralization
• requires a high receiver density where the IR is
expected to change fast with movement or head
direction
8
Y
Model and receiver
grid examples : 1/4
10m
A church walkthrough, 80 receivers, plan view
41
31
30
32
29
40 49
33 15
28
141312
11
16 0203 10 27
34 17
09
A0
57
08
35 18
19 0001 07 26
06 25
2122
36 20
0405
39 65
37
24
38
23
73
X
42
43
44
45
46
47
48
50
51
52
53
54
55
56
58
59
60
61
62
63
64
66
67
68
69
70
71
72
74
75
76
77
78
79
80
Y
9
Model and receiver
grid examples : 2/4
The church walkthrough, 3D view
10
Model and receiver
grid examples : 3/4
The church walkthrough, CATT-Walker™ view
11
Model and receiver
grid examples : 4/4
A smaller room walkthrough, 3D model view
42 41
43 2726
44 28
56 45
29 12
09
08
10
30 14
1311
01
73 66 57 46 31
00
15
A0
16
07
020305
17
47 32
06
18
58
19
33
04
2
0
67
2
3
2
2
1
74
48
34
25
83 79
59
49 353637383924
68
50
60
75
51 52 53 5440
80
69
61
84
62 63 64 65 55
76
70
81
71
77
72
85
78
82
86
12
Current options : 1/2
 multiple source simulation at no extra CPU cost
(assumes that sound input is common for all sources
such as in a PA system)
 choice of HRTFs for the binaural decode
 choice of headphone eq. for the binaural decode
 variable walking speed
 optional TCP/IP control via the Walker Steer API
 optional trade-off for use with slower PCs (latency
and/or horizontal only i.e. based on WXY)
13
Current options : 2/2
Optional grid and WXYZ FIR view
14
Future options : 1/2
 detailed models with textured graphics imported from
programs such as 3ds max
 head-tracking at no extra CPU cost (not crucial in
front of a PC screen)
 multiple independent sources (higher CPU demand)
 Doppler effects (difficult with FIR interpolation, not
crucial for room acoustics)
 direct B-format output for external decoding to any
loudspeaker array (lower CPU demand)
15
Future options : 2/2
 ambisonic output for direct loudspeaker replay (lower
CPU demand)
 use of measured instead of predicted B-format FIRs
(can be measured by a Soundfield microphone)
 use of 2nd order B-format FIRs (higher CPU demand,
not crucial for a binaural down-mix)
16
Applications
 everyday use (processing only longer due to a higher
number of receivers)
 presentations to clients
 presentations in architecture competitions
 research projects exploring the possibility to control
via TCP/IP. Two example EC-projects:
 “POEMS” at Chalmers University, Gothenburg
 “Wayfinding” at LIMSI, Paris
17
Summary
 a technique for real time walkthrough auralization has
been described:
 based on B-format FIRs and binaural downmix
 is in itself general and can as well be based on
measured responses
 no special shortcuts made for the real time option
 future improvements of prediction and auralization
methods will directly carry over to the walkthrough
auralization
18