Transcript Slide 1

Illustrative Volume Rendering
on
Consumer Graphics Hardware
Roy van Pelt
Exam Committee:
dr. A. Vilanova (BMT)
dr. ir. H.M.M. van de Wetering (CSE)
dr. ir. M. Chaudron (CSE)
1/40
Project motivation
 Improve comprehensibility
 Useful for context visualisations
 Similar to illustrations in physiology books
2/40
Introduction: Illustrative Volume
Rendering
 Volume Rendering
 3D dataset, i.e.: CT / MRI
Images by kind courtesy of R. Brecheisen (2007) (MVR
framework)
3/40
Introduction: Illustrative Volume
Rendering
 Illustrative Volume Rendering
 Depict dataset as a drawing/sketch
Stippling
Hatching
Contours
4/40
Introduction: Illustrative Volume
Rendering
 Existing framework: VolumeFlies
 Traditional illustration techniques
 Based on particle systems
5/40
Project objectives
 Design and implement VolumeFlies, using
consumer graphics hardware, such that
real-time interaction with the data is
possible.
 Integrate VolumeFlies with a direct volume
rendering approach.
6/40
Background Info: GPU pipeline
• Massively parallel
– Adjust algorithms to use parallelism
– Unified shader design
• Programmability
– Stream programming model using shaders
Geometry
Vertex
Processing
Shader
Geometry
Shader
Fragment
Rasterization
Shader
Fragment
Operations
7/40
Background Info: GPU pipeline
• Massively parallel
– Adjust algorithms to use parallelism
– Unified shader design
• Programmability
– Stream programming model using shaders
Geometry
Vertex
Processing
Shader
Geometry
Shader
Fragment
Rasterization
Shader
Fragment
Operations
8/40
Background Info: General GPU
approach
Source buffer = “Input array”
Geometry
Vertex
Processing
Shader
Proxy geometry = “Array index”
Destination buffer = “Output array”
Geometry
Shader
Fragment
Rasterization
Shader
Fragment
Operations
Transform feedback
Discard fragment shader
9/40
Background Info: Particle system
 Particle: an element described by properties
For example particle positionp
in the volume
Movie source:
http://www.runevision.com
10/40
Framework modules
1 Initialiser
Initialises particle set
2 Behaviou
r
Alters particle set
3 Filter
Removes unwanted particles
4 Visualise
r
Illustratively depicts particles
11/40
Framework modules
1 Initialiser
Initialize particles near the ISO-surface
Related to marching cubes
12/40
Framework modules
1 Initialiser
3D Texture
Volume
Vertex
Shader
Particle positions
(x,y,z) = (r,g,b)
Geometry
Shader
Fragment
Shader
Proxy geometry
13/40
Framework modules
1 Initialiser
2 Behaviou
r
3 Filter
4 Visualise
r
14/40
Framework modules
2 Behaviou
r ij r pi  p j
Eij (| rij |)
pj
Minimise
the total energy
within the particle system
m
60
Ei   Eij (| rij |)
rviijijthe
|
m evenly |on
Redistribute
particles
j 1, j i
40
surfacepii
E  E
E E Ei
20
vi  ( i , i , Energy
)
minimisation
i 1
xi yi zi
0.2
0.4
0.6
0.8
1
| rij |
“Robust particle systems for curvature dependent sampling of implicit surfaces” – Meyer et al.
(SMI 2005)
15/40
Framework modules
2 Behaviou
r
ni
vi
tangent plane
iso-surface
Redistribute particles evenly on the
surface
Two step particle displacement
“Robust particle systems for curvature dependent sampling of implicit surfaces” – Meyer et al.
(SMI 2005)
16/40
Framework modules
2 Behaviou
r
X
1
Y1
X
2
Y2
X
3
Y3
X
4
Y4
X
5
Y5
X
6
Y6
X
7
Y7
X
8
Y8
X
9
Y9
X
10
Y 10
X
11
Y 11
X
12
Y 12
Z1
Z2
Z3
Z4
Z5
Z6
Z7
Z8
Z9
Z 10 Z 11 Z 12
Bin 0 Bin 1 Bin 0 Bin 0 Bin 0 Bin 1 Bin 2 Bin 1 Bin 3 Bin 2 Bin 3 Bin 2
1. Sort particles by their bin number
Redistribution
Solution:
hard to
port to GPU:
Odd-even merge
sort
X
X
X
X
X
X
Create
Addressing
aXYbin-structure
all
neighbours
to XYcompute
isXY XY XY
Y
Y
Y
Y
Y
Y
X1
Y1
3
4
5
2
6
8
7
10
12
9
11
3
4
5
2
6
8
7
10
12
9
11
Z1
Z3
Z4
Z5
Z2
Z6
Z8
Z7
Z 10 Z 12 Z 9
Z 11
Bin 0 Bin 0 Bin 0 Bin 0 Bin 1 Bin 1 Bin 1 Bin 2 Bin 2 Bin 2 Bin 3 Bin 3
computationally
energy minimisation
expensive
locally.
2. Create a lookup table for the bins
Binary search
0
4
7
10
3. Redistribute particles by repulsion
Energy minimisation approach
17/40
Framework modules
2 Behaviou
Verify the change of the total energy
r
m
E   Ei
Apply reduction operation on the GPU
E1 E2 E3 E4 E5 E6 E7 E8
i 1
VolumeFlies +applied a fixed number of iterations
'
'
'
'
E
E
E
E
2
4
1
3
A new stop-criterion
is introduced
+
E ''1E ''2
+
E '''1
Total system energy
18/40
Framework modules
2 Behaviou
r
Odd-Even Merge Sort
GPU approach more elaborate.
Binary Search
Still much faster than CPU approach.
Energy Minimization
Particle system is generic.
Verify stop-criterion
19/40
Framework modules
Behaviou
2 Initializer
1
r
3 Filter
4 Visualize
r
20/40
Framework modules
3 Filter
a) Create the ISO surface
Cone splatting:
Cones directed to the viewing plane
Cones are scaled to prevent edge overlap
21/40
Framework modules
3 Filter
b) Hidden 
surface
removal

Off-screen buffer:

Splat rgb = Particle xyz
Visible when particle position occurs as a colour
22/40
Framework modules
Initialiser
1 Filter
3
2 Behaviou
r
4 Visualise
r
23/40
Density-based stippling
4 Visualise
r
Per particle random threshold
Based on basic diffuse
Vary particle
lighting density to change tone
Initial density resembles darkest tone
Executed by vertex shader
User can control contrast
24/40
Scale-based stippling
4 Visualise
r
Scale point representation
Based on basic diffuse
Vary
point size to change tone
lighting
Larger points create darker areas
Executed by vertex shader
User can control contrast
25/40
Direction-based hatching
4 Visualise
r
Single hatches
Hatch tracing in a single direction
Single and cross hatching
Cross hatches
26/40
Direction-based hatching
4 Visualise
r
iso-surface
Hatch tracing in a single direction
Segments projected in tangent plane
27/40
Direction-based hatching
4 Visualise
r
Particle positions
1
3D Texture: Volume
Vertex
Shader
Proxy geometry
Hatch segments
Geometry
Shader
Fragment
Shader
Generate hatches in fixed direction
Hatch segments
2
Vertex
Shader
Proxy geometry
Geometry
Shader
Fragment
Shader
Visualise generated hatches
28/40
Direction-based hatching
4 Visualise
r
1st level
2nd level
Again apply basic diffuse lighting
Two-level threshold
29/40
Curvature-based hatching
4 Visualise
r Differential structure in volume required
1. Filtering approach for derivative reconstruction
2. Optimize
Emphasize
for GPU
curvature
by usingwith
interpolation
hatches capabilities
Fast curvature
calculation approach
3. Compute
principal curvature
“Fast Third-Order Texture Filtering” – Sigg, Hadwiger (GPU
Gems 2)
Re-implemented in high-level shading language
Based on general GPU-approach
30/40
Curvature-based hatching
4 Visualise
Variation
Maximum
theminimum
normal vector
change
when
describe
moving
thea principal
small distance
r ofand
curvature
Emphasize curvature with hatches
Defining the principal
curvature
ni
k1i
k 2i
tangent plane
iso-surface
Image by Eric
Gaba
31/40
Curvature-based hatching
4 Visualise
 r Measure first partial derivates:


gradient g
Measure second partial derivates: Hessian H
S
Extract curvatures by eigen analysis on the shape operator
Emphasize curvature with curvature
Calculate principal curvature
n
k1
tangent plane
n  g / | g |
P  I  nn
iso-surface
T
g
S  PHP / | g |
“Curvature-Based Transfer Functions for Direct Volume Rendering Methods and Applications” – Kindlmann et al. (IEEE
Visualization 2003)
32/40
Curvature-based hatching
4 Visualise
r
1D Transfer function
-1
1
k1
0
1
2
k2
33/40
Curvature-based hatching
4 Visualise
r
Messy hatch results using curvature directions
Smooth the field directions and weight the
trace
Fixed direction
when
surface is
unreliable
34/40
Curvature-based hatching
4 Visualise

r
2
1
Principal curvature can be calculated in real-time
Demonstrated by real-time curvature colour
mapping
35/40
Contours
4 Visualise
r
Contour: locations where
normal
is perpendicular
to the
view
Draw ‘hatches’
for particles
near
the contour
Segments trace the direction of the contour
36/40
Results: Performance
VolFliesGPU
VolumeFlies
Load Volume
Brute-force initialiser
253.34
Redistribution
Smooth Field
52.65
Hatch generation (Direction)
53.05
Hatch generation (Scale)
Hatch visualisation
Contours
0
0
2
2
44
6
6
88
10
10
12
12
14
14
16
16
18
18
Profiling time (seconds)
37/40
20
20
Results: Demonstration movie
38/40
Conclusions & Future work
 Completely GPU-based
 General GPU approach
 Two generic components:
 Particle system
 Real-time curvature estimator
 Integration with direct volume rendering
 More elaborate memory management
 Improvement of modules / Zooming
 New applications and styles: DTI, animation, ghosting, exploded view…
39/40
Thank you for your attention!
Questions
?
40/40
Particle repulsion
 Energy minimisation
E  E
m
i
i 1
Ei 
m

j 1, j i
vi  (
Eij (| rij |)
r ij  pi  p j
Ei Ei Ei
,
,
)
xi yi zi
 Two step particle displacement
T
pi  pi  ( I  ni ni )v i
gi  f ( pi )
pi  pi  f ( pi )
gi
| g i |2
41/40
Stippling
 Density-based
Ld (i)  max(ni  ei,0)
vi  ( Ld (i))c
 Scale-based
Ld (i)  max(ni  ei,0)
Si   (
3
2
(1  Ld (i)))c
42/40
Hatching
 Smooth field
 2  1
s  arctan

 2  1
2
0, if | 1 |  and |  2 | 

 (1 ,  2 )  
1
1

|
2(|
s
|

) |, otherwise

2

wk

s  (w )
w
 )
w (
j
i
T
1j
 (1  wT ) sT
j
j
T
b
43/40
Hatching
 Tracing
h0  s i
(1  w j )h j  w j  j k1 , | k1  h j || k2  h j |
j
j
j

h j 1 
(1  w j )h j  w j  j k2 j , | k1 j  h j || k2 j  h j |
j
wj 
s
44/40
Contours
d  2 (k2 e)k1  1 (k1 e)k2
Image by D.
DeCarlo
45/40
Curvature estimation 1/3
 Measure partial derivatives by
convolution filtering with cubic B-spline
46/40
Curvature estimation 2/3
 Optimise for the GPU by using interpolation
Weights lookup texture
i-1
i
Colour texture
x
i+1
i+2
47/40
Curvature estimation 3/3
 Compute principal curvature
n  g / | g |
P  I  nn
T
S  PHP / | g |
 Eigen analysis on S
S k 1  1 k 1
S k 2  2 k 2
48/40