Transcript Practical Implementation of High Dynamic Range Rendering

Practical Implementation of
High Dynamic Range
Rendering
Masaki Kawase
BUNKASHA GAMES
BUNKASHA PUBLISHING CO.,LTD
http://www.bunkasha-games.com
http://www.daionet.gr.jp/~masa/
Agenda
•
•
•
•
•
•
•
What can be done with HDR?
Dynamic Range
Implementation on DX8 hardware
Implementation on DX9 hardware
Multiple Gaussian Filters
HDR in games
References
What can be done with HDR?
• Dazzling light
• Dazzling reflection
• Fresnel reflection
– bright reflection in the normal direction
• Exposure control effects
• Realistic depth-of-field effects
• Realistic motion blur
Dazzling Light
Dazzling Reflection
HDR Fresnel
Bright reflection
off low-reflectance surfaces
Exposure Control
HDR Depth-of-Field
Future perspective
HDR Motion Blur
Future perspective
Dynamic Range
• The ratio of the greatest value to the smallest
value that can be represented
• Displayable image
– 28
Low dynamic range (LDR)
• Frame buffer of absolute luminance
– Render the scene in absolute luminance space
– >232
represents all luminances directly
• Frame buffer of relative luminance
– Apply exposure scaling during rendering
– >215~16 dark regions are not important
HDR Frame Buffers
• For glare generation
• When rendering with relative
luminances:
– Ideally, more than 215~16
– In games
• 212~13 (4,000~10,000) is acceptable
HDR Environment Maps
• Very important for representing:
– Realistic specular reflection
– Dazzling specular reflection
• Specular reflectance of nonmetals
– Reflectance in the normal direction is typically
less than 4%
– Bright light remains bright after such low reflection
• To maintain dazzles after reflection of ~1-4%
– Dynamic range of more than 10,000 or 20,000 is
necessary
Implementation on DX8 Hardware
• We have no choices
• Pixel Shader 1.x
– Integer operations only
• HDR buffer formats
– Low-precision buffers only
– Use the alpha channel as luminance information
– Fake it to achieve believable appearance
• Accurate calculation is not feasible
Fake HDR Pixel Shader
ps_1_1
Glossy reflection material
// v0.rgb : Diffuse color of primary light
// v1.rgb : Color for other lights/ambient
//
mad r0.rgb,
v0, t2,
v1
pre-scaled by (exposure * 0.5)
//
// v1.a
tex t0
tex t1
tex t2
: Specular reflectance (Fresnel)
+mul t0.a,
v1.a, t0.a
//
//
//
//
//
Scale the primary diffuse color by
shadow/light map, and add the result of
other per-vertex lighting
Scale the specular reflectance
by gloss map
//
// t0.rgb : Decal texture (for diffuse)
// t0.a
: Gloss map (for specular)
mul r0.rgb, t0, r0
+mul r0.a, t0.a, t1.a
// Modulate diffuse color with decal texture
// r0.a = specular reflectance
* envmap luminance
mul t1.rgb, t1, c0
+mul r1.a, r0.a, t1.a
// Modulate envmap with specular color
// Envmap brightness parameter
// r1.a = specular reflectance
* envmap luminance * gloss map
// t1.rgb : Envmap color
// t1.a
: Envmap luminance
// t2.rgb : Shadow/light map
//
// c0.rgb : Specular color
// c0.a
: Clamp(gloss * 2, 0, 1)
lrp r0.rgb, t0.a, t1, r0 // Reflect the envmap by specular reflectance
mul r1.a, r1.a, c0.a
// Envmap brightness parameter
// r1.a = specular reflectance
* envmap luminance * gloss map
* Clamp(gloss * 2, 0, 1)
lrp r0.rgb,
+lrp r0.a,
r1.a,
r1.a,
t1, r0 //
//
//
//
t1.a, r0.a
Output color
Interpolate the envmap color and the
result of LDR computation, based on
the envmap brightness parameter (r1.a)
// Output luminance information
Generating Displayable Image
• Extract high-luminance regions
 1  Threshold

Glaresrc  Buffer.rgb  
 Buffer.a 
16


Threshold : ~0.4-0.5
• Generate glare
– Reference:
• Kawase, Masaki, “Frame Buffer Postprocessing Effects in DOUBLES.T.E.A.L (Wreckless)”
• Generate a displayable image
– Calculate the luminance from the frame buffer


Lumexposed  Buffer.rgb  Buffer.rgb Buffer.a2 16  2
– Add the result of glare generation to the luminance
Notes on DX8 Implementation
• Accurate calculation is not feasible
– How to make it believable by faking
– Based on appearance rather than theory
Implementation on DX9 Hardware
• There are currently many limitations
– Choose implementations accordingly
• Pixel Shader
– Pixel Shader 2.0 or later
– Pixel Shader 1.x
• Buffer formats for HDR
– High-precision integer/float buffers
– Low-precision integer buffers
Issues with High-Precision Buffers
• Memory usage
– At least twice as much memory as the
conventional full-color buffer is needed
• Limitations
– Alpha blending cannot be used
– Texture filtering cannot be used with floating-point
formats
• Some systems don’t support them
• The situation is not good…
Use Low-Precision Buffers
• Make use of low-precision buffers
– A8R8G8B8 / A2R10G10B10 etc.
– Low memory consumption
– Alpha blending can be used
Compression with Tone Mapping
• Render directly to displayable format
• Nonlinear color compression
Buffer( x, y) 
Lumexposed ( x, y)
Lumexposed ( x, y)  1
– Effectively wide dynamic range
– Reference:
• Reinhard, Erik, Mike Stark, Peter Shirley, and Jim Ferwerda,
“Photographic Tone Reproduction for Digital Images”
• The alpha channel is not used
– Can be used for any other purpose
Environment Map Formats
• Relatively low resolution
• Alpha channel/blending is not very important
• Use the 16-bit integer format if enough
memory storage is available
– Treat it as having an interval of [0, 256] or [0, 512]
– Texture filtering can be used
• In the future
– Do it all with A16B16G16R16F
Low-Precision Environment Maps
• Use them when:
– High-precision buffers are not supported, or
– Memory storage is limited
• If the fill-rate of your system is relatively low
– Use the same format as used in DX8 fake HDR
• If the fill-rate is high enough:
– Nonlinear color compression
• Similar to tone mapping
– Store exponents into the alpha channel
• More accurate operations are possible
• Using it just as a scale factor is not enough
– Even the DX8 fake HDR has a much bigger impact
Color Compression
• Similar to tone mapping
• Encode when rendering to an environment map
Buffer( x, y) 
Lumexposed ( x, y)
Lumexposed ( x, y)  Offset
Offset : luminance curve controlling factor (~2-4)
• A bigger offset means:
– High-luminance regions have higher resolutions
– Low-luminance regions have Lower resolutions
• Decode when rendering to a frame buffer
– From the environment map fetched
Lumexposed ( x, y ) 
Buffer( x, y )  Offset
1 δ Buffer( x, y )
d : a small value to avoid divide-by-zero
Color Compression
• Use carefully
– Mach banding may become noticeable on
reflections of large area light sources
• e.g. Light sky
E8R8G8B8
• Store a common exponent for RGB into the
alpha channel
• Use a base of 1.04 to 1.08
offset : ~64-128
Lumexposed  Buffer.rgb baseBuffer.α256offset
– Base=1.04 means dynamic range of ~23,000
(1.04256)
• A bigger base value means:
– Higher dynamic range
– Lower resolution (Mach banding becomes
noticeable)
E8R8G8B8 Encoding (HLSL)
// a^n = b
#define LOG(a, b)
( log((b)) / log((a)) )
#define EXP_BASE
#define EXP_OFFSET
(1.06)
(128.0)
// Pixel Shader (6 instruction slots)
// rgb already exposure-scaled
float4 EncodeHDR_RGB_RGBE8(in float3 rgb)
{
// Compute a common exponent
float fLen = dot(rgb.rgb, 1.0) ;
float fExp = LOG(EXP_BASE, fLen) ;
// More accurate encoding
#define EXP_BASE
(1.04)
#define EXP_OFFSET (64.0)
// Pixel Shader (13 instruction slots)
float4 EncodeHDR_RGB_RGBE8(in float3 rgb)
{
float4 ret ;
// Compute a common exponent
// based on the brightest color channel
float fLen = max(rgb.r, rgb.g) ;
fLen = max(fLen, rgb.b) ;
float fExp = floor( LOG(EXP_BASE, fLen) ) ;
float4 ret ;
ret.a = clamp( (fExp + EXP_OFFSET) / 256, 0.0, 1.0 ) ;
ret.rgb = rgb / pow(EXP_BASE, ret.a * 256 - EXP_OFFSET) ;
float4 ret ;
ret.a = (fExp + EXP_OFFSET) / 256 ;
ret.rgb = rgb / fLen ;
return ret ;
}
return ret ;
}
E8R8G8B8 Decoding
// Pixel Shader (5 instruction slots)
float3 DecodeHDR_RGBE8_RGB(in float4 rgbe)
{
float fExp = rgbe.a * 256 - EXP_OFFSET ;
float fScale = pow(EXP_BASE, fExp) ;
return (rgbe.rgb * fScaler) ;
// If R16F texture format is available,
// you can use texture to convert alpha to scale factor
float3 DecodeHDR_RGBE8_RGB(in float4 rgbe)
{
// samp1D_Exp: 1D float texture of 256x1
//
pow(EXP_BASE, uCoord * 256 - EXP_OFFSET)
float fScale = tex1D(samp1D_Exp, rgbe.a).r ;
}
return (rgbe.rgb * fScale) ;
}
• Encoding/decoding should be done using
partial-precision instructions
– Rounding errors inherent in the texture format
are much bigger
Rendering with Tone Mapping
Glossy reflection material
struct PS_INPUT_GlossReflect
float4 PS_GlossReflect(PS_INPUT_GlossReflect vIn) : COLOR0
{
float4 vDecalMap = tex2D(samp2D_Decal, vIn.tcDecal) ;
float3 vLightMap = tex2D(samp2D_LightMap, vIn.tcLightMap) ;
{
float2 tcDecal
: TEXCOORD0 ;
float3 tcReflect
: TEXCOORD1 ;
float2 tcLightMap
: TEXCOORD2 ;
float2 tcFresnel
: TEXCOORD3 ;
float3 vDiffuse = vIn.cPrimaryDiffuse * vLightMap + vIn.cOtherDiffuse ;
vDiffuse *= vDecalMap ;
// HDR-decoding of environment map
float3 vSpecular = DecodeHDR_RGBE8_RGB(
texCUBE(sampCUBE_EnvMap, vIn.tcReflect) ) ;
float3 vRoughSpecular = texCUBE(sampCUBE_DullEnvMap, vIn.tcReflect) ;
// Exposure-scaled lighting results
float fReflectance = tex2D( samp2D_Fresnel, vIn.tcFresnel ).a ;
fReflectance *= vDecalMap.a ;
// Use TEXCOORD to avoid clamping
float3 cPrimaryDiffuse : TEXCOORD6 ;
float3 cOtherDiffuse
vSpecular = lerp(vSpecular, vRoughSpecular, fShininess) ;
float3 vLum = lerp(vDiffuse, vSpecular, fReflectance) ;
: TEXCOORD7 ;
} ;
// HDR tone-mapping encoding
float4 vOut ;
vOut.rgb = vLum / (vLum + 1.0) ;
vOut.a = 0.0 ;
return vOut ;
}
Generating Displayable Image
• Extract high-luminance regions
 Buffer.rgb  Threshold 
Glaresrc  max
, 0
1  Threshold


Threshold : ~0.5-0.8
• Divide by (1 - Threshold) to normalize
• Generate glare
– Use an integer buffer to apply texture filtering
• Hopefully, a float buffer with filtering…
• Generate a displayable image
– Add the glare to the frame buffer
Notes on DX9 Implementation
• High-precision buffers
– Consumes a lot of memory
– No blending capability
• Low-precision buffers
– Pixel shaders are expensive
• Consider fake techniques like DX8
– High performance
– Low memory consumption
– Very effective
Multiple Gaussian Filters
• Bloom generation
• A single Gaussian filter does not give
very good results
– Small effective radius
– Not sharp enough around the light position
• Composite multiple Gaussian filters
– Use Gaussian filters of different radii
– Larger but sharper glare becomes possible
Multiple Gaussian Filters
e
r 2
e  ( 0.25 r )  2e  ( 0.5r ) 
2
4e r  8e ( 2 r )  16e ( 4 r )
2
r x y
2
2
distancefrom theorigin
2
2
2
Multiple Gaussian Filters
Original image
e
r
2
e
 ( 0.25 r ) 2
4e
r 2
 2e
 8e
 ( 0.5 r ) 2
( 2 r ) 2

 16e
( 4 r ) 2
Multiple Gaussian Filters
• A filter of large radius is very expensive
• Make use of downscaled buffers
– A large radius means a strong low-pass filter
• Apply a blur filter to a low-res version of the image and
magnify it by bilinear filtering  The error is unnoticeable
• Change the image resolution rather than the filter radius
–
–
–
–
–
1/4 x 1/4
1/8 x 1/8
1/16 x 1/16
1/32 x 1/32
…
(1/16 the cost)
(1/64 the cost)
(1/256 the cost)
(1/1024 the cost)
• Even a large filter of several hundred pixels square can
be applied very quickly
Applying Gaussian Filters to
Downscaled Buffers
1/4 x 1/4 (256x192 pixels)
1/8 x 1/8 (128x96 pixels)
1/32 x 1/32 (32x48 pixels)
1/64 x 1/64 (16x12 pixels)
1/16 x 1/16 (64x48 pixels)
Bilinear Filtering and Composition
• Magnify them using bilinear filtering and
composite the results
– The error is almost unrecognizable
+
+
+
=
Good Enough!
Notes on Filter
• Use high-precision formats for low-res
buffers
• Don’t take too many samples
– Very expensive, especially for high-res
images
HDR in Games
• Should be appealing rather than accurate
– Accuracy is not important for players
– Even an inaccurate scene can be appealing
• Cost and performance
– The key is to understand the effects of HDR
– Devise a fake technique that is fast enough and
produces believable results
• Accurate HDR rendering is still hard in games
– Use HDR only for effects that give a large impact
• Use sprites if you want to generate glare only for the sun,
which is much faster and gives high quality results
HDR in Games
• Integer formats have very limited
dynamic range
– Render in relative luminance space
– Apply exposure scaling during rendering
– High precision is not needed for the dark
regions after exposure scaling
• As those regions remain dark in the final image
– Carefully choose the range to maximize
effective precision
HDR in Games
• Future perspective
– All operations can be done in float
– Effective use of HDR
• Depth-of-field with the shape of aperture stop
• Motion blur with high luminances not clamped
References
• Reinhard, Erik, Mike Stark, Peter Shirley, and Jim Ferwerda,
“Photographic Tone Reproduction for Digital Images”
• Mitchell, Jason L., “Real-Time 3D Scene Post-Processing”
• DirectX 9.0 SDK Summer 2003 Update, “HDRLighting Sample”
• Debevec, Paul E., “Paul Debevec Home Page”
• Kawase, Masaki, “Frame Buffer Postprocessing Effects in
DOUBLE-S.T.E.A.L (Wreckless)”