Stupid Spherical Harmonics (SH) Tricks Peter-Pike Sloan Microsoft Corporation What’s not in this talk? • Whitepaper has details: – www.ppsloan.org/publications/StupidSH35.pdf – Will update a bit after.
Download ReportTranscript Stupid Spherical Harmonics (SH) Tricks Peter-Pike Sloan Microsoft Corporation What’s not in this talk? • Whitepaper has details: – www.ppsloan.org/publications/StupidSH35.pdf – Will update a bit after.
Stupid Spherical Harmonics (SH) Tricks Peter-Pike Sloan Microsoft Corporation What’s not in this talk? • Whitepaper has details: – www.ppsloan.org/publications/StupidSH35.pdf – Will update a bit after GDC as well • Great visuals – Lots of pictures of spheres + spherical functions… – other talks… SH at GDC 2008 Thursday 4-5pm, 2004 West Hall Lighting and Material of HALO 3 Hao Chen (Bungie) Friday 2:30-2:50pm, 3004 West Hall Lightmap Compression in HALO 3 Yaohua Hu (Bungie) Monday Room 2018 Practical Methods for PRT using SH Manchor Ko (Naughty Dog) Talk Overview • • • • Overview/Background SH Representing/Extracting lighting models “Ringing” and what to do about it SH Products, what they are and how they can be used Harmonic Functions? • Solutions of Laplaces equation – Divergence of gradient = 0 df 2 df 2 df 2 f 2 2 2 dx dy dz – Potential theory: fluids, e&m, gravity • Fourier basis is solution on unit circle • Spherical Harmonics are solution on the sphere – Also eigenfunctions of Angular Momentum operators in Quantum Chemistry/Physics SH for Graphics? [Kajiya84] [Cabral87] [Westin92] [Dobashi95] [Sillion91] [Stam95] SH for Interactive Graphics [Ramamoorthi01] Irradiance Environment Maps Reflected light for diffuse objects SH for Interactive Graphics [Ramamoorthi01] Irradiance Environment Maps Reflected light for diffuse objects [Sloan02] Precompued Radiance Transfer (PRT) Models shadowing, inter-reflections limited glossy surfaces SH for Interactive Graphics [Ramamoorthi01] [Sloan02] SH for Interactive Graphics [Kautz02] [Lehtinen03] General BRDF’s [Ramamoorthi02] Unshadowed [Ramamoorthi01] [Sloan02] [Sloan03] SH for Interactive Graphics [Kautz02] [Lehtinen03] [Sloan03] General BRDF’s [Ramamoorthi02] Unshadowed [Ramamoorthi01] [Sloan02] [Sloan03] Texture Details [Sloan05] [Sloan06] SH for Interactive Graphics [Kautz02] [Lehtinen03] [Sloan03] General BRDF’s [Ramamoorthi02] Unshadowed [Ramamoorthi01] [Sloan02] [Sloan03] Texture Details [Annen04] [Oat05] [Sun05] [Habel08] [Wang07] [Green07] [Han07] [Sloan05] [Sloan06] SH for Interactive Graphics [Kautz02] [Lehtinen03] [Sloan03] General BRDF’s [Ramamoorthi02] Unshadowed [Ramamoorthi01] [Sloan02] [Sloan03] [Sloan05] [Sloan06] Texture Details [Annen04] [Oat05] [Sun05] [Habel08] [Zhou05] [Wang07] [Green07] [Han07] [Ren06] Dynamic Scenes [Sloan07] [Pan07] What aren’t SH good for? • High frequency signals • Global support • Some other basis functions can be better Conventions • Functions f s • Coefficients fi • Bold for column vector ys , y • BOLD for matrix M Spherical Harmonics Spherical Harmonics y00 s L0 ylm s L is band, m is function in band (-L..L) L 1 y11 s y10 s y11 s 2 y L 2 2 s y21 s y20 s y12 s y22 s Spherical Harmonics y s 0 0 L0 Band L has (2*L+1) functions Order N: bands through N-1, N2 functions L 1 y11 s y10 s y11 s 2 y L 2 2 s y21 s y20 s y12 s y22 s Spherical Harmonics 1 2 3 sin sin 2 15 2 sin cos sin 2 15 sin sin cos 2 3 2 3 cos 1 5 4 3 cos sin 2 cos 2 15 2 cos sin cos 15 4 sin cos sin 2 2 2 Spherical Harmonics 1 2 3 y 2 15 2 xy 15 yz 2 3 2 5 4 3z 3 x 2 z 2 1 15 xz 2 15 4 x 2 y2 Zonal Harmonics 1 2 3 y 2 15 2 xy 15 yz 2 3 2 5 4 3z 3 x 2 z 2 1 15 xz 2 15 4 x 2 y2 Properties of SH • Single index ylm s yi s i l l 1 m • Reconstruction ~ f s f i yi s f T ys • Orthonormal basis y s y s ds – Projection is integration – Integral of product i j ij f i f s yi s ds ~ ~ f s g s ds f T g Projection … Projection We evaluate integrals of lighting*BRDF (smooth) … PROJECTON/BF DEMO SH Rotational Invariance rotate SH SH rotate Rotational Invariance SH SRBF “good” “bad” ROTATION DEMO SH Rotation Structure C L L L L L L L L L Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q 1 Y Z X Q YX Q YZ Q 3Z 2 1 Q XZ Q X 2 Y 2 O(n3) ZH Rotation Structure C L L L L L L L L L Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q 1 Y Z X Q YX Q YZ Q 3Z 2 1 Q XZ Q X 2 Y 2 O(n2) What’s that column? Rotate delta function so that z → z’ : • Evaluate delta function at z = (0,0,1) z z ( s) • Rotating scales column C by dl z’ – Equals y(z’) due to rotation invariance Clmdl ylm z z ( s) What’s that column? Rotate delta function so that z → z’ : • Evaluate delta function at z = (0,0,1) 2l 1 dl z (s) yl 0 s ds yl 0 z 4 z z ( s) • Rotating scales column C by dl z’ – Equals y(z’) due to rotation invariance Clmdl ylm z Clm ylm z dl z ( s) Efficient ZH Rotation z g(s) Efficient ZH Rotation z g(s) 34 0 0 3 2 3 4 3 gl yl 0 s g s ds Efficient ZH Rotation z g(s) 34 0 0 3 2 3 4 3 gl yl 0 s g s ds z’ g’(s) Efficient ZH Rotation z g(s) 34 0 0 3 2 3 4 3 gl yl 0 s g s ds z’ g’(s) g G* y z G* diag g 0* , g1* , g1* , g1* , Efficient ZH Rotation z g(s) 34 0 0 3 2 3 4 3 gl yl 0 s g s ds z’ g’(s) g G* y z G* diag g 0* , g1* , g1* , g1* , gl 4 g gl dl 2l 1 * l Convolution • h(z) (ZH) convolved with f(s) (SH) h * f m l 4 0 m hl f l 2l 1 • Irradiance Environment Maps – h(z) is clamped cosine – f(s) is distant lighting * Convolution • h(z) (ZH) convolved with f(s) (SH) h * f m l 4 0 m hl f l 2l 1 • Irradiance Environment Maps – h(z) is clamped cosine – f(s) is distant lighting * Convolution • h(z) (ZH) convolved with f(s) (SH) h * f m l 4 0 m hl f l 2l 1 • Irradiance Environment Maps – h(z) is clamped cosine – f(s) is distant lighting * Irradiance Environment Maps Irradiance Environment Maps Lighting Projection into SH • Compute: f s ys ds float f[],s[]; float fWtSum=0; Foreach(cube map face) Foreach(texel) float fTmp = 1 + u^2+v^2; float fWt = 4/(sqrt(fTmp)*fTmp); EvalSHBasis(texel,s); f += t(texel)*fWt*s; // vector fWtSum += fWt; f *= 4*Pi/fWtSum; // area of sphere Analytic Light Source • Directional light – Evaluate SH in direction r • “Sphere” light source – Use ZH a zl d y s ds l 0 – Integrals closed form • “Cone” light source – Explicit P r sin d Analytic Light Source • “soft” cone – “ease spline” fall off 2 x 3 3x 2 1 – See whitepaper Cone Soft Cone Analytic Light Source • “soft” cone – “ease spline” fall off 2 x 3 3x 2 1 – See whitepaper Cone Soft Cone Practical Issues • Evaluation – Generate ZH vector from light parameters – Rotate to desired direction • Normalization – Reflected radiance for a normal pointing at light should be “a” h z cz l a c T a T h z zl LIGHT DEMO Extracting Lights from SH • Why? – Old (pre-shader) hardware – Glossy materials, conventional direct lighting – Compression • Single directional light light reflected with normal N from lighting directional environment light Directional Light • SH to the rescue – Point-wise squared difference is error we want – But this is just product of 2 SH functions – Differentiate and solve for zero Directional + Ambient • Solve for direction after removing DC – DC is only basis function that can contribute to ambient • Given directional intensity, solve for ambient intensity Picking Directions • “Optimal Linear” direction – Weighted “brightest” direction • SH are smooth, distinct peaks can only be so close • Look at main lobe of delta function (clear peak), generate points on the sphere so that any peak would have at least one point Picking Directions • Climb “up hill” from each point – Maxima f = minima –f • Keep N points that have largest value and are “far enough” (based on peak spacing) as candidates when fitting Picking Directions • Climb “up hill” from each point – Maxima f = minima –f • Solve for light intensities in given directions (see whitepaper for equations.) EXTRACTION DEMO Ringing Ringing • Functions with discontinuities – Square wave – Hemisphere, visibility • Oscillations near discontinuity – No matter how many terms • Truncation also causes problem Ringing • Functions with discontinuities – Square wave – Hemisphere, visibility • Oscillations near discontinuity – No matter how many terms • Truncation also causes problem 5th degree Polynomial Ringing • Functions with discontinuities – Square wave – Hemisphere, visibility • Oscillations near discontinuity – No matter how many terms • Truncation also causes problem 9th degree Polynomial Ringing • Functions with discontinuities – Square wave – Hemisphere, visibility • Oscillations near discontinuity – No matter how many terms • Truncation also causes problem 17th degree Polynomial Ringing • Functions with discontinuities – Square wave – Hemisphere, visibility • Oscillations near discontinuity – No matter how many terms • Truncation also causes problem 201st degree Polynomial Ringing • Functions with discontinuities – Square wave – Hemisphere, visibility • Oscillations near discontinuity – No matter how many terms • Truncation also causes problem Windowing/Sigma-factors • Scale the Fourier coefficients k sinc m 1 m 1sin k m 1 k • Scale by sinc is convolution by box in the spatial domain • Raise to power – Repeated convolution with a box=bsplines Windowing/Sigma-factors • Scale the Fourier coefficients k sinc m 1 m 1sin k m 1 k • Scale by sinc is convolution by box in the spatial domain • Raise to power – Repeated convolution with a box=bsplines Windowing/Sigma-factors • Scale the Fourier coefficients k sinc m 1 m 1sin k m 1 k • Scale by sinc is convolution by box in the spatial domain • Raise to power – Repeated convolution with a box=bsplines Windowing/Sigma-factors • Scale the Fourier coefficients k sinc m 1 m 1sin k m 1 k • Scale by sinc is convolution by box in the spatial domain • Raise to power – Repeated convolution with a box=bsplines Windowing/Sigma-factors • Scale the Fourier coefficients k sinc m 1 m 1sin k m 1 k • Scale by sinc is convolution by box in the spatial domain • Raise to power – Repeated convolution with a box=bsplines Windowing/Sigma-factors • Scale the Fourier coefficients k sinc m 1 m 1sin k m 1 k • Scale by sinc is convolution by box in the spatial domain • Raise to power – Repeated convolution with a box=bsplines Windowing/Sigma-factors • Scale the Fourier coefficients k sinc m 1 m 1sin k m 1 k • Scale by sinc is convolution by box in the spatial domain • Raise to power – Repeated convolution with a box=bsplines Windowing/Sigma-factors • Scale the Fourier coefficients k sinc m 1 m 1sin k m 1 k • Scale by sinc is convolution by box in the spatial domain • Raise to power – Repeated convolution with a box=bsplines Windowing/Sigma-factors • Scale the Fourier coefficients k sinc m 1 m 1sin k m 1 k • Scale by sinc is convolution by box in the spatial domain • Raise to power – Repeated convolution with a box=bsplines Windowing/Sigma-factors • Scale the Fourier coefficients k sinc m 1 m 1sin k m 1 k • Scale by sinc is convolution by box in the spatial domain • Raise to power – Repeated convolution with a box=bsplines Windowing/Sigma-factors • Scale the Fourier coefficients k sinc m 1 m 1sin k m 1 k • Other “windows” can be used • Hanning: k 0.51 cos m 1 Windowing/Sigma-factors • Scale the Fourier coefficients k sinc m 1 m 1sin k m 1 k • Other “windows” can be used • Hanning: k 0.51 cos m 1 SH Lancosz Equation 𝜋𝑥 sin 𝑤 𝜋𝑥 𝑤 W=6 W=10 SH Hanning Equation 1 + cos 2 𝜋𝑥 𝑤 W=6 W=10 SH Delta Function Hanning Lancosz Ringing SH No Windowing Hanning 6 Hanning 10 Ringing SH Order 3 Order 6 No Windowing Hanning 6 Other Error Function • Projection minimizes squared error – HVS doesn’t… • Penalize oscillations when approximating – “curvature” – squared laplacian – easy to compute using SH • Solve for weight – Reduce curvature measure by fixed amount Other Error Function Δ2 =0.1 Δ2 =0.5 Composite WINDOWING DEMO Content Sensitive Windowing • In graphics we don’t look at the whole function – Integrate against a somewhat compact smooth function (BRDF, possibly visibility) • Can we window only when we know we are looking “at the rings”? – Keep highlights/shadows sharp, blur rings – Applicable for lighting environments dominated by single light Content Sensitive Windowing 6th order reflection Windowed (Hanning 7) Content sensitive windowing SH Products SH Product Uses • Punch “hole” in skylight model – Large flying object – Ground plane – Big buildings • Accumulate visibility functions – Dynamic scenes • Scale light probe – Clouds [Zhou05] SH Product • Product of two SH functions projected back to SH • Can be expressed in terms of triple product tensor G • If f is fixed, can compute product matrix M SH Product • Product degree is sum of functions degree – Multiplying polynomials • Truncate the product – Commutes but doesn’t associate! • Accumulate at higher degree Special Cases • One function constant • One function analytic – Zero out ground plane – Symbolically compute • One function ZH – rotate into ZH frame, multiply, rotate back – 6th order around 3x faster • Mixed orders – Output quadratic Technique Mul Add 6th order 2527 1995 Product Matrix 1296 1260 Ground Plane 106 57 ZH(*) 380 249 3rd out 933 676 PRODUCT DEMO Conclusions • SH have lots of uses in interactive applications • Other investigations – Solve for windowing to eliminate/minimize negative values – Experiment with content sensitive windowing (PRT) – More aggressive optimization to pull out light sources (including area models) – Use windowing when pulling out area light sources (anneal windowing out – start with smoother function) • DX SDK: – Evaluation, Rotation, Products, Projection (cube map), Cone/Sphere lights – PRT, Irradiance Environment Map demos • Companion white paper: www.ppsloan.org/publications Acknowledgments • John Snyder, Microsoft Research • Images: Jim Kajiya, Nelson Max, Stephen Spencer, Yoshinori Dobashi, Jos Stam, Ravi Ramamoorthi, Jan Kautz, Chris Oat, Kun Zhou, Ralf Habel, Manchor Ko, Hao Chen, Xinguo Liu, Tone DeRose • Co-Authors: Jan Kautz, John Snyder, Xinguo Liu, Heung-Yeung Shum, Jesse Hall, John Hart, Ben Luna, Zhong Ren, Rui Wang, Bo Sun, Hujun Bao, Qunsheng Peng, Baining Guo, Naga Govindaraju, Derek Nowrouzezahrai • Light Probes: Paul Debevec • Discussion: Hao Chen, Arn Arndt, James Grieves, Clint Hanson, Loren McQuade, Dan Baker, Alex Evans, Tom Forsyth, Willem de Boer, Machor Ko, Naty Hoffman, Chris Oat, Jason Sandlin, Jon Steed, Shanon Drone Demos • What SH look like – Visualize basis functions – 3 ways? • Rotation (compare with valve) – Same code base as above • Various light models • Extracting lights • Windowing demos – Includes data dependent windowing • Product demo – Analytic skylight, have clouds zooming by • Main Demo – – – – PRT Visualize SH functions Windowing Light models • Skylight + products demo – Forked version of above?