Fourier Analysis of Stochastic Sampling For Assessing Bias and Variance in Integration Kartic Subr, Jan Kautz University College London

great sampling papers

Spectral analysis of sampling must be IMPORTANT!

BUT WHY?

numerical integration, you must try

assessing quality: eg. rendering Shiny ball, out of focus Shiny ball in motion … pixel multi-dim integral

variance and bias High varianceHigh bias

bias and variance High varianceHigh bias predict as a function of sampling strategy and integrand

variance-bias trade-off High variance High bias analysis is non-trivial

Abstracting away the application… 0

numerical integration implies sampling 0 sampled integrand (N samples)

numerical integration implies sampling 0 sampled integrand

the sampling function integrand sampling function sampled integrand multiply

sampling func. decides integration quality integrand sampled function multiply sampling function

strategies to improve estimators 1. modify weights eg. quadrature rules

strategies to improve estimators 1. modify weights eg. importance sampling 2. modify locations eg. quadrature rules

abstract away strategy: use Fourier domain 1. modify weights2. modify locations eg. quadrature rules analyse sampling function in Fourier domain

abstract away strategy: use Fourier domain 1. modify weights a. Distribution eg. importance sampling) 2. modify locations eg. quadrature rules sampling function in the Fourier domain frequency amplitude (sampling spectrum) phase (sampling spectrum)

stochastic sampling & instances of spectra Sampler (Strategy 1) Fourier transform draw Instances of sampling functionsInstances of sampling spectra

assessing estimators using sampling spectra Sampler (Strategy 1) Sampler (Strategy 2) Instances of sampling functionsInstances of sampling spectra Which strategy is better? Metric?

accuracy (bias) and precision (variance) estimated value (bins) frequency reference Estimator 2 Estimator 1 Estimator 2 has lower bias but higher variance

overview

related work signal processing assessing sampling patterns spectral analysis of integration Monte Carlo sampling Monte Carlo rendering

stochastic jitter: undesirable but unavoidable signal processing Jitter [Balakrishnan1962] Point processes [Bartlett 1964] Impulse processes [Leneman 1966] Shot noise [Bremaud et al. 2003]

we assess based on estimator bias and variance assessing sampling patterns Point statistics [Ripley 1977] Frequency analysis [Dippe&Wold 85, Cook 86, Mitchell 91] Discrepancy [Shirley 91] Statistical hypotheses [Subr&Arvo 2007] Others [Wei&Wang 11,Oztireli&Gross 12]

recent and most relevant spectral analysis of integration numerical integration schemes [Luchini 1994; Durand 2011] errors in visibility integration [Ramamoorthi et al. 12]

recent and most relevant spectral analysis of integration numerical integration schemes [Luchini 1994; Durand 2011] errors in visibility integration [Ramamoorthi et al. 12] 1. we derive estimator bias and variance in closed form 2. we consider sampling spectrum’s phase

Intuition (now) Formalism (paper)

sampling function = sum of Dirac deltas + + +

Review: in the Fourier domain … primalFourier Dirac delta Fourier transform Frequency Real Imaginary Complex plane amplitude phase

Review: in the Fourier domain … primalFourier Dirac delta Fourier transform Frequency Real Imaginary Complex plane Real Imaginary Complex plane

amplitude spectrum is not flat = primalFourier = Fourier transform

sample contributions at a given frequency Real Imaginary Complex plane At a given frequency sampling function

the sampling spectrum at a given frequency sampling spectrum Complex plane centroid given frequency

the sampling spectrum at a given frequency sampling spectrum instances expected centroid centroid variance given frequency

expected sampling spectrum and variance expected amplitude of sampling spectrumvariance of sampling spectrum frequency DC

intuition: sampling spectrum’s phase is key without it, expected amplitude = 1! –for unweighted samples, regardless of distribution cannot expect to know integrand’s phase –amplitude + phase implies we know integrand!

Theoretical results

Result 1: estimator bias bias reference inner product frequency variable S S f f sampling spectrumintegrand’s spectrum Implications 1.S non zero only at 0 freq. (pure DC) => unbiased estimator 2. complementary to f keeps bias low 3.What about phase?

Result 1: estimator bias bias Implications 1.S = pure DC => unbiased estimator 2.S complementary to f keeps bias low 3.What about phase?

expanded expression for bias bias

expanded expression for bias reference bias phase amplitude S f f S

omitting phase for conservative bias prediction reference bias phase amplitude S f f S

new measure: ampl of expected sampling spectrum ours periodogram

Result 2: estimator variance variance frequency variable inner product S S || f || 2 sampling spectrumintegrand’s power spectrum

the equations say … Keep energy low at frequencies in sampling spectrum –Where integrand has high energy

case study: Gaussian jittered sampling

1D Gaussian jitter samples jitter using iid Gaussian distributed 1D random variables

1D Gaussian jitter in the Fourier domain real Imaginary Complex plane Fourier transformed samples at an arbitrary frequency Jitter in position manifests as phase jitter centroid

derived Gaussian jitter properties any starting configuration does not introduce bias variance-bias tradeoff

Testing integration using Gaussian jitter random points binary functionp/w constant functionp/w linear function

bias-variance trade-off using Gaussian jitter bias variance Gaussian jitter random grid Poisson disk low-discrepancy Box jitter

Gaussian jitter converges rapidly Log-number of primary estimates log-variance Gaussian jitter Random: Slope = -1 O(1/N) Poisson disk low-discrepancy Box jitter

Conclusion: Studied sampling spectrum sampling spectrum integrand spectrum integrand sampling function

Conclusion: bias sampling spectrum integrand spectrum integrand sampling function bias depends on E( ).

Conclusion: variance sampling spectrum integrand spectrum integrand sampling function bias depends on E( ). variance is V( ). 2

Acknowledgements

Take-home messages relative phase is key Ideal sampling spectrum No energy in sampling spectrum at frequencies where integrand has high energy

Questions?

Sorry, what? Handling finite domain? Integrand = integrand * box

conclusion

Fourier Analysis of Stochastic Sampling Strategies For Assessing Bias and Variance in Integration Kartic Subr, Jan Kautz University College London

Theory

a simple estimator

the estimator in the Fourier domain

sampling error accumulates as DC

summary: the quantities involved integrand

image reconstruction: related work radiance Y X pixel footprint actual radiance sample green surface at any (X,Y) location, reconstruct at circle centers

related work signal processing Jitter [Balakrishnan1962] Point processes [Bartlett 1964] Impulse processes [Leneman 1966] Shot noise [Bremaud et al. 2003]

related work signal processing Jitter [Balakrishnan1962] Point processes [Bartlett 1964] Impulse processes [Leneman 1966] Shot noise [Bremaud et al. 2003] assessing sampling patterns Point statistics [Ripley 1977] Frequency analysis [Dippe&Wold 85, Cook 86, Mitchell 91] Discrepancy [Shirley 91] Statistical hypotheses [Subr&Arvo 2007] Others [Wei&Wang 11,Oztireli&Gross 12] spectral analysis of numerical integration numerical integration schemes [Luchini 1994; Durand 2011] errors in visibility integration [Ramamoorthi et al. 12]

reconstruct image at pixel centers radiance Y X pixel footprint actual radiance using radiance samples at sparse (X,Y) locations

Image reconstruction: well studied problem radiance Y X pixel footprint actual radiance

image reconstruction problem image from [Soler et al 09] 5D samples: space + angle + time

what if radiance samples are approximate? radiance pixel variance 2D space

image reconstruction using integration estimates time directions aperture pixel area image from [Belcour et al 13]

image reconstruction using integration estimates time directions aperture pixel area image from [Belcour et al 13] we focus on integration

accuracy and precision of estimators estimated value (bins) frequency histogram of estimates correct value of integral expected value of estimator

accuracy and precision of estimators estimated value (bins) frequency bias variance

2 has lower bias but higher variance estimated value (bins) frequency reference Estimator 2 Estimator 1

we derive estimator bias and variance bias variance closed form! ? integrand sampling spectrum

we derive estimator bias and variance bias variance closed form! integrand sampling spectrum ?

Intuition: non-weighted samples

Review: in the Fourier domain … primalFourier Dirac delta p Fourier transform Frequency Real Imaginary Complex plane amplitude phase

dissecting the sampling spectrum Real Imaginary Complex plane

Review: in the Fourier domain … primalFourier constant amplitude phase depends on p Dirac delta p Fourier transform

dissecting the sampling spectrum Real Imaginary Complex plane amplitude phase

periodogram is even more conservative amplitude of expected spectrum periodogram samples (expected power spectrum)

Summary of results for low bias –amplitude of expected sampling spectrum –keep orthogonal to integrand’s Fourier spectrum for low variance –variance of sampling spectrum –keep orthogonal to integrand’s power spectrum

Acknowledgements Royal Society’s Newton International Fellowship Sylvain Paris, Cyril Soler, Fredo Durand Anonymous SIGGRAPH reviewers

quantitative experiments