Why simple quadrature is just as good as Monte Carlo

A Appendix: Direct comparisons to other MC based integration methods?

We attempt to make some direct comparisons between simple quadrature and Markov chain Monte Carlo (MCMC), importance sampling, stratified sampling, and Latin hypercube sampling using this analysis. We use comparisons of these methods to MC from literature and then compare them to simple quadrature using (3.11).

The variance of a MCMC estimate is

where τ is the integrated autocorrelation time. When the samples are all drawn i.i.d., one finds τ=1, which is MC. Due to the Markov chain nature of MCMC, the samples tend to have positive autocorrelation that exponentially decay, Cov⁡(fi,fj)∼exp⁡(-|j-i|/τ), which means that τ tends to be greater than one, i.e., it tends that VarMC≤VarMCMC. Using (3.11), this implies that the expected variance of rectangular and midpoint quadrature is also less than the variance of MCMC.

The other sampling methods are difficult to compare directly with simple quadrature for a number of reasons. The variance of importance sampling in the general case is not directly compared to MC in the literature because the “importance sampling distribution” h⁢(x→) is chosen somewhat arbitrarily in practice. It is known that the optimal choice of h⁢(x→) is h*⁢(x→)=|f⁢(x→)|/〈|f⁢(x→)|〉; however, this is self-defeating as it requires effectively knowing the value of the integral in the first place, 〈|f⁢(x→)|〉. This causes practitioners to rely on heuristics or guess and check strategies for choosing h⁢(x→). Thus, by not having a rigorous general comparison to MC, it also lacks a rigorous comparison to simple quadrature. Latin hypercube sampling (LHS) and stratified sampling (SS) have been compared with MC, but because VarLHS≤VarMC (for monotonic functions at least) and VarSS≤VarMC, while E⁡[Var𝒬]≤VarMC, one cannot determine from these inequalities alone whether or not simple quadrature is less than the other methods or vice-versa – for general f⁢(x→). Making general direct comparisons with these methods is outside the scope of the present article.

B Appendix: Simulation results

For 𝒬R and 𝒬M, we simulate the standard deviation of the error under their respective piecewise spline interpolant assumptions. Using N=216=65,536 samples, we simulated the approximate error of the integral using

where {fd(n𝒬)⁢(c𝒬,i)} are randomly generated from a uniform distribution having σfd(n𝒬) (these distributions should be assigned based on the prior knowledge, and the choice of its functional form does not affect our result much – here uniform was chosen a priori). The error is randomly simulated 100 times, {ϵ^𝒬(1),…,ϵ^𝒬(100)}, and the unbiased standard deviation of the error is estimated using

as the expected value of the error is zero. If the expected error is not zero because 〈∑i=1N∑d=1Dfd(n𝒬)⁢(c𝒬,i)〉≠0, then one can use the knowledge of this bias to correct estimate (4.2).

The exponent χ in N-χ from the standard deviation of the error (4.3) is what we seek to validate in the simulation. We estimate χ from the simulation results by using

for 𝒬R and 𝒬M, which in theory is χ=n𝒬D+12. The ± values on the simulated χ^ values in Table 1 and Table 2 were generated by repeating the whole experiment 10 times and reporting their average χ^±σ^χ^ of the estimates. Once one realizes that the function derivatives can be treated as random numbers, the simulation effectively becomes the exercise of demonstrating the well-known result that the standard deviation of the average of a set of random numbers decreases ∝N-12.