FRACTIONAL TREND OF THE VARIANCE IN CAVALIERI SAMPLING

Abstract

Cavalieri sampling is often used to estimate the volume of an object with systematic sections a constant distance T apart. The variance of the corresponding estimator can be expressed as the sum of the extension term (which gives the overall trend of the variance and is used to estimate it), the 'Zitterbewegung' (which oscillates about zero) and higher order terms. The extension term is of order T2m+2 for small T, where m is the order of the first non-continuous derivative of the measurement function f, (namely of the area function if the target is the volume). A key condition is that the jumps of the mth derivative f (m) of f are finite. When this is not the case, then the variance exhibits a fractional trend, and the current theory fails. Indeed, in practice the mentioned trend is often of order T2q+2, typically with 0 <q <1. We obtain a general representation of the variance, and thereby of the extension term, by means of a new Euler-MacLaurin formula involving fractional derivatives of f. We also present a new and general estimator of the variance, see Eq. 26a, b, and apply it to real data (white matter of a human brain).