Distribution of sample variances

If random samples , are drawn from an distribution, then and, by the Central Limit Theorem, . Standardising, we have and .

Next, we notice that

$$x_i - \mu = x_i + \bar{x} - \bar{x} - \mu$$

so that

$$(x_i - \mu)^2 = (x_i + \bar{x} - \bar{x} - \mu)^2 = (x_i - \bar{x})^2 + (\bar{x} - \mu)^2 + 2\bar{x}(x_i - \bar{x})$$

Hence, we have

$$\sum_i{(x_i - \mu)^2} = \sum_i{(x_i - \bar{x})^2} + \sum_i{(\bar{x} - \mu)^2} + 2\bar{x}\sum_i{(x_i - \bar{x})}$$

However,

$$\sum_i{(x_i - \bar{x})} = \sum_i{x_i} - \sum_i{\bar{x}} = \sum_i{x_i} - n\bar{x} = 0$$

since . So

$$\sum_i{(x_i - \mu)^2} = \sum_i{(x_i - \bar{x})^2} + n (\bar{x} - \mu)^2$$

We now have an alternative expression for the sample variance,

$$S^2 = \frac{1}{n} \sum_i{(x_i - \bar{x})^2} = \frac{1}{n} \sum_i{(x_i - \mu)^2} - (\bar{x} - \mu)^2$$

and so

$$n \frac{S^2}{\sigma^2} = \sum_i\frac{(x_i - \bar{x})^2}{\sigma^2} - \frac{(\bar{x} - \mu)^2}{\sigma^2/n}$$

Since and , we have

$$\frac{(X_i - \mu)^2}{\sigma^2} \ensuremath{\sim \mathrm{\chi^2}(1)}$$

and

$$\frac{(\bar{X} - \mu)^2}{\sigma^2/n} \ensuremath{\sim \mathrm{\chi^2}(1)}$$

This means that

$$\sum_i\frac{(X_i - \mu)^2}{\sigma^2} \ensuremath{\sim \mathrm{\chi^2}(n)}$$

and so

$$\sum_i\frac{(X_i - \mu)^2}{\sigma^2} - \frac{(\bar{X} - \mu)^2}{\sigma^2/n} \ensuremath{\sim \mathrm{\chi^2}(n-1)}$$

Therefore,

$$n\frac{S^2}{\sigma^2} \ensuremath{\sim \mathrm{\chi^2}(n-1)}$$

If, instead, we a dealing with the unbiased estimate of the population variance,

$$\hat{\sigma}^2 = \frac{1}{n-1} \sum_i (x_i - \bar{x})^2$$

This is clearly equivalent to

$$\hat{\sigma}^2 = \frac{n}{n-1} S^2$$

It now follows that

$$n S^2 = (n - 1) \hat{\sigma}^2$$

Hence,

$$(n - 1) \frac{\hat{\sigma}^2}{\sigma^2} \ensuremath{\sim \mathrm{\chi^2}(n - 1)}$$