Let $\hat{\theta}_m$ be an unbiased estimate the average value $\theta$ of Bernoulli distribution $X_i$
$$\hat{\theta}_m = \frac{1}{m}\sum_{i=1}^{m}X_i$$
$$\theta = \mathbb{E}(X_i)$$
Therefore the Variance of $\hat{\theta}_m$ is as follows:
$$Var(\hat{\theta}_m) = Var(\frac{1}{m}\sum_{i=1}^{m}X_i)$$
$$ = \frac{1}{m^2}Var(\sum_{i=1}^{m}X_i)$$
$$ = \frac{1}{m^2}\sum_{i=1}^{m}Var(X_i)$$
$$ = \frac{1}{m}Var(X_i)$$
$$ = \frac{1}{m}\theta(1 - \theta)$$
Note that for this example as $m$ increases the variance of the estimator approaches zero in the limit, so long as the value for $\theta$ is defined.

Example for the Variance of an Estimator

The variance of an unbiased estimator $\hat{\theta}$ is defined as:
$$Var(\hat{\theta}) = \mathbb{E}((\hat{\theta} - \mathbb{E}(\hat{\theta}))^2)$$
where $\mathbb{E}$ is the expected value function.

The Standard Error of an unbiased estimator $\hat{\theta}$ is simply the positive square root of the Variance:
$$SE(\hat{\theta}) = \sqrt{Var(\hat{\theta})}$$

University of Toledo

• Variance: The variance of the data is the average squared distance between the mean and each data value.

• Standard deviation: Standard deviation is calculated as the square root of the variance (the average squared deviation from the mean).



variance and standard deviation

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Learn Before

Related

Learn After