Sampling distribution of estimator

The estimator for a parameter based on a sample could be different from the true value of the parameter, depending on what the sample is. We can estimate the sampling distribution for the estimator in order to predict how good the estimator is.

• Error: $\hat{\Theta} - \theta$
• Bias: $b^\theta = \mathbb{E}^\theta[\hat{\Theta} - \theta]$
• Variance: $v^\theta = \mathbb{E} \left[ (\hat{\Theta} - \mathbb{E}^\theta[\hat{\Theta}])^2 \right]$
• Standard error: $\mathrm{se}^\theta = \sqrt{v^\theta}$
• MSE: $\mathrm{MSE}^\theta = (b^\theta)^2 + v^\theta$

1. Estimate parameters of the distribution that generated the data, $\hat{\theta}$.
2. Create a simulated data set from the estimated parametric distribution. Calculate new parameters from this simulated data set. Repeat many times to generate an empirical distribution $\hat{\mathbb{Q}}$.
3. Calculate the bias and standard deviation of the empirical distribution $\hat{\mathbb{Q}}$.

General bootstrap does not involve estimating the parameters and instead takes samples of size $n$ with replacement from a sample of size $n$.

1. Calculate the estimated parameters $\hat{\theta}$ from the data.
2. Draw $n$ samples from the data (size $n$) with replacement. Use the samples to calculate estimated parameters $\hat{\theta}$. Repeat many times to create an empirical distribution $\hat{\mathbb{Q}}$.
3. Calculate the bias and standard deviation of $\hat{\mathbb{Q}}$.