Methods of estimation

Given some data, we may want to estimate the parameter(s) of the true probability distribution they come from. There are three methods: plugin, feature matching, and maximum likelihood.

For the plugin estimator, simply plug in the data, weighting each data point by its associated probability.

$$ \mu = \mathbb{E}[X] $$

$$ \hat{M} = \frac{1}{n} \sum_{i=1}^{n} X_i = \hat{\mathbb{E}}[X] $$

$$ v = \mathbb{E}\left[\left(X - \mathbb{E}[X] \right)^2 \right] $$

$$ \hat{V} = \frac{1}{n} \sum_{i=1}^{n} (X_i - \mu)^2 $$

$$ a = \mathrm{median}(\mathbb{P}) $$

$$ \hat{A} = \mathrm{median}(\hat{\mathbb{P}}) $$

A feature is a property of a distribution, such as mean, variance, or median.