Fisher information

The Fisher information roughly describes how much information a random variable gives about an unknown parameter of its distribution. It is defined as:

$$I(\theta) = {\rm Var}[\ell'(\theta)] = -\mathbb{E}[\ell''(\theta)]$$

From the Fisher information, we can derive the asymptotic variance of the parameter.

$$\sqrt{n}(\hat{\theta}_n^{MLE} - \theta^*) \xrightarrow[n \to \infty]{(d)} \mathcal{N}(0, \frac{1}{I(\theta^*)})$$

where $\theta^*$ is the true parameter, and $\hat{\theta}_n^{MLE}$ is the MLE of the parameter.