Hypothesis testing

Hypothesis testing involves deducing the quantity of a hypothesis $H$, which takes on one of the values $H_0, H_1, \dots$ from a measurement $R=r$.

We can do this by making the decision that minimizes the probability of error *conditional* on the measurement $R = r$.

• If $P(H_1|R = r) > P(H_0|R = r)$, that is, if it is more likely that $H = H_1$ than $H = H_0$ given that $R = r$, we decide '$H_1$'.
• Otherwise, if $P(H_1|R = r) < P(H_0|R = r)$, that is, if it is more likely that $H = H_1$ than $H = H_0$ given that $R = r$, we decide '$H_0$'.

The resulting conditional probability of error is:

$$ P(\mathrm{error}|R = r) = \min\{1 - P(H_0|R = r), 1 - P(H_1|R = r)\} $$

The conditional probabilities $P(H_1|R = r)$ and $P(H_0|R = r)$ are the a posteriori probabilities, as opposed to $P(H_1)$ and $P(H_0)$, the a priori probabilities.

The a posteriori probabilities can be calculated using Bayes' rule:

$$ P(H_0|R = r) = \frac{P(H_0) f_{R|H}(r|H_0)}{f_R(r)} $$

$$ P(H_1|R = r) = \frac{P(H_1) f_{R|H}(r|H_1)}{f_R(r)} $$

where $f_{R|H}$ is the conditional PDF of the random variable $R$ given a certain $H$, and $f_R$ is the PDF of $R$.

Since we are just comparing $P(H_0|R = r)$ and $P(H_1|R = r)$, we can cancel out the $f_R(r)$ on both sides, so it is equivalent to comparing $P(H_0) f_{R|H}(r|H_0)$ and $P(H_1) f_{R|H}(r|H_1)$:

• If $P(H_0) f_{R|H}(r|H_0) > P(H_1) f_{R|H}(r|H_1)$, then announce '$H_0$'.
• If $P(H_0) f_{R|H}(r|H_0) < P(H_1) f_{R|H}(r|H_1)$, then announce '$H_1$'.

The likelihood ratio $\Lambda(r)$ is defined as:

$$ \Lambda(r) = \frac{P(H_1) f_{R|H}(r|H_1)}{f_{R|H}(r|H_0)} $$

We can compare this likelihood ratio to the threshold $\eta$, which is the ratio between the a priori probabilities:

$$ \eta = \frac{P(H_1)}{P(H_0)} $$

If $ \Lambda{r} > \eta $, then announce $`H_1'$. Otherwise, announce $`H_0'$.

Probability of miss (probability we announce $H = H_0$, when in reality $H = H_1$):

$$ P_M = P('H_0'|H_1) $$

Probability of false alarm (probability we announce $H = H_1$, when in reality $H = H_0$):

$$ P_{FA} = P('H_1'|H_0) $$

Probability of detection (probability we announce $H = H_1$ given that $H = H_1$):

$$ P_D = P('H_1'|H_1) $$

True negative rate/specificity (probability we announce $H = H_0$ given that $H = H_0$):

$$ 1 - P_{FA} = P('H_0' | H_0) $$

Positive predictive value (probability that $H = H_1$ given that we announce $H = H_1$):

$$ P(H_1| 'H_1') $$

Negative predictive value (probability that $H = H_0$ given that we announce $H = H_0$):

$$ P(H_0| 'H_0') $$