On the topic of probability, just like most amateurs (myself included), we are mainly concerned with posterior distribution, given prior information. For example, given an urn with 2 blue balls, and 99 black balls, what is the probability of selecting a pink ball? (LOL). But a good deal of real life problems transcend such simple distributions. That brings up the exciting subfield of hypothesis testing. Whether it’s testing the efficacy of a Covid-19 vaccine or studying the correlation between synthetic drugs and teenage pregnancy, hypothesis testing is one the powerful and useful tools of probability.
The set up is pretty easy, given the following
prior information
Data or the present Distribution
Hypothesis to be tested
given the elementary coxrule we can express the probabilities and their equivalence as
finally
We can notice many wonderful things in this simple equation, the term is the posterior probability after considering the hypothesis in light of new information
given the prior information
. And the logical prior probability
is updated with the dimensionless term
. Tomake calculations faster andaccount very tightly packed probablities (e.g 0.9999 and 0.9999999) we can express the equation in terms of logarithms in base 10, as shown below.
We can further represent the original equation in terms of “odds” i.e
bringing back the logarithms and representing called the expectation.We can then write
and if the data is comprised of mutliple mutually exclusive data on H, we may as well write
The above equationslooks very elegant and it’s easy to fall into the trap of extending it to non-binary hypothesis i.e for
To see that it’s not possible, we consider a simple three hypothesis problem with the following properties
all the hypotheses are mutually exclusive such that
we can write the Hypothesis posterior probability as
Trying to rewrite the denominator of the second term in term of the other two hypotheses followsthe following tedious process (since it’s not a binary case).
The probability of “not” the first hypotheses implies either the second or third, so we can write further
since the hypotheses are mutually exclusive, we can go further
replacing back in the original equation we get
Now imagine for hypotheses testing with , the calcualtions become a lot more messier and complication, showing very simply that Non-trivial extension of the binary case in Hypothesis testing is not possible or permissible.