Letters to the Editor
-
stats
DavidNN,
Generally a test will distinguish between false positives (no steroids but test read positive) and false negative (steroids but test read negative). Unfortunately, you normally have a minimax problem -- it is generally impossible to reduce the probability of the former without increasing the probability of the latter. For example, you may decide that a ratio of two types of testosterones should be in the range [a,b]; if you increase the range, you reduce the probability of false positives while increasing the probability of false negatives. These are generally formally called accuracy and sensitivity.
In any case, Jon was attempting to use Bayes' Rule which incorporates your prior beliefs as well as the new information; this is believed by some statisticians to be superior to what I was taught (p values, hypothesis testing, etc). Basically, Bayes tried to quantify how one should change prior beliefs in light of new evidence; the school of statistics I learned (and I don't know of a catchy name for them -- I just called them hypothesis testing and p-values) ignores prior beliefs. For something like cheating via drugs, Bayes seems like the better choice given the difficulty of measuring true cheating because of flawed tests and new drugs.
Bayes:
posterior = (likelihood * prior) / (marginal likelihood)
or P(cheated | positive test ) = [ P(positive test | cheated ) * P(cheated) ] / [ P(positive test) ]
using the standard notation.
Now:
P(cheated | positive test) = what we want to know
P(positive test | cheated) = 0.95 (ie if cheated 95% chance of correctly detecting it)
P(cheated) = x = unknown (comes from beliefs)
marginal = P(test positive | cheated)*P(cheated) + P(test positive| didn't cheat) * P(didn't cheat) = 0.95 * x + 0.05*(1-x)
Thus our formula is (0.95 * x) / ( 0.95*x + (1-0.95)(1-x)).
So, this says that if we believe (for pairs of P(cheated) =x and P(cheated | positive test))
(0.1, 0.68)
(0.2, 0.83)
(0.5, 0.95)
(0.9, 0.99)
So if you only believe there is a 10% chance Floyd cheated, then you would believe there is only a 68% chance the test correctly pegged him as guilty, and so on.
Obviously, you could also change the sensitivity of the test, and 0.95 might be optimistic. Also, I don't know if these exact details are generally disclosed, though a thorough explanation of the statistics behind testing might create further public confidence in these tests. In any case, I would generally have a problem stripping people of titles for any probability below
HTH.
earl