Friday, March 30, 2007

Can Doctors Calculate Statistics?

Here's a simple problem. Let's say that there's a disease that strikes one person in a thousand. And let's also say that there's a test for the disease that, on average, mistakenly indicates that fifty healthy people in a thousand have this disease. Now you take this medical test, and the result is "positive".

Now for the question: What is the probability that you have this disease?

Well, we know that in our population of one thousand, this test will result in a "positive" for 51 people, of which only one will have the disease. So, the answer is that the probability is one out of 51, or just under 2%.

These are conditional probabilities (or, if you prefer, Bayesian reasoning), and if understand this concept, then you probably know more than your doctor:

Hoffrage and Gigerenzer (1998; Gigerenzer, 1996) tested 48 physicians on four standard diagnostic problems, including mammography. When information was presented in termsof probabilities, only 10% of the physicians reasoned consistently with Bayes’ rule

For instance, Eddy (1982) asked physicians to estimate the probability that a woman with a positive mammogram actually has breast cancer, given a base rate of 1% for breast cancer, a hit rate of about 80%, and a false-alarm rate of about 10%. He reported that 95 of 100 physicians estimated the probability that she actually has breast cancer to be between 70% and 80%, whereas Bayes’ rule gives a value of about 7.5%.

One obvious (to me, at least) question is this: With these sort of medical tests, it seems like the outcome is "healthy" regardless of the test result. So, what's the point of the test?

No comments: