From Mahalanobis, a link to a story following up on medical research findings. From the CNN.com article:

New research highlights a frustrating fact about science: What was good for you yesterday frequently will turn out to be not so great tomorrow.

The sobering conclusion came in a review of major studies published in three influential medical journals between 1990 and 2003, including 45 highly publicized studies that initially claimed a drug or other treatment worked.

Subsequent research contradicted results of seven studies — 16 percent — and reported weaker results for seven others, an additional 16 percent.

That means nearly one-third of the original results did not hold up, according to the report in Wednesday’s Journal of the American Medical Association.

This is interesting, but I’d like to hear more. If we think of effects as being continuous, then I’d expect that “subsequent research” would find stronger results half the time, and weaker results the other half the time. I imagine their dividing line relates to statistical significance, but that criterion can be misleading when making comparisons.

I’m not saying there’s anything wrong with this JAMA article, just that I’d like to see more to understand what exactly they found. They do mention as an example the notorious post-menapausal hormone study.

P.S. For the name-fans out there, the study is by “Dr. John Ioannidis, a researcher at the University of Ioannina.” I wonder if having the name helped him get the job.