Dale Lehman points us to this news article by Paul Basken on a study by Joanna Le Noury, John Nardo, David Healy, Jon Jureidin, Melissa Raven, Catalin Tufanaru, and Elia Abi-Jaoude that investigated what went wrong in the notorious study by Martin Keller et al. of the GlaxoSmithKline drug Paxil.
Lots of ethical issues here, but what’s interesting to me here is something about the data analysis in the original study. Here’s Basken:
[The biggest problem was] routine professional disagreements over how exactly to classify patient behaviors.
Patients who showed some form of suicidal behavior were not included in Dr. Keller’s final count, the analysis concluded, because of failures to transcribe all adverse events from one database to another and the use of “an idiosyncratic coding system.”
Such breakdowns are widely seen in clinical trials. The effect, “wittingly or unwittingly,” is to hide the adverse effects of medications being tested, said an author of the analysis, Jon N. Jureidini, a professor of psychiatry and pediatrics at the University of Adelaide, in Australia.
It’s called the garden of forking paths. If you get to choose your data-exclusion rule, you get to win the “p less than .05 game,” you get to publish your articles in top journals, and if you’re really lucky you get $$$.
Also this, which will resonate with regular readers of our blog:
Another editorial, by Peter Doshi, an associate editor of the journal, repeated emphatic criticisms of Glaxo, Dr. Keller and his co-authors (and their universities for failing to publicly rebuke them), and the journal that published their study back in 2001, the Journal of the American Academy of Child and Adolescent Psychiatry. Mr. Doshi also described turmoil within the academy, which recently elected one of Dr. Keller’s co-authors, Karen D. Wagner, a professor of psychiatry and behavioral sciences at the University of Texas Medical Branch at Galveston, to serve as its president, beginning in 2017.
Remember, Ed Wegman received the Founders Award from the American Statistical Association.
“It is often said that science self-corrects,” Mr. Doshi wrote. “But for those who have been calling for a retraction of the Keller paper for many years, the system has failed.”
Eternal vigilance is the price of liberty.
Full disclosure: I do regular consulting for Novartis.
So horrible it’s funny; so funny it’s horrible
Basken got this amazing, amazing quote:
Dr. Keller contacted The Chronicle on Wednesday to insist that the 2001 results faithfully represented the best effort of the authors at the time, and that any misrepresentation of his article to help sell Paxil was the responsibility of Glaxo.
“Nothing was ever pinned on any of us,” despite various trials and investigations, he said. “And when I say that, I’m not telling you we’re like the great escape artists, that we’re Houdinis and we did something wrong and we got away with the crime of the century. Don’t you think if there was really something wrong, some university or agency or something would have pinned something on us?”
Wow. Call me gobsmacked. Does anyone really talk like that? He sounds like the bad guy in a Columbo episode, somewhere after he stops pretending that he doesn’t know anything about the crime, and just about the time he turns to the detective and says how, even if he had done it, there’s no possible proof.
P.S. Here’s Doshi’s editorial. Worth reading. As Doshi writes, “It’s often argued that fairness in journalism requires getting ‘both sides’ of the story, but in the story of Study 329, the “other side” does not seem interested in talking.” Reminds me of Weggy. Much worse, of course, but the same principle of stonewalling.