Last week we reported on the push to get the data released from that controversial PACE study on chronic fatigue syndrome.
Julie Rehmeyer points to a news article with background on the story:
Patients rapidly discovered serious scientific problems with the 2011 Lancet paper. Despite these errors, the study, known as the PACE trial, went on to inform recommendations from such influential bodies as the Centers for Disease Control and Prevention, the Mayo Clinic, and the British National Health Service. . . .
But just days before the new study was released, on Oct. 21, the San Francisco journalist David Tuller published a major investigation exposing deep methodological flaws in the entire PACE trial that put its validity in serious doubt.
And this time, the new study has been met with intense criticism from outside the world of patients and advocates. On Friday, six researchers, including prominent scientists such as virologist Vincent Racaniello of Columbia University and geneticist Ronald Davis of Stanford University, released an open letter to the Lancet demanding an independent review of the PACE trial.
“The whole study is unbelievably amateur,” says Jonathan Edwards, a biomedical researcher at University College London who signed the letter. “The trial is useless.”
Rehmeyer reports on a lag between the scientific community and medical practice:
The PACE trial has exerted a strong influence on American physicians: If you ask your doctor about CFS, odds are good you’ll hear that cognitive behavioral therapy (the flavor of psychotherapy used in the trial) and exercise are the only proven treatments for CFS.
The American scientific research community, on the other hand, has rejected the psychiatric model that PACE epitomizes and is instead looking for physiological explanations for the disease.
And then the data story:
Starting in 2011, patients analyzing the study filed Freedom of Information Act requests to learn what the trial’s results would have been under the original protocol. Those were denied along with many other requests about the trial, some on the grounds that the requests were “vexatious.” The investigators said they considered the requests to be harassment.
And a garden of forking paths:
The study participants hadn’t significantly improved on any of the team’s chosen objective measures: They weren’t able to get back to work or get off welfare, they didn’t get more fit, and their ability to walk barely improved. Though the PACE researchers had chosen these measures at the start of the experiment, once they’d analyzed their data, they dismissed them as irrelevant or not objective after all. In addition, the patients researching the study found statistical errors, actions that might have pumped up the subjective ratings, measurement problems that allowed participants to deteriorate without being detected, conflicts of interest, and more.
And here’s a quote from one of my colleagues:
“The Lancet needs to stop circling the wagons and be open,” says Bruce Levin, a biostatistician at Columbia University who signed the open letter. “One of the tenets of good science is transparency.”