Skip to content
 

PACE study and the Lancet: Journal reputation is a two-way street

Sopranos_ep211b

One thing that struck me about this PACE scandal: if this study was so bad as all that, how did it taken so seriously by policymakers and the press?

There’s been a lot of discussion about serious flaws in the published papers, and even more discussion about the unforgivable refusal of the research team to share their data. But the question I want to address here is, how did they get into the position where this research got taken seriously in the first place?

As David Tuller, a public health and journalism lecturer at UC Berkeley, put it:

The thing about PACE that has astonished me is how papers with such obvious flaws were accepted immediately by the entire UK academic/psychiatric/medical establishment, including the Lancet. It’s a case of mass confirmation bias. The emperor really has no clothes, and the patients have known it for years and have been screaming about it. But they have been dismissed as irrational and dangerous. I got interested because a friend has been sick for 20+ years and he nudged me to look into PACE.

No paper with an analysis in which you can get worse and be counted as improved should ever be published. Their outcome thresholds for being “recovered” or “within normal range” on their two primary outcomes of fatigue and physical function demonstrated worse health than the entry scores required to demonstrate disability for both measures [see here for details]. It’s absurd. And yet the study has been presented as definitive for years—in the literature, public health agencies, and in public understanding.

Again, if it’s that bad, how did it stand for so long? One possibility is that that analysis wasn’t so horrible, but given everything I’ve been hearing—and the lack of any convincing defense from the paper’s authors—I’m inclined to think that Tuller’s correspondents are probably right, that the study is fatally flawed.

So, again, how did it stay afloat for so long.

Part of it must be that it was a large, well-funded study on an important problem. I guess there’s no other comparable study to knock it off its perch.

Another factor is the reputation of the Lancet, characterized by Wikipedia as follows:

The Lancet is a weekly peer-reviewed general medical journal. It is one of the world’s oldest and best known general medical journals, and has been described as one of the most prestigious medical journals in the world. . . . In the 2014 Journal Citation Reports, The Lancet’​s impact factor was ranked second among general medical journals, (at 45.217), after The New England Journal of Medicine (55.873).

Getting published in any top journal, Lancet or Science or Nature or PPNAS or any other, is a crapshoot. Any paper is hard to get published in these highly competitive journals, but whatever does get published is pretty random. But then once the paper is there, many people will consider it as correct unless there’s a huge backlash against it.

What I’m saying is, I suspect that the Lancet’s brand name gave this paper a pass.

In this way, the Lancet (and other high-profile journals such as PPNAS) play a role in science publishing, that is similar to the Ivy League in universities: It’s hard to get in, but once you’re in, you have that Ivy League credential, and you have to really screw up to lose that badge of distinction.

Or, to bring up another analogy I’ve used in the past, the current system of science publication and publicity is like someone who has a high fence around his property but then keeps the doors of his house unlocked. Any burglar who manages to get inside the estate then has free run of the house..

In this case, the “fence” is the requirement that a research paper overcome a series of hurdles: there should be a statistically significant p-value, ideally there should be some random sampling or random assignment, there should be publication in a top journal. Once the hurdles are overcome, it takes a lot a lot a lot to shake faith in it. Sure, there are some famous examples of published papers that have become laughingstocks (ESP, himmicanes, ovulation and voting, air pollution in China, children of beautiful parents, etc.), but that’s my point: it took a lot of slamming to discredit even these hugely-flawed studies.

Let me put it another way: Yes, it’s hard to get a paper published in the Lancet, Science, Nature, Psych Science, PPNAS, etc. Really hard. But the difficulty of acceptance does not imply that the papers that do get accepted are always any good. Publication is a random process. Newsworthiness counts for a lot, and newsworthiness can at times get in the way of science. An attitude of certainty counts for a lot (remember all those p-values), and certainty can get in the way of science too.

What about the Lancet’s review process for the now-notorious PACE paper? Tuller writes:

In explaining The Lancet’s decision to publish the results, Horton told the interviewer [for Australian radio] that the paper had undergone “endless rounds of peer review.” Yet the ScienceDirect database version of the article indicated that The Lancet had “fast-tracked” it to publication. According to current Lancet policy, a standard fast-tracked article is published within four weeks of receipt of the manuscript.

Endless rounds of review in four weeks. Shades of Zeno here. Or maybe the fast-tracking procedure has changed since 2011, when the paper was published.

Anyway, even if the paper had hadn’t been fast-tracked in four weeks and even if the review really did have “endless rounds,” it’s still just a few people reviewing, and typically these reviewers have neither access to the raw data nor the time for careful reanalysis.

That’s fine—reviewers are busy, and they’re reviewing for free. I review dozens of submitted journal articles a year, I consider this part of the “service” aspect of my job. It’s not the reviewer’s job to catch every problem.

But that’s my point. So what if the paper had endless rounds of peer review?

As Dan Kahan might say, what do you call a flawed paper that was published in a journal with impact factor 50 after endless rounds of peer review? A flawed paper.

And that’s fine too, it’s ok for Lancet to publish flawed papers. If you run a journal, you’ll publish flawed papers. Mistakes are inevitable. What I can’t excuse is the journal editor’s dogged defense of a flawed paper. What’s that all about?

Two-way street

And this brings us to the topic of the current post.

Journalists and policymakers are trained to believe things that appear in top journals. On the other hand, reputations change. Psychological Science used to be considered a top journal, and maybe it will be considered a top journal again, but right now it’s notorious for junk science. The American Sociological Review is considered the top journal in that field, but my own experience is that they refused to run a correction. Why? Because they don’t run corrections. That doesn’t give me so much confidence in the papers in that journal that I haven’t happened to look at. PPNAS has the himmicanes and hurricanes and other such studies. As for the Lancet . . . there’s this study and then there was that Iraq deaths paper from a few years back. These are the two papers that first come to mind when I hear “The Lancet.” Not such good news for the journal’s reputation.

My point is not that Lancet papers are worse than those in other journals. My concern is that Lancet papers are inappropriately taken more seriously than they should. Publishing a paper in Lancet is fine. But then if the paper has problems, it has problems. At that point it shouldn’t try to hide behind the Lancet reputation, which seems to be what is happening. And, yes, if that happens enough, it should degrade the journal’s reputation. If a journal is not willing to rectify errors, that’s a problem no matter what the journal is.

Remember Newton’s third law? It works with reputations too. The Lancet editor is using his journal’s reputation to defend the controversial study. But, as the study becomes more and more disparaged, the sharing of reputation goes the other way.

I can imagine the conversations that will occur:

Scientist A: My new paper was published in the Lancet!

Scientist B: The Lancet, eh? Isn’t that the journal that published the discredited Iraq survey, the Andrew Wakefield paper, and that weird PACE study?

A: Ummm, yeah, but my article isn’t one of those Lancet papers. It’s published in the serious, non-politicized section of the magazine.

B: Oh, I get it: The Lancet is like the Wall Street Journal—trust the articles, not the opinion pages?

A: Not quite like that, but, yeah: If you read between the lines, you can figure out which Lancet papers are worth reading.

B: Ahhh, I get it.

Now we just have to explain this to journalists and policymakers and we’ll be in great shape. Maybe the Lancet could use some sort of tagging system, so that outsiders can know which of its articles can be trusted and which are just, y’know, there?

Long run, reputation should catch up to reality. But before the long run comes, there are a few people out there with chronic fatigue syndrome who don’t feel like waiting.

P.S. I don’t often edit Wikipedia, but this time I was moved to round off those ridiculously over-precise numbers in the quote near the top of this post. I also added an item on the PACE study to the “Controversy” section of the page.

164 Comments

  1. Q says:

    “…if this study was so bad as all that, how did it taken so seriously by policymakers and the press?”
    Hell, you obviously haven’t seen the average climate science papers that get published in the top journals like Nature & Science.

  2. I think another reason that PACE was taken seriously is that the results of the trial were plausible (at least to some people), given what we already know about other types of psychiatric illnesses, such as somatization disorder, depression, anxiety, conversion disorder, and other conditions that may present with strong bodily symptoms (symptoms that patients may not readily attribute to some underlying psychological state).

    • TS says:

      What exactly do we know about these conditions, other than that they exist in the minds of psychiatrists?

    • Fiona says:

      I’m not sure it’s that simple. You say ‘other types of psychiatric illness’ but why do some people even think of M.E as a Psychiatric illness? Largely because of the PACE trial and psychiatrists saying it is. The evidence doesn’t in fact support it. On the contrary, There is a wealth of evidence supporting the fact that it is a physical illness as M.S is. The NIH has recently stepped out and Supported this view. Studies which suggest it’s a physical illness have been hugely drowned out by PACE supporters.

      • Point taken. I should not have used the word “other”. I’m not advocating a particular position with regard to the nature of the illness.

      • A.B. says:

        I’m not sure there is any evidence for the existence of psychogenic illness.

        • Oh come now. Have you never heard of a panic attack? People sweat, have chills, tremble, feel numbness or tingling, tightness of breath, racing heart, blurry vision, have paranoia or fear…

          You can give someone all those kinds of symptoms by dosing them with various types of drugs too. The fact of the matter is basically that due to psychological factors, the people are dosing themselves with high levels of “drugs” (naturally produced hormones secreted under control of the brain). It produces real physical effects.

          Long term Psychological stress such as financial stress and relationship stress etc can make people produce various hormones that suppress the immune system, which then leads to them getting more and more frequent viruses and/or bacterial infections, alters inflammation response, can cause changes in the biota of the gut or other areas, and can be associated with things like irritable bowel disorder or higher risk of heart attack. The fact is, the brain is a physical control system that is connected to the rest of the body, and it’s capable of making the body do things that are harmful to the body. End of story. You can’t dismiss that.

          Yes, you can demand that people study psychogenic illness scientifically, and give explanations for the mechanism and the cause, and the potential cures, and you can easily claim that PACE and others are dismissive of real problems because they’ve come up with a bogus model for how it works which they use to blame the patient… but that doesn’t mean the brain can’t make the body sick.

          • A.B. says:

            Panic attacks are widely believed to be caused by the mind… but where is the evidence for it? Can we definitely exclude a physical cause? No. Can we prove a mental cause? No. You’re simply assuming that a panic attack is caused by the mind. That’s a personal belief, not a scientific fact. History shows that this sort of reasoning is always wrong. There are a large number of diseases and healh problems that were claimed to be caused by the mind, until science progressed to a point where their physical basis could be understood. I fail to see why claims of psychogenic causation should be a given a free pass and be accepted as truth despite there being no concrete evidence that this is the correct explanation.

            • I’m not a dualist as you apparently are. There is no “mind” there is only the body, some of the body is the brain. External stimuli (such as your bank account level, or whether your wife is about to divorce you) impinge on the sensory organs that connect to the brain, and neurons in the brain induce electrical impulses that impinge on various parts of the body including glands that secrete hormones. That’s all.

              • Curious says:

                Daniel:

                I agree with your perspective. I think the critique of perspectives similar to those of the PACE researchers is what I wrote in response to Rahul below:

                “… the presumption is that there is actually nothing wrong with the patient and that they are simply creating this fatigue in their mind as a result of a psychological problem which is then dismissed as being irrelevant. The conclusion is they are just ‘sad’ and they will get over it, or they should get over it, or they should pull themselves up by their bootstraps, etc.”

                It is the perspective of being dismissive of psychological problems as if they are a choice and a function of one’s will. Historically this perspective was grounded in morality and specifically in religious morality, though it is not limited to religion. It is a naive belief about the functioning of the brain based on moral stricture rather than anything approaching the reality and complexity of the human brain.

          • Spoonseeker says:

            All the so-called psychogenic illnesses really have in common is that they do not have a physical cause which is currently understood. To ascribe their cause to the stresses of life is only conjecture.

            What history tells us ,however, is that over the years, time after time as medical knowledge has grown, conditions such as epilepsy, multiple sclerosis, Parkinsons disease, and many others – originally thought to be psychogenic or somatized or hysterical or whatever the current buzzword was – have been shown instead to have a physical cause. So time after time the psychogenic explanation has been shown to be wrong. The growing body of biomedical research into ME – coupled with the overdue exposure of the many flaws of the PACE Trial – is now revealing what has been clear to anyone who could be bothered to read the research for a long time – that this too has a physical cause.

            The false labeling of conditions which are not understood as ‘psychogenic’ has served only to divert attention from medical ignorance. It is time we learned from history and put a stop to it before any more patients suffer.

            • I’m sure you’re correct. It sounds very plausible that fools have dismissed physical ailments as “purely imaginary” due to lack of knowledge or whatever.

              That being said, it’s also entirely plausible that real illnesses caused by psychological conditions can be dismissed as “not real” by failing to acknowledge that the brain can cause illness in the body. So for example, if a soldier has PTSD and as a result has a difficult time taking care of his basic needs, becomes anorexic, and has the start of kidney failure… you shouldn’t dismiss the idea that some kind of psychological therapy could potentially restore this person’s full health just because you “don’t believe that the mind can cause physical illness” or whatever.

              Dismissing serious ailments such as MS or Parkinsons or whatever as “merely psychological” is just that, dismissive… but so is dismissing the whole idea that brains can cause physical symptoms directly or indirectly.

              • Julie says:

                It’s not that those of us who are pointing out that, as of yet, there is no scientific evidence of somatization (or psychological causes to physical ailments) that this means that there can’t ever be found that in the future, for some.

                What we see and say is, there is assumption that this is fact, and is currently supported by science. As of yet, it really isn’t.

                There’s much less rigor asked (and in general a much lower bar expected for psychological science) of studies in support of somatization, and a very high bar and resistance against solid scientific study that shows against somatization in ME.

                It’s very crazy making. We’re called anti-science, when all were really asking for is good science.

      • Rahul says:

        @Fiona

        I’m even confused as to why we think of M.E. as an illness at all.

        i.e. What’s the unifying feature of all the M.E. cases / symptoms etc. When I read the pathophysiology it sounded more like a catch all for diverse symptoms. What links all the M.E. cases together?

        Why is M.E. a distinct “disease” rather than an assorted collection of patients exhibiting “long term debilitating fatigue of unknown origins”? Something like the “others” or “unknown origin” catch all in ICD coding.

        I admit I’m naive about this, so am curious to know more.

        • Valentijn says:

          @Rahul – It does have a rather unique symptom, in the form of post-exertional malaise (PEM). Some ME/CFS criteria omit PEM or make it optional, despite near-universal objections from patients, their advocates, and their doctors. PACE used the worst of those objectionable criteria, by describing ME/CFS as simply being debilitating chronic fatigue.

          PEM, on the other hand, results in ME/CFS patients getting very sick starting approximately 24 hours after normal or even trivial amounts of exertion. It features an exacerbation of immune, neurological, and muscular symptoms. So lymph nodes often get more painful and palpably swollen, sore throat appears or worsens, ataxia can intensify, sensitivity to light and sound intensify, orthostatic intolerance worsens, headache becomes more prominent, full body muscle soreness starts (not just the muscles used in the exertion), muscle weakness, etc.

          Researchers have started to document the occurrence of PEM, using the CardioPulmonary Exercise Test (CPET). It’s a very mainstream test which has been around 50+ years, and is the gold standard for proving disability (though not pathology), and is also used by elite athletes to assess the effectiveness of their training methods. Basically you get on a treadmill or exercise bike, attach some electrodes, wear a mask, and pedal until exhaustion as the resistance slowly increases. The measured exchange of gases is used to calculate the Resiratory Quotient (RQ), and if the RQ is high enough, it’s objective indication that maximal effort was exerted.

          Some research has shown altered gene expression 24 hours after a maximal CPET for ME patients when compared to sedentary controls. More research has shown that ME patients do much worse in a 2nd CPET 24 hours after the first one (in the acute PEM period), whereas sedentary controls do about the same or a bit better on the 2nd CPET. These results have been independently replicated by several groups, though not on a large sample yet.

          So not only does ME/CFS have a unique symptom, it is one which has been verified in objective manner. It makes the psychogenic theories and treatments for the disease especially ridiculous. It also suggests that one of those treatments, Graded Exercise Therapy, is indeed just as harmful as patients have reported it to be.

    • Alex Young aka alex3619 says:

      The problem with psychogenic disorders is they are all hypothetical, and defended using rhetorical argument. Somatization and conversion disorder are unproven constructs. Their roots are in the work of Charcot and Freud. In many instances there are a host of physical issues with these patients, just as with MUS patients, and invocation of mental causation relies on magical argument. No validated mechanisms exist.

      I do think that plausibility, that is having some congruence with existing attitudes and beliefs in the medical world, is an issue. However the errors are so blatant in the PACE trial that I would expect, in a perfect world (which it isn’t), that only a small percentage of doctors and medical bureaucrats would fall for it. I suspect part of the problem is doctors are too busy to read papers critically. I suspect another part is most doctors have limited skills for evaluating papers. Gigerenzer has written extensively on a pervasive failure of doctors to understand even simple statistics, with a fail rate of about 80% in the US. Evaluating papers require more skills than just statistics though. I also suspect many doctors are not comfortable interpreting psychiatric papers.

      However all this in meshed in problems with institutionalized central authorities advising doctors, and a failing investigative journalism sector, and doctors (similarly to journalists) are expected to do more and more in less and less time, and consuming less and less resources. There are economic and political influences here too.

      Once you invoke somatization and conversion disorder, you then invoke the question: how can anyone take either seriously? Something is terribly wrong in medical training. How has pseudoscience become entrenched in medicine?

      • John says:

        I just wanted to reply to “like” this article and this comment in particular.

        Theoretically all of the checks and balances of peer review, editors zealous for their journals’ reputations etc. should work. But what happens in practice when the authors have huge institutional authority and are in influential social and political circles, often overlapping with the journal editors? What happens when grants from contrary perspectives invariably get refused because the reviewers are sure to include at least one influential person from the “establishment view.” ( E.g. All Ron Davis’s grant applications in ME got refused because the reviewers are made up of people who ascribe to the British Psychogenic Hypothesis )

        I think it becomes a situation where is it impractical to gain funding to design and run studies that challenge these authors assumptions.

        Two quotes:

        “Truth never triumphs — its opponents just die out.”
        —Max Planck

        “In science, one is happy; in science one rises above all others.”
        —Louis Pasteur

        For science to be an open arena where ideas and premises can be challenged and refuted, as well as research and conclusions, it can’t be the situation where the people who review grant applications and who review studies all have a vested interest in a certain theory or philosophy.

        • Andrew says:

          John:

          I agree with your general point but I disagree with this statement of yours, “Theoretically all of the checks and balances of peer review, editors zealous for their journals’ reputations etc. should work.” I disagree because, even in theory, refereeing is only an imperfect screen.

          • John says:

            You’re absolutely right.

            I suppose what I wanted to get across is that it has failed extremely dramatically in the case of Myalgic Encephalomyelitis ( in a way that I think would not be observed to a similar degree in most other fields ). One would think that a few bad papers would slip through in every field, but not a torrent. I think I wanted to get across that this process, in my view, in Myalgic Encephalomyelitis has become self-sustaining, as many of the authors of studies like PACE have very high institutional authority.

          • Rahul says:

            Andrew:

            So what do you think of stuff like “Boiler Codes” etc. Those are still written by people who are fallible and yet manage to do get fairly good outcomes when practicing engineers follow their prescriptions.

            No peer review can be perfect but it sure can be a lot better than it is right now. But by sacrificing quality for quantity in peer review we aren’t helping improve the process. I’d love it had you advised people to review fewer papers but to do a better job at reviewing them. e.g. demand to see raw data & the codes etc.

            Neither does it help (I think) when you say publication is a random process. It may have some elements of randomness in it, sure, but that doesn’t mean it is entirely a random process. At least it shouldn’t be and that’s not some quixotic state but only if we try harder it can be so. Recommendations in boiler codes aren’t random advice.

            • Andrew says:

              Rahul:

              Code can and should be reviewed. Clinical studies can be defined pretty clearly and data and code should be public. Research papers in poli sci or sociology, a bit less so; they’re more open-ended. I really don’t like the current model of terse 3-page Science/Nature papers focused on grabby graphs and statistical significance. If someone sends me a paper such as the PACE trial, it’s gonna be hard for me to review in detail. If they want me or someone else to replicate the analysis, look at it from all angles, evaluate all the claims in the paper, etc., they’re gonna have to pay me or motivate me in some other way. An email saying “Please review this paper for us and the review will never be published and it will be just one element in the editor’s decision,” this won’t cut it. I used to sometimes write long referee reports. Now, if I was going to do such a thing, I’d instead post my long report as a blog entry, thus making general points to an audience of thousands rather than specific points to an audience of 2 (the author and the editor).

              The other advantage of post-publication review is that effort is focused on papers that matter (such as the PACE trial) or on papers that get attention (such as those ovulation papers). Suppose the APSR sends me a paper to review and I say: Hey, this is the one paper a year I want to review, please send me all the data and I will replicate the analysis etc etc. It’s likely I’ll have wasted my time because the paper’s a dud. I think published, signed, post-publication review makes much more sense.

              • Rahul says:

                Frankly, I don’t care much about the Poli Sci / Socio studies.

                But, yes, stuff like the PACE study matters a lot. Real decisions get taken based on what the article concludes.

                Frankly, I don’t know why we can’t pay reviewers for their time. What must have been the total money expended on the PACE trials? What’s the cost of subsequent policy that will be taken based on it? Millions probably.

                Why are we so stingy about paying reviewers for their time?!

              • Keith O'Rourke says:

                Rahul:

                > What must have been the total money expended on the PACE trials?

                Likely not enough for them to thoroughly check and document their work (at least if they wanted to present their fantastic results at conferences).

                The larger issue was discussed before on this blog, its not clear/settled on who should or would step up (provide resources) to take responsibility for quality control management of academic output: Not the journals as not in the immediate/intermediate interests (though Andrew argues it is in the longer term interests), not the universities as public relations is booming and being seen as of exponentially increasing importance, not funding agencies as they don’t want their funding decisions subject to such stringent review (who does), maybe governments (Obama admin has made some moves).

                I speculatively suggested alliances between certain larger universities to make the branding of quality of their output credible (given cross-review between them).

                Also, I suggested independent random audits of academic output which upset some readers.

                But it is not a simple matter – quality problems of published research have been known for some time (though only topically in the last 5 or so years).

              • Martha says:

                Keith’s ideas sound good:

                “I speculatively suggested alliances between certain larger universities to make the branding of quality of their output credible (given cross-review between them).

                Also, I suggested independent random audits of academic output which upset some readers.”

              • Interested Party says:

                5 million pounds (UK). That’s how much was spent on the PACE trial.

            • jrkrideau says:

              Have people seen Ben Goldacre’s site http://compare-trials.org/ ? It looks like pre-registration can supply reviewers with some information of use at least in the area of the Garden of Forking Paths or phacking.

              As a light suggestion for a different approach to peer review let me post something I posted in the comments section of another blog (Respectful Insolence) regarding peer review.

              I remember that years ago one of my very first grad courses was in how to (figuratively) rip apart papers. It was excellent training for reading the literature and, hopefully, avoiding some of the major mistakes others had committed.

              Perhaps as a replacement for the current peer review process we could hold competitive review sessions with small rewards (Smarties, Mars Bars?) with starving graduate students or hungry & underpaid post-docs? Perhaps free pizza, beer and bragging rights with the group, … ?

              This is likely to have the advantages that the reviewing process is likely to be a bit more of a novel activity, may be an excellent learning experience for the reviewers who may have to do a bit of digging and (grad students forgive me) the students are as likely or more likely to have time to do thorough reviews than overworked PIs.

      • Martha says:

        Lots of good points in Alex’s comment.

      • Take one group of people and randomly assign half of them to be taxed very heavily so that they are just barely getting by on their income and the slightest perturbation in conditions will cause near-bankruptcy….

        Evidently this will cause “mental stress”. Do you find it hard to believe that there would be physical symptoms?

        I understand your general point that we want mechanisms and science and soforth, but it just isn’t that hard for me to imagine mechanisms whereby mental conditions cause physical symptoms. There are all kinds of hormone effects and soforth that can mediate the effect.

        Yes, people invoking mental->physical causation should investigate models and mechanisms, but that doesn’t mean “mental causes physical” is necessarily “magical argument”.

        That being said, I have nothing to say one way or another about PACE specifically and the hypotheses in question here. Just pointing out that “mental causes physical” isn’t really that implausible on the whole.

        • Anoneuoid says:

          It sounds like sickness behavior: https://en.wikipedia.org/wiki/Sickness_behavior

          When you have the flu (it is thought) you feel tired, etc not directly because the virus is doing something, but as part of your body’s response to the illness. It is not implausible that the brain can get stuck in this “sickness” state.

        • Frederick Guy says:

          What Daniel Lakeland said, +1.
          The well-known efficacy of placebos being a case in point.

          • A.B. says:

            Actually placebos are mostly powerless, their power being an artifact of poor research methods that fail to control for regression to the mean and forms of bias.

            Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment.
            http://www.ncbi.nlm.nih.gov/pubmed/11372012

            • Look, I promise you I can consistently induce significant measurable physical changes in people by purely sensory (visual, audio, smell) means, just find a few 18 year old males and show them pictures of naked women, or find a population of 18 year old females and have them listen to the audiobook version of some romance novel, or get the two populations in the same room and waft the smell of bacon into the room.

              “mental” IS “physical” end of story (or at least, show me massive quantities of evidence to the contrary, Bem’s ESP paper isn’t going to do it!). The only magical thinking here is that maybe there’s something called “the mind” that is different from “the brain embedded in a soup of chemicals flowing through the bloodstream”

              • If you don’t like those examples find any old person and show them a Zombie film or one of those stupid axe-murderer type films that Hollywood constantly makes.

                Why do we make axe-murderer films? Because some people like “taking” the drugs that their body produces at the doses they are produced at when shown such films.

              • A.B. says:

                Erections are not disease. These kind of examples from which one is supposed to deduce that psychogenic disease must surely exist just demonstrate that proponents of psychogenic illness have no evidence that the mind can cause disease.

              • A.B. says:

                A horror movie will raise pulse cause physiological changes in the brain and body. So what? Where is the disease? It only becomes a disease part a certain threshold. We’re not talking about minor everyday physiological reactions, but about debilitating disease that destroys lives.

                Yes there are diseases where people experienced tachycardia and anxiety and the like… but where is the evidence that they are experiencing these as result of negative thinking? The mere possibility that it might be psychogenic isn’t evidence that it actually is.

                Yes old people might complain about chest pain when watching a horror movie because their hearts can’t handle the increased heart rate. Take away the old age and there would be no such problem. What is really the cause of chest pain in this scenario?

              • So, I’ve argued that “things that happen in the brain” can cause “things that happen in the rest of the body” and you’re in 100% agreement, but you still do not believe that “disease” can be caused by “things that happen in the brain”? (let’s for the moment assume we’re outside the realm of serious psychosis such as schizophrenia)

                So, I’m going to have to get you to define “disease” because it’s obviously a subset of “things that happen in the body” but your claim is that “these disease things” are not, or have no evidence for being caused by “things that happen in the brain”.

                Or is it simply that you request that people who believe that specific disease X is caused by specific mental patterns Y should give specific evidence for that claim? Because if that’s your point, I’m in 100% agreement. But I think how much evidence is required will vary a lot based on reasonable priors for the effect.

                For example, my prior on “panic attack” in the absence of recreational, prescription, or OTC drug use is that it’s caused by “mental processes” related to stressors in the environment such as relationships, money, career, etc, and NOT viruses, bacteria, food allergies, etc. I base this on the fact that large numbers of people who report panic attacks do so at times when there are clear presence of these stressors, and there’s an obvious mechanism by which the effect could be mediated (hormones of various types) and indeed those hormones can be observed to be elevated, and their levels change when people are told stressful things (like they are going to be arrested or go bankrupt or whatever). And yet you seem to argue that this is a remarkable leap of faith.

                My strongest point is that there is no clear distinction between “mental” and “physical” and that “mental” is just a subset of all the various physical things that happen to atoms that happen to be incorporated into big jiggling bunches called “humans”.

              • A.B. says:

                Mind is not brain. I never said anything the brain. Can you show me a brain? Yes you can. Can you show me a mind? No can’t. The mind is a metaphysical concept outside of the realm of science. That’s why claims of psychogenic disease are unscientific. There is no way to examine the mind to see if it’s actually causing the disease as claimed.

                As for your panic attack example, you list a long list of things that must be excluded before it can be considered to be psychogenic. Unless you believe that our knowledge of the human body and disease is complete, and our diagnostic methods perfect, then you cannot logically argue that we have the ability to know that none of these physical causes is present. It has been estimated that there exist 320000 vertebrate viruses.

                You might think the psychogenic explanation is plausible. But there is difference between plausibility and a proven fact.

                That panic attacks correlate with exposure to stressors says nothing about the ultimate cause. It’s actually a good reason to believe that must be other factors than these stressors, or else everyone would have panic attacks under the same conditions. Panic attacks are not random occurences but tend to afflict particular individuals who often repeatedly experience them.

                Bone fractures are associated with walking but not every fall will result in a bone fracture. But if there is osteoporosis then falling is much more likely to result in fracture.

                By the way, one known cause of panic attacks and anxiety that is frequently misdiagnosed is postural orthostatic intolerance, a disease affecting the autonomic nervous system. Gain-of-function adrenergic receptor antibodies have been proposed as cause for POTS (and I believe also found in some studies) which would nicely explain why sufferer’s nervous system overreacts to ordinary stressors.

                The thing is, these sort of discoveries are not made when people are content with evoking some metaphysical explanation.

              • A.B. says:

                I meant postural orthostatic tachycardia syndrome. I always get these mixed up.

              • “mind is not brain” is exactly what I mean by “dualism”. I reject it. “mind is brain” end of story. You will have a very long row to hoe to convince me otherwise. It appears that you yourself do not believe in a “metaphysical mind” which puts us in agreement?

              • Rahul says:

                @Daniel:

                Isn’t that like saying “All Biology is Chemistry which is essentially Physics & Physics is essentially Maths”?

                I mean, it’s true deep down but till we are at the point where we have all the machinery mapped out we do need a Biologist and a Physicist and they can’t work with the laws of Physics alone but need their own “dirty” abstractions and simplifications.

                So also, the mind is indeed the brain, but till we know every fine detail of how thoughts and consciousness work we are forced to work with some other levels of abstraction.

                We may come to the point when a patient with debilitating schizophrenia gets his DNA modified in the lab & re-injected to make him perfectly fine, but till then we are forced to operate through the clumsy, blunt, ignorant tinkering of counselors and psychiatric sessions.

              • I have no specific dog in the panic attack fight. The way to determine this is to run unethical experiments: (ie. subject some group of people to severe mental distress and document panic attack incidence while monitoring a randomized control group). I also reject the idea that there has to be a SINGLE explanation for panic attack. Since the experiments can’t be done, it’s hard to definitively determine.

                I think we agree more than disagree. You seem to agree with:

                1) Brains can cause problems for the body

                2) In many instances where a psychological connection to illness is hypothesized there are other good competing hypotheses that need testing.

                3) In many instances of disease there are mediating factors (not everyone gets panic attacks under the same circumstances etc). And so causality is generally more complicated than X causes Y.

                I agree with all of that. I think where we disagree is:

                1) you believe that when people say “mind” they refer to a “spirit” of some sort and this is magical thinking, whereas I believe that whatever the common usage is, the mind is an emergent property of the brain, and every “Mind state” comes out of some kind of “brain state” and so even if some people do some kinds of magical thinking, it can be un-magicified by remembering that minds are just basically brains talking about themselves, and brains can cause problems for bodies.

                2) You seem to reject that serious chronic disease can be caused by some kind of “bad thinking”. I consider this to be an empirical question that is plausible because I know there are ways that the brain can alter the body in the short term as demonstrated by my examples, and so the long-term should also be studied but I do agree with you it should NOT be exclusively the route of study in any given context.

                3) You seem to have a definition of disease that excludes things that I might consider disease, and so we may be talking past each other on definitions. Do you dismiss things like malnutrition caused by excessive levels of gambling or sex addiction or serious depression as “diseases”? Do you reject those states as “mental disorders?” I’m happy enough with “thinking about stuff” can “cause things to happen in the brain” which can “cause things to happen in the body” and it is POSSIBLE and sufficiently plausible that it is deserving of study that “thinking about things” can cause relatively permanent or severe somatic disorders. And this is NOT magical thinking, but also certainly NOT ever to be taken as the only hypothesis.

              • @Rahul: my main point is just that it is in principle possible to reduce “mind state” to “brain state” and so there’s nothing magical about “the mind could cause X” because it’s the same as “physical processes in the brain could cause X” and physical processes are real.

                It seems as if A.B. claims that “mind is a metaphysical concept outside of science” and I disagree strongly and have a reductionist argument why that isn’t true.

                yes, we need to still have biologists and psychologists, but we don’t need psychologists because “mind is an incorporeal spirit entity not subject to the laws of physics” but because “the physics is really F’ing complicated so we need to work at a higher level of abstraction” It doesn’t mean the physics isn’t there.

              • Rahul says:

                @Daniel

                Yes, I think we are in agreement on the fundamentals.

                But at a crass, utilitarian level isn’t a lot of this about whether we send the patients to a Psychiatrist or an Internal Medicine specialist? So that’s where the abstraction matters & that’s what we are fighting about.

              • The biggest issue I have here is that “talk therapy” and other forms of “mental health treatment” not involving drugs seems to be very effective in solving important problems in portions of the population. That includes things like anorexia and bulemia, PTSD, drug addiction, depression, etc. But it may also include other forms of illness (panic attacks? inflammatory diseases?) or be a contributing factor to improvements in other diseases (cancer? infertility, who knows?)

                It has been shown as far as I know that counseling and talk therapy induces a variety of physical changes in the brain and in circulating hormones in immune system cells and in secreted molecules etc (citation needed, but I’m told this has been demonstrated by people who have studied this issue). And that these changes are somewhat orthogonal to the ones we can do with psychiatric drugs, so that combinations of drugs and talk therapy are expected to be the best available treatment for these types of diseases.

                So, calling mind->body causality “magical thinking” is potentially very damaging, because it’s not, this kind of therapy is a form of altering the brain and thereby altering the body. And this ancient Greek view of a duality of spirit and body needs to stop so that we can get respect for a potentially important medical tool (but NOT the only tool!)

              • urbantravels says:

                Daniel Lakeland: I think that extraordinary claims require extraordinary evidence. This idea that “states of mind” can cause serious chronic disease is not a fascinating new hypothesis that deserves a fair shake. It’s as old as the hills. It’s had its chance to prove itself, and it has rather consistently failed to stand up to close scrutiny.

                Yet the a priori belief that there MUST be a causal link between state of mind and bodily health is so powerful that people let all kinds of dodgy claims pass, because they reinforce a set of socially acceptable prejudices. Nobody likes to think that debilitating disease could strike the undeserving. The afflicted MUST be made to share responsibility for their own illness. If they aren’t actively faking it because they want attention or don’t like to work, then the alternative is to patronizingly assume that they have no insight into the mentality which is the “real” cause of their distress.

                You resort finally to claiming that there is scientific evidence that talk therapy can effect all kinds of things in the body, things that are objectively measurable, but I think you will find this “evidence” is generally based on just the kind of puffery, p-hacking, etc that we see in PACE. James Coyne has been writing about this problem for years; he didn’t just suddenly discover that it’s a problem when he looked at the PACE paper. Instead, when he looked at the PACE paper he saw an unfortunately familiar pattern: poor scientific practice being used to prop up an unlikely hypothesis, and only getting away with it because the standard of “proof” in psychological research of this sort is so dismally low.

                CBT is a form of therapy designed to correct “wrong thinking” in the patient. This then is purported to have all kinds of salutary effects on health, far beyond a small and temporary bump in subjective well-being. It seems a lot of people really, really WANT this kind of effect to be real – so much so that they keep trying to prove it exists. See James Coyne’s extensive debunking of, for instance, the claim that CBT improves survival times in cancer. (Spoiler: It doesn’t.) Or claims that CBT is as effective as antipsychotic drugs in treating schizophrenia. (Guess.)

                One thing that CBT *is* very effective at is training the patient in what kinds of answers they are supposed to give on questionnaires. That is what the results of the PACE trial really showed – that the needle can move a little bit, on subjective measures, in the short term. (The long-term follow-up study showed that there was NO significant difference between the different interventions studied in the trial, even on subjective measures!)

                What PACE did not show in the slightest was the kind of result you would expect if people who were merely ill because they “believed” they were ill and were just deconditioned and fearful. If that were really the cause of the disease, you would expect that therapy and encouragement would gradually let people strengthen themselves up and return to normal. After all, people do this all the time after even the worst deconditioning caused by other illnesses, injury, etc.

                Absent an underlying, ongoing organic disease process that would actively interfere with resuming normal levels of activity, what is the extremely sophisticated “mind/body” link that would cause this syndrome? If you can’t propose a mechanism, you had better show REALLY robust evidence that this “mind/body” link actually affects bodily systems in this way. And I mean directly affects it – in an objectively provable, reproducible way, just as you would have to show that a certain drug is more effective than placebo.

                While the regular man on the street might like to believe in the “mind/body connection” because it’s such a good story (there’s a reason why “psychosomatic” illnesses are beloved in fiction – they offer a convenient, not to say cheap, way to externally dramatize a character’s internal conflict) — that in itself might be relatively harmless, like astrology or tarot. But something much bigger and more ominous happens when the “mind/body connection” is invoked to reinforce certain power relationships and economic policies. The basic concept underlying PACE – that people are sick because of their state of mind, and that state of mind is correctable – just happens to be extremely convenient when you’ve promised to provide certain social benefits to sick people, but would really rather not have to actually provide them, at least on a long-term basis. You can dress up your actions in rhetoric that sounds vaguely compassionate – “Of course we’re not saying that these people aren’t REALLY sick — they are really sick, poor dears, but they don’t understand that their sickness is being caused by something that can go away if they just believe hard enough that they can get well, and we’re going to help them understand that.”

                The “mind/body” hypothesis, because it has no root in provable scientific fact, is unfortunately very prone to abuses and exploitation of this sort. It is an extremely powerful cultural myth: fundamentally an argument for moral influence on the physical body, which is why it can’t be un-politicized. I think you could throw the whole thing out the window and never miss it, if what you’re really most interested in is reducing human suffering and effectively treating disease.

              • urbantravels: no doubt you’re right about PACE. I’m asking that we not perpetuate the myth that there exists a separate incorporeal “mind” and that therefore because the “mind” is incorporeal that it therefore can’t affect the body.

                I totally agree that “psychosomatic” illness has been abused, but I also think that there are conditions where things done to the “mind” (ie. brain) cause illness. It seems so unlikely that there isn’t a SINGLE illness in a single person whose primary cause is “this person was repeatedly abused as a child, and then got into a terrible abusive relationship as an adult, and is now facing bankruptcy and the loss of her oldest child” none of which is the same as “there’s a virus, bacterium, fungus, autoimmune, or inherent congenital issue”

                If you put rats in a cage where they have good quality food and water, but randomly are either shocked with an electric shock or have bright flashing lights going off, or they hear loud sounds, or smell predator smells are you going to bet for or against them having an increased lifespan vs control rats?

              • Martha says:

                @Daniel:

                For your arguments to be convincing, you need to be able to point to high quality studies that give convincing evidence that:

                1. ““talk therapy” and other forms of “mental health treatment” not involving drugs” are indeed “very effective in solving important problems in portions of the population.”, and that those portions are substantial proportions of the populations having the problems in question;

                and

                2. “counseling and talk therapy” indeed “induces a variety of physical changes in the brain and in circulating hormones in immune system cells and in secreted molecules etc” and that “these changes are” indeed “somewhat orthogonal to the ones we can do with psychiatric drugs, so that combinations of drugs and talk therapy are expected to be the best available treatment for these types of diseases.”

                Until such time as you do so, your arguments are just based on hearsay.

            • John Peters says:

              As a patient with ME, I agree with Daniel, that there is no distinction between physical and psychological. There is no ghost in the machine.

              My view is that this is a red herring.

              ME is acknowledged as real by all parties. The illness is experienced as physical. It is accepted that the illness starts with a virus. There are measurable physical changes in the body. If someone is going to assert that the illness is psychogenic, then the burden of proof is on them to provide evidence for that claim. And no one has been able to.

              Furthermore, every time the theory on which this model has been tested, the theory has been proved wrong: it’s not a form of depression; the belief it is caused by a virus has no effect; patients do not fear exercise; ‘physiological deconditioning’ is not a factor.

              • John Peters says:

                And does anyone really think that Robin Soderling has false beliefs and can be treated with gradual exposure to activity?
                http://www.bbc.co.uk/sport/tennis/35170939

              • Rahul says:

                This is very interesting. I had no clue that the illness starts with a virus. Has the causative virus been isolated? If not, how are we sure that a virus is responsible?

              • And I agree with you. Whatever this disease or group of diseases is, we don’t have a clear explanation, and the “mental -> physical” model is not by any means established, but neither is any other clear model (such as “it’s virus X” or “it’s an auto-immune disease triggered by exposure to chemical Y”).

                But, there are lots of other areas of illness that might well be caused or at least contributed to by mental issues, and throwing out all of “mental -> physical” because a few charlatans ran the PACE trial is also wrong.

                Finally, there are plenty of conditions where causality is more complicated than single thing X causes Y.

                One I’m familiar with is chronic rhino-sinusitis (CRS). It seems likely that physical restrictions in the sinuses combined with exposure to allergens and/or respiratory viruses leads to treatment of a small number of acute sinus infections with antibiotics. Repeated doses of antibiotics then lead to significantly altered biota of the sinuses which leads to chronic inflammation via immune system cells which both attack the existing microbes and damage the mucosa in the process, and this leads to repeated cycles of acute infection followed by further antibiotics in a sort of mathematical attractor point of the dynamics.

                Existing crappy measures of biota content (basically “culturing”) fail to detect this condition. But DNA based measurement of 8000 bacterial strains which is only a recently available measurement technique detects consistent results. In addition, alteration of the sinus biota may involve unicellular fungal growth in the absence of competing bacteria and this may mediate some of the heavy inflammation.

                So, is this “viruses cause CRS”? “antibiotics cause CRS?” “allergies cause CRS?” “having kids in daycare (exposure to repeated viruses) causes CRS?” “the immune system causes CRS?”

                stupid simplistic models are just that… stupid and simplistic.

                Some citations:

                On DNA based identification of differences in microbiome:

                http://stm.sciencemag.org/content/4/151/151ra124

                On fungal involvement:

                http://www.mayoclinic.org/departments-centers/otorhinolaryngology-ent/minnesota/research/chronic-sinusitis

              • A.B. says:

                @Rahul the role of viruses in the disease is debated. The majority of patients (but not all) report a flu-like illness that coincides the onset of diseases, or precedes it by a few days to weeks.

                A number of different pathogens have been associated to the disease in this manner. A mass outbreak in Norway was seemingly caused by contamination of drinking water with giardia lambia.

                There are people who think the infectious agent is still present hidden somewhere, there are people who think the infection causes some sort of immune dysfunction that persists long after the pathogen is gone, there are people who the infection might just trigger some form of autoimmunity.

              • Rahul says:

                @A.B.

                “flu-like illness” sounds quite vague. And then again M.E. itself is such a vague combination of symptoms that I don’t even know how accurately patient would be at self reporting the point in time we could label as “first onset of disease”.

                Isn’t the incidence of flu itself fairly common among the entire population? If you make the time window broad enough, what’s the chance that someone having, say, a car accident had a “flu-like illness” in the 12 weeks before the accident?

                Further I assume practically none of the patients would have heard of ME / CFS at the point in time where the self report the “first onset” date post hoc? There’s a fair likelihood that their reading about their disease throws up the viral link and there’s some bit of priming happening in hindsight when they do report the correlated “flu-like illness” and first onset?

              • A.B. says:

                @Rahul The term flu-like illness is a medical diagnosis meaning possible flu, or a disease with the same symptoms. Doctors don’t run tests for every flu because as you said, it’s a common event.

                Mononucleosis is one infection that is specifically associated with CFS.

                Many patients can tell you the day they got ill because the onset is so sudden and definite.

                That CFS is often triggered by or preceded by an infection is an uncontroversial statement everyone agrees on. No reason to doubt this.

              • John Peters says:

                First, it’s worth pointing out that we do not know that all patients diagnosed with ME have the same illness.

                Second, it is accepted by all sides that for a substantial proportion of patients the illness started with a virus.

                Third, there is no evidence that the virus link is post-hoc. As I mentioned, an attempt to link belief in a virus to outcome of treatment failed.

                Many report onset just as Soderling did. I was diagnosed with a virus and then ‘post-viral syndrome’.

                The psychogenic explanation is that CFS is a kind of psychosis: trauma, including from a virus, may make a person ill and then after this trauma has ended (ie there is no longer any ongoing biological cause) the patient continues to feel ill but that this illness is now simply the belief the person is ill.

                There is no single causative virus but there are certain viruses which are linked to the illness much more than others.

                My experience was very close to that of Soderling: I had a virus and I tried to run it off. It’s my belief that the failure to rest properly meant the virus overwhelmed the immune system and penetrated the nervous system. My symptoms are that I pick up infections very easily and take longer to recover, and, devastatingly, a brain fog, massively impaired cognitive functioning and a brain which doesn’t function as an organ in properly regulating my body. I cant think, process information, remember; I am hormonal; I feel the cold and am sensitive to noise.
                My symptoms are found in encephalitis and in inflammation of the brain and in post-inflammation syndrome.

              • Lidia says:

                I do agree with a lot of what has been said by everyone.

                However, it is essential that we differentiate between whether ME/CFS is a psychiatric illness or a biomedical one.

                Why does it matter?

                Eleanor Stein MD FRCP(C):

                1. If ME/CFS is a psychiatric condition then psychiatric treatments should help. i.e. by using a combination of psychotherapy and psychotropic drugs.
                2. If ME/CFS is a bio-medical condition then we should keep looking for better treatments for infection, autonomic, endocrine and other systems.
                3. In those patients where there is a combination of both, there should be an intergration of the two.

                IF THE WRONG ASSUMPTIONS ARE MADE, TREATMENTS WILL BE FLAWED AND PEOPLE WILL NOT GET BETTER.

                ME/CFS is not a psychiatric disorder.

                Evidence of difference:
                4. Looking at large groups of people (epidemiology) shows differences between ME/CFS and psychiatric disorders.
                5. Looking at individuals (clinical) shows differences.
                6. Looking at cell and body system function (pathophysiology) shows differences.

                Epidemiological Evidence:
                7. Rates of psychiatric disorder in CFS/ME are similar to rates in other chronic medical conditions.
                8. Rates of personality disorder in CFS/ME are not elevated.
                9. The genetics of depression and ME/CFS are independent.
                10. Illness severity and not psychological factors predict outcome.

                ME/CFS is a biomedical disease.

                Pathophysiological Evidence:
                The most replicated research repeatedly shows that thare are problems with
                11. Endocrine function
                12. Autonomic function
                13. Cardiac function
                14. Muscle metabolism
                15. Homeostatic function
                16. And an infectious etiology is often present
                (… and there is much more.)

                CBT and GET is to ME/CFS what CBT and an instruction to daily increase sugar intake (at the same time as not prescribing insulin), would be to someone suffering from Type 1 Diabetes.

                THAT IS WHY ME/CFS PATIENTS ARE SO UPSET WITH THE PACE TRIAL. By using the Oxford criteria (= very heterogeous patient group), not including any severely affected patients with ME/CFS, including only relatively high functioning patients with genuine ME/CFS (there were probably less than 50%) and actively removing those with obvious neurological symptoms (yes, they really did!), the PACE Trial investigators were expecting their CBT and GET interventions to show improvements, thus adding support to their psychiatric agenda. BUT EVEN AFTER ALL THIS EFFORT, the data still did not look as favourable as they expected. And what they did next is the subject of David Tuller’s investigation.

                I can only conclude that they are fully aware of what they are doing … and it’s very sinister.

            • two random studies on psychological stress and its effect on platelet aggregation (clot formation?) and on wound healing, just the first things I found while googling:

              http://www.ncbi.nlm.nih.gov/pubmed/24460512

              http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3052954/

              I’m sure there are tons of other studies with similar results. Psychological situations cause measurable problems in the rest of the body.

              Are there methodological problems with some of these studies? Yes I’m sure, medicine is full of methodology problems. But, there are many clear mechanisms by which this kind of thing can occur, and in animal models we can actually induce stress and measure outcomes, even those that require killing the mouse to measure. Can’t do that in humans.

              My wife (who works as a research biologist with mice) tells me that her recollection (without specific citation) is that all kinds of “chronic stress” experiments have been done, and typically you can tell these mice a mile away, they have hair loss, sores on their skin, poor digestion, weight loss, etc etc.

              Is “psychosomatic illness” a charlatan’s/puritan’s playground? YES I’m sure it is. But, just as we shouldn’t tell people that AIDS or MS or whatever are “psychosomatic” clearly, we shouldn’t tell people who have lots of psychological issues such as the victims of abuse or soldiers or aid workers in disaster zones that we won’t pay for any kind of “talk therapy” to try to improve their irritable bowel syndrome or severe allergies or glaucoma or whatever because logically “the mind, which is incorporeal” can’t possibly affect “the body” which is real and physical.

              Ok, that’s it. I’ve said my bit, I think overall my position is clear. I guess I’ve had enough of this particular discussion and will move on to the next topic now.

              • Interested Party says:

                Just because theoretically something could be true, doesn’t make it good policy, or even a good experiment. For that you need:

                1) A definite specific model of how this would work in the particular disease on the biochemical level. I have never seen one of these.

                There are models, sure, but they employ some sort of handwaving between the nebulous concept of “stress” (what kind of stress? Emotional distress? Nutritional deficiencies due to bad eating habits secondary to emotional distress? Wear and tear on the cells due to infection, mechanical torsion, etc.? Just one of these? All of them and more?) and the vague concept of “immune and/or neurological changes”. I haven’t seen one that can get from specific types of stress with appropriate molecules identified, to specific types of neurological or immune damage, without a handwaving gap.

                Secondary point: For the sake of argument, just because something could theoretically be caused by, for instance, the effects of emotional distress, it doesn’t logically follow that psychological therapy could necessarily reverse this effect (supposing there were a known appropriate psychological therapy). Not all chemical reactions have an even flow between alternate states, and may require a different kind of input to react in reverse.

                OR

                2) Some kind of empirical evidence that it works. I have never seen any.

                On the contrary, I have seen a great deal of evidence that it does not work (no matter which patient population we’re talking about) to reduce physiological disease symptoms.

                Yes, therapy which is designed for that purpose can improve coping (which can in some cases improve the way the patient manages the disease–provided other appropriate treatment is also available and provided), and reduce emotional distress. It should be speced for that, and provided to patients who ask for it for that purpose.

                (CBT/LP/similar which is designed to induce denial doesn’t improve coping.)

                However offering CBT in the absence of any medical treatment–particularly without acknowledging that medical treatment is lacking–is mainly going to make the patient feel patronized.

                My two cents.

            • @Martha:

              Apparently I’m not making my argument very well. So I’m going to try to do it in a very clear more formal way. Here are my assertions:

              1) There is no “incorporeal mind” there is only “brain”. I take this as given.

              2) It is plainly clear that brains can cause physical issues in the rest of the body. (I’ve pointed to physical arousal, horror films, and a couple of studies of induced stress in mice (clotting, and inflammation etc) as just examples of how the brain can do real physical things to the body, whether in short or long term, I’ve also suggested that there are a wide variety of additional mice experiments that people can look up).

              3) Therefore, we can not dismiss by logic alone, without study, or dismiss any proposal to study, how “talk therapy” or other psychological interventions might help physical symptoms.

              THAT is the entirety of my logical argument.

              Everything else is just as you say anecdotal/hearsay suggestions that there are sources of information already out there to suggest that psychological interventions can improve physical symptoms in a way that is orthogonal to psych drugs. That’s not my specialty, I’ve heard about these studies from people who are in psychiatry, I’ve heard about studies in which mouse models of physical disease treatment were called into question when it was discovered that the two groups were cared for by different vet techs, and the smell of male humans induced noticeable stress in one group whereas the smell of female humans didn’t… I’ve heard (hearsay) about a wide variety of fairly complicated psych->physical mechanisms… If anyone is interested they should go research all that. This hearsay type stuff is meant mainly to point out that there is a whole field of study IN THE LAB where people have observed psych->physical. If there are methodology problems… then fine, but the effects are widespread and someone who wants to argue that psych->physical causation is really impossible would have to debunk that work very thoroughly, not just dismiss all of it as having a low bar for proof like all of psychology… or whatever.

              I’m NOT arguing that “talk therapy etc” IS VERY EFFECTIVE, I’m arguing that there’s not logical reason to assume that it CAN’T be effective, and there is some prior evidence that mechanisms exist, so we shouldn’t shut down that whole line of inquiry just because some people have dismissed physical diseases as “psychosomatic” and been very wrong.

              I hope that’s crystal clear now.

      • Kerbox says:

        The problem is you are throwing the baby out with the bath water, just because ME/CFS has been unfairly hijacked around these issues, it does not mean somatization or conversion disorders have no relevance to other situations. It is like pretending that someone with split personalities who has suffered horrendous abuse, is just putting it on, or the diagnoses is just theory, the background of the person irrelevant and its curable by biological or neurological intervention, right?

        There is no doubt there is incredible pitfalls in assessing what is psychological in patients and what is biological, there is an incredible grey area and no easy answers, you can change the brain by a simple change in behaviour, you can change the brain by a simple change in nutrition. Welcome to the dichotomy and complex nature of life.

        The quest again the London shrinks and PACE is justifiable, but the resentment and quest against the whole profession is unbalanced; and I am afraid due to too many jaded patience. Meanwhile any sensible discussion around these issues, or someone claiming some success from psychological intervention is ran off any forum or threads locked. We want to talk about that psychology?

        • Rahul says:

          What’s your opinion about all the micro-classifications for disorders in DSM-5 etc.? Do we really have the data to support all these mini diseases based on often broad and subjective diagnostic criteria?

          That’s one point that makes me view the profession with suspicion.

        • K says:

          wow. well this thread got heated.

          I would argue that a lot of the disagreement on this topic really comes down to terminology.
          Namely psychological vs. psychiatric, vs. psychogenic / psychosomatic.
          as well as disease vs. illness vs. symptom

          I don’t think anyone here would argue that psychological or psychiatric diseases aren’t ‘real’ or can’t be devastating or life altering.
          But lets be clear that psychogenic / psychosomatic have a very specific meaning
          While mind obviously interacts with body, ‘psychogenic’ and psychosomatic have a very specific meaning that differentiate them from psychiatric or psychological. Ie- that unconscious emotions / trauma / mental distress are somehow translated into persistent physical symptoms. A theory that has yet to pan out in any chronic, systemic condition  in all of medical history. There are certainly mechanisms by which mental states effect the body. Eg anxiety leading to hyperventilation. But the idea of that state being constantly sustained for several decades to the extent that someone cannot bathe or feed themselves without assistance, (and often without any history of psychiatric illness or trauma) is rather another story. And one that should require an equal level of evidence as any other causal hypothesis. 

          The salient argument here more generally is with the “God-of-the-gaps” mentality that crams a psychogenic explanation into every gap in knowledge and then stubbornly refuses to budge that explanation when contrary evidence is found.
          It’s a pattern that has been repeated for everything from MS to ulcers, and it often has very real consequences in terms of blocking progress towards meaningful research and treatment. Eg.  – https://goo.gl/at11LA
          Not to mention access to benefits, assistance, and  appropriate medical care.

          It is certainly possible that psychosomatic disease exists. but it should be possible (and necessary) to validate scientifically. and as yet this doesn’t seem to have happened. Patients, clinicians and researchers who deal with ME tend to take a pretty hard line on psychogenic theory because as you have seen in this thread it has done a great deal of harm in the form of diagnostic overshadowing, access to care, innappropriate treatments, abuse of patients etc. This hardline extends outside ME&CFS to the “free pass” these theories have been in other illnesses, because these same mistakes have been repeated endlessly in numerous diseases. At some point we have to stop this cycle and ask for proof outside of vague handwaving, word-games and Charcot/Freud based theory.

          For some good rebuttals of psychosomatic theories:
          https://www.researchgate.net/publication/283476227_Psychogenic_explanations_of_physical_illness_Time_to_examine_the_evidence
          and
          https://aeon.co/opinions/bad-thoughts-can-t-make-you-sick-that-s-just-magical-thinking

          • Yay, progress in this discussion.

            I agree with you, and elsewhere I mention two “types” of situations

            1) where doctors can’t figure out what is wrong and “blame psych”

            2) where doctors/psychiatrists clearly see that there’s a psych problem, and then wonder if physical illness that manifests after a duration of psych problems might be amenable to psych treatment.

            It’s my impression that (1) is a charlatan’s game where doctors try to maintain a power position by asserting that they do in fact know what’s wrong when in fact they don’t. Patients are rightly angry.

            (2) on the other hand goes the other way, insurance companies and govt agencies argue: “we’re not going to pay for your psych therapy because there’s no way it could possibly improve your recurrent skin rashes or elevated allergies or bleeding stools or receding gums or nighttime incontinence or increased rate of falling or periodic racing heart beat or hypertension….”

            what’s terrible is when the argument goes: “look at all this history of (1) it proves that there’s no such thing as (2)” and so the rightful indignation at (1) can harm patients in situation (2)

            Valid arguments against (1) are “you have no evidence whatsoever that the mind is involved here it’s just your stupid default explanation perpetuating a power relationship in which you pretend to know what you’re talking about”

            But not “the mind is an un-physical thing that is therefore unscientific and can not ever cause disease therefore you are all charlatans in case (1), and by the way this argument also applies equally to (2)”

            • K says:

              Daniel I certainly agree wholeheartedly that psychological services should be available to anyone who wants them and might benefit from them.
              But…I still think you are still making a bit of a category error here.

              The conditions you mention are not considered psychosomatic. They are posited by some as having a psychological component . Asserting that patients may be able to relieve some of their subjective suffering by alleviating stress or other psychological factors is very different from asserting that their disease is solely caused and maintained by their own faulty cognition and behaviour.
              Psychosomatic theories relieve the medical field from responsibility in finding underlying pathology and developing treatments, put the blame for failure of any treatment squarely on the patient, and often result in missing any further pathology due to the fact that once a “somatization” label is applied it can be used to explain any future symptoms or signs.
              Where psychosomatic thinking gets dangerously unscientific is that this notion is unfalsifiable.
              For an illustrative example of this mindset, one of the PACE investigators (Sharpe) comments here on a woman who died of Prion disease (confirmed by autopsy), previously misdiagnosed as somatization disorder http://ajp.psychiatryonline.org/doi/full/10.1176/appi.ajp.160.2.391
              Sharpe defends the diagnosis, explaining that even though her symptoms are explained by Prion disease she could still ALSO have a conversion disorder.
              Its this sort of irrational post-hoc logic that makes people question psychosomatic theories as a whole. Once a theory is unfalsifiable it is no longer science it is superstition.

              While as I said above it is POSSIBLE that a wholly psychosomatic disease exists, I have personally seen no convincing evidence of it. Which does not in any way make psychological or psychiatric conditions less valid.

              In terms of allergies I have never heard of a psychological explanation. Are we talking about IGE allergies and/or anaphylaxis that both show up on blood / scratch tests? or unexplained reactions? Could you provide a reference? (I’m not trying to be adversarial, I am genuinely curious)

              • That’s why I split things into category (1) and (2). The point is just that arguing against (1) along the lines of “the mind is incorporeal and can not ever cause these diseases” (which is along the lines of arguments postulated elsewhere on this page) is also an argument against things in category (2). But, the argument is fallacious, and therefore does not provide a real argument against (1) and yet there are good arguments against (1) which are not fallacious and do not apparently harm people in category (2).

                I totally agree that category (1) which is “classic” psychosomatic type baloney is a problem and is unscientific. I just think we need to argue that without completely discounting the idea that “mind which is really brain can cause directly or indirectly problems with the body”

                As for example conditions, they are mostly just made-up examples taken from categories that I’ve seen discussed in this context. In particular there are real scientific hypotheses with mechanisms associated to:

                immune system and inflammation related diseases
                heart-disease (possibly through inflammation or through adrenal hormones etc)
                digestive diseases (possibly through immune mediation)
                motor control (falling, weakness, etc)

                For example here’s a paper where people are interested in investigating igE related problems as having psychological stress as contributing factors:

                http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3264048/

                they give an overview of the general concept. I don’t want to claim that there is any STRONG evidence or anything like that, only that we shouldn’t discount this area of research by arguing against category (1) in ways that are erroneous and also provide arguments against category (2)

                I’ve even seen arguments and evidence as to how gut flora produce neurotransmitter molecules which alter mood and psychological conditions, so that it’s entirely plausible that for example psychological stress induces immune response that induces inflammation that alters gut function which then alters behavior (eating for example) and that alters the gut-bacterial production of neuro-active molecules which further alters psychology and can lead to a “latch” type feedback loop.

                here’s a starting point example for gut biota -> psych feedback: http://www.nytimes.com/2015/06/28/magazine/can-the-bacteria-in-your-gut-explain-your-mood.html?mwrsm=Email

                The world is really really interesting and complicated!

              • K says:

                oh dear me. that allergy paper you linked is really illustrating my point though.
                (I am guessing you just quickly looked for something related to allergies and somatization, but this goes to show the quality of the average psychosomatic paper.)
                It is an opinion piece. and it references as “fact” discredited psychosomatic nonsense like asthma being caused by “histrionic mothers”.
                Then there is a whole chain of “evidence” which confuses correlation with causation and presumes facts not in evidence.
                I confess I didn’t make it all the way through this before I had had enough, but this sentence alone should be enough to give one pause: “Various sources have estimated that up to 75% of all visits to physician’s offices are stress-related.” really? which sources? (none are cited) what is the evidence?

                The problem in a nutshell is that this field has not progressed from its roots in stage-magic and theories about wandering uteruses .
                There is currently as much internal logic and evidentiary basis in this field as there is in homeopathy.
                Yet clinicians and policy-makers are frequently treating psychosomatic disease as a proven fact.

                Perhaps I can turn this question around . What evidence convinces you that psychosomatic disease exists?
                Not psychiatric disease or conditions exacerbated by stress, but physical, chronic diseases that are both caused and perpetuated by repressed trauma and/or other psychological distress?

              • “What evidence convinces you that psychosomatic disease exists?
                Not psychiatric disease or conditions exacerbated by stress, but physical, chronic diseases that are both caused and perpetuated by repressed trauma and/or other psychological distress?”

                What part of a ton of posts did I say that “psychosomatic disease exists”??

                All I’m saying is: there *are* mechanisms by which brains can cause bodily problems, ie. physical conditions in which psychiatric conditions play a contributory role…. that’s it. Previously on other portions of this page people have argued that there is no scientific basis for such.

                and yes, you’re right, I did just google up something on allergy, and you’re right that opinion piece could well be terrible.

              • also, the bits about histrionic mothers etc I took to be essentially “history of early thought on connections between allergy and psychology” not “it’s a fact that histrionic mothers cause allergy”

                You have to get past the intro and background stuff before you get to a variety of citations on studies where they show that stress modulates allergen response in experiments, where psychiatric stress is related to incidence of hospitalizations for various things, where psychiatric stress is related to eosinophil reactions etc etc etc.

              • Basically, the best thing you could do with my allergy paper is to go to the citations and look for articles that aren’t weird or from the 1800’s like:

                13. Golden SH. A review of the evidence for a neuroendocrine link between stress, depression and diabetes mellitus. Curr Diabetes Rev. 2007 Nov;3(4):252–259. [PubMed]

                14. Blanchard EB, Lackner JM, Jaccard J, et al. The role of stress in symptom exacerbation among IBS patients. J Psychosom Res. 2008 Feb;64(2):119–128. [PubMed]

                17. Wainwright NW, Surtees PG, Wareham NJ, Harrison BD. Psychosocial factors and incident asthma hospital admissions in the EPIC-Norfolk cohort study. Allergy. 2007 May;62(5):554–560. [PubMed]

                21. Liu LY, Coe CL, Swenson CA, Kelly EA, Kita H, Busse WW. School examinations enhance airway inflammation to antigen challenge. Am J Respir Crit Care Med. 2002 Apr 15;165(8):1062–1067. [PubMed]

                etc etc.

                For whatever reason it seems consistently on this page, people read-into my argument stuff that they expect me to believe because they’re so used to their particular dialog on “psychosomatic” illness with obnoxious defensive doctors or researchers or whatever.

                Brains can cause or contribute to bodily symptoms. That’s the essence of my argument.

              • The cited article on exams and asthma for example, while I haven’t read the full article, the abstract sounds like a perfectly reasonable experimental study with maybe naive frequentist statistics, but still highly suggestive of a real effect of psychological stress on asthma, on eosinophils on interleukine, etc etc

                http://www.ncbi.nlm.nih.gov/pubmed/11956045

              • Rahul says:

                A lot of this long argument seems like asking how many angels can dance on the head of a pin…..

                Is there any utilitarian implication?

              • K says:

                That “School examinations enhance airway inflammation to antigen challenge” example is exactly what I mean by mistaking correlation for causation. any number of things that weren’t controlled for could cause eosinophil levels to rise at a particular time of year. This study didn’t even use any control subjects. By the same logic I could argue that ice-cream consumption causes shark attacks (they both increase in the summer and this is a famous example of the correlation/causation mistake where on a graph they can be made exactly correlated)

                I really don’t think it is worth picking apart these papers / references one by one.
                And I get from your last posts that you are not really going to bat for psychosomatic theories.

                I think it is just important to clinicians, researchers and patients to point out that the level of evidence and the internal logic currently used in this field is unacceptable.
                I think a large part of the problem is people assuming there must be some convincing evidence somewhere down the logical chain, (otherwise how would anyone take this stuff seriously) but if you follow the logic/citations/assumptions backwards there isn’t any real evidence supporting it.

                I get that you want to point out that psychosomatic disease is theoretically possible, and I agree. but only in so far as the existence of big-foot is also possible, I can’t disprove it, and I can infer from the existence of other 2-legged upright mammals that it is possible, but experience and a lack of convincing evidence tells me that it is highly improbable.

              • @Rahul: as a cost savings measure many of the psych clinics are being forced into acting essentially like dispensaries. Especially because reimbursement for pills is relatively easy (thanks to massive lobbying from pharma??), but getting people to pay for person-hours for counseling is not easy. So skilled counselors become pill pushers with 15 or 30 minute appointment durations.

                @K yes, I’m not going to bat for “psychosomatic” as you defined it (ie. essentially the assumption that a given disease must have a purely psych origin because no-one has yet found a good physical explanation). But I go farther than you in believing that psych as contributor to a wide variety of physical issues has good scientific evidence.

                It’s not very ethical to induce chronic psych issues in a controlled experiment to see how it affects humans physically, so there is no way to do carefully controlled experiments rather than observational. But in animal models it’s been done with restraint, predator, and other types of stress induced in mice etc, and my understanding is that after such stress all sorts of bio-markers for immune and soforth issues can be seen. I believe it’s been done both for acute and chronic (long term) stress. If you’re interested take a look but for now I think I’ve had enough for the moment.

              • Kyla says:

                Yes. Agree this yes gone wildly off topic and I will bow out. But first I feel compelled to point out that there is nothing unethical about control groups in the types of studies we were discussing. The control group I mentioned as being missing in that allergy paper would simply have been a demographically matched group with the same diagnosis, who were NOT taking exams, but had the same measurements taken in the same month and at the same time of day. Basic trial design from which no purportedly medical study should be exempt.

            • Rahul: treating asthma in college students by psychological counseling is probably like the tiniest epsilon towards the point.

              In general wider availability of psych treatment and acceptance of research funds towards how psych treatment might help chronic illness especially in those who are diagnosed with chronic psych issues FIRST (as opposed to forcing psych treatment as the only treatment down the throats of people who have fully somatic disorders).

              I know people in community mental health, the funding and acceptance of the need for such clinics etc is constantly in peril.

              • Rahul says:

                @Daniel

                Interesting. I didn’t realize this was an uphill battle still. I assumed using counselling for such purposes was mainstream & accepted.

                And I agree: In a lot of scenarios counselling & pharmacological approaches give a synergy superior to sticking to just one.

    • Zoe Wilding says:

      For anybody who has ME, here’s a link – http://www.hummingbird.com. I printed off their pages and asked my gp to read them. He was enraged because I had the temerity to google it, after years. He became very aggressive, asked what they wanted to sell to me (nothing). I suggested that maybe he may like to go onto the website when he had a moment. Refused point blank. Too many people sitting at desks writing “things” to publish. I have ME. There is a massive difference between ME and Chronic Fatigue Syndrome. My GP lumps them together and have not been referred to a neurologist. ME is caused by damage to the brain. Mine is down to whiplash followed by meningitis. I have a history of depressive bouts and other conditions. I have waited for 6 months to get an appointment for the opthamologist at the hospital after visiting an optician. I have huge glaring white lights, imagine looking at a jagged hole in the ice on a pond, but instead of water the hole is filled up with glaring white light and you cannot see. Yesterday I went to A&E where I was told that I could see the emergency opthamologist tomorrow. This may be my way in to get seen by a neurologist. My gp took a batch of tests for all organs, and bone inflammation. Results were all fine except that I tested positive for Lupus. Apparently I have not got Lupus although I have many of the symptoms. People become depressed with this horrendous condition, of course they do, but it is secondary to the disease. Many people with ME have a whole spectrum of other illnesses which are all kicked off by a vastly “depressed” immune system. Cognitive Behaviour Therapy is not the answer. I was attempting to achieve the targets that were set and had a massive relapse. My heart is now damaged. This is an ME “thing”. I have always been very active and slim and used to walk for about 6 miles a day with my dogs. I have now had to employ a dog walker, I shop online and just keep my fingers crossed that the plug’s not pulled. I’m 57, I’ve had this disease for about 9 years. This hidden disease is a nightmare, nobody can see what’s going on. No funds are put into research. The NHS does not want people to be diagnosed due to the repercussions financially.

    • David says:

      Its important to emphasise the importance of scientific rigour integrity and high scientific standards in all research. This is especially required in the controversial areas of science where people get quite emotional and have vested interests in certain outcomes. Many have mentioned the PACE trial in Britain and this is a subject which will require very careful and detailed analysis and scrutiny. There are egos, commercial agendas, professional agendas, personal agendas, etc. involved in this PACE controversey. I and some others prefer to look at the scientific and medical facts, and draw rational and logical conclusions from them.

      From a scientific & medical perspective and legal perspective, the following needs to be scrutinised by many independent parties in the scientific and medical communities, and analysis communicated to all doctors and medical professionals.

      A scientific analysis of the PACE trial – http://www.me-ireland.com/bogus.htm#pace
      – The documented harms caused by exercise and exertion in cases of ME and CFS – http://www.me-ireland.com/bogus.htm#harm
      – Scientific and medical evidence – http://www.me-ireland.com/scientific.htm
      – The legal implications for doctors and medical professionals – http://www.me-ireland.com/bogus.htm

  3. Alex says:

    Andrew, I’m curious what you think is the role of the reviewer if not to catch every problem. That’s at least the goal, right? Obviously the goal is difficult, perhaps so difficult that no reviewer practically aims for it, because of lack of time, access to data, etc, but aren’t reviewers supposed to catch problems and either propose solutions or declare the paper untenable?

    • Andrew says:

      Alex:

      The reviewer should definitely try to catch problems. I just don’t think it’s reasonable to expect that reviewers will catch all problems. As we’ve seen over and over again, even the top journals publish papers with huge errors. Himmicanes, anyone?

      Beyond that, is the problem that top journals select based on importance of the findings, not on correctness.

      • Rahul says:

        Andrew:

        It can be very hard indeed to catch some deep or hard to find error.

        But by all the comments I’ve read the error in the PACE articles were obvious. At least it seems so.

        At that point it becomes hard to excuse how three reviewers (plus a few editors and house statisticians) let them pass.

        Of course, if we all start out without aiming to do a conscientious job of reviewing, and putting quantity over quality, publishing Himmicane papers is natural.

    • Martha says:

      Alex,
      I’ve heard reviewers (typically called “referees”) of journal submissions in math say that it is not their job to catch mistakes — that that is the author’s responsibility. Indeed, I have had two papers accepted for publication that had errors. In the first case, I caught the error myself before publication, withdrew that paper, and was able to correct it and resubmit the paper. In the second case, a colleague pointed the error out to me before publication — and it was a serious error that I was not able to correct. (The colleague and another person later gave a correct proof and insisted in adding my name as coauthor, on the somewhat dubious grounds that they would not have looked at the problem if I hadn’t written the paper.)

      When I have refereed papers, I certainly try to catch errors (as well as make suggestions to improve exposition if needed), but realize I can’t expect to find all errors.

  4. Adrian says:

    I wonder if there is an additional problem in that reviewers for this type of work may be drawn from a small pool of interconnected researchers. One of the problems with not knowing who the reviewers are is that we don’t know if they gave the paper an easier time as they have common beliefs or if they were from within the same small community.

  5. Jane Colby says:

    I note with sadness that the previous post refers to “other types of psychiatric illnesses” as if chronic fatigue syndrome was itself a psychiatric condition. Some patients within the “chronic fatigue” bracket may indeed have depression, but as those who have worked with experts such as the great microbiologist Elizabeth Dowsett will know, caught under the “chronic fatigue” umbrella are those who would once have been diagnosed withmyalgic encephalomyelitis, a potentially very severe and chronic neurological disease classified as such by WHO. While this is as far as you can get from a condition of the mind, it is of course a physical disease centred in the brain, and much evidence has accrued to identify it as a persistent viral infection. It was an unscientific disaster when this mixing of of patient groups was allowed to occur and until the damage to science us unravelled we will continue to see seriously sick children damaged with exercise treatment and their educational rights ignored. As executive director of Tymes Trust I am not prepared to ignore such a travesty.

    • Well said, Jane Colby. This mixing of patient groups has been an unmitigated disaster for those of us with Ramsay-defined myalgic encephalomyelitis (ME). We are all, of course, now well accustomed to the ugly politics of ME in the UK, the cronyism, the irresponsible conflation of ME with ‘unexplained fatigue’, but I will never understand the unblinkered willingness with which health journalists and editors have sycophantically promoted the PACE trial and its supporters. The harm done to ME patients through misrepresentation is truly immeasurable.

  6. Adrian says:

    I think it is also worth saying that since publication the Lancet has been vigorously defending the PACE trial. They do not seem to have responded to David Tuller’s articles or to an associated letter asking for a review. In the Australasian Radio interview mentioned in the article Richard Horton was very critical of patients for daring to criticize the trial.

    Accompanying the paper the lancet also published an editorial which further exaggerated the results claiming a 30% ‘recovery rate’ one that was even higher than the one that the PACE authors later claimed in their recovery paper where they changed the protocol to have a somewhat perverse definition of recovery (below the entry criteria). The lancet defended this editorial against a complaint brought to the press complaints commission which adjudicated the authors or the editorial were expressing an opinion although criticized the Lancet for not making this clear. They said that publishing letters pointing out the recovery definition of recovery was not used and a letter from the authors saying that they were not commenting on recovery offered sufficient remedial action. I believe the authors of the PACE article had seen the editorial prior to publication.

    http://www.pcc.org.uk/cases/adjudicated.html?article=ODQwOQ==

  7. Graham McPhee says:

    These are pretty fair questions, Andrew, but one of the worries is what happens next? What happens when a faulty paper is published? How does it get corrected?

    There are lots of intelligent people with ME/CFS who spotted major faults with PACE. Many of them have strong backgrounds in science, mathematics, statistics, even psychology. These faults weren’t obscure technicalities: it doesn’t take a genius to spot that something is seriously wrong when, for a key assessment, a score of 65 or less is described as being “abnormal function” only one paragraph after describing scores of 60 or more as being within “normal range” (where 0 is very poor, 100 very good). This happened on page 3 of the PACE “recovery paper”, and this measure was fundamental to the definition of recovery.

    Patients’ comments have little effect, even when the logic is simple and the evidence utterly clear. It is only now, 6 years on, when we get researchers from other areas to look at the evidence, that the problem is beginning to be recognized. Until then, only the small, specialized group of researchers involved in the area were the ones who handled any criticisms and they refused to reveal any further details. In this case patients who asked questions or who express scientific doubts were tagged as vexatious. It is pretty obvious to me that most of the other people defending the trial (e.g. in a discussions in Parliament) had not even looked at the detail, but had relied (as you imply) on the status of the researchers and of the journals involved.

    Following up Alex Chernavsky’s comments, does anyone realize that the conjecture that ME/CFS has a major psychological component has never been tested? That the amount of biomedical research over the last 35 years into the illness has been utterly minimal, with a number of small studies funded by charities coming up with interesting findings, but a complete lack of substantial funding for definitive trials? The assumption of the illness having a major psychological certainly has contributed to the poor attitude of a section of the medical profession towards analysis provided by patients.

    So if only those closely involved in a small research area are permitted to comment or to criticize, there is little hope that such work can be trusted.

    • Andrew says:

      Graham:

      I think open data and post-publication review are the way to go. I’m distressed by the Lancet editor’s defensive attitude, which seems contrary to the interests of science or policy. And, I argue, ultimately contrary to the reputation of the Lancet as well.

  8. Andrew: as a numerate person, I don’t find the statistic of about 400,000 people killed per year by medical mistake too implausible, but I suspect I also wouldn’t find either 200,000 or 800,000 too implausible either. Basically my prior is kind of broad. What’s your take on this issue:

    http://www.healthcareitnews.com/news/deaths-by-medical-mistakes-hit-records

    Instead of lives lost, I’d really like to see a QUALY/80 count, basically the same scale as lives (80 years is a good scale value for life expectancy) but would include a lot of pain and suffering as well as death.

    Is crappy medical research in the same boat as “we gave this guy a drug his chart said he was allergic to and he died” or “we diagnosed this guy with a panic attack and sent him home, but he was actually having a heart attack”?

    • Nick Menzies says:

      “I’d really like to see a QUALY/80 count”

      The calculation would be very different. To count up the deaths in the way they have, all you need to do is observe a medical mishap followed by a death. To count the QALYs, you need to simulate future life course with and without the medical mishap. Then you need to defend your modelling methods, and defend the use of QALYs. I agree that QALYs would be a better measure of burden, but there are good reasons why an advocacy campaign like this will talk about deaths.

      • Right, sure. the counterfactual is pretty difficult, but just starting with dealing with the deaths as a loss of 80-X life years, and the discomfort as a loss of NQ life years where N is the duration of the illness that should have been treated correctly and Q is the quality loss… and then putting a big constant out in front and arguing that the constant is somewhere between 0.1 and 1 would be a start. :-)

        in other words, it’d just be useful to get someone to do the ACCOUNTING part ;-)

  9. Jonathan says:

    Andrew: As I see it the problem with the peer review process was simply that psychiatry has inadequate standards for trials. Any clinical pharmacologist sent this paper for review would have said that it was not providing reliable information: period. This is not an issue of missing subtle problems. It is a barn door issue of a trial that is methodologically a non-starter. Unfortunately, it seems that in psychiatry people simply do not understand how bias has to be dealt with in order to produce a useful result. And I have had emails from eminent psychiatrists, who may well have reviewed this, indicating just that – they do not seem to understand. So the answer to your question is probably fairly simple. Psychiatry needs to get its act together.

    • Andrew says:

      Jonathan:

      This is related to the problem we discussed a couple years ago with that ovulation-and-clothing study that had been published in Psychological Science. The estimates effects were ridiculously implausible, and the measurement of peak fertility was both biased and noisy. Even setting aside the garden of forking paths, the study, and others like it, were dead on arrival. But the researchers didn’t seem to care about how bad the measurements were. They’d been trained to believe that if you have statistical significance, you’re good to go. I found the whole interaction very frustrating in that they didn’t even want to engage with concerns about bias and variance of measurements. They were holding on to that statistical significance and not letting go.

    • jrkrideau says:

      I think one of the problems is that medical doctors are not trained as researchers.

      Below is a quote from the Respectful Insolence blog where Orac (the blogger and an MD/Ph.D) discusses this. http://scienceblogs.com/insolence/2015/09/24/most-doctors-are-not-scientists/

      Medical training is not the same as scientific training. Yes, science is a prominent part of medical training, but not in the same way it is for scientists. In actuality, although medicine is based in science, it is an applied science. The vast majority of physicians do not do scientific investigation or contribute to scientific knowledge. Rather, they apply known science to the treatment of patients.

      This is not a knock on them, or an insult, or a criticism. It is simply an acknowledgement of what most medicine is. Medical school is very much more like a trade school, in which students are taught how to take care of patients, than a school teaching how to do science. Basically, medicine as a profession resembles engineering far more than a scientific specialty in that the vast majority of physicians apply science to the problem of diagnosing and treating illness, as engineers apply science to the problem of building things.

      Again, this is not a knock on either physicians or engineers. How they apply existing science to solve problems (or, in the case of engineers, to build things) can involve incredibly clever feats of mental prowess, but it is nonetheless a very different process than doing science to produce new scientific knowledge.

      • Yep, and even when you have an MD PhD combo, that person typically has spent a lot more time finding out how to treat patients than a person who has a pure PhD. The result is that many (but not all) “doctor researchers” do things that a 2nd year PhD student would recognize as a bad research technique. For example, running a gel and finding that all the lanes have some band… but they didn’t run a negative control! maybe all the lanes were contaminated! And more importantly, designing experiments that fail to actually rule out other obvious possibilities so that at the end of an expensive experiment you know very little more than you did going in.

        Unfortunately, funding for Doctor Researchers is a separate category, and generally quite large, so lots of university research is carried out by doctor researchers who probably should just be treating patients instead, and people who have heavy-lifting capabilities when it comes to research, but have no knowledge of “treating patients” are forced to scramble for much smaller pools of research funds than otherwise.

  10. Mike Dean says:

    The authors of the invited editorial that accompanied the 2011 PACE paper are highly likely to have been referees – normal journal practice. Bleijenberg and Knoop share the PACE authors’ beliefs about the psychosomatic nature of the illness and, for good measure, they even exaggerated the benefits of the touted therapies: http://www.ncbi.nlm.nih.gov/pubmed/21334060

    • Dr John L Whiting says:

      Do we ever get to know the identities of these referees?

      Surely, corrupt or unduly biased referees are just as culpable (if not more culpable) of errors in the publication process?

      It is not only the alleged sanctity of the Lancet that is a worry, but the safeguarding process employed by such prestigious journals in whom we hitherto have put our trust. If I am going to rig a boxing match, I pick the right people to ensure that I get the results I paid for.

      • Andrew says:

        John:

        I think open refereeing would be a good thing. Then when the paper comes out and is questioned, people can ask the referees what they were thinking. And the referee might say, I only looked at it for a few minutes and I didn’t notice that flaw. I don’t think it’s fair to blame the referees, who are typically doing their best for zero pay and near-zero credit.

        • Sasha says:

          Prof. James Coyne recently made the point that there are a huge number of PACE authors and that the UK is a small world when it comes to research. He doubted that it would have been possible for the PACE trial to have been independently reviewed by UK reviewers, given that a lot of the people familiar with the therapies and issues in PACE would have been either cronies of the authors or people whose positions would be vulnerable if they gave critical reviews and their identities were known (or could be guessed).

          Here’s the post:

          http://blogs.plos.org/mindthebrain/2015/11/25/was-independent-peer-review-of-the-pace-trial-articles-possible/

          So although I agree with open refereeing in some ways, I think it has disadvantages in others. Coyne suggests that maybe UK studies should have overseas reviewers and says (if memory serves) that getting overseas reviewers is the norm in some countries.

        • Fernando says:

          Andrew:

          In general I have some scepticism about people doing things for zero pay and credit.

          Referring can be a powerful tool to block out competing theories, push for citations, or do favors. Even in an pseudo anonymous context many authors work in narrow fields and do not have a hard time guessing who is reviewing. Presumably the reviewers know this. And so you have the basis of a repeated game where faction can coordinate and gate keep.

          Admittedly this is all hard to measure but from my experience I think it likely.

          • Andrew says:

            Fernando:

            I don’t disagree with you on this. In my own reviewing, what keeps me sane is thinking of the following principles:

            1. As a reviewer, I’m not making the decision, I’m just providing information for the editor, who’s the one who will decide.

            2. Every paper gets published somewhere. So no need to be upset if a bad paper gets published in journal X or a good paper is refused publication in journal Y.

            3. What if they’d never sent me the paper to review in the first place? Something would’ve happened to it. So I shouldn’t feel a crushing responsibility as reviewer.

            But I do remember a few years ago when I remarked on the blog that when I review a paper, I typically spend less than 15 minutes. Some commenters got really angry!

            • Rahul says:

              Spending less than 15 mins on reviewing a paper is great…………so long as you recommend rejection!!

              OTOH, yes, I’d be more than a bit annoyed if you really do recommend publication after not even 15 mins of reading. Do you really do that?! I still cannot believe you are serious!

              I find your arguments justifying this iffy too: Should a cop not take seriously a call he was dispatched on because someone heard a lady scream? I mean what if he had not been dispatched? Something would have happened. It’s not a crushing responsibility. No need to put his best effort into investigating.

              How about serving on the admissions committee at Columbia or the faculty selection committee? No need to do a good job because I guess the candidates will get selected at *some* University anyways. So might as well select them at Columbia. No need to be upset if Columbia gets a crappy candidate.

      • Alex says:

        I don’t know if they do it across all their journals, but Frontiers in Human Neuroscience (part of the Frontiers in umbrella) publishes the reviewers along with the paper. I think it should happen more often.

  11. a physicist says:

    The Lancet fast-tracks a lot of stuff, and seems to often choose papers for their controversy value. They did a special issue on the Global Burden of Disease and accepted all the papers before the final numbers (i.e. results) had been put into the papers.

    • Mike Dean says:

      You’re right that the Lancet courts controversy, but PACE was fast-tracked at the request of the PI, Peter White. This emerged in the course of Prof. Malcolm Hooper’s complaint to the Lancet. http://www.meactionuk.org.uk/Comments-on-PDW-letter-re-PACE.htm

      Hooper also reported some information on statistical peer-review at the Lancet which is relevant to Andrew’s blog:

      ‘Given that the PACE Trial article was fast-tracked, the question arose as to whether or not there had been sufficient time for scrupulous checking of the data by The Lancet’s own statisticians. For the avoidance of doubt, when on 29th March 2011 the executive editor responsible for the publication of the PACE Trial articles was asked how The Lancet’s statisticians could have let such conflicting interpretation of the data be published in a journal of its reputation, he confirmed that he had taken Peter White on trust, saying (verbatim): “We can only do what we can do. We have to take things on trust. We don’t get the statisticians to go round and check every calculation that’s been done. It’s not up to the statisticians to advise on all the adding up”.’

  12. jimells says:

    “Newsworthiness counts for a lot”

    But corruption and connections count for a lot more:

    http://erythos.com/gibsonenquiry/Report.html

    “CFS/ME is defined as a psychosocial illness by the Department for Work and Pensions (DWP) and medical insurance companies. Therefore claimants are not entitled to the higher level of benefit payments. We recognise that if CFS/ME remains as one illness and/or both remain defined as psychosocial then it would be in the financial interest of both the DWP and the medical insurance companies.”

    -UK Parliamentary Group on Scientific Research into M.E. (Gibson Inquiry, 2006)

    This same report also called for investigation of the “Wessely School”:

    “…numerous cases where advisors to the DWP have also had consultancy roles in medical insurance companies. Particularly the Company UNUMProvident. Given the vested interest private medical insurance companies have in ensuring CFS/ME remain classified as a psychosocial illness there is blatant conflict of interest here.”

    Ten years after release of the report, we are *still* waiting for the investigation.

  13. Leveller says:

    I think that the Lancet’s defence of the PACE trial needs to be understood in the context of the culture of the British Establishment and particularly i) the desire to emphasise the extent to which those with health problems can overcome their limitations through rehabilitation and management, and thus do not truly need financial support from the state or insurance companies and ii) the prolonged campaign to present CFS patients’ concerns about the quality of CBT/GET research as stemming from their own fears of the stigma of mental health, or simply their own stupidity. The consensus view of the British Establishment seems to be that patients concerned about the PACE trial are unreasonable troublemakers and that their views do not even need to be properly considered.

    I think that Horton saw his publication and defence of the PACE trial as a brave stand against militant, anti-science patient pressure groups. Unfortunately, he’d failed to spot the serious problems with what had been published, and his reputation is now tied to his smears against the PACE trial’s critics. Also, it seems that his defence of PACE was built upon a real lack of understanding, and having fallen for the PACE researchers spin, he has been less careful with what he says publicly.

    eg: On radio Horton said:

    “Yeah, I mean adaptive pacing therapy essentially believes that chronic fatigue is an organic disease which is not reversible by changes in behaviour. Whereas cognitive behaviour therapy obviously believes that chronic fatigue is entirely reversible. And these two philosophies are kind of facing off against one another in the patient community and what these scientists were trying to do is to say, ‘Well, let’s see. Which one is right?'”

    http://www.abc.net.au/rn/healthreport/stories/2011/3192571.htm

    Horton places recovery claims at the heart of the PACE trial’s findings, as did most media reports on the trial, and I think that it is recovery claims that are of most interest to patients too. In the Lancet commentary which accompanied the PACE trial it was claimed that:

    “PACE used a strict criterion for recovery:
    a score on both fatigue and physical function within
    the range of the mean plus (or minus) one standard
    deviation of a healthy person’s score. In accordance with
    this criterion, the recovery rate of cognitive behavioural
    and graded exercise therapy was about 30%—although
    not very high, the rate is significantly higher than that
    with both other interventions.”

    http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(11)60172-4/abstract

    This ‘strict criterion for recovery’ in fact only required that patients report and SF36-PF score of 60 or more, and a likert Chalder fatigue score of 18 or less. Patients could enter the trial with an SF36-PF score of 65 and a likert Chalder fatigue score of 12 (bimodal score of 6) – so they could report a worsening of scores for both questionnaires used as primary outcomes in this non-blinded trial, and yet still be classed as fulfilling a ‘strict criterion for recovery’. I think it is understandable for patients to be angry about this, and to want to see the release of results for the far more stringent recovery criteria laid out in the trial’s protocol. ‘Which one is right?’ If Horton was really interested in finding out, I would expect him to be calling for the release of the trial data, not working behind the scenes to keep it away from independent researchers.

    Horton presented patients’ concerns about the PACE trial like this:

    “Today we met lead scientists behind the PACE trial. The campaign against this study was virulently anti-science, adding £750K to the costs.
    “This story of waste, vexatious lobbying, and ad hominem attack should be front-page news.”

    https://twitter.com/richardhorton1/status/54959875133939712
    https://twitter.com/richardhorton1/status/38371958055772160

    This is one example of many, and since then, we have seen considerable promotion in the UK media of quite unpleasant prejudices about those CFS patient who are concerned about the PACE trial and the claims made about the value of CBT and GET. If it is now acknowledged that this ‘virulent anti-science campaign’ was actually a result of honest, reasonable and diligent patients spotting real problems with the quality of the research which was profoundly affecting their lives, then this will be more than a little embarrassing for Horton and the Lancet. Publishing poor quality and misleading claims about a stigmatised health condition, and then smearing those patients attempting to draw attention to the problems with the research, should be unacceptable. He has placed himself in a position where it is now in his interests to defend the PACE trial to the end, regardless of the validity of the criticisms being made.

  14. Andrew says:

    Leveller:

    “Vexatious” lobbying, huh? I think “vexatious” is going to be one of our new catchphrases, along with “evilicious,” “replication bullies,” and, of course, “gremlins.”

    People questioning a study published in a top journal—what could be more vexatious than that?!

  15. John Peters says:

    The Lancet is so heavily invested in PACE that when the US IOM report on ME/CFS essentially endorsed everything patients have been saying for 25 years and rejected the approach of the people involved in PACE, the Lancet published a dishonest editorial claiming ‘the results of the PACE trial paved the way for this IOM report’. There is no mention of PACE in the report. I emailed Ellen Clayton, who chaired the committee which made the report, and she said unequivocally that as far as she is concerned PACE did not.
    Many patients have challenged the Lancet on this point and I have emailed the editor. There has been no response.

    • A.B. says:

      Amusingly PACE trial main investigator Peter White later commented on the IOM report:

      > But perhaps the most unhelpful aspect of this report is its over-emphasis on the physical, with an almost complete absence of psychological and social aspects of the illness. We have a whole chapter reviewing the potential roles of immune, endocrine and infectious causes, but little if any mention of the potential roles of life events, stress, emotions, beliefs, and behaviour. This report took $1 million to produce, and the authors have missed a golden opportunity to integrate the findings of both mind and body to move us away from the sterile, dualistic understanding that still dominates this illness.

      http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=10052432&fileId=S0033291715002366

      To which I respond that these psychological factors were not discussed because the evidence for their causative role in CFS is weak and not credible.

      • Martha says:

        The quote makes Peter White sound almost like L. Ron Hubbard.

      • urbantravels says:

        It is wearying in the extreme to see, again and again, this claim on the part of the British psychiatric lobby that their view of the disease is more valid because it is “holistic” and respects some kind of mystical “mind-body” connection that most people, including especially patients, are not sophisticated enough to understand.

        This notion of the “mind-body connection” as a primary driver of physical chronic disease states always reminds me of the famous New Yorker cartoon with a complicated equation on the blackboard of which “Step 2” is “Then, a miracle occurs.” This “holistic” view of disease amounts to fuzzy-mindedness at best, and a particularly vicious strain of victim-blaming at worst.

  16. A.M. says:

    Thankyou for covering this issue.

    With the case of the “normal range” for fatigue and physical function, arguably the most questionable and controversial post-hoc change to the protocol, which overlapped with trial eligibility criteria for severe disabling fatigue:

    This is not just a case of a peer reviewer accidentally missing a flaw in the paper. In this case, a peer reviewer actually introduced it or at least inspired it, and then waved it through knowingly. According to correspondence between Professors Peter White et al. and Malcolm Hooper (with Richard Horton as mediator), “The normal range analysis was plainly stated as post hoc, given in response to a reviewer’s request.” ( http://www.meactionuk.org.uk/whitereply.htm )

    It is unclear what exactly the peer reviewer suggested e.g. whether they demanded the exact normal range that was used, or they simply made a general suggestion which gave the authors some freedom in defining the thresholds for the normal range. Either way, the reviewer must have approved the change. White et al. consider asking questions about this issue to be “vexatious”, so we may never know what happened.

    One more point. The “normal range” is described as a post-hoc analysis and there is no mention of changes to the recovery criteria in the final version of the statistical analysis plan that was approved shortly before the unblinding of trial data. Yet White et al. asserted to David Tuller that the revised recovery criteria (which includes this “normal range”) was “pre-specified”. While it may be true that they made changes to the recovery criteria before calculating the recovery rates, the statistical analysis plan indicates that they had access to a wide range of data for the Lancet paper that was apparently prepared before the normal range was first introduced at the peer review stage. In other words, it appears likely that they were familiar with every single component of the recovery criteria before the major changes to the recovery criteria were made. But again, they consider asking questions to clarify the timing of these changes to be “vexatious”.

    • Rahul says:

      Isn’t the information the authors refuse to release within the reach of a Freedom of Information request?

      Assuming they used University email address or University PCs etc?

      • JohnCB says:

        Yes. No.

        “Isn’t the information the authors refuse to release within the reach of a Freedom of Information request? “

        FoI requests have been made and refused, appealed, supported on appeal and still refused, as I understand it.

        It has been covered by Coyne and Tulley in their (necessarily) lengthy articles.

  17. Alex Gamma says:

    Many medical doctors have the unfortunate tendency to not take (somatic) symptoms with a presumed psychological origin very seriously. It has to do with the impossibility of physically measuring such symptoms (fatigue, pain…). Consequently, they are taken to be somehow “less real”. The patients, in addition to suffering from the symptoms, also suffer from the stigma that such an attitude creates: they are treated as if their condition is questionable, maybe just imagined; as if it springs from a weakness of character and they’re to blame for it.

    No wonder that many patients fight for their conditions to be placed firmly within the somatic domain, but they do it mostly, I suspect, to get taken as seriously as patients with plainly somatic conditions. While such a “reductionist” move is all right for patients, it is mistaken as a general stance towards mental or “psychogenic” disorders. The mind is real, consciousness is real. Many symptoms, including those of somatic disorders, manifest on the level of conscious experience (pain being the paradigmatic example). It is eminently reasonable to expect that the mind and body interact, including in the production of disorders. There’s nothing disreputable about a “mind-body connection” or about mental causation (if there were no mental causation, how could you ever know and report the fact that you’re conscious, for example?).

    CFS/ME might be a purely somatic disorder, or it might not be. What matters is that every condition that produces suffering and possibly impairment deserves to be taken seriously qua disorder. There are not two categories of illnesses: the real somatic ones and the somehow less real mental or psychogenic ones. The way to go is to destigmatize the mental as a source of suffering (see e.g. pain research, where the notion of pain without “objective” correlate has come to be accepted). Yes, victim-blaming needs to go, but not by declaring the possible mental origin of disorders to be fishy.

    • A.B. says:

      The claim that the mind can cause a debilitating illness and severe ongoing physical symptoms has no scientific basis. Diseases are declared to be caused by mental factors not because there is evidence for it, but because a physical cause cannot be identified. Which doesn’t mean that no physical cause exists. The history of medicine is full of examples of diseases being considered to be caused by mental factors only for those diseases to be later proven to be caused by physical factors. eptic ulcer is the one example everyone is familiar with… but there are many other examples such as asthma, rheumatoid arthritis, diabetes, cancer, AIDS, Grave’s disease, dystonia. Claims of psychogenic causation are best understood as unsubstantiated belief that serves the purpose of giving the illusion of knowledge and control.

  18. Rahul says:

    Andrew, why do you give the Editors and the Reviewers a pass? To me those are absolutely the critical steps that failed.

    By reviewing dozens of papers for free and by not doing a good job at it aren’t you just passing the problem elsewhere? Why not focus on building a good fence? Maybe a good series of fences?

    The whole attitude of regarding publishing as a random process sounds so nihilistic and counterproductive to me.

    Finally doctors must get their clinical guidelines from somewhere? Do we expect them to trawl the whole, huge literature for every condition they need to treat clinically and then develop their own considered opinions each time?

    If they can’t trust “authoritative” sources who should they trust? If we excuse reviewers and editors for doing a shoddy job gratis, so then we have to spend money elsewhere to erect another set of good fences. Who will that be? The FDA? The Physicians Associations? We could just stop the charade of Journals and peer review and just post to arxiv and then the “users” (FDA, physicians etc.) can do their own due diligence.

    Isn’t it just paying lip service to the “service” aspect of the job if you are not going to do a really conscientious job reviewing? No, reviewers must not catch *every* problem: They only have to catch *one* (non-trivial) problem and that ought to be enough to send a paper back.

    If reviewers need access to raw data they must demand it. Refuse to review a paper till you have such access. If time is short demand time. Or refuse to review.

    If reviewers are too busy let the Journals figure out a better way to compensate them for their time. I’d rather more people refused to review than did a crappy job at it. The world might be better off having fewer papers published.

    I think the buck must stop somewhere. If it is so obvious that the PACE paper was so shoddy in hindsight let the reviewers & editors also share some shame. By saying no fence can be perfect we are just encouraging bad, halfhearted reviewing.

    • Andrew says:

      Rahul:

      What disappoints me in all this is the non-engagment of the editors and reviewers in post-publication review. I’ve published 3 posts on this and had dozens of comments, but nothing from anyone who refereed or edited the paper justifying their decisions. What really bugs me is that attitude that, once something’s published, it’s supposed to be on some plateau that makes it invulnerable to criticism. It’s amazing to me that, among all the comments on these posts, I don’t recall a single one being supportive of the paper. I’d hope that the editors and reviewers would either give a substantive defense of their decision, or say they made a mistake, or say something, not just sit tight and hope the storm will blow over. As if the problem is criticsm, not bad science.

      • Rahul says:

        Andrew:

        But you seem to be more disappointed by the non-engagement than by the fundamental fact that the review process itself was crappy?

        Further, there can be errors in any process but by your comments it sounds like you are taking crappy, half-hearted reviewing as a matter of fact, routine condition rather than just a rare error.

        Why are you so averse to *improving* the review process itself?

        • Andrew says:

          Rahul:

          I’m not averse to improving the review process, I’m just inclined to favor moving toward more post-publication review rather than tinkering with the existing pre-publication review process.

          To me, the biggest challenge in moving away from the current system is that the current system, for all its flaws, makes use of a huge amount of goodwill and volunteer labor. If we drop pre-publication review entirely, it’s not clear how many of the hours that currently go into pre-publication review, will instead go into post-publication review.

          For example, my current idea of a good system would be for researchers to dump all their papers into a sort of beefed-up Arxiv, and then for journals such as JASA, APSR, JPSP, etc., to transform into recommender systems: instead of producing a journal and publishing papers, JASA (say) would produce a monthly list of recommended papers and maybe do some extra things like organize discussions of these already-posted papers.

          This would all seem much better to me. But it would require that all the volunteer effort that currently goes into refereeing and editing, would instead go into selection and post-publication review. And maybe that wouldn’t happen, I just don’t know.

          • Rahul says:

            So what would be your advice to clinicians? How are they supposed to decide treatments if not by believing articles in “authoritative” Journals?

            If I were a clinician and trusted the PACE study to treat patients what could I be doing to be more robust to crappy studies?

            • Andrew says:

              Rahul:

              In my ideal world, there would still be authoritative journals, they’d just be pointers to already-posted studies.

              But, in any case, clinical research is different from the usual papers in poli sci, psychology, econ, etc. A clinical study has some patients and typically has to be approved by some regulatory body, so it makes sense that all clinical studies can be preregistered and reported with all their raw data in public repositories. If the topic is important, others can analyze the data too, so that puts less of a burden on the original data analysts, indeed it allows for a separation of hypothesis generation, study design, data collection, data analysis, and policy recommendations.

              I would definitely recommend there being some official or quasi-official organizations (comparable to today’s JAMA, Lancet, NIH, etc.) that would evaluate studies and analyses. I’m not expecting clinicians to wade through hundreds of published reports in their subfields.

              • Mike Dean says:

                Andrew, There is a general presumption that in the absence of a very large “definitive” clinical trial, individual trial results count for less than systematic reviews and meta-analyses. PACE was meant to be “definitive” but was not – at least not in the direction that the trialists expected. This might be one reason why the PACE authors are conducting systematic reviews of their own trials. Therapist allegiance is noted as a conflict of interest by the Cochrane Collaboration, but not necessarily elsewhere. James Coyne has just blogged about this problem with a planned systematic review involving White.
                https://jcoynester.wordpress.com/2016/01/05/undisclosed-conflicts-of-interest-in-a-systematic-review-protocol-of-interventions-for-medically-unexplained-symptoms/

              • Sasha says:

                I think we need both good pre-publication review and open review thereafter. The problem with PACE is that the reviewers, for whatever reason, didn’t catch the major errors (the absurd “normal ranges” for recovery) or correct the massive spin in interpretation (the ignoring of the pressure on patients to sway their subjective ratings of health in the direction of the experimenters’ favoured treatments). Once PACE was published in the Lancet, plenty of people wrote in to point out these problems (post-publication review, if you like) and it was clear that the “normal range” analyses should have been retracted and a correction published, but the editor refused to do it. This issue even went to the Press Complaints Commission, who gave a lame-duck response (the PCC was later closed down for its lame-duckness).

                So a clinician who goes to the Lancet and reads that PACE paper has no clue, unless they read all that correspondence (which I doubt anyone does), won’t realise that a careful reading shows that it’s a terrible study.

                In an ideal world, editors would respond to such criticism and whack a big retraction/correction notice on such a paper but this is rare. Their “I must hide my mistake” attitude kicks in.

                How do we create a system where pre-publication reviewers are motivated to do a good job? And where post-publication review/critique has the influence it should?

          • Ed Hagen says:

            For example, my current idea of a good system would be for researchers to dump all their papers into a sort of beefed-up Arxiv, and then for journals such as JASA, APSR, JPSP, etc., to transform into recommender systems: instead of producing a journal and publishing papers, JASA (say) would produce a monthly list of recommended papers and maybe do some extra things like organize discussions of these already-posted papers.

            I had the same idea. The US and European governments already have most of the IT infrastructure in place in the form of PubMed and Europe PubMed. All PubMed would need to add is a way for authors to upload papers, and then a way for individuals to form “journals”, i.e., curated lists of pubs.

            • + 10^6

              And not just PubMed. Let’s just create “archive of american scientific thought” and/or “archive of european scientific thought” and let anyone and everyone upload whatever they want (maybe 1 pdf per day per author, to avoid pure spamming).

              Furthermore, let’s just require by law that any research funded with public money publish exclusively in this archive.

  19. Sasha says:

    An interesting point (in my opinion) on how to motivate reviewers.

    In clinical trials, the people most motivated to get at the truth of the study are patients. A lot of patients are scientists. Being a scientist is no protection against getting sick: lots of scientists have cancer, diabetes… and ME/chronic fatigue syndrome. Many of us are qualified to critique clinical trials and highly motivated to ensure that authors don’t get away with bullshit and spin. And many patients have studied clinical trial methodology since they got sick, especially so they can critique (Tom Kindlon, with a background in mathematics before he got ME/CFS did so and he has played a huge role in getting the problems with PACE acknowledged).

    But we only get to do post-publication letters to the editor, not pre-publication review. Maybe we need a pool of patient-scientists whom journals can call on for pre-publication review.

    By the time a paper is published, it’s too late for the patients who’ll be the target of the treatment intervention. We can’t afford to have our health damaged by this crap while we’re waiting for the post-publication review community to fix the deficiencies of the pre-publication review community.

  20. Ed says:

    I would say that journalists and, especially, policy makers like this stuff for the same reason they like the questionable work on mindfulness or CBT for various psychiatric conditions*. It offers cheap solutions that require little infrastructure to implement. If conditions can really be treated with a round of exercise or a bit of mindfulness, policy makers don’t have to put much money into treating these conditions.

    *James C. Coyne has been engaged in similar battles with researchers over some flawed work on CBT for schizophrenia.

  21. Rahul says:

    Isn’t the information the authors refuse to release within the reach of a Freedom of Information request?

    Assuming they used University email address or University PCs etc?

    • John Peters says:

      Simple answer: yes, this is subject to FOI.

      There is a long history regarding FOI. Many requests have been dismissed as vexatious, upheld by the FOI commissioner. The PIs of PACE used ‘vexatious FOI requests’ in a Science Media Centre-orchestrated campaign in the media complaining of ‘harassment’ of the PIs. It gained a lot of coverage by, amongst others, the BBC, including the Today programme and ‘The Times’: A small group of militant, anti-science ME patients were stopping the PIs from carrying out important work to help those very same patients. It is a perception which still prevails.

      Peter White, the PACE leader, has gone on record in supporting an exemption for all university research from the FOI Act.

      A more recent FOI request was again rejected by Queen Mary University, but this time the FOI Commissioner sided with the request. QMUL are appealing.

      One PACE study was published by PLoS One and Prof Coyne has asked for the data as part of the PLoS publishing agreement. Kings Coll, London, rejected it as ‘vexatious’. Prof Coyne has appealed to the PLoS board and we await their findings. More on that at Prof Coyne’s blog here https://jcoynester.wordpress.com/

  22. Jo Best says:

    I haven’t managed to read all the comments yet so I’m sure the point has been made that one reason the PACE paper was so readily accepted and widely (mis)reported is that it is the only medical research to be part-funded by the UK government’s Department of Work and Pensions. The trial clearly set out to prove that people with ‘CFS/ME’ may return to work if on sick leave or stop claiming disability benefits after treatment with the PACE trial protocol for cognitive behaviour therapy and graded exercise. Results were anticipated by NICE when forming their guideline for CFS/ME long before the trial had finished and was used as a reason for not updating the guideline when review was due. Occupational Health guidelines for ill-health retirement similarly routinely require that applicants undergo these supposed evidence-based treatments before consideration. This would be ok if there were no risk of harm of doing so, but there is known risk of harm so that the only impediment was a delay in the process, but there is risk of the person getting more ill, sometimes irrevocably so. Then there are the dreadful consequences of parents accused of factitious disorder by proxy on the assumption that there is no underlying organic disease process causing chronicity. Thank you Andrew for your observations on this issue. I really enjoy and appreciate your clarity. I like The Sopranos photo, which reminds me that I call the PACE trial clan (which extends imo to the UK CFS/ME Research Collaborative) as the ME Mafia, somewhat ironic considering that ME patients are freely and routinely described as dangerous activists. Best wishes for 2016.

  23. Rahul says:

    @Keith:

    Aren’t there some things researchers like Andrew can start doing on their own as a first step to improve the situation:

    e.g.

    Post all your work online at something like arxiv. Ideally much before you submit to a Journal.

    Refuse to referee papers that won’t submit data or code.

    Try to do a more thorough job while reviewing each paper. If that means fewer reviews so be it.

    I cannot understand this practice of doing cursory, one-hour reviews (of “dozens” of papers every year) just because it is a gratis job and no reviewer is perfect and there’s always post publication review.

    • Keith O'Rourke says:

      @Rahul

      I did not mean to sound like that joke “if you want to get there, you shouldn’t be starting out from here” and there are (always have been) folks who take the high road. If you you don’t have inside information about who those are – you really can’t make sense of any collection of academic outputs (e.g. clinical trails on a given treatment) and that requires a system wide change.

      > I cannot understand this practice of doing cursory, one-hour reviews
      One of the hardest lessons for me to learn was not to agree to do things without having adequate resources to do it well, while realizing repeatedly doing things is who you are (becoming). For instance, in an academic setting, if you spend too much time reviewing papers you might be putting your position at risk. But, it is not actually a gratis job in a academic setting, as you are expected to do some, its just seldom well tracked.

      • Rahul says:

        In hindsight, apparently “one hour reviews” was a mistake.

        As Andrew mentioned elsewhere in a comment, he typically devotes “less than 15 minutes” to reviewing a paper.

        Do you think Andrew is an outlier or are 15 minute reviews typical among academics?

        15 minute rejections don’t surprise me, but to recommend an article for publishing in “less than 15 minutes” makes a sad joke of the whole peer review process, in my opinion.

        I agree 100% with you: People ought to learn to say no.

        • I can imagine three main scenarios:

          1) They’ve obviously done something dumb, Reject, < 15 mins

          2) They haven't obviously done something dumb, and I can't think of any subtle issues that should be addressed, accept 15 mins, accept if the issue can be addressed.

          Perhaps Andrew, in his field, and given his reputation, etc tends to get either 1, or 2, and rarely 3.

          In my experience, Biologists almost always are in the situation 1, or 3, and 2 is almost never the case. In Engineering, I think 1,2,3 are all possibilities.

          • ack, stupid blog and its eating less than signs.

            1) They’ve obviously done something dumb, reject, < 15 mins
            2) They haven’t obviously done something dumb, and I can’t think of any subtle issues that should be addressed, accept < 15 mins
            3) They haven’t obviously done something dumb, but I can think of a subtle issue, here it is… accept if the issue can be addressed, > 15 mins

            • Rahul says:

              But can you really do #2 in less than 15 minutes? I’m very skeptical.

              • Martha says:

                I don’t think I could — indeed, I often don’t see “subtle issues that should be addressed” on first reading. But maybe some people are more competent than I at this.

              • Rahul says:

                @Martha

                It’d need superhuman competence in my opinion. To do a good review job in 15 mins.

                Hell, it takes me longer to read undergrad student project reports.

  24. Rahul says:

    @Danel:

    Things like CFS / ME make me wonder as to what a “disease” is.

    Here we have a set of diffuse symptoms, that vary greatly from patient to patient, we have no clue of any causal mechanism.

    No one organ or system seems the target, even the triggering event is inchoate. No definition can be formulated based on a measurable set of physiological parameter changes.

    I’m not at all denying the suffering of patients but why is this a specific “disease” rather than an unrelated bunch of diverse cases we don’t know where to classify into.

    • Curious says:

      I do not think the current state of knowledge is quite what you describe: http://www.cdc.gov/cfs/causes/index.html

      Also, it is not uncommon for sets of symptoms to be lumped together into a syndrome or disorder, etc. The critic of the approach taken by the PACE researchers is that the presumption is that there is actually nothing wrong with the patient and that they are simply creating this fatigue in their mind as a result of a psychological problem which is then dismissed as being irrelevant. The conclusion is they are just ‘sad’ and they will get over it, or they should get over it, or they should pull themselves up by their bootstraps, etc.

    • A.B. says:

      The widely used Fukuda case definition of CFS is also widely considered to be inadequate due to being too broad and vague. Patients have been complaining about this for a long time. There exist stricter case definitions used in research. There also exists the Oxford definition, only used by PACE authors and colleagues, and it’s even worse than the already poor Fukuda definition.

      Misdiagnosis is also common. Patient get diagnosed with CFS while they have something else such as pituitary or vascular disease.

      Pretty much everyone agrees that we need biomarkers and more research to separate CFS into subgroups that make more sense.

      Patients talk about disease because it’s shorter than the alternatives.

      We can’t say anything definitive yet but there are reasons to believe that number of subgroups might not be as large as the diversity in symptoms would suggest. Researchers are definitely capable of finding abnormalities shared by a substantial number of patients.

      The CPET findings are one and Valentijn wrote a good comment further above on this page.

      Another interesting line of research is that (conservatively) half of patients selected according to a stricter definition respond to a drug used to treat autoimmune disease.
      Benefit from B-lymphocyte depletion using the anti-CD20 antibody rituximab in chronic fatigue syndrome. A double-blind and placebo-controlled study.
      http://www.ncbi.nlm.nih.gov/pubmed/22039471

    • Martha says:

      @Rahul

      Remember that in 1982, the CDC defined “a case of AIDS as a disease, at least moderately predictive of a defect in cell-mediated immunity, occurring in a person with no known cause for diminished resistance to that disease.” (See Editorial Note at end of http://www.cdc.gov/mmwr/preview/mmwrhtml/00001163.htm) That’s pretty vague, but when the HIV virus was identified, things fell into place and work on treatments could begin.

      • A.B. says:

        And there were also people who proposed that AIDS was some form of mental disorder, with the reduction in cell-mediated immunity being an expression of depression.

        The group-fantasy origins of AIDS.
        http://psycnet.apa.org/psycinfo/1985-14989-001

        • Here I think is where we agree 100%. If you propose a mental origin for a disease you need to provide evidence. For example, can you show that some mental processes cause T-cells to die off? Can you show that altering the mental processes causes them to come back? etc etc.

          But the need to demonstrate your model does not preclude “mental -> physical” as a category.

          If you want to go further and say “mental -> physical” is a category of explanation that has been overrun with charlatans and that this background should give us low prior probability for believing the model in most cases… I probably would be fine with that, lots of medicine seems to be overrun with charlatans (see Tamiflu discussion recently) ;-)

  25. Leveller says:

    I’ve just realised that my second twitter link was to the wrong Horton attack on PACE critics – above he was criticising the ME Association, complaining that they “object to rigorous research findings because they challenge fixed assumptions? A disservice to patients.”

    Here is the correct link:

    “This story of waste, vexatious lobbying, and ad hominem attack should be front-page news.”
    https://twitter.com/richardhorton1/status/54960152406786051

    Andrew: “People questioning a study published in a top journal—what could be more vexatious than that?!”

    Some of them being mere patients seems particularly galling.

  26. Zach says:

    I can understand how, under the current flawed publishing system, some (even some major) methodological or analytical problems sometimes escape the scrutiny of peer reviewers and/or editors.

    But what I don’t understand is how everyone at the Lancet thought it was no problem to publish a major high-profile medical trial in which all of the primary endpoints had been switched to post-hoc endpoints, without this being mentioned, explained or explored in detail within the paper.

    Another major issue that seems to have escaped the Lancet is that this was an open label-trial, without a placebo control arm, that relied purely on self-report primary outcomes in a trial of interventions that aimed to change patients’ interpretation of their symptoms. Almost zero attention was given to the potential for bias in such a study.

    If this was a high-profile multi-million dollar pharmaceutical trial, the methodology would have been laughed at, and the evidence from the trial would have been deemed to be of inadequate quality upon which to base medical decisions.

  27. Lidia says:

    ME/CFS is a very serious illness in which a biological mechanism has been seen to underpin every one of its associated symptoms. Here is a summary paper, ‘Deviant Cellular and Physiological Responses to Exercise in ME/CFS’, which lists some of the most interesting findings to date:
    http://www.jacobspublishers.com/images/Physiology/J_J_Physiology_1_2_007.pdf
    The footnotes are links to each of the articles cited.

    In contrast, the Principal Investigators on the PACE Trial believe that ME/CFS patients complain of physical symptoms that do not result from underlying physical disease but are the consequence of abnormal illness beliefs, and that the abnormal beliefs are responsible for the perpetuation of the perceived disability.

    There is a huge difference between the 2 models and this has an enormous impact on how patients are perceived and treated.

    Some analysts have suggested that less than 50% of patients on the PACE Trial are likely to have actually had ME/CFS, due to the broad selection criteria and methods used to recruit patients, and because the most seriously ill were actively excluded from the trial. The ‘results’ are nevertheless frequently being applied to all with genuine ME/CFS. (Those of you who have read the research papers above will, begin to understand why this is so disastrous to patients.)

    Many researchers believe that if the PACE Trial data is properly reanalysed (even allowing for the fact that not everyone on the trial has the disease), it will actually demonstrate that the ‘somatisation’ model on which the trial was founded, is in fact incorrect. This is why we believe that the authors have gone to such bizarre lengths to reinterpret and obfuscate the real findings … and presumably why after over 150 requests for release of the data, they still haven’t done so. (They have spent over £75,000 in lawyers fees in the process.)

    It is of great importance to the patient community that the pressure is kept up on this … especially since a recent ME Association report has found that 74% of patients have reported harm from GET, one of the therapies that the PACE trial is promoting. This would never be permissible if this had been a drug trial.
    http://www.meassociation.org.uk/how-you-can-help/fundraising-support/

    So thank you for this post Andrew. It is much appreciated. Please keep up the pressure.

  28. Alex Gamma says:

    A.B. wrote:
    “The claim that the mind can cause a debilitating illness and severe ongoing physical symptoms has no scientific basis. Diseases are declared to be caused by mental factors not because there is evidence for it, but because a physical cause cannot be identified. Which doesn’t mean that no physical cause exists. The history of medicine is full of examples of diseases being considered to be caused by mental factors only for those diseases to be later proven to be caused by physical factors. eptic ulcer is the one example everyone is familiar with… but there are many other examples such as asthma, rheumatoid arthritis, diabetes, cancer, AIDS, Grave’s disease, dystonia. Claims of psychogenic causation are best understood as unsubstantiated belief that serves the purpose of giving the illusion of knowledge and control.”

    It’s more complicated. First, the fact that doctors resort to “psychogenic” when no physical cause is known does not mean that mental origins of diseases do not exist. Second, a mental origin of a condition does not preclude a physical cause. The problem here is how to think about the relationship between the mental and the physical. It is generally accepted by scientists that the physical world is causally closed, so there will always be a physical cause for any physical effect. Given, however, that the mind is acknowledged to be real (and certainly consciousness as the central aspect of the mind is undoubtedly real), the question becomes how the mind and body relate to each other. (Unless you want to claim that consciousness is an epiphenomenon, but as I wrote, then you can’t explain how you can know and report the fact that you are conscious.) One possibility would be a relation of instantiation, like for example when the molecular structure of a crystal is seen as the lower-level instantiation of the crystal’s hardness (a higher-level property). Some neural activity would then, for example, be the lower-level instantiation of a higher-level property such as a mental/conscious state.

    This is only hinting at some of the difficulties, but you certainly cannot get away with claiming that mental origins of bodily conditions have no scientific basis. The mental, in particular consciousness, are undoubtedly real, so to assume there could be a causal or other kind of connection to the body is only rational, given that there is no contrary evidence (and, to repeat, the presence of a physical cause cannot be considered contrary evidence). As a matter of fact, our conscious experiences are, strictly, the only things we can be absolutely sure are real, so the denial of mental causation and the insistence on physical stuff as the only real stuff becomes quite paradoxical.

    • Yes, yes, yes. In particular, the essence of all the stuff I’ve been yammering on about above is: “you certainly cannot get away with claiming that mental origins of bodily conditions have no scientific basis”

      You can at the same time easily get away with saying that “the assertion that specific disease X is caused by mental conditions has no scientific basis”. It’s only the broad categorical dismissal of all mental origins that is unwarranted.

      There are pretty much two different major classes of situation:

      1) where doctors observe some physical ailment, can’t come up with a model for the physical cause, and resort to some kind of default and “dismissive” assertion of “psychosomatic” illness. Most likely this is going to be repeatedly wrong. There’s no evidence of psychosomatic cause in this case, it’s not motivated by theory, it’s just “we ruled out everything we could think of… must be the brain?”

      2) Where doctors observe that people with particular mental conditions (soldiers, aid workers, victims of abuse, victims of disasters, whatever) tend to eventually develop particular physical conditions, and the working hypothesis is that the mental conditions, which were prior to the physical, led to or contributed to the physical. Here we have actual scientific type thought, and there are mechanisms which are known (hormones etc) and lab experiments (on rats etc) that are highly suggestive. Dismissing this line of inquiry because (1) is so prevalent is unwarranted.

      And also, damn you WordPress for making it so hard to figure out where to reply to nested comments… so there’s a bunch of stuff up above that’s all swimming around on the page.

  29. JohnCB says:

    “it’s still just a few people reviewing, and typically these reviewers have neither access to the raw data nor the time for careful reanalysis.”

    I think there is a huge assumption here. There are very few names in the top elechons publishing on this subject in the UK. Look which name is missing from the list of authors … It is extremely difficult for researchers in the biomedical field to publish on CFS/ME.

    The climate in the UK is very different to the USA. This is not an issue of The Lancet alone. The BMJ has also been criticised. There was a very interesting paper highlighting this published in The Journal of Chronic Fatigue Syndrome, a copy can be viewed here –
    http://freespace.virgin.net/david.axford/JCFS.pdf

Leave a Reply