Skip to content
 

You can read two versions of this review essay on systemic exertion intolerance disease (chronic fatigue syndrome)

Julie Rehmeyer wrote a book, “Through the Shadowlands: A Science Writer’s Odyssey into an Illness Science Doesn’t Understand,” and my review appeared in the online New Yorker, much shortened and edited, and given the title, “A memoir of chronic fatigue illustrates the failures of medical research.” My original was titled, “Systemic exertion intolerance disease: The Human cost and the battle over the science.”

The New Yorker editors are pros, so I can only assume their version is better than mine. But here on this blog there are no space limitations—pixels are free—and so I thought I’d share the original, which I completed two months ago:

1. “Through the Shadowlands: A Science Writer’s Odyssey Into an Illness that Science Doesn’t Understand”

Julie Rehmeyer was a science journalist leading an active, outdoorsy life in New Mexico when, over the period of a few months, she lost most of her strength, endurance, and confidence, along with the ability to live a normal life. This all happened over fifteen years ago, and her new book, Through the Shadowlands, chronicles her struggles since then, her search for a cure or at least some amelioration of her condition, and the steps she went through to get to this point.

Spoiler: As of the book’s publication in 2017, Rehmeyer currently has a mild form of her condition, enabling her to go about her days but but requiring an extraordinary level of care and caution. There are many twists and turns in the story, and the book is structured around the theories, relationships, and physical possessions that she needed to discard to get from there to here.

In a way, this is the story of your life and mine–if we get lucky enough to live it all the way through. We become weaker as we age, at times gradually and at times suddenly, you don’t know what you’ve got till it’s gone. But along the way we adapt ourselves to fit our diminished capabilities and we adapt our environment to fit us; we make new connections and struggle to make sense of the world, to clear the trail for others. Rehmeyer’s struggles with her health and well being gave an urgency to the search for understanding of self and nature that we all must undertake in our lives.

Rehmeyer’s physical condition, which she shares with an estimated million other Americans, is called chronic fatigue syndrome, a term that unfortunately doesn’t explain much of anything: its definition from the Mayo Clinic, for example, is “a complicated disorder characterized by extreme fatigue that can’t be explained by any underlying medical condition” and has also been called myalgic encephalomyelitis and, more recently, systemic exertion intolerance disease.” This last phrase seems to me to be the most descriptive, so I will use it in the subsequent discussion. A particularly frustrating aspect of the disease is that the usual principles do not apply: the seemingly sensible plan of gradually ramping up exercise level can lead not to a gradual rebuilding of strength but to a collapse. The strategy that would be most natural for a physically active person recovering from an injury or an illness turns out to be counterproductive, and no treatment has been found to be generally effective.

Here is the outline of the book’s narrative: Following her physical breakdown, Rehmeyer quickly went through the usual steps of conventional and alternative medicine, but her condition continued to worsen. Somewhere along the line she moved to a different state, struggled to keep up with her work, and broke up with a partner who was not willing or able to be with her in sickness or in health. She eventually got in touch with a online community that recommended avoidance of even the smallest exposure to mold. Following this severe regimen, which required her to get rid of her house and much of its contents, she cobbled together a functional, if wary, life. More recently she trained herself to reduce her sensitivity to low concentrations of mold, but she remains subject to exposures that can suddenly weaken her. At the same time, however, Rehmeyer has struggled to not only understand her world but to change it, and along with this effort came new friends, a new partner, and an involvement with the scientific literature and controversy on exertion intolerance disease.

Rather than retelling each episode of this story, I will give some quotes that illustrate some episodes in the external events and in the development of the narrator’s ideas.

“Although I [Rehmeyer] certainly wasn’t a Christian Scientist as my mother was, her religion had fueled the belief in me that diligent attention to one’s internal feelings and attitudes was the best place to start in solving a problem, allowing one to act more powerfully in the world and sometimes opening the doors to change in ways you couldn’t predict.”

“Clearly, something had been going wrong inside my body, and medicine’s failure to understand what that something was didn’t change that fundamental reality. . . . But without an explanation, the illness felt a bit like Schrodinger’s cat, neither dead nor alive, neither physical nor psychological–and yet rich with possibility.”

“When my mother first died, I’d felt like the sole survivor of a catastrophe that had wiped out not just her, but my entire culture, as if I were an ancient Indian who was the only remaining speaker of my native language.”

“When I read the scientific literature, though, it felt like tissue paper to me, the findings fragile, with great gaps between them. There were thousands of studies out there, may of which identified abnormalities, but the abnormalities seemed barely related, and the studies were generally so small I didn’t trust them any way.”

“I was struck by key role a ‘good story’ had in my decision-making. . . . what made it a good story for me was that it cast me as adventurer rather than victim . . . I also noticed that when I contemplated this expedition to the desert, a sort of current ran through me, an energy that felt different from personal excitement.”

At this point, I couldn’t get the image from Breaking Bad out of my mind, that image of Walter White in that camper out in the desert.

“When Jones described the state’s effort to save the devils, I felt a pang of envy. Tasmania was spending many millions of dollars every year . . . If the CDC had responded as forcefully to the outbreak of ME/CFS in the 80’s, would I have been all alone in dealing with this damn illness? I wondered how much the Tasmanian devils were helped by the vulnerability of those poteroos and bandicoots and bettongs . . . maybe what we needed was a Looney Toons character based on us . . . Taz might have few similarities to real devils, but he’d certainly created worldwide awareness of them.”

And then there was the mysterious swim cure, what Rehmeyer describes as “a reliable miracle.” Early on in her story, when it seemed that nothing could stop her life from falling apart, Rehmeyer found that when she was at her weakest, she could jump in the pool, will her body to swim, and her weakness would go away for the rest of the day—“but it only worked if I went in the middle of an episode, not beforehand.” Beyond the practical challenges that this entailed, it also frustrated Rehmeyer at an intellectual level, as she could never find any explanation for why the swim cure worked, even to this day, and it remains a loose end in her story.

Rehmeyer’s progress was far from linear: there was a period for a year or so where her condition nearly disappeared, only to return more severely than before. Her rigorous strategy of avoiding all mold gave her some control over her symptoms, but then eventually she was able to handle greater exposures. She concludes by writing, “I’m doing very well, but I’m also not cured. . . . At times, I’m close to 100% well, but other times are more difficult. I’m able to live a full life. . . . I got lucky.” She adds: “A number of ME/CFS patients have tried mold avoidance, influenced by me, and their results have been mixed.”

While focused on Rehmeyer’s own experiences, the book touches in many places on the larger questions of the diagnosis and treatment of systemic exertion intolerance disease, and I’ll return to some of these issues in a moment.

But first, what makes this book so appealing and the underlying story so thought-provoking? I see two factors.

First, Rehmeyer is the hero of her own story: literally knocked off her feet by a still-mysterious condition, she needed to elicit from others the help that she required, working through theories of what was going on. Full of life, curiosity, and ingenuity, this the book that Richard Feynman would’ve written, had he been afflicted with a disease that the scientific establishment couldn’t understand.

Second, systemic exertion intolerance disease is a medical mystery affecting a million or more people, with a corresponding scientific controversy. Can psychiatric treatment help sufferers of systemic exertion intolerance disease? If so, does this mean the condition is “all in the head”? To what extent can we trust conclusions published in medical journals? I have argued above that some of these controversies arise from thinking in black-and-white terms and not recognizing variation in the condition and its treatment, but in any case this has become a fascinating story of a struggle between patient groups and the medical establishment, and also within a scientific community that has often had difficulty balancing the goals of publication, certainty, and snappy conclusions with real-world uncertainty and variation.

2. Systemic exertion intolerance disease and the controversy over the Pace trial

As Rehmeyer so powerfully demonstrates in her book, there is no known cure for systemic exertion intolerance disease. Indeed, there is not even any generally effective therapy for ameliorating the condition. But there are a lot of theories floating around, and some of them have been incorporated into interventions that have been tested in clinical trials.

Most prominent of these studies is the Pace trial, an experiment conducted in 2005 on 641 patients in the United Kingdom who were randomly assigned to four treatments: adaptive pacing therapy, cognitive behavioral therapy, graded exertise therapy, or nothing beyond medical care. One reason for the influence of the study is that the U.K., through its National Health Service, must make decisions about what treatments to approve and fund, and controlled clinical trials are generally taken to offer the best evidence.

There has been much dispute about the results of the Pace trial, along with a struggle over data availability. Before getting to the controversy, I will summarize the findings as presented in the original paper on the study, published in the English medical journal The Lancet in 2011. This article, bylined by a team of 19 authors led by Peter White, reported results after 12 weeks, 24 weeks, and one year of followup, which were encouraging for two of the four treatments in the study:

– Cognitive behavior therapy (“This theory regards chronic fatigue syndrome as being reversible and that cognitive responses (fear of engaging in activity) and behavioural responses (avoidance of activity) are linked and interact with physiological processes to perpetuate fatigue.”), and

– Graded exercise therapy (“GET was done on the basis of deconditioning and exercise intolerance theories of chronic fatigue syndrome. These theories assume that the syndrome is perpetuated by reversible physiological changes of deconditioning and avoidance of activity. . . . The aim of treatment was to help the participant gradually return to appropriate physical activities, reverse the deconditioning, and thereby reduce fatigue and disability.”).

The results were less positive for the other two treatments that were considered:

– Adaptive pacing therapy (“APT was based on the envelope theory of chronic fatigue syndrome. This theory regards chronic fatigue syndrome as an organic disease process that is not reversible by changes in behaviour and which results in a reduced and finite amount (envelope) of available energy. . . . This adaptation was achieved by helping the participant to plan and pace activity to reduce or avoid fatigue, achieve prioritised activities and provide the best conditions for natural recovery.”), and

– Specialist medical care (“SMC was provided by doctors with specialist experience in chronic fatigue syndrome. . . . Treatment consisted of an explanation of chronic fatigue syndrome, generic advice, such as to avoid extremes of activity and rest, specific advice on self-help, according to the particular approach chosen by the participant (if receiving SMC alone), and symptomatic pharmacotherapy (especially for insomnia, pain, and mood).”) SMC was given to all participants in the study, hence it can be considered a control condition in this experiment.

The major outcomes presented were based on self-reported fatigue and physical function scores. Fatigue was based on a questionnaire which put people on a 0-33 scale with patients’ average initial scores of around 28. This score dropped during the year (that is, a reduction in fatigue) by an average of 7.4 and 7.6 points under CBT and GET, the two more effective treatments, but only 5.4 and 4.5 points under APT and SMC. Physical function was based on a questionnaire which put people on a 0-100 scale with patients’ average initial scores of around 38. This score increased during the year (an increase in physical function) by 19.2 and 21.0 under CBT and GET, but only 8.7 and 11.6 under APT and SMC. Improvements of 2 points in the fatigue score or 8 points in the physical function score were considered “clinically useful,” hence the authors reported that CBT and GET were effective, while APT was no more effective than the control (that is, APT plus SMC was no better than SMC alone).

On the other hand, as Julie Rehmeyer wrote in a news article a couple years ago, “The study participants hadn’t significantly improved on any of the team’s chosen objective measures: They weren’t able to get back to work or get off welfare, they didn’t get more fit, and their ability to walk barely improved. Though the PACE researchers had chosen these measures at the start of the experiment, once they’d analyzed their data, they dismissed them as irrelevant or not objective after all.” Rehmeyer was suspicious of the subjective reports, imagining herself as a participant: “I come in and I’m asked to rate my symptoms. Then, I’m repeatedly told over a year of treatment that I need to pay less attention to my symptoms. Then I’m asked to rate my symptoms again. Mightn’t I say they’re a bit better—even if I still feel terrible–in order to do what I’m told, please my therapist, and convince myself I haven’t wasted a year’s effort?”

The 2011 Pace paper and its followups were much criticized among activist groups and among the larger scientific community, leading to a stern editorial and several responses in the Journal of Health Psychology. As summarized by Caroline Wilshire in that discussion, “Cognitive behavioural therapy and graded exercise therapy had modest, time-limited effects on self-report measures, but little effect on more objective measures such as fitness and employment status. Given that the trial was non-blinded, and the favoured treatments were promoted to participants as ‘highly effective’, these effects may reflect participant response bias.”

In 2015, a team of researchers led by biologist Ronald Davis, and including my Columbia biostatistics colleague Bruce Levin, wrote an open letter criticizing how the Pace team reported its results. Davis and his colleagues wrote: “In an accompanying Lancet commentary, colleagues of the PACE team defined participants who met these expansive ‘normal ranges’ as having achieved a ‘strict criterion for recovery.’ The PACE authors reviewed this commentary before publication.”

in later correspondence, the Pace authors wrote, “our paper did not report on recovery,” but they did use this term in other writings. For example, there’s a 2007 paper, “Is a full recovery possible after cognitive behavioural therapy for chronic fatigue syndrome?”, which reports: “Using the most comprehensive definition of recovery, 23% of the patients fully recovered.” And the aformentioned Lancet commentary, written by collaborators of the Pace team, stated, “the recovery rate of cognitive behaviour therapy and graded exercise therapy was about 30%.”

As a statistician, I don’t see much value in using hard thresholds, but if you are going to talk about recovery as a yes-or-no outcome, then it’s important to use consistent definitions. One criticism of the analysis of Pace trial is that the research team changed their definition of recovery after collecting their data, weakening it so much that patients could get worse on two of four criteria (which were the central measures–fatigue and physical function) over the course of he trial and still be considered “recovered.”

One difficulty in understanding all these comparisons is that changes are happening to individuals, but the published article reports only averages. This is one of many reasons why outsiders have requested the patient-level data from the experiment, and this has created its own controversy: when the experiments refuse to share the data, is it to protect patient confidentiality or simply to protect themselves from criticism?

I don’t have access to the raw data myself but I can make some inferences based on general statistical principles. The first is that there is always variation: some people get better, some people get worse, others improve on some measures and decline on others. In a setting such as systemic exertion intolerance disease where the treatments are so speculative and the condition itself is not clearly defined, we would expect any treatment to be more effective than control only on some subset of people.

Based on the treatment descriptions given above, it makes sense that each of the three options—CBT, GET, or APT—would help some sufferers of systemic exertion intolerance disease and not others. Beyond that, patients can get much better or much worse because of unforeseen other factors, as happened various times in Julie Rehmeyer’s story.

To get an idea of what we’re struggling with, here’s an unhelpful quote from a radio interview with Lancet editor Richard Horton: “adaptive pacing therapy essentially believes that chronic fatigue is an organic disease which is not reversible by changes in behaviour. Whereas cognitive behaviour therapy obviously believes that chronic fatigue is entirely reversible. And these two philosophies are kind of facing off against one another in the patient community and what these scientists were trying to do is to say, ‘Well, let’s see. Which one is right?'”

The problem with this attitude is that systemic exertion intolerance disease, or ME/CFS, is a diverse condition, or set of conditions lumped under a common diagnosis. It is completely reasonable to think that different therapies could work for different people, and that the condition has different sources for different people. Even for any particular person, the condition could have a mix of causes and be amenable to a mix of therapies. So the attitude that it’s one or the other, can be a serious mistake even for a given patient, let alone when trying to characterize a broadly-diagnosed syndrome in the general population.

It should be no surprise that cognitive behavioral therapy and exercise therapy can help people. The success of these therapies for some percentage of people, does not at all contradict the idea that many others need a lot more, nor does it provide much support for the idea that “fear avoidance beliefs” are holding back those people with systemic exertion intolerance disease who, like Rehmeyer, were in no condition to increase their exercise level.

As Simon Wessely, one of the supporters of the Pace trial and a pillar of the English medical establishment, writes, “there were a significant number of patients who did not improve with these treatments. Some patients deteriorated, but this seems to be the nature of the illness, rather than related to a particular treatment. . . . PACE or no PACE, we need more research to provide treatments for those who do not respond to presently available treatments.”

I was bummed that the New Yorker editor cut the line about “Breaking Bad” and other parts of my book review—but, yeah, yeah, I know, “kill your darlings.” On the plus side, the editor, Sharan Shetty, pushed me to end more clearly with a discussion of the bigger picture of the larger implications for medical research. i told Shetty that, following academic rules, I’d like to add him as coauthor but he said, no, that’s not how they do things in the world of general-interest publishing. So I’ll thank him here.

P.S. Gary Greenberg points out an error in my New Yorker article, which states, “Traditionally, the controlled trial has been considered the gold standard of medical evidence: you gather a bunch of patients and randomly assign some to a control group, which receives no treatment, and some to an experimental group, which does.”

That’s not quite right. “Control” can include a placebo or indeed any alternative treatment, so it was not correct to define “control” as “no treatment.”

15 Comments

  1. Andrew, I really appreciate your wondering into the fray. Personally, I find the PACE investigators get ugly and retaliatory when criticized. I requested their data that was promised to be available as a condition for publishing in PLOS One. 18 months later I have not gotten the data, only an Expression of Concern posted by PLOS One. But the PACE investigators have succeeded in getting me from commenting on their trial at my popular PLOS blog,Mind the Brain, because, they claim it would be a conflict of interest.

    I am writing here though, because your article in the NYer states “For conditions like S.E.I.D., then, the better approach may be to gather data from people suffering “in the wild,” combining the careful methodology of a study like pace with the lived experience of thousands of people. Though most may be less eloquent than Rehmeyer, each may have his or her own potential path to recovery.”

    Did you yourself write that? The PACE is a miserably poor study in its design and execution. Although the original protocol describes the study as a “randomized controlled trial.” subsequent publications dropped the “controlled” because it is unclear what the comparison group controls. There is a marked imbalance with the CBT and GET conditions in terms of all the nonspecific {placebo) factors being loaded in the active treatment groups, not the control group. And then there is the business of the switched scoring of subjective self-report primary outcomes and suppressed objective primary outcomes.

    best

    • Andrew says:

      James:

      The above post has my version of the article; I just did a search and did not find the word “careful.” So I suspect the phrase “careful methodology of a study like pace” did not come from me. I can’t be sure, though, as the article went through a bunch of edits. It doesn’t sound like something I’d say, but who knows. I don’t really know enough about the Pace study to characterize it as careful or not.

      • I assumed so. After all, I learned a lot from the devastating critique here at your blog of the meditiational analysis the PACE investigators did.

        • Andrew says:

          James:

          I looked through my files and found an intermediate version with this phrase: “careful data collection and protocol of a study like Pace.” So that part seems to have come from me. I’m guessing that it was the New Yorker editor who changed “data collection and protocol” to “methodology” in that sentence.

          So I guess I was ok with characterizing the data collection and protocol of the Pace study as “careful.” And I can see how an editor could’ve changed that to “methodology” without realizing the implications.

  2. Phil says:

    This is only thematically related but I’ll say it anyway: some people might enjoy “The Sound of a Wild Snail Eating”, as I did. Here’s a review. The author of TSOAWSE didn’t have Systemic Exertion Intolerance Disease, but she did have a disease that made it impossible for her to work or walk or, for some long stretches, even to sit upright. A friend gave her a snail in a small terrarium, and she got interested in what it did in its little world; she gives it a lot of credit for helping her cope with her debilitating illness.

  3. mia says:

    There’s another thing that’s interesting about that Horton quote: “adaptive pacing therapy essentially believes that chronic fatigue is an organic disease which is not reversible by changes in behaviour. Whereas cognitive behaviour therapy obviously believes that chronic fatigue is entirely reversible. And these two philosophies are kind of facing off against one another in the patient community and what these scientists were trying to do is to say, ‘Well, let’s see. Which one is right?’”

    In the Lancet commentary which accompanied PACE, it was claimed that “PACE used a strict criterion for recovery: a score on both fatigue and physical function within the range of the mean plus (or minus) one standard deviation of a healthy person’s score. In accordance with this criterion, the recovery rate of cognitive behaviour therapy and graded exercise therapy was about 30%”. This ‘strict criterion’ was actually so loose that trial participants could start the trial being classed as suffering from severe and disabling fatigue, report a worsening for every outcome during the trial, and yet still fulfil this ‘strict’ criteria for ‘recovery’ at the end of the trial.

    In contrast, the Wilshire reanalysis which used the PACE protocol criteria for ‘recovery’ found that the addition of neither CBT nor GET led to significant improvements in recovery rates (with all groups reporting recovery rates of under 7%): http://www.tandfonline.com/doi/abs/10.1080/21641846.2017.1259724?journalCode=rftg20

    (The Wilshire paper also explains how the two outcome measures used to define ‘recovery’ in the Lancet commentary, the Chalder fatigue scale and SF36-PF, can overlap for ‘recovery’ and ‘severe and disabling fatigue’ within the PACE trial.)

    So for what Horton seems to consider to be the key issue to be addressed by this expensive and controversial piece of research, he has managed to mislead himself, and anyone else who trusts what they read in the Lancet. Horton never seemed to understand the PACE trial, or why so many patients were concerned about it, but instead just seemed to enjoy posing as a valiant defender of rigorous research against ‘anti-science’ patients. eg: https://web.archive.org/web/20170711034848/https://twitter.com/richardhorton1/status/38371958055772160

    It is difficult to think of examples of the group which is the subject of research being so condemned for raising concerns about the quality of the research itself. Gay rights activist Peter Tatchell has commented that: “Attempt to stigmatise ME/CFS #PACEtrial campaigners reminds me of when I protested aversion therapy for LGBTs” https://twitter.com/PeterTatchell/status/772035205695672320

    So Horton has tied his own career and reputation to the PACE trial. It’s difficult to see how he could now acknowledge the problems with this study without having to resign. It is not a surprise that he is refusing to publish critical letters from independent researchers: http://www.virology.ws/2016/08/29/once-again-lancet-stumbles-on-pace/

  4. Erik Johnson says:

    I’m the guy who took Julie on “The CFS History Mold Tour” (page 155)

    I’d like to make a couple of quick points.

    1. The mold phenomenon at “Ground Zero for CFS” has never been investigated by any CFS researcher.
    2. The CFS syndrome was coined due to immune abnormalities. Not for tired people or to study symptoms.

    As I so annoyingly told Julie, if anyone wants to understand this syndrome, they must revisit the incident and evidence that started it.

    (And isn’t “Check your premise – Start at the beginning” the way science used to normally be done?)

    • Keith O'Rourke says:

      > way science used to normally be done?

      The way science is normally done is highly problematic and often harmful – how science ought to be done (which everyone struggles with as its open ended and subtle), if it was, would be helpful in the long run.

      Horton does seem to point out this instance of highly problematic and often harmful way of doing science (unintentionally?) – “these two philosophies are kind of facing off against one another in the patient community”

      I once used a quote from General Patton to put the sad position of patients with conditions currently being researched to students at Duke as part of research ethics “You don’t want to be in a clinical trail, you want other poor bastards to have been in trialS earlier – so they know what to do for you.”

  5. michael allen, phd says:

    This is an improvement on what the NYer published but I still have major problems. 1. Gellman holds you up as a rather heroic figure for tackling your disease and eventually improving. This makes the rest of us franky look like losers. I’ve had ME for 25 years and have tried just about everything and it’s infuriating when the illness is covered in this way. 2. He buys into the possibility that CBT and GET might help some. Does he realize that CBT is to rid one of false illness beliefs? This scientific incoherence by scientists must stop. It’s a multisystemic disease. Period. Stop. There’s is nothing psychological about it and any attempt to even suggest this is scientific malfeasance. (I’m a Phd clinical psychologist). The name systemic intolerance is just wrong and misguided. It’s a repeat of the “fatigue” mistake, i.e. defining a disease by one symptom. The ICC has well defined diagnostic for Myalgic Encephalomyelitis. The medical establishment has a vested interest in not accepting this because it would be forced to admit a 35 year mistake. I am furious because I’m still waiting for a major national publication print an in depth article by someone who actually knows something about it.

  6. Having read “Through the Shadowlands,” I was surprised to see a review in the New Yorker focused so thoroughly on the PACE trial, which only takes up one chapter of the memoir. To me the book was about life — how and why we survive and struggle and find meaning — and science plays a part in that life, but can’t be expected to fulfill or explain all aspects of our existence. That said, I think it’s wonderful to have people talking about these essentially fake trials. Anyone who’s ever been a guinea pig in a lab (I have) or known people who go out of their way to become human lab rats (I knew a particularly odious one) can attest that this great, important, authoritative, peer-reviewed thing we colloquially refer to when we say “SCIENCE” knows that the process is rife with human error and manipulation. We should not be shocked to discover that many peer-reviewed studies are problematic or just plain false.

  7. zbicyclist says:

    Of course, misdiagnosis of CFS is common. A co-worker suffered from “CFS” for years, even dropping out of his PhD program. Then he had a child, who failed to thrive until the child was diagnosed as being allergic to gluten. Viola! He also gave up all gluten — his “CFS” was really a misdiagnosed gluten allergy.

    Clarifying: I am NOT saying all CFS is something else, just that this condition is mysterious and may not be one thing. This obviously complicates diagnosis and research.

  8. Trevor Butterworth says:

    It’s not about killing your darlings, but about the editors’ perceived view of their readers appetite for detail relative to the space available and the need for concision and eloquence. I think it’s fantastic that the New Yorker is even writing about statistical issues in clinical trials—that’s a big step forward!

  9. Julie Rehmeyer says:

    Hi Andrew,

    Thanks so much for this review! Three points:

    1. There was an important error in the New Yorker version. It says:
    One of their chief objections is that the pace research team, while collecting its data, changed the main metric of “recovery” from objective measures, such as fitness and employment status, to subjective ones, such as the patients’ self-evaluations over time.

    The primary outcomes were always the subjective ones, so this isn’t accurate. However, they weakened their thresholds on the subjective measures for “recovery” (as well as for every other benchmark in the trial) enormously — so much so that (as you point out in your longer version) patients could get worse on fatigue and physical function and still be considered “recovered.” When the objective measures didn’t improve, the researchers dismissed them, even though they themselves had chosen them.

    2. You make the point that RCTs tell you less when you’re looking at heterogeneous groups. But in the PACE trial, the researchers chose a broad definition of ME/CFS that exacerbated this problem, including a far more diverse population than was necessary or appropriate.

    While it’s true that ME/CFS (or SEID) may well be a cluster of related illnesses, it’s also the case that definition really matters. The Mayo Clinic definition that you offer is pretty much the worst one out there — it negates itself, giving a medical explanation by demanding that there is no medical explanation. The one the PACE trial used isn’t quite that bad, but it’s close, requiring only six months of disabling fatigue as the primary symptom. That can include a significant percentage of depressed people, which is most likely part of why the treatments appeared effective — increasing activity is a good treatment for depression. And that definition doesn’t require the hallmark symptom of the disease, exercise intolerance (as the term SEID emphasizes).

    Though you se the term SEID, you didn’t use the SEID definition, which requires exercise intolerance (the hallmark of the disease) along with fatigue, unrefreshing sleep, and either cognitive problems or difficulties regulating heart rate or blood pressure when standing. The SEID definition is designed to be simple for clinicians to use and able to catch all patients with the disease, at the cost of including some others. That’s useful for clinical practice, but not so useful for research. The best current definition for research (IMO) is the Canadian Consensus Criteria, which is more restrictive and also more complex: http://me-pedia.org/wiki/Canadian_Consensus_Criteria. It adds requirements for pain and neuroimmune symptoms.

    Even among patients who meet the Canadian Consensus Criteria, there is likely heterogeneity, but far, far less so than simply looking at really tired folks.

    The PACE investigators and their intellectual heirs persist in using very broad definitions to this day.

    3. Finally, I just wanted to point out that actually you do have access to a chunk of the data. Patients managed to get the data relevant to evaluating recovery by the original defintion, which they made publicly available here: https://sites.google.com/site/pacefoir/pace-ipd_foia-qmul-2014-f73.xlsx?attredirects=0.

    The data showed that accordingly to the original definition of recovery, the researchers had null results. That is, neither CBT nor GET led a statistically significant number of patients to “recover.”

  10. N says:

    “I come in and I’m asked to rate my symptoms. Then, I’m repeatedly told over a year of treatment that I need to pay less attention to my symptoms. Then I’m asked to rate my symptoms again. Mightn’t I say they’re a bit better—even if I still feel terrible–in order to do what I’m told, please my therapist, and convince myself I haven’t wasted a year’s effort?”

    I always wondered how and how serious psychology research treats the issue of social desirability bias, especially when therapeutic approaches like CBT apply lots of pressure on the patient to agree with the therapy’s premises.

Leave a Reply