How to design future studies of systemic exercise intolerance disease (chronic fatigue syndrome)?

Someone named Ramsey writes on behalf of a self-managed support community of 100+ systemic exercise intolerance disease (SEID) patients. He read my recent article on the topic and had a question regarding the following excerpt:

For conditions like S.E.I.D., then, the better approach may be to gather data from people suffering “in the wild,” combining the careful methodology of a study like PACE with the lived experience of thousands of people. Though most may be less eloquent than Rehmeyer, each may have his or her own potential path to recovery.

Ramsey asks:

From your perspective, are there particular design features to such an approach that one should prioritize, in order to maximize its usefulness to others?

Here’s the challenge.

The current standard model of evaluating medical research is the randomized clinical trial with 100 or so patients. This sort of trial is both too large and too small (see also here): too large in there is so much variation in the population of patients, and different treatments will work (or not work, or even be counterproductive) for different people; too small in that the variation in such studies makes it hard to find reliable, reproducible results.

I think we need to move in two directions at once. From one direction, N=1 experiments: careful scientific evaluations of treatment options adapted to individual people. From the other direction, full population studies, tracking what really is happening outside the lab. The challenge there, as Ramsey notes, is that a lot of uncontrolled information is and will be available.

I’m sorry to say that I don’t have any good advice right now on how future studies should proceed. Speaking generally, I think it’s important to measure exactly what’s being done by the doctor and patient at all times, I think you should think carefully about outcome measures, and I think it’s a good idea to try multiple treatments on individual patients (that is, to perform within-person comparisons, also called crossover trials in this context). And, when considering observational studies (that is, comparisons based on existing treatments), gather whatever pre-treatment information that is predictive of individuals’ choice of treatment regimen to follow. For SEID in particular, it seems that the diversity of the condition is a key part of the story and so it would be good to find treatments that work with well-defined subgroups.

I hope others can participate in this discussion.

9 thoughts on “How to design future studies of systemic exercise intolerance disease (chronic fatigue syndrome)?

  1. For starters, I assume you’re only talking about lifestyle change treatments and not recommending that experimental drugs be released into the wild and studied observationally instead of in a clinical trial setting?

    • Z:

      It’s not really up to me. I don’t see a sharp dividing line between “lifestyle change treatments” and “experimental drugs.” So let me just say that I’m talking about evaluation of whatever treatments people might be trying, without addressing issues of approvals of new drugs.

  2. I don’t see clinical trials going away any time soon (nor do I think they should stop). But there is a desperate need for large scale (population) observational data. It would need to be anonymized, but that should to pose too much of a problem. But clinicians are faced today with two extremes: careful (but not necessarily correct!) trial data and their own personal anectdotal evidence. What is not readily available is the cumulative experience of the thousands of clinicians. For a particular disease, wouldn’t it be nice to know how many people with a given set of characteristics have the disease, what treatments they have received, and what the outcomes were? The potential for misinterpretation is great – but not any greater than the current system which relies totally on published research (and we know about the problems with that) and personal experience.

    There are no technical hurdles to the collection and dissemination of this information. It is a human-invented problem. The US health care industry and “competitive” insurance industry is really the roadblock. And HIPPA has become a legal shield that does more to protect the industry than to protect individuals.

      • Thanks, Keith. This is indeed a step in the right direction and a valuable resource in its own right. I think it underscores my point about the primary obstacle not being technological, but human-caused. If you look at where the Sentinel data is coming from, it is currently from some (albeit large) insurers. It is also limited to things that fall under the FDA’s purview. Including all insurers and all conditions, with outcome data, would make this immensely more useful. I believe that is the goal, but getting there in the US fragmented insurance/provider health care market will be more difficult (note that it took 8 years to get this Sentinel system working) than it would in a single payer system. The single payer system comes with its own set of problems – which I don’t want to minimize – but I think is the only way that such population data is likely to be assembled and disseminated.

  3. I’m not sure I understand this sentence:

    “And, when considering observational studies (that is, comparisons based on existing treatments), gather whatever pre-treatment information that is predictive of individuals’ choice of treatment regimen to follow.”

    I read this in two ways, each of which seems more applicable to the large population studies rather than N=1.

    1) By “individuals’ choice of treatment regimen to follow”, you mean the treatment they stick with, possibly after trying several others, having found that it works for them (at least, better than the alternatives). Naturally, you want to see how efficacy correlates with pre-treatment information that could be analyzed for future patients in order to recommend a treatment for them.

    2) You want to deal with selection bias: maybe a particular subgroup of the patient population seems to respond better to treatment, but it’s mostly because they tend to ask for or receive a treatment that is effective for them more often that other subgroups do.

    Which one of these is closer to the mark?

  4. It seems the argument is that (a) the optimal treatment is patient-specific, and (b) that this can be known in some way, either through preferences that could be elicited ex ante or through trial and error , or both. If this is the case, it seems that the trials should be evaluating approaches to providing interventions, rather than the interventions themselves. Thus, the trial arms would be defined by the menu of options available, how these are presented to the patient, and guidance on switching rules. The impact of any individual intervention can no longer be separated out, but that is okay if it varies so much at the individual level.

  5. At the very least, average treatment effect estimates and standard errors from clinical trials should provide fairly tight informative priors for the mean about which subgroup effects vary in a hierarchical model fit to observational data.

Leave a Reply to Z Cancel reply

Your email address will not be published. Required fields are marked *