I know Stephen agrees with this point “once we have N[studies]=2 we can fit a hierarchical model but we’ll need strong prior info on the between-study variance parameter” as I ran that by him when I was putting this post together https://andrewgelman.com/2017/10/05/missing-will-paper-likely-lead-researchers-think/ in particular this point “its the banning of informative priors altogether – forcing there to be a discrete decision to either completely ignore or fully incorporate very noisy between study variation”.

As for patients instead of studies, I also think he would agree but I should re-read his work on this e.g. Understanding Variation in Sets of N-of-1 Trials http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0167167

I do think there has been a blind spot to the opportunities of learning from multiple N of 1 trails that might be overcome with secret Bayesian thinking. To actually get anywhere in particular application ares (in places I have been waiting 20 years) anti-Bayesian sentiments have to die out.

]]>can you help me with grant money? ]]>

Consider a simple study with N/2 people in the treatment group and N/2 in the control, for simplicity a binary outcome with probabilities near 0.5. Then the estimated treatment effect is y_bar_1 – y_bar_2, and its standard error is approximately sqrt(0.5^2/(N/2) + (0.5^2)/(N/2)) = sqrt((0.5)^2*4/N) = 1/sqrt(N).

Now suppose you’re studying an interaction, comparing two groups which are evenly split in the sample. The estimated interaction is y_bar_1a – y_bar_1b – (y_bar_2a – y_bar_2b), and its standard error is approximately sqrt(0.5^2/(N/4) + (0.5^2)/(N/4) + 0.5^2/(N/4) + (0.5^2)/(N/4)) = sqrt((0.5^2)*16/N) = 2/sqrt(N).

The estimate of the interaction has twice the standard error of the estimate of the main effect.

]]>At about 36:40 Andrew starts to explain why the estimate of the interaction has twice the standard error of the main effect. Yes, I was one of the virtual n00bs who didn’t understand how/why sigma/sqrt(N) explains this. Many thanks in advance for any links.

]]>Sad.

]]>This is more a limitation of the data analysis method than of the general idea. Of course it’s possible to have models where you account for the effect of earlier treatments, but it requires more from the model.

]]>In my opinion, the ‘N of 1’ name is a misnomer. In rare diseases research, ‘N of 1 trials’ usually consist of more than 1 subject and inference is pooled over the small number of subjects. A better name might be repeated crossover where each subject receives trts multiple times in some random order. The limitations are similar to crossover trials – you need to have a stable, chronic disease which returns to baseline during the washout periods. A limitation of N of 1 trials for even these diseases is that patients often drop out when they notice something is ‘working’, especially if the test drug is on the market.

]]>The hostile reception that this topic got from some commenters surprised me. I can only guess their confidence to comment publicly is unconstrained by their limited knowledge of drug trial methodology. It would be interesting to hear what Stephen “secret Bayesian” Senn thinks.

]]>Yes, for sure.

]]>I never said “sophisticated”; that came from you. Beyond this, yes, if you do something on one person and estimate its effect, you’re doing an N=1 trial. If you do something on two people and estimate its effect, you’re doing an N=2 trial. That doesn’t mean it’s a *good* N=1 or N=2 trial, just that considering the study as a N=1 or 2 trial can be helpful framing of the problem, especially if you’re interested in generalizing to others in the general population.

Some chemotherapy is giving him a few more weeks (maybe months) of a good life.

The oncologist finds his response to the protocol very interesting and not quite as expected. Better in some clinical ways, puzzling when she looks at the blood chemistry. She modifies the protocol from week to week depending on what she sees (physical exam and lab results).

I’ve joked with her that she may get a paper out of this, as well as helping Pippin in his last days. Now I can tell her she’s doing an N-of-1 study.

]]>What I learned from my brief reading of the guide is that there is indeed a place for N of 1 trials and that they are amenable to statistical analysis that makes them different than RCT or clinical practice. What I don’t have a feel for is the extent to which the scope for such studies is large or small. I do worry that we may see a swarm of power pose N of 1 studies – eminently publishable since it has this new headline grabbing jargon. It would be nice to have a sense of just how applicable the technique is to real world situations.

]]>The purpose of an N-of-1 trial is to identify the most effective treatment for the 1 patient in the trial. There is no attempt to generalize the results to any other patient. If you omit the person-level variance component, you are simply absorbing its value for this particular patient into the constant term of the model. If you include a person-level variance component with a prior distribution you are giving yourself the apparent ability to infer what might happen if other patients were in the trial–but that is beside the point of the trial.

What am I missing here?

]]>