Jason Yamada-Hanff writes:
I’m a Neuroscience PhD reforming my statistics education.
I am a little confused about how you treat confidence intervals in the book and was hoping you could clear things up for me. Through your blog, I found Richard Morey’s paper (and further readings) about confidence interval interpretations. If I understand correctly, the main idea is that it is not clear that anything can be said about any given CI—that is, interpreting an individual CI is a mistake because all of the relevant properties are in the long-range behavior of the CI-generating procedure, not the CIs themselves. Andrew has written various posts addressing the common misinterpretations.
What I don’t quite get is that there are various recommendations in your regression book [with Jennifer Hill] that do seem to advocate interpreting individual CIs—indeed, what else is possible if you decide to calculate one? For example, you outline how to generate CIs in the early part of the book and routinely judge whether a given regression coefficient indicates an effect by looking at whether +/-2*SE crosses 0.
How do I align these interpretations with the scolding from Morey and you to not interpret CIs that way? I imagine I’m just missing some subtlety, but I can’t see it. It wouldn’t be the first time in stats! Perhaps the coefficient estimate/standard errors can be thought of as Bayesian posteriors, even though they aren’t calculated that way?
I replied: Jennifer and I are working on the 2nd edition of our book (volume 1 scheduled to come out at end of 2017, volume 2 in 2018) and we are removing all references to statistical signficance. Also we’re discussing a bit the role of prior distributions and the ways in which conf intervals can be misinterpreted. Our thinking has changed a lot in the past 10 yrs (and I take a lot of the blame for the stat siginf stuff in our book, as Jennifer was always skeptical).
Jason then asked:
Just so I understand you correctly, you are basically disclaiming the interpretation of CIs in this first edition, yes?
Without giving away the farm on the second edition, is there a quick version of a good way to interpret the standard errors? As the standard deviation of the distribution for the coefficient estimate, the standard error does seem to be some sort of a measure of precision… although what sort is not so clear.
And I replied: s.e. represents variation in the point estimate, but conf interval has usual Bayesian interpretation only with flat prior.