Rink Hoekstra writes:
A couple of months ago, you were visiting the University of Groningen, and after the talk you gave there I spoke briefly with you about a study that I conducted with Richard Morey, Jeff Rouder and Eric-Jan Wagenmakers. In the study, we found that researchers’ knowledge of how to interpret a confidence interval (CI), was almost as limited as the knowledge of students who had had no inferential statistics course yet. Our manuscript was recently accepted for publication in Psychonomic Bulletin & Review, and it’s now available online (see e.g., here). Maybe it’s interesting to discuss on your blog, especially since CIs are often promoted (for example in the new guidelines ofPsychological Science), but apparently researchers seem to have little idea how to interpret them. Given that the confidence percentage of a CI tells something about the procedure rather than about the data at hand, this might be understandable, but, according to us, it’s problematic nevertheless.
I replied that I agree that conf intervals are overrated, a point I think I discussed briefly here. We used to all go around saying that all would be ok if people just ditched their p-values and replaced them with intervals. But, from a Bayesian perspective, the problem is not with the inferential summary (central intervals vs. tail-area probabilities) but with those default flat priors, which are particularly problematic in “Psychological Science”-style research where effect sizes are small and estimates are noisy.