I was given the opportunity to briefly comment on the paper, A Bayesian approach to complex clinical diagnoses: a case-study in child abuse, by Nicky Best, Deborah Ashby, Frank Dunstan, David Foreman, and Neil McIntosh, for the Journal of the Royal Statistical Society. Here is what I wrote:
Best et al. are working on an important applied problem and I have no reason to doubt that their approach is a step forward beyond diagnostic criteria based on point estimation. An attempt at an accurate assessment of variation is important not just for statistical reasons but also because scientists have the duty to convey their uncertainty to the larger world. I am thinking, for example, of discredited claims such as that of the mathematician who claimed to predict divorces with 93% accuracy (Abraham, 2010).
Regarding the paper at hand, I thought I would try an experiment in comment-writing. My usual practice is to read the graphs and then go back and clarify any questions through the text. So, very quickly: I would prefer Figure 1 to be displayed in terms of standard deviations, not variances. I find variances difficult to interpret, and I’m always taking mental square roots (0.09 is 0.3 squared, and so forth). Figure 3 is appealing but I don’t like the visual emphasis of the endpoints of the 95% intervals. From a Bayesian standpoint, there is nothing special about the 2.5th and 97.5th percentiles of the posterior distribution, and I think it goes against the spirit of the article to emphasize these arbitrary endpoints. I also think that, with some care, the graphs in Figures 3, 4, and 5 could be compactly re-expressed to show comparisons more effectively (as in Gelman, Pasarica, and Dodhia, 2002). Tables 2 and 3 I think are useless: why should a reader care that the 10th percentile point of the distribution for a particular probability os 0.164 or whatever? Again, this seems to me to contradict the decision-analytic focus of the applied research.
These brusque comments on display may seem peripheral but to me they are important. Communication is a central task of statistics, and ideally a state-of-the-art data analysis can have state-of-the-art displays to match.
References
Abraham, Laurie (2010). Can you really predict the success of a marriage in 15 minutes? Slate, 8 March. http://www.slate.com/articles/double_x/doublex/2010/03/can_you_really_predict_the_success_of_a_marriage_in_15_minutes.html
Gelman, Andrew, Pasarica, C., and Dodhia, R. (2002). Let’s practice what we preach: turning tables into graphs. American Statistician 56, 121-130.
I don’t see a problem with marking intervals in Fig. 3, at least with that figure design. It’s hard to convert greyscales to numerical probabilities, so the 95% interval markers give some idea of the shape of the distribution. What’s the alternative? You could plot the full pdf curve for each estimate, but it might be hard to compress all those curves into one figure for comparison. Besides, even when I plot pdfs I like to put some interval markers for guidance below since it can be hard to interpret tail areas.