Hadley Wickham asks:

I was wondering if you knew of any good articles on the use of error bars. I’m particularly looking for articles that discuss the difference between error of means and error of difference in the context of models (e.g. mixed models) where they are very different. I suspect every applied field has a couple of good articles, but it’s really hard to search for them.

Can anyone help on this? My only advice is to get rid of those horrible crossbars at the ends of the error bars. The crossbars draw attention to the error bars’ endpoints, which are generally not important at all. See, for example, my Anova paper, for some examples of how I like error bars to look.

Marianne Zawitz and I wrote a report on this topic in 1998, when I was a visiting fellow at the Bureau of Justice Statistics. Titled "Displaying Violent Crime Trends…," it's at http://bjs.ojp.usdoj.gov/content/pub/pdf/dvctue.p…

How about this: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC20641…

Error bars in Experimental Biology, by Cumming, FIdler and Vaux

Hello,

maybe these references could help:

Afshartous, D. and Wolf, Michael (2007). Avoiding Data Snooping in Mixed and Multilevel Models, Journal of the Royal Statistical Society, Series A, vol. 170, 1035-1059.

Cumming, G. (2009). Inference by eye: Reading the overlap of independent confidence intervals, Statistics in Medicine, vol. 28(2), 205-220.

Jackson, C. (2008). Displaying Uncertainty with Shading, The American Statistician, vol. 62, 340-347.

Cheers

In some plots error bars of different estimates overlap. Without crossbars you can’t see where the intervals end. Of course, you might say that a plot where error bars overlap is wrong to begin with.

In some plots, error bars of different estimates overlap. Without crossbars you can't see then where the intervals end. Of course you might say that a plot in which error bars overlap is wrong to begin with.

A few more:

Goldstein, H., Healy, M.J.R., 1995. The graphical presentation of a collection of means. Journal of the Royal Statistical Society: Series A 158, 175–177.

Senn, S.J., Auclair, P., 1990. The graphical representation of clinical trials with particular reference to measurement over time. Statistics in Medicine 9,1287–1302.

Afshartous D, Preston RA (2010). “Confidence intervals for dependent data: equating nonoverlap with statistical significance," Computational Statistics and Data Analysis. 54(10): 2296-2305.

General references that may be of use or interest.

T.N. Thiele discussed error bars in 1889. See p.181 of Lauritzen's book on Thiele. Anyone know an earlier reference?

W.S. Cleveland's The elements of graphing data has a good discussion.

Harvey Motulsky's Intuitive biostatistics may also be useful.

We discuss the issue here:

Rouder, J. N. & Morey, R. D. (2005). Relational and arelational confidence intervals: A comment on Fidler et al. (2004). Psychological Science, 16, 77-79.

Hi all,

Thanks for the great suggestions. For the benefit of anyone who comes back to this page in the future, here’s the complete list of articles I’ve found so far:

senn:1990

The graphical representation of clinical trials with particular reference to measurements over time

Senn, S. and Auclair, P. and Johnson, S.

Statistics in Medicine

9

1287–1302

(1990)

sdf1995

The Graphical Presentation of a Collection of Means

Goldstein, H. and Healy, M. J. R.

Journal of the Royal Statistical Society. Series A (Statistics in Society)

158

175-177

(1995)

http://www.jstor.org/stable/2983411

When a study produces estimates for many units or categories a common problem is that end-users will wish to make their own comparisons among a subset of these units. This problem will occur, for example, when estimates of school performance are produced for all schools. The paper proposes a procedure, based on the graphical presentation of confidence intervals, which enables such comparisons to be carried out while maintaining an average required type I error rate.

maltz:1998

Displaying Violent Crime Trends Using Estimates from the National Crime Victimization Survey

Maltz, M. D. and Zawitz, M. W.

(1998)

http://bjs.ojp.usdoj.gov/content/pub/pdf/dvctue.pdf

schenker:2001

On Judging the Significance of Differences by Examining the Overlap Between Confidence Intervals

Schenker, N. and Gentleman, J. F.

The American Statistician

55

182-186

(2001)

masson:2003

Using confidence intervals for graphically based data interpretation.

Masson, M. E. J. and Loftus, G. R.

Canadian Journal of Experimental Psychology/Revue canadienne de psychologie experimentale

57

203

(2003)

payton:2003

Overlapping confidence intervals or standard error intervals: What do they mean in terms of statistical significance?

Payton, M. E. and Greenstone, M. H. and Schenker, N.

The Journal of Insect Science

3

(2003)

fidler:2005

Still Much to Learn About Confidence Intervals

Fidler, F. and Thomason, N. and Cumming, G. and Finch, S. and Leeman, J.

Psychological Science

16

494-495

(2005)

http://pss.sagepub.com/content/16/6/494.short

belia:2005

Researchers misunderstand confidence intervals and standard error bars.

Belia, S. and Fidler, F. and Williams, J. and Cumming, G.

Psychological Methods

10

389

(2005)

cumming:2005

Inference by Eye: Confidence Intervals and How to Read Pictures of Data.

Cumming, G. and Finch, S.

American Psychologist

60

170

(2005)

cumming:2007

Error bars in experimental biology

Cumming, G. and Fidler, F. and Vaux, D. L.

Journal of Cell Biology

177

7–11

(2007)

afshartous:2007

Avoiding “data snooping” in multilevel and mixed effects models

Afshartous, D. and Wolf, M.

Journal of the Royal Statistical Society: Series A (Statistics in Society)

170

1035–1059

(2007)

jackson:2008

Displaying Uncertainty With Shading

Jackson, C. H.

The American Statistician

62

(2008)

http://www.mrc-bsu.cam.ac.uk/personal/chris/papers/denstrip.pdf

cumming:2009

Inference by eye: Reading the overlap of independent confidence intervals

Cumming, G.

Statistics in Medicine

28

205–220

(2009)

http://dx.doi.org/10.1002/sim.3471

afshartous:2010

Confidence intervals for dependent data: Equating non-overlap with statistical significance

Afshartous, D. and Preston, R. A.

Computational Statistics & Data Analysis

54

2296 – 2305

(2010)

Hmmm, that didn’t format so well (what happened to the preview??). Here’s the bibtex:

@article{senn:1990,

Author = {Senn, SJ and Auclair, P. and Johnson, S.},

Journal = {Statistics in Medicine},

Number = {11},

Pages = {1287–1302},

Title = {The graphical representation of clinical trials with particular reference to measurements over time},

Volume = {9},

Year = {1990}}

@article{sdf1995,

Author = {Goldstein, Harvey and Healy, Michael J. R.},

Journal = {Journal of the Royal Statistical Society. Series A (Statistics in Society)},

Number = {1},

Pages = {175-177},

Title = {The Graphical Presentation of a Collection of Means},

Volume = {158},

Year = {1995}}

@techreport{maltz:1998,

Author = {Maltz, Michael D. and Zawitz, Marianne W.},

Institution = {Bureau of Justice Statistics},

Number = {167881},

Title = {Displaying Violent Crime Trends Using Estimates from the National Crime Victimization Survey},

Year = {1998}}

@article{schenker:2001,

Author = {Schenker, Nathaniel and Gentleman, Jane F},

Journal = {The American Statistician},

Number = {3},

Pages = {182-186},

Title = {On Judging the Significance of Differences by Examining the Overlap Between Confidence Intervals},

Volume = {55},

Year = {2001}}

@article{masson:2003,

Author = {Masson, M.E.J. and Loftus, G.R.},

Journal = {Canadian Journal of Experimental Psychology/Revue canadienne de psychologie experimentale},

Number = {3},

Pages = {203},

Title = {Using confidence intervals for graphically based data interpretation.},

Volume = {57},

Year = {2003}}

@article{payton:2003,

Author = {Payton, Mark E. and Greenstone, Matthew H. and Schenker, Nathaniel},

Journal = {The Journal of Insect Science},

Title = {Overlapping confidence intervals or standard error intervals: What do they mean in terms of statistical significance?},

Volume = {3},

Year = {2003}}

@article{fidler:2005,

Author = {Fidler, Fiona and Thomason, Neil and Cumming, Geoff and Finch, Sue and Leeman, Joanna},

Journal = {Psychological Science},

Number = {6},

Pages = {494-495},

Title = {Still Much to Learn About Confidence Intervals},

Volume = {16},

Year = {2005}}

@article{belia:2005,

Author = {Belia, S. and Fidler, F. and Williams, J. and Cumming, G.},

Journal = {Psychological Methods},

Number = {4},

Pages = {389},

Title = {Researchers misunderstand confidence intervals and standard error bars.},

Volume = {10},

Year = {2005}}

@article{cumming:2005,

Author = {Cumming, G. and Finch, S.},

Journal = {American Psychologist},

Number = {2},

Pages = {170},

Title = {Inference by Eye: Confidence Intervals and How to Read Pictures of Data.},

Volume = {60},

Year = {2005}}

@article{cumming:2007,

Author = {Cumming, Geoff and Fidler, Fiona and Vaux, David L.},

Journal = {Journal of Cell Biology},

Number = {1},

Pages = {7–11},

Title = {Error bars in experimental biology},

Volume = {177},

Year = {2007}}

@article{afshartous:2007,

Author = {Afshartous, D. and Wolf, M.},

Journal = {Journal of the Royal Statistical Society: Series A (Statistics in Society)},

Number = {4},

Pages = {1035–1059},

Title = {Avoiding “data snooping” in multilevel and mixed effects models},

Volume = {170},

Year = {2007}}

@article{jackson:2008,

Author = {Jackson, Christopher H.},

Journal = {The American Statistician},

Number = {4},

Title = {Displaying Uncertainty With Shading},

Volume = {62},

Year = {2008}}

@article{cumming:2009,

Author = {Cumming, Geoff},

Journal = {Statistics in Medicine},

Number = {2},

Pages = {205–220},

Title = {Inference by eye: Reading the overlap of independent confidence intervals},

Volume = {28},

Year = {2009}}

@article{afshartous:2010,

Author = {Afshartous, David and Preston, Richard A.},

Journal = {Computational Statistics & Data Analysis},

Number = {10},

Pages = {2296 – 2305},

Title = {Confidence intervals for dependent data: Equating non-overlap with statistical significance},

Volume = {54},

Year = {2010}}