Skip to content
 

Any good articles on the use of error bars?

Hadley Wickham asks:

I was wondering if you knew of any good articles on the use of error bars. I’m particularly looking for articles that discuss the difference between error of means and error of difference in the context of models (e.g. mixed models) where they are very different. I suspect every applied field has a couple of good articles, but it’s really hard to search for them.

Can anyone help on this? My only advice is to get rid of those horrible crossbars at the ends of the error bars. The crossbars draw attention to the error bars’ endpoints, which are generally not important at all. See, for example, my Anova paper, for some examples of how I like error bars to look.

10 Comments

  1. Mike Maltz says:

    Marianne Zawitz and I wrote a report on this topic in 1998, when I was a visiting fellow at the Bureau of Justice Statistics. Titled "Displaying Violent Crime Trends…," it's at http://bjs.ojp.usdoj.gov/content/pub/pdf/dvctue.p

  2. Michael Lew says:

    How about this: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC20641
    Error bars in Experimental Biology, by Cumming, FIdler and Vaux

  3. ahuri says:

    Hello,

    maybe these references could help:

    Afshartous, D. and Wolf, Michael (2007). Avoiding Data Snooping in Mixed and Multilevel Models, Journal of the Royal Statistical Society, Series A, vol. 170, 1035-1059.

    Cumming, G. (2009). Inference by eye: Reading the overlap of independent confidence intervals, Statistics in Medicine, vol. 28(2), 205-220.

    Jackson, C. (2008). Displaying Uncertainty with Shading, The American Statistician, vol. 62, 340-347.

    Cheers

  4. Anonymous says:

    In some plots error bars of different estimates overlap. Without crossbars you can’t see where the intervals end. Of course, you might say that a plot where error bars overlap is wrong to begin with.

  5. Titus v.d. Malsburg says:

    In some plots, error bars of different estimates overlap. Without crossbars you can't see then where the intervals end. Of course you might say that a plot in which error bars overlap is wrong to begin with.

  6. David Afshartous says:

    A few more:

    Goldstein, H., Healy, M.J.R., 1995. The graphical presentation of a collection of means. Journal of the Royal Statistical Society: Series A 158, 175–177.

    Senn, S.J., Auclair, P., 1990. The graphical representation of clinical trials with particular reference to measurement over time. Statistics in Medicine 9,1287–1302.

    Afshartous D, Preston RA (2010). “Confidence intervals for dependent data: equating nonoverlap with statistical significance," Computational Statistics and Data Analysis. 54(10): 2296-2305.

  7. Nick Cox says:

    General references that may be of use or interest.

    T.N. Thiele discussed error bars in 1889. See p.181 of Lauritzen's book on Thiele. Anyone know an earlier reference?

    W.S. Cleveland's The elements of graphing data has a good discussion.

    Harvey Motulsky's Intuitive biostatistics may also be useful.

  8. Richard D. Morey says:

    We discuss the issue here:
    Rouder, J. N. & Morey, R. D. (2005). Relational and arelational confidence intervals: A comment on Fidler et al. (2004). Psychological Science, 16, 77-79.

  9. Hi all,

    Thanks for the great suggestions. For the benefit of anyone who comes back to this page in the future, here’s the complete list of articles I’ve found so far:

    senn:1990

    The graphical representation of clinical trials with particular reference to measurements over time
    Senn, S. and Auclair, P. and Johnson, S.
    Statistics in Medicine 

    1287–1302 
    (1990)

    sdf1995

    The Graphical Presentation of a Collection of Means
    Goldstein, H. and Healy, M. J. R.
    Journal of the Royal Statistical Society. Series A (Statistics in Society) 
    158 
    175-177 
    (1995)
    http://www.jstor.org/stable/2983411
    When a study produces estimates for many units or categories a common problem is that end-users will wish to make their own comparisons among a subset of these units. This problem will occur, for example, when estimates of school performance are produced for all schools. The paper proposes a procedure, based on the graphical presentation of confidence intervals, which enables such comparisons to be carried out while maintaining an average required type I error rate.

    maltz:1998

    Displaying Violent Crime Trends Using Estimates from the National Crime Victimization Survey
    Maltz, M. D. and Zawitz, M. W.
     
     
     
    (1998)
    http://bjs.ojp.usdoj.gov/content/pub/pdf/dvctue.pdf

    schenker:2001

    On Judging the Significance of Differences by Examining the Overlap Between Confidence Intervals
    Schenker, N. and Gentleman, J. F.
    The American Statistician 
    55 
    182-186 
    (2001)

    masson:2003

    Using confidence intervals for graphically based data interpretation.
    Masson, M. E. J. and Loftus, G. R.
    Canadian Journal of Experimental Psychology/Revue canadienne de psychologie experimentale 
    57 
    203 
    (2003)

    payton:2003

    Overlapping confidence intervals or standard error intervals: What do they mean in terms of statistical significance?
    Payton, M. E. and Greenstone, M. H. and Schenker, N.
    The Journal of Insect Science 

     
    (2003)

    fidler:2005

    Still Much to Learn About Confidence Intervals
    Fidler, F. and Thomason, N. and Cumming, G. and Finch, S. and Leeman, J.
    Psychological Science 
    16 
    494-495 
    (2005)
    http://pss.sagepub.com/content/16/6/494.short

    belia:2005

    Researchers misunderstand confidence intervals and standard error bars.
    Belia, S. and Fidler, F. and Williams, J. and Cumming, G.
    Psychological Methods 
    10 
    389 
    (2005)

    cumming:2005

    Inference by Eye: Confidence Intervals and How to Read Pictures of Data.
    Cumming, G. and Finch, S.
    American Psychologist 
    60 
    170 
    (2005)

    cumming:2007

    Error bars in experimental biology
    Cumming, G. and Fidler, F. and Vaux, D. L.
    Journal of Cell Biology 
    177 
    7–11 
    (2007)

    afshartous:2007

    Avoiding “data snooping” in multilevel and mixed effects models
    Afshartous, D. and Wolf, M.
    Journal of the Royal Statistical Society: Series A (Statistics in Society) 
    170 
    1035–1059 
    (2007)

    jackson:2008

    Displaying Uncertainty With Shading
    Jackson, C. H.
    The American Statistician 
    62 
     
    (2008)
    http://www.mrc-bsu.cam.ac.uk/personal/chris/papers/denstrip.pdf

    cumming:2009

    Inference by eye: Reading the overlap of independent confidence intervals
    Cumming, G.
    Statistics in Medicine 
    28 
    205–220 
    (2009)
    http://dx.doi.org/10.1002/sim.3471

    afshartous:2010

    Confidence intervals for dependent data: Equating non-overlap with statistical significance
    Afshartous, D. and Preston, R. A.
    Computational Statistics & Data Analysis 
    54 
    2296 – 2305 
    (2010)

    • Hmmm, that didn’t format so well (what happened to the preview??). Here’s the bibtex:

      @article{senn:1990,
      Author = {Senn, SJ and Auclair, P. and Johnson, S.},
      Journal = {Statistics in Medicine},
      Number = {11},
      Pages = {1287–1302},
      Title = {The graphical representation of clinical trials with particular reference to measurements over time},
      Volume = {9},
      Year = {1990}}

      @article{sdf1995,
      Author = {Goldstein, Harvey and Healy, Michael J. R.},
      Journal = {Journal of the Royal Statistical Society. Series A (Statistics in Society)},
      Number = {1},
      Pages = {175-177},
      Title = {The Graphical Presentation of a Collection of Means},
      Volume = {158},
      Year = {1995}}

      @techreport{maltz:1998,
      Author = {Maltz, Michael D. and Zawitz, Marianne W.},
      Institution = {Bureau of Justice Statistics},
      Number = {167881},
      Title = {Displaying Violent Crime Trends Using Estimates from the National Crime Victimization Survey},
      Year = {1998}}

      @article{schenker:2001,
      Author = {Schenker, Nathaniel and Gentleman, Jane F},
      Journal = {The American Statistician},
      Number = {3},
      Pages = {182-186},
      Title = {On Judging the Significance of Differences by Examining the Overlap Between Confidence Intervals},
      Volume = {55},
      Year = {2001}}

      @article{masson:2003,
      Author = {Masson, M.E.J. and Loftus, G.R.},
      Journal = {Canadian Journal of Experimental Psychology/Revue canadienne de psychologie experimentale},
      Number = {3},
      Pages = {203},
      Title = {Using confidence intervals for graphically based data interpretation.},
      Volume = {57},
      Year = {2003}}

      @article{payton:2003,
      Author = {Payton, Mark E. and Greenstone, Matthew H. and Schenker, Nathaniel},
      Journal = {The Journal of Insect Science},
      Title = {Overlapping confidence intervals or standard error intervals: What do they mean in terms of statistical significance?},
      Volume = {3},
      Year = {2003}}

      @article{fidler:2005,
      Author = {Fidler, Fiona and Thomason, Neil and Cumming, Geoff and Finch, Sue and Leeman, Joanna},
      Journal = {Psychological Science},
      Number = {6},
      Pages = {494-495},
      Title = {Still Much to Learn About Confidence Intervals},
      Volume = {16},
      Year = {2005}}

      @article{belia:2005,
      Author = {Belia, S. and Fidler, F. and Williams, J. and Cumming, G.},
      Journal = {Psychological Methods},
      Number = {4},
      Pages = {389},
      Title = {Researchers misunderstand confidence intervals and standard error bars.},
      Volume = {10},
      Year = {2005}}

      @article{cumming:2005,
      Author = {Cumming, G. and Finch, S.},
      Journal = {American Psychologist},
      Number = {2},
      Pages = {170},
      Title = {Inference by Eye: Confidence Intervals and How to Read Pictures of Data.},
      Volume = {60},
      Year = {2005}}

      @article{cumming:2007,
      Author = {Cumming, Geoff and Fidler, Fiona and Vaux, David L.},
      Journal = {Journal of Cell Biology},
      Number = {1},
      Pages = {7–11},
      Title = {Error bars in experimental biology},
      Volume = {177},
      Year = {2007}}

      @article{afshartous:2007,
      Author = {Afshartous, D. and Wolf, M.},
      Journal = {Journal of the Royal Statistical Society: Series A (Statistics in Society)},
      Number = {4},
      Pages = {1035–1059},
      Title = {Avoiding “data snooping” in multilevel and mixed effects models},
      Volume = {170},
      Year = {2007}}

      @article{jackson:2008,
      Author = {Jackson, Christopher H.},
      Journal = {The American Statistician},
      Number = {4},
      Title = {Displaying Uncertainty With Shading},
      Volume = {62},
      Year = {2008}}

      @article{cumming:2009,
      Author = {Cumming, Geoff},
      Journal = {Statistics in Medicine},
      Number = {2},
      Pages = {205–220},
      Title = {Inference by eye: Reading the overlap of independent confidence intervals},
      Volume = {28},
      Year = {2009}}

      @article{afshartous:2010,
      Author = {Afshartous, David and Preston, Richard A.},
      Journal = {Computational Statistics & Data Analysis},
      Number = {10},
      Pages = {2296 – 2305},
      Title = {Confidence intervals for dependent data: Equating non-overlap with statistical significance},
      Volume = {54},
      Year = {2010}}