Hadley Wickham asks:
I was wondering if you knew of any good articles on the use of error bars. I’m particularly looking for articles that discuss the difference between error of means and error of difference in the context of models (e.g. mixed models) where they are very different. I suspect every applied field has a couple of good articles, but it’s really hard to search for them.
Can anyone help on this? My only advice is to get rid of those horrible crossbars at the ends of the error bars. The crossbars draw attention to the error bars’ endpoints, which are generally not important at all. See, for example, my Anova paper, for some examples of how I like error bars to look.
Marianne Zawitz and I wrote a report on this topic in 1998, when I was a visiting fellow at the Bureau of Justice Statistics. Titled "Displaying Violent Crime Trends…," it's at http://bjs.ojp.usdoj.gov/content/pub/pdf/dvctue.p…
How about this: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC20641…
Error bars in Experimental Biology, by Cumming, FIdler and Vaux
Hello,
maybe these references could help:
Afshartous, D. and Wolf, Michael (2007). Avoiding Data Snooping in Mixed and Multilevel Models, Journal of the Royal Statistical Society, Series A, vol. 170, 1035-1059.
Cumming, G. (2009). Inference by eye: Reading the overlap of independent confidence intervals, Statistics in Medicine, vol. 28(2), 205-220.
Jackson, C. (2008). Displaying Uncertainty with Shading, The American Statistician, vol. 62, 340-347.
Cheers
In some plots error bars of different estimates overlap. Without crossbars you can’t see where the intervals end. Of course, you might say that a plot where error bars overlap is wrong to begin with.
In some plots, error bars of different estimates overlap. Without crossbars you can't see then where the intervals end. Of course you might say that a plot in which error bars overlap is wrong to begin with.
A few more:
Goldstein, H., Healy, M.J.R., 1995. The graphical presentation of a collection of means. Journal of the Royal Statistical Society: Series A 158, 175–177.
Senn, S.J., Auclair, P., 1990. The graphical representation of clinical trials with particular reference to measurement over time. Statistics in Medicine 9,1287–1302.
Afshartous D, Preston RA (2010). “Confidence intervals for dependent data: equating nonoverlap with statistical significance," Computational Statistics and Data Analysis. 54(10): 2296-2305.
General references that may be of use or interest.
T.N. Thiele discussed error bars in 1889. See p.181 of Lauritzen's book on Thiele. Anyone know an earlier reference?
W.S. Cleveland's The elements of graphing data has a good discussion.
Harvey Motulsky's Intuitive biostatistics may also be useful.
We discuss the issue here:
Rouder, J. N. & Morey, R. D. (2005). Relational and arelational confidence intervals: A comment on Fidler et al. (2004). Psychological Science, 16, 77-79.
Hi all,
Thanks for the great suggestions. For the benefit of anyone who comes back to this page in the future, here’s the complete list of articles I’ve found so far:
senn:1990
The graphical representation of clinical trials with particular reference to measurements over time
Senn, S. and Auclair, P. and Johnson, S.
Statistics in Medicine
9
1287–1302
(1990)
sdf1995
The Graphical Presentation of a Collection of Means
Goldstein, H. and Healy, M. J. R.
Journal of the Royal Statistical Society. Series A (Statistics in Society)
158
175-177
(1995)
http://www.jstor.org/stable/2983411
When a study produces estimates for many units or categories a common problem is that end-users will wish to make their own comparisons among a subset of these units. This problem will occur, for example, when estimates of school performance are produced for all schools. The paper proposes a procedure, based on the graphical presentation of confidence intervals, which enables such comparisons to be carried out while maintaining an average required type I error rate.
maltz:1998
Displaying Violent Crime Trends Using Estimates from the National Crime Victimization Survey
Maltz, M. D. and Zawitz, M. W.
(1998)
http://bjs.ojp.usdoj.gov/content/pub/pdf/dvctue.pdf
schenker:2001
On Judging the Significance of Differences by Examining the Overlap Between Confidence Intervals
Schenker, N. and Gentleman, J. F.
The American Statistician
55
182-186
(2001)
masson:2003
Using confidence intervals for graphically based data interpretation.
Masson, M. E. J. and Loftus, G. R.
Canadian Journal of Experimental Psychology/Revue canadienne de psychologie experimentale
57
203
(2003)
payton:2003
Overlapping confidence intervals or standard error intervals: What do they mean in terms of statistical significance?
Payton, M. E. and Greenstone, M. H. and Schenker, N.
The Journal of Insect Science
3
(2003)
fidler:2005
Still Much to Learn About Confidence Intervals
Fidler, F. and Thomason, N. and Cumming, G. and Finch, S. and Leeman, J.
Psychological Science
16
494-495
(2005)
http://pss.sagepub.com/content/16/6/494.short
belia:2005
Researchers misunderstand confidence intervals and standard error bars.
Belia, S. and Fidler, F. and Williams, J. and Cumming, G.
Psychological Methods
10
389
(2005)
cumming:2005
Inference by Eye: Confidence Intervals and How to Read Pictures of Data.
Cumming, G. and Finch, S.
American Psychologist
60
170
(2005)
cumming:2007
Error bars in experimental biology
Cumming, G. and Fidler, F. and Vaux, D. L.
Journal of Cell Biology
177
7–11
(2007)
afshartous:2007
Avoiding “data snooping” in multilevel and mixed effects models
Afshartous, D. and Wolf, M.
Journal of the Royal Statistical Society: Series A (Statistics in Society)
170
1035–1059
(2007)
jackson:2008
Displaying Uncertainty With Shading
Jackson, C. H.
The American Statistician
62
(2008)
http://www.mrc-bsu.cam.ac.uk/personal/chris/papers/denstrip.pdf
cumming:2009
Inference by eye: Reading the overlap of independent confidence intervals
Cumming, G.
Statistics in Medicine
28
205–220
(2009)
http://dx.doi.org/10.1002/sim.3471
afshartous:2010
Confidence intervals for dependent data: Equating non-overlap with statistical significance
Afshartous, D. and Preston, R. A.
Computational Statistics & Data Analysis
54
2296 – 2305
(2010)
Hmmm, that didn’t format so well (what happened to the preview??). Here’s the bibtex:
@article{senn:1990,
Author = {Senn, SJ and Auclair, P. and Johnson, S.},
Journal = {Statistics in Medicine},
Number = {11},
Pages = {1287–1302},
Title = {The graphical representation of clinical trials with particular reference to measurements over time},
Volume = {9},
Year = {1990}}
@article{sdf1995,
Author = {Goldstein, Harvey and Healy, Michael J. R.},
Journal = {Journal of the Royal Statistical Society. Series A (Statistics in Society)},
Number = {1},
Pages = {175-177},
Title = {The Graphical Presentation of a Collection of Means},
Volume = {158},
Year = {1995}}
@techreport{maltz:1998,
Author = {Maltz, Michael D. and Zawitz, Marianne W.},
Institution = {Bureau of Justice Statistics},
Number = {167881},
Title = {Displaying Violent Crime Trends Using Estimates from the National Crime Victimization Survey},
Year = {1998}}
@article{schenker:2001,
Author = {Schenker, Nathaniel and Gentleman, Jane F},
Journal = {The American Statistician},
Number = {3},
Pages = {182-186},
Title = {On Judging the Significance of Differences by Examining the Overlap Between Confidence Intervals},
Volume = {55},
Year = {2001}}
@article{masson:2003,
Author = {Masson, M.E.J. and Loftus, G.R.},
Journal = {Canadian Journal of Experimental Psychology/Revue canadienne de psychologie experimentale},
Number = {3},
Pages = {203},
Title = {Using confidence intervals for graphically based data interpretation.},
Volume = {57},
Year = {2003}}
@article{payton:2003,
Author = {Payton, Mark E. and Greenstone, Matthew H. and Schenker, Nathaniel},
Journal = {The Journal of Insect Science},
Title = {Overlapping confidence intervals or standard error intervals: What do they mean in terms of statistical significance?},
Volume = {3},
Year = {2003}}
@article{fidler:2005,
Author = {Fidler, Fiona and Thomason, Neil and Cumming, Geoff and Finch, Sue and Leeman, Joanna},
Journal = {Psychological Science},
Number = {6},
Pages = {494-495},
Title = {Still Much to Learn About Confidence Intervals},
Volume = {16},
Year = {2005}}
@article{belia:2005,
Author = {Belia, S. and Fidler, F. and Williams, J. and Cumming, G.},
Journal = {Psychological Methods},
Number = {4},
Pages = {389},
Title = {Researchers misunderstand confidence intervals and standard error bars.},
Volume = {10},
Year = {2005}}
@article{cumming:2005,
Author = {Cumming, G. and Finch, S.},
Journal = {American Psychologist},
Number = {2},
Pages = {170},
Title = {Inference by Eye: Confidence Intervals and How to Read Pictures of Data.},
Volume = {60},
Year = {2005}}
@article{cumming:2007,
Author = {Cumming, Geoff and Fidler, Fiona and Vaux, David L.},
Journal = {Journal of Cell Biology},
Number = {1},
Pages = {7–11},
Title = {Error bars in experimental biology},
Volume = {177},
Year = {2007}}
@article{afshartous:2007,
Author = {Afshartous, D. and Wolf, M.},
Journal = {Journal of the Royal Statistical Society: Series A (Statistics in Society)},
Number = {4},
Pages = {1035–1059},
Title = {Avoiding “data snooping” in multilevel and mixed effects models},
Volume = {170},
Year = {2007}}
@article{jackson:2008,
Author = {Jackson, Christopher H.},
Journal = {The American Statistician},
Number = {4},
Title = {Displaying Uncertainty With Shading},
Volume = {62},
Year = {2008}}
@article{cumming:2009,
Author = {Cumming, Geoff},
Journal = {Statistics in Medicine},
Number = {2},
Pages = {205–220},
Title = {Inference by eye: Reading the overlap of independent confidence intervals},
Volume = {28},
Year = {2009}}
@article{afshartous:2010,
Author = {Afshartous, David and Preston, Richard A.},
Journal = {Computational Statistics & Data Analysis},
Number = {10},
Pages = {2296 – 2305},
Title = {Confidence intervals for dependent data: Equating non-overlap with statistical significance},
Volume = {54},
Year = {2010}}