Helen DeWitt links to this blog that reports on a study by Scott Bateman, Carl Gutwin, David McDine, Regan Mandryk, Aaron Genest, and Christopher Brooks that claims the following:
Guidelines for designing information charts often state that the presentation should reduce ‘chart junk’–visual embellishments that are not essential to understanding the data. . . . we conducted an experiment that compared embellished charts with plain ones, and measured both interpretation accuracy and long-term recall. We found that people’s accuracy in describing the embellished charts was no worse than for plain charts, and that their recall after a two-to-three-week gap was significantly better.
As the above-linked blogger puts it, “chartjunk is more useful than plain graphs. . . . Tufte is not going to like this.”
I can’t speak for Ed Tufte, but I’m not gonna take this claim about chartjunk lying down.
I have two points to make which I hope can stop the above-linked study from being slashdotted and taken as truth.
1. The non-chart-junk graphs in the paper are not so good. Figure 1 is a time series of dollars that is unhelpfully presented as a bar chart and which is either unadjusted for inflation or, if adjusted, is not indicated as such. Figure 2a is a lineplot that whose y-axis should go down to 0, but doesn’t. Both graphs also use the nonstandard strategy of labeling the y-axis on the right rather than the left. Figure 2b is an impossible-to-read pie chart with one of the wedges popping out of the circle. Regular readers of this blog will know what I think of that. Figures 2c and 2d are blurry and have no axis labels. Figure 2d is particularly bad because it’s a time-series graph in which time is presented on the y-axis; it also has the problem with inflation adjustment noted earlier. Figures 4-9, presenting their own findings, are not particularly easy to read either.
Chartjunk aside, it’s hard to make good graphs, so I can’t really blame Bateman et al. for their performance here. They’re doing about as well as might be expected in routine psychology research. And maybe they’re right that crappy chartjunk graphs are better than crappy non-chartjunk graphs. But I don’t think it’s appropriate to generalize to the claim that chartjunk graphs are better than good graphs.
2. This brings me to my second point, which is that a huge, huge drawback of chartjunk is that it limits the amount of information you can display in a graph. If all you want is to display a sequence of 5 numbers, then, sure, go for the chartjunk, I don’t really care. But why limit yourself to only displaying 5 numbers? Consider the graphs in Red State, Blue State (or in our other research publications, or on this blog). Sure, you can do pretty instead of plain (see this discussion with examples), but here the graphics design is used to enhance the points made in the graph, not as a distraction.
"Figure 1 is a time series of dollars that is unhelpfully presented as a pie chart". Is this a typo? "Monstrous Costs" of house campaigns is a bar chart, at least in the version I downloaded. Because the observations are two years apart, that's not necessarily the worst format.
Figure 2 is interesting. The "standard" 2b is a truly horrible pie chart, and it's easy to see that the "chartjunk" 2b — with the wedge coming out of the pie chart shown as a woman's bright red lips — would lead to better memory of the fact that 8 cents of every cosmetic dollar is spent on ingredients.
This blog points out that a careful reading reveals many flaws in the original paper:
http://junkcharts.typepad.com/junk_charts/2010/05…
Zbicyclist: Yes, it's a bar chart. I fixed that typo.
Did they really test understanding the data? It sounded to me like they just tested for remembering the slogan. Two graphs are called "Diamonds *were* a girl's best friend", but only one shows a picture of a girl and some diamonds. Two graphs are called "Monstrous costs" but only one shows a monster. Two graphs are called "The Cosmetics Dollar", but only one has a picture of some cosmetics.
The common features here are that 1) a picture is a mnemonic that lets the viewers remember 2) an assertion. Funny how the data isn't even considered in the comparison. It could have been completely absent from the jazzed-up graphs, or even contradicted their message, without affecting the message at all.
But it's *because* plain graphs and tables are duller as push propaganda that they're better as factual communication.
I worked a few years ago with a group of epidemiologists who had completed an experimental study of features to improve interpretation of graphs. Not published in peer-reviewed literature but it is easily available:
http://www.health.nsw.gov.au/pubs/2006/better_gra…
One interesting thing I remember was the correct interpretation of pie graphs. Respondents without a university education did much better at correctly estimating the magnitude of a difference in a pie graph than those with an university education.
I had a quick look at Bateman et al and I couldn't find any information on their study population – not very helpful!
My experience from years in the marketing research industry is that audiences generally don't like the kind of data-rich graphs that you, me, and Ed Tufte find so valuable.
By the way, I'm reading The Pinch by David Willetts (the Science and University Minister in the new British coalition government), which is the hot book of the moment in British wonkdom. It's quite good, with lots and lots of fascinating data. (Did you know that in Britain, 50% of middle class children have a television in their bedrooms vs. 90% of poor children?) It hasn't been published in the U.S. yet, but you'd like it a lot.
Except … tBut there isn't a single graph in the entire book. Willetts laboriously converts all his data on quantitative subjects like pension contributions over generations into paragraphs of words.
Why?
Books like The Pinch aren't really written to be read, they are written to be reviewed and talked about in the broadsheet press. And the reviewers are primarily ex-English majors who are allergic to quantitative graphs.
Just browsing through comments on our paper and I ran into this one. I thought I'd draw your attention to a few little points. First, we didn't claim that one or the other kind of chart is "better", simply that those that incorporated chartjunk were more memorable (significantly so with respect to the theme of the chart, but trending so for other features). Otherwise, we found that answering simple questions about the graphs was not apparently more difficult with chartjunk (something that we found surprising) and that this result was despite the fact that people spent more time looking at the "valueless" junk regions of the graph when they were present. As noted in the blog linked in this article, we make no claims about analysis. That's a different kettle of fish. We were interested in memorability. If you open USA today and glance at a graph, will you remember it better if you see a Holmes-style graph or a plain-Jane graph? And does the inclusion of that chartjunk interfere with your ability to understand the gist of the graph's message?
Thanks for your interest in the paper!