Skip to content
 

“Earlier you had waxed nostalgic for the days when people sent you bad graphs . . .”

Nadia Hassan writes:

Earlier you had waxed nostalgic for the days when people sent you bad graphs. This [from Javier Zarracina] is not a stand-out on that front, but it is far from ideal:

bush-trump-cuts.0

A lot of buzz in recent years about data journalism or quantitative journalism. There is a lot of issues to be worked out, but seriously clear graphics can help get ideas across and deserve to be prioritized to some degree.

I agree.

As part of my educational mission, I will list some problems with this graph:

– Using areas to denote tax cuts is risky, because if there’s ever a tax increase in any category, it would have to be indicated by a negative area.

– The whole thing is hard to read because of the mix of numbers and pictures. Someone once wrote that a graph isn’t much of a graph if you also have to write every number that’s being plotted.

– Separation of groups into disjoint balls is illogical because income is a continuum. This goes with a confusing arbitrariness regarding which quantiles to use as dividing points.

– Bizarre lack of parallelism comparing Bush’s plan in 2015 to Trump’s in 2016. I guess the numbers from the two plans are coming from different press releases. But couldn’t the Vox reporter call up the organizations that supplied the numbers and get the info on both plans for a common year?

And the accompanying news article, by Dylan Matthews shows a distressing lack of “push.” For example, Matthews writes:

The left-leaning tax analysis shop Citizens for Tax Justice . . . conclud[es] that the plan would cost a whopping $10.8 trillion in its first decade. . . . The right-leaning Tax Foundation estimates that Bush’s tax plan would cost $3.6 trillion over 10 years on a static basis . . .

Ummm, what? It’s the job of a reporter at a site like Vox to evaluate such claims, not just let them sit there, no? If one group said the plan cost $4.1 trillion and the other said $3.8 trillion, sure, you can attribute this to slightly different assumptions around the edges. But, $10.8T compared to $3.6T? In a story that’s explicitly about the effects of these (hypothetical) tax plans, this seems like something you’d want to look into.

P.S. I do respect that infographics need to grab eyeballs, so I’m not necessarily saying that a simple lineplot would be the best choice here. Creativity is great; go for it! But this graph is so cartoonish, I think it is a loss of opportunity to share some information.

12 Comments

  1. Rahul says:

    I’ve reached the conclusion that critiquing graphs is almost pointless:

    First, it rarely affects or changes the people producing bad graphs. Their motives are usually different from clarity of communication etc.

    Second, the prescriptions for good graphs are so damn subjective that even among those people I’d consider as “experts” will wildly disagree about what makes for a good graph.

    Finally, most “experts” in these areas are pretty much espousing their own pet opinions, or even worse, prejudices. There has been very little empiricism or objective measurement on what constitutes a good graph.

    • Andrew says:

      Rahul:

      I disagree. I think there are a lot of people out there who’d like to do better. Criticisms such as mine serve as ways to help people think more generally and systematically about the goals they have in graphing. Yes, different experienced researchers will make graphs in different ways, but it can still be valuable to think about what a graph is doing. Instead of just using the default settings or doing a graph that looks cool. In the example above, my primary audience is not the graph-maker at Vox, but rather all the people out there who are interested in using graphs to make statistical comparisons.

  2. Chris G says:

    > It’s the job of a reporter at a site like Vox to evaluate such claims, not just let them sit there, no?

    It should be but actual performance is erratic. For example, my experience is that Sarah Kliff is reliably good about reality-checking claims whereas Matthews is hit or miss. He can be good but he boots it occasionally.

    • Rahul says:

      My impression of Vox itself is pretty poor. Most of their articles seem like a smart intern was sent to compile information from other sources.

      There’s very little depth, insight or novelty in most Vox pieces.

      • Chris G says:

        With the exception of Kliff, I agree. Her health-care-related reporting has been consistently good over the past 5ish(?) years. She was good before she moved to Vox and, with a SSS caveat, that doesn’t appear to have changed. Overall though Vox is a good example of how being smart is not the same as being thoughtful.

        • Chris G says:

          Brad Plumer is also reliably good. Dave Roberts has written some decent pieces and Max Fisher a few too. They have some people who produce good journalism sometimes but overall they do tend to publish nicely-wrapped low depth, low insight pieces. The “explainer” wrapper suggests that you’re about to read something with depth and insight but what you get is the journalistic equivalent of a ricecake.

  3. Jessica H. says:

    I would add to your critique that circular area is not a very effective visual encoding for numeric data, where effective is defined as by Mackinlay 1986: how well the visual encoding exploits the abilities of the output medium and human visual system (or more simply, how accurately people can read the data values from it, based on the results of graphical perception experiments). Position or length tend to be more accurate. There’s a lot of variance between people but most tend to overestimate proportional relationships between circular areas in graphs, e.g., see the smaller circle on the bottom for 1% Bush as being a bigger percentage of the larger 1% circle for Trump than it is. Having the values labeled can help correct for the errors when there’s some good reason to use a less effective encoding, but its not clear what is being gained from it here.

  4. mpledger says:

    Is it a graph though? Isn’t it really a table with graphical embellishments?

  5. Thomas Lumley says:

    > Someone once wrote that a graph isn’t much of a graph if you also have to write every number that’s being plotted.

    There may well be dozens of independent inventions of that point dating back decades, but one of them is me.

Leave a Reply