So.
Farewell then
Rainbow color scheme.
You reigned in Matlab
Far too long.
But now that
You are no longer
The default,
Will we
miss you?
We can only
Visualize.
E. T. Thribb (17 1/2)
Here’s the background. Brad Stiritz writes:
I know you’re a creator and big proponent of open-source tools. Given your strong interest in statistical visualization, I thought you might still be interested in Matlab’s new default color map, “parula”, which replaces their rainbow-spectrum map called “jet”. This blog post presents a series of quiz-questions (with answers) that you might enjoy & this white paper presents background research justifying the big change.
One reason I find this exciting is that the new color map is based on the Lab color space, which was my favorite for natural-looking photo adjustments back when I was seriously into Photoshop. Lab space is just extremely cool both theoretically & practically.
p.s. regarding open-source vs. close-source environments, the rising popularity of R & Python has forced Mathworks to finally start offering low-cost entry into their world. I wonder though how much this can really bend the adoption curve, especially given that they’re trying not to cannibalize full-price sales too much ..?
My reply: Yes, rainbow is well known to be problematic. I’m surprised Matlab stuck with it for so long. I guess the message is that defaults, once set, are hard to change. For example, R base graphics has some notorious default problems, such as tick marks that are too big and axis labels that are too far from the axes, to the extent that graphs are full of whitespace (a problem that is exacerbated when making grids of graphs). Everybody has to know by now how bad these defaults are, but nobody changes them. So good for Matlab for changing its default colors, even if they made the choice 20 years or so too late!
Does anyone know of research on the perceptual pros and cons of “one-color” (e.g. white-to-black) vs. “two-color” (e.g. blue-to-white-to-orange) heatmaps? It seems likely to me that because “two-color” maps really involve three colors there is potential for perceptual artifacts or difficulties.
Colorblind-unfriendly red-green heatmaps in molecular biology are another unfortunate entrenched default. Plus the convention is red = “genes up”/green = “genes down”, contrary to the many people’s intuition (based on analogy to traffic lights).
Relevant to Brad’s postscript, displacing matlab is one of the major goals of the julia project, and a major influence on the syntax of that language.
https://github.com/stevengj/julia-mit/
This is my favorite reference on the subject: http://www.sandia.gov/~kmorel/documents/ColorMaps/.
Thanks Michael, that paper is quite interesting, especially the part about stretching and compressing the regions. I wonder why blue-red heatmaps are not more popular. I hope someone gives that guy (or some psychologists or whomever) money for empirical follow-up studies.
I like this tool. Meant for cartography but still very useful for selecting color schemes.
http://colorbrewer2.org/
Rahul:
The tool you link to is well-organized, but for most purposes I much prefer continuous or near-continuous color schemes such as we used in our MRP maps (for example, some of the slides in this presentation).
Andrew:
Personally, I like continuous color schemes too. But do we differ in what we mean by “continuous”?
e.g. Here’s a color scheme I often use from Color Brewer. http://i.imgur.com/bUQsy1L.jpg Is this not what you’d call continuous?
Or this scheme for instead: http://i.imgur.com/6OtvOXM.jpg
Rahul:
No, I think these schemes just have 5 discrete levels. For many purposes this can be ok but I just prefer a continuous color range so I don’t have to worry about boundary effects.
Ah! Gotcha. I see what you mean.
OTOH, when you heat map by state etc. you are already dealing with arbitrary boundaries aren’t you? Or take the typical survey that your data comes from. Were respondents queried on a continuous scale or a discrete Likert scale.
So, often, I think the continuity you can get from these graphs is an illusion. It’s like measuring with a large Least Count instrument but displaying the results fine grained.
@Andrew:
Did you look at that MATLAB white paper? Continuous color ranges can introduce boundary artifacts — that seems like the main reason they’re switching their defaults. I’ve always found the old MATLAB defaults really annoying — they stand out a mile away in NIPS papers as being “I used a heatmap from MATLAB that looks cool but is misleading”.
I didn’t see the continuous color scales in that enormous (15MB) slide show you linked. You mean that blue to red for Romney/Obama? Or the red-state blue-state stuff that used shades of red and shades of blue? So are you advocating single colors for each continuous scale? That will save you from getting into the troubles MATLAB pointed out.
Bob:
The continuous color ranges I’m talking about are on pages 19, 28, and 29 of that linked presentation. I’m not talking about a continuous range of hues, I’m talking about a continuous range of intensities (I think that’s the word?) within each hue, going from dark blue to light blue, for example.
100% agree that R has a problem with lousy defaults. stringsAsFactors=TRUE comes to mind. I sometimes wonder if we could write a script that would update any existing R packages that did not specify parameters to explicitly use the previous defaults, then just update the defaults for future work. That should mean that there would be no compatibility issues (if you wrote the updating script well enough) and I assume that these defaults stay in place, because too many libraries have been written that assume them and not because anyone is really in favour of the bad margins, big ticks and converting strings into factors on principle.
Jon:
Yeah, I used to put stuff in my R initialization file (I can’t remember now what it’s called; Rconsole?) but then every time I updated R, I’d need to recopy that file, and that seemed like an ugly way to go. Maybe we should just redefine all the base graphics commands with my favorite defaults and put them into the “arm” package. We could just call them plot.() etc., for example.
I suppose one reason the bad defaults have lasted so long is that the serious effort is going into ggplot2 (and lattice, perhaps), so probably the R experts consider base graphics to be a bit of a joke anyway.
No, the serious effort now is going into ggvis, rCharts, gridSVG, etc.
ggplot2 has reached v. 1.0.0, and is thus stable is some sense.
Speaking of lattice, the default pink-blue colors are exactly the same hue, and thus indistinguishable to the colorblind. I’ve been meaning to try the defaults in the ‘mosaic’ package.
I’d use ‘arm’ functions with slightly different defaults. Just for quick exploratory plots though—it would be too idiosyncratic for shared scripts.
What I’d like is the Andrew default for ggplot2. As confusing as I find the ggplot2 doc, I find base graphics even more confusing. More importantly, ggplot2 is consistently organized and produces default plots I really like. Andrew has some issues and I’ve been meaning to sit down and extract his guidelines and code them up in ggplot2.
When we write the Stan book, we’ll have to do it for my sanity in coding and Andrew’s pickiness for graphics (not that there’s anything wrong with being picky about graphics).
Bob, I want to belatedly encourage you and Andrew to do this. Inertia associated with ggplot2 default settings has prevented me from really getting into it. I assume that Andrew does not really use ggplot?
Maybe this would be a good way of getting the manifesto “codified”.
http://statmodeling.stat.columbia.edu/2014/10/15/statistical-communication-graphics-manifesto/
When do imagine the Stan book will come out? How will it move beyond the J Stat Soft paper? Could it be a full-fledged sequel to ARM?
stringsAsFactors may be hard on beginners, but for intermediate users, it’s not so bad as a default.
For one thing, string processing can be orders of magnitude slower than factors for some operations.
Based on the etymology of the word ‘default’ it would be appropriate to say that bad defaults are a sin, and that statistics is the science of minimizing deficiencies: http://etymonline.com/index.php?term=default
It’s E.J. Thribb. (Sorry, I grew up reading Private Eye.)
Robin,
I hate to have to explain the joke, but . . . it’s “E. T.” in honor of Ed Tufte. My earlier obit in this style was by “R. A. Thribb.”
I missed the earlier one somehow. But now you have another data point on the intersection of Bayesians and Private Eye readers.
Another crappy default used to be the ghastly purple-ish color Excel chose for the very first series you plotted. I think they have changed it now.
Definitely an improvement. I never liked “jet”. (In other news, I never cared for Fuji Velvia either.)
Once upon a time I used Igor Pro for graphing and data analysis. The images on their website aren’t impressive but it was really easy to create appealing plots – no more or less control over appearance than with MATLAB but I found the interface much easier to work with.
Another -1 for Fuji Velvia. I took a gazillion medium format transparencies in India with it before I came to my senses and switched back to Ektachrome. A-B comparisons are so misleading — it’s reminiscent of garishly over-contrasty and bright defaults on flat-panel TVs — having them color calibrated makes a huge difference for long-term viewing even if it doesn’t pop as much in side-by-side comparison. (Same goes for American vs. French wines, if I may say so; sure, the American ones smack you upside the head, but you can really sit down for a conversation with a Burgundy.)
I also really loved the both Ilford Delta 100 and T-Max 100 — the grayscale tone was just gorgeous.
But in the end, my inner hipster took over and I switched to almost exclusively using T-Max 3200 (really ISO 800 pushed 2 stops by default if I remember correctly). Probably because it invoked the Tri-X I grew up with, but was easy to use without a flash on my 35mm Ricoh point and shoot. That Ricoh had a fantastic lens for the price, even if it couldn’t stand up to my all-time fave, a Canon 1.4L 24mm I used to do food photography in tight kitchens. That lens could see in the dark.
jet for all its faults was RG color-blind readable. The discrete-looking transitions can be misleading, but at least I know that artifact is there. As an RG color blind person ggplot2 defaults & lab are unusable.
Can someone help sort out the copyright issue?
In the comments to the matlab blog post, Steve Eddins claims that “the colormap is, however, MathWorks intellectual property, and it would not be appropriate or acceptable to copy or re-use it in non-MathWorks plotting tools.”
Surely, Parula is simply a specification of a trajectory through the colorspace? How on earth could MathWorks claim intellectual ownership to an equation? Could they really sue someone (say, an R user) who created a small script using the Parula mapping?
(actually, I might post this comment on the matlab blog post as well..)
+1. I want to see MathWorks sue someone for that one.
Anyone know of a similar colormap to the copyrighted parula, that one can use in matplotlib?
>So good for Matlab for changing its default colors, even if they made the choice 20 years or so too late!
Hi Andrew,
I don’t understand exactly what you meant.. “too late” for what?
Brad:
I mean that 20 years or more ago, they could’ve already realized this was a bad default color scheme.
For an enormous collection of color maps (palette tables) see http://soliton.vm.bytemark.co.uk/pub/cpt-city/
Specifically, http://soliton.vm.bytemark.co.uk/pub/cpt-city/jjg/cbcont/index.html has continuous versions of the Brewer sequential and diverging color schemes; these, plus Brewer’s own qualitative set (for distinguishing different categories) provide a more than adequate collection for many purposes, though I also like
http://soliton.vm.bytemark.co.uk/pub/cpt-city/jmn/index.html
and
http://soliton.vm.bytemark.co.uk/pub/cpt-city/km/index.html
(I am not the site’s owner but have contributed).
I certainly am glad to see this change in MATLAB: there are far too many “sea of blue” and “sea of green” plots.