Lowess is great

I came across this old blog entry that was just hilarious–but it’s from 2005 so I think most of you haven’t seen it.

It’s the story of two people named Martin Voracek and Maryanne Fisher who in a published discussion criticized lowess (a justly popular nonlinear regression method).

Curious, I looked up “Martin Voracek” on the web and found an article in the British Medical Journal whose the title promised “trend analysis.” I was wondering what statistical methods they used–something more sophisticated than lowess, perhaps?

They did have one figure, and here it is:

vorm2338.f1.gif

Voracek and Fisher, the critics of lowess, are fit straight lines to data to clearly nonlinear data! It’s most obvious in their leftmost graph. Voracek and Fisher get full credit for showing scatterplots, but hey . . . they should try lowess next time! What’s really funny in the graph are the little dotted lines indicating inferential uncertainty in the regression lines–all under the assumption of linearity, of course. (You can see enlarged versions of their graphs at this link.)

As usual, my own house has some glass-based construction and so it’s probably not so wise of me to throw stones, but really! Not knowing about lowess is one thing, but knowing about it, then fitting a straight line to nonlinear data, then criticizing someone else for doing it right–that’s a bit much.

4 thoughts on “Lowess is great

  1. But are they doing statistical tests for inference where they only model slope and intercept? If so, maybe it makes sense to only plot what they're testing? I've wondered about this in my own work. If you're not using a GAM or something with a spline in the model structure, should you make a big deal of the nonlinearity by plotting curves in your graphics? Presumably the conclusions in the paper, especially if you've got a model that's more complex than just what you can plot in a single graph, should be based on the strength of the coefficients in the model, not on what you happen to see in a graph plotting marginals?

  2. Voracek and Fisher's conclusions were that, yes, there are trends in these variables over time – the estimated linear trend is depicted in the graphs. While perhaps not optimal, linear regression is a valid method for this form of basic inference, and is at least easily understood.

    Perhaps more importantly, the data is on body shapes of Playboy centerfolds, and comes from the traditionally-jokey Christmas issue of the BMJ. Voracek and Fisher's criticism of lowess (elsewhere) may be bogus, but getting seriously worked up about this example? — that's a bit much.

  3. You see this stuff all over the place though: isn't the idea that there's no reason to fit curves as a straight line is the best linear approximation to the conditional expectation very popular in econometrics at the moment?

  4. Freddy:

    You can see the linear trend using lowess too! But I agree that the linear regressions pick up the linear trends just fine.

    My problem with Voracek and Fisher is not with their analysis in that paper. My problem with Voracek and Fisher is their ill-informed attack on lowess in their discussion of Seth Roberts's paper. They were concerned about "the overuse of the loess procedure" but they gave no good reason for their concern. I'm not happy with people who go around criticizing statistical methods for no good reason.

Comments are closed.