A friend/colleague sent me some comments on my recent article with Kaiser Fung on Freakonomics. My friend gave several reasons why he thought we were unfair to Levitt.
I’ll give my reply (my friend preferred that I not quote his email, but you can get a general sense of the questions from my answers). But first let me point you to my original post, Freakonomics 2: What went wrong?, from a couple years ago, in which I raised many of the points that ultimately went into our article.
And here’s my recent note. (I numbered my points, but the email I was replying to was not numbered. This is not a point-by-point rebuttal to anything but rather just a series of remarks.)
1. Both Kaiser and I are big fans of Freakonomics. It’s only because Levitt can (and has) done better, that we’re sad when he doesn’t live up to his own high standards. If we didn’t convey this sense of respect in our American Scientist article, that is our failing.
2. I think it was at best tacky and at worst irresponsible for Levitt to repeatedly promote the work of his economist colleague on sex ratios and then, when the work was retracted, praise the economist who had made the mistake, without ever acknowledging the scientists who had been right all the time.
3. I think it was at best tacky etc. for Levitt to non-respond to Pierrehumbert’s comments by labeling these as an “intentional misreading” and then, referring to his own writings on the subject, to write, “I’m not sure why that is blasphemy.” Nobody ever said Levitt’s writings were “blasphemy” ant it just seems silly to me for Levitt to say that.
4. I have come across some other of Myhrvold’s writings and he seems like a bit of a crank. He has a habit of applying the concept of albedo in all sorts of inappropriate settings.
5. Considering points 2-4 above, I think it’s fair to say that in his recent writings, Levitt has been a bit too quick to trust the work of his friends.
6. There are also a bunch of other examples that we didn’t have room for in our article (we had to cut a lot for space limitations), for example in October 2008 endorsing his U of C colleague Casey Mulligan’s “economic arguments as to why things are not so bad.” (Mulligan wrote, “The current unemployment rate of 6.1 percent is not alarming.”) I can’t try to evaluate Mulligan’s arguments, but my point here is that Levitt is perhaps too quick to trust his friends. I have never met Levitt, but I have heard from others that he’s a very nice guy. And in his writings, he portrays himself in a humble manner. The flip side of this, however, is that he’s a bit too uncritical of friends. Perhaps if Levitt had a higher self-regard, he’d be more willing to apply critical standards to others.
7. Levitt is the one who called himself a rogue (or at least acquiesced in the portrayal). I don’t think there’s anything wrong with us pointing out the contradictions and difficulties that arise from this role. For that matter, I don’t think there’s anything wrong with not being a rogue. I have the same sort of Harvard-MIT background that Levitt has. I freely admit to not being a rogue, even though there have been times and places (e.g., UC Berkeley in the early 1990s) where I happened to be an outsider. Overall, I’ve always been part of the establishment, and there are a lot of advantages to that, even beyond being paid well. Levitt’s part of the establishment too. He’s used his advantages well, and I don’t begrudge him that one bit.
8. To return to point 1 above, I think Levitt did a great service by writing Freakonomics (and doing the research on which the book was based) and also continuing with the Freakonomics blog. I particularly like how Levitt and Dubner convey the intellectual excitement of research and the back-and-forth between theory and data. Commendably, they don’t put themselves above the fray but rather are open about the active nature of scientific theorizing.
But then, when they falter, they disappoint me. For example, getting back to item 2 above, Levitt and Dubner were sitting on a wonderful story about how a smart economist made a mistake and how that economist refused to admit error for a couple of years even when the experts in the field showed evidence that she was wrong. But instead Levitt flattened the story by making the economist the hero and never mentioning the others. This is Washington-chopping-down-the-cherry-tree history, it’s a missed opportunity, and I think it’s disrespectful to the researchers in that field who were right all along.
To put it another way:
About a year ago, I gave my talk, “Of Beauty, Sex, and Power,” at the meeting of the National Association of Science Writers. At one point I mentioned Freakonomics and the audience groaned. Steve Levitt is not a popular guy with this crowd. And that’s the typical reaction I get: “Freakonomics” is a byword for sloppy science reporting, it’s a word you throw out there if you want an easy laugh. Even some defenders of Freakonomics nowadays will say I shouldn’t be so hard on it, it’s just entertainment.
Now go back a few years. In 2005, Freakonomics was taken seriously. It was a sensation. Entertaining, sure, but not just entertainment—rather, the book represented an exciting new way of looking at the world. There was talk of the government hiring Levitt to apply his Freakonomics tools to catch terrorists.
That’s what Kaiser and I meant when we asked “What went wrong?” Freakonomics was once a forum for a playful discussion of serious, important ideas; now it’s more of a grab-bag of unfounded arguments. There’s some good stuff there but seemingly no filter.
It’s not news that Freakonomics has problems. See, for example, the reviews by Felix Salmon, Ariel Rubenstein, John DiNardo, and Daniel Davies.
In reporting and discussing some cases of questionable judgment on the part of the Freakonomics team, Kaiser and I were not trying to shoot down Freakonomics but to explore what went wrong: What moved them on the track from serious popular science to a walking punch line?
We identify the problem as one of quality control: lots of material and nobody checking it. That’s how you get an uncritical trust in an economist with a wrong theory of sex ratios, an albedo-obsessed software executive, an economist who claimed in October 2008 that “things are just not that bad,” Daryl Bem, and . . . a certain Reader in Management at the London School of Economics.
My guess is that Levitt takes his academic writing seriously but think of the Freakonomics books and blogs as just for fun. Although Levitt and Dubner could probably hire someone to fact-check their books and blogs, they don’t really see the point, since their goal is to get people thinking rather than to get the right answer.
But if they would admit their mistakes (rather than either ignoring them or going on the offensive when people point out problems), maybe they could move back to their former role of serious communicators of social science. This is what Kaiser and I talk about in our article: the tensions between accuracy and readability.