Skip to content
 

No, Michael Jordan didn’t say that!

The names are changed, but the song remains the same.

First verse. There’s an article by a journalist,

to which Andrew responded in blog form,

Second verse. There’s an article by a journalist,

to which Michael Jordan responded in blog form,

Whenever I (Bob, not Andrew) read a story in an area I know something about (slices of computer science, linguistics, and statistics), I’m almost always struck by the inaccuracies. The result is that I mistrust journalists writing about topics I don’t know anything about, such as foreign affairs, economics, or medicine.

P.S. from Andrew: The story is perhaps not so clear; see the comment by Lee Gomes and the subsequent discussion.

19 Comments

  1. Chris G says:

    A few things:

    1) While I get IEEE Spectrum by virtue of being an IEEE member I don’t often read it because content is often on the lite side.

    2) If you just ignore the interviewer’s attempts to lead Jordan and just read Jordan’s comments, his comments are very good.

    3) With regard to the potential “big-data winter” that Jordan notes, I’m reminded of the Gartner Hype Cycle – http://en.wikipedia.org/wiki/Hype_cycle

    4) I think the potential pitfall with Big Data, which Jordan alludes to, is that data mining doesn’t address causality. As Tim Harford put it in ‘Big data: are we making a big mistake’, “[B]ig data do not solve the problem that has obsessed statisticians and scientists for centuries: the problem of insight, of inferring what is going on, and figuring out how we might intervene to change a system for the better.” Link = http://www.ft.com/cms/s/2/21a6e7d8-b479-11e3-a09a-00144feabdc0.html#axzz3H1IOEfso

    5) With respect to trusting or mistrusting journalists, I believe the answer is be an educated consumer. Do some homework to get yourself grounded in the topic you’re reading about and then identify some knowledgeable and honest journalist who write on it. That said, depending upon the topic it can take a long time (months? years?) to figure out who’s truthworthy and who isn’t.

    • Janne Sinkkonen says:

      The absence of causal inference in “big data” (hype) comes from the passivity of big data: typically the data become collected as a side effect of a process, big data is a passive track. Then causal inference in the sense of either generalisable insight or less generalisable operational predictions is difficult, for both needs trials, intervening with the system producing the data.

      Therefore my guess is that big data gradually shifts to big experiments. But they are harder than passive data collection, and require more from systems and organisations from which the data arise.

      Meanwhile, even passive big data can be used as covariates in small-scale experiments. One can find correlative structure from big data, reduce its dimension by LDA for example, then do small-scale experiments or active learning with the small-dimensional representations as covariates.

      BTW, I have had very similar feelings of media than Bob have. It is really hard to know what to trust, when all the things you know about have inaccuracies or are distorted, typically towards unwarranted optimism and hype. And I know mostly about science, which is supposed to objective compared to, e.g., politics!

    • Rahul says:

      Apart from causality the big worry is external validity. How robust are your findings outside the particular dataset?

  2. Fernando says:

    Well, I actually read the Jordan interview and I could not tell a glaring difference between the title and intro, and the text of the interview. Sure, the title is catchy and exagerated but surely you expect that from a journalist.

    Moreover I feel Jordan is being unfair to the journalist. The gist of his blog post is that he meant something different than what he actually said. Well, you can hardly blame the journalist for taking him at his word.

    I say this as someone who thinks many academic’s obsession with the media is pathetic. But if you play with fire then live win the consequences.

    PS I think Andrew’s case is different in that the interview was (a) not textual and (b) there appears to have been some technical errors in reporting e.g. about whatbis a p value etc.

  3. Rahul says:

    Why can’t they just run the drafts by the interviewees? It’d save a lot of this I-didn’t-say-what-they-said-I-said angst.

  4. question says:

    “Whenever I (Bob, not Andrew) read a story in an area I know something about (slices of computer science, linguistics, and statistics), I’m almost always struck by the inaccuracies. The result is that I mistrust journalists writing about topics I don’t know anything about, such as foreign affairs, economics, or medicine.”

    For the most part I find it is better to just believe the opposite of what I hear/read in the news. Also, this is one of the reasons I really like wikipedia. The problem isn’t so much that wikipedia is an unreliable source, but that people treat other sources without enough incredulousness.

  5. Paul says:

    In my teens I began to notice how far off the mark journalists always were in areas I knew something about. … A few years later I was really impressed by a 1982 Time magazine cover story on arcade games that was pretty much on the money. I still have no idea how often this happens but I suppose there must be a few good reporters.

  6. Tom Dietterich says:

    There is currently a burst of research on causality that is motivated in large part by A/B testing on web sites (surely a case of “big data”). So not all big data is purely observational, and the companies doing A/B testing are extremely interested in making causal inferences. I find this exciting: causality used to be a purely “academic” topic that was entirely ignored by machine learning practitioners. Now it is front and center for many of them.

  7. Lee Gomes says:

    I am the author of the IEEE Spectrum interview in question, and your headline is spectacularly wrong. Michael Jordan did indeed say every single word he was quoted as saying. The interview was on tape. I typed up a transcript, edited it, and, as per our agreement, sent the text to Dr. Jordan. He sent it back with a few minor corrections. EVERYTHING in the article, except the headline and lead-in paragraph, he saw before-hand and approved of.

    His complaint was about the headline.

    “Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Other Huge Engineering Efforts
    Big-data boondoggles and brain-inspired chips are just two of the things we’re really getting wrong”

    To evaluate its accuracy, let’s review a few of the things Dr. Jordan said.

    * It is “patently false” to think that there is any neuroscience behind current Deep Learning and machine language techniques.

    * We are “not yet in an era” where we can use an understanding of the brain to create intelligent computer systems.

    * “It’s pretty clear” that the brain doesn’t work via the methods used in machine learning, notably back propagation.

    * “There is no clear reason the hope should be borne out,” in reference to building useful silicon based on how a particular researcher views the brain.

    * “There are many, many hard problems” in vision that remain to be solves.

    * The notion of “error bars” is missing “in much of the machine learning literature.”

    I’ll stop now, and I haven’t even gotten to the part where he predicts a “Big Data Winter” in response to the current hype cycle.

    In an earlier post about this issue, I asked readers to do what Dr. Jordan asks them to do in his post. “Read the title and the first paragraph and attempt to infer what’s in the body of the interview. Now go read the interview and see what you think about the choice of title.”

    Fernando, in one of the comments above, did that, and reported he “could not tell a glaring difference between the title and intro, and the text of the interview. Sure, the title is catchy and exaggerated but surely you expect that from a journalist.”

    I, of course, disagree that it was “exaggerated.” I certainly think it was “lively,” but in a way appropriate to what the interview actually said. And it did what a headline is supposed to do: Make you want to read the article.

    One of the ironies of this disagreement is that it was Dr. Jordan’s headline, “Big Data, Hype, the Media and Other Provocative Words to Put in a Title,” and not mine, that was in fact hype-y and provocative.

    Lee Gomes
    lee@leegomes.com
    San Francisco

    • Rahul says:

      If you did send Prof. Jordan a draft of the article & he okayed it, then now I find myself blaming him more.

      Was there a reason why you didn’t also send him the title & lead-in paragraph?

      PS. Is it possible to post the actual interview tapes online?

  8. Lee Gomes says:

    I wrote the introduction; one of the IEEE editors wrote the headline. Never in our wildest dreams did we expect Dr. Jordan to object to them; in fact, when I first saw the headline, I sent an email to the editor congratulating him for capturing the essence of Dr. Jordan’s remarks. In retrospect, we would have happily sent them to him; changing a word or two would have avoided this whole mess, along with the sliming of the article and my reputation that is accompanying it. I can’t post the tape of the interview online for many reasons, the main one being this was not a podcast-style interview, and the transcript barely covered half of what we talked about. Please note, though, that Dr. Jordan does not contest the accuracy of the interview itself; he says in his post that “at least the core of the article was actually me in my own words.” The simple fact of the matter is that he didn’t like a few words in the headline, and used that fact to launch an inflammatory attack against the entire enterprise.

    • Rahul says:

      Yeah that sounds pretty sad and unfair to you. I’m sorry.

      I’ve seen analogous situations where a person gave an interview frankly but then in hindsight did not like the ring of what he said then when it appeared in a formal, permanent, public form.

      Not sure if that’s what is at play here.

  9. Igor Carron says:

    Andrew,

    I am sure you have noticed that this discussion has continued on Yann Lecun’s page. Please note the comments.

    https://www.facebook.com/yann.lecun/posts/10152348155137143

    Cheers,

    Igor.

    • Andrew says:

      Igor:

      Thanks for the link. But now I’m curious: why did you say “I am sure you have noticed that this discussion has continued on Yann Lecun’s page”? I had no awareness of this page until you posted this link. How would I have known it? Was it linked to from some other place that I should’ve known about?

      I’ll have to say that I think I’m missing the point of the whole discussion. What Lecun wrote, and what Jordan wrote, and what Gomes wrote, all seem reasonable to me. They all seem to be saying pretty much the same think. I guess it looks that way to me because I am not so close to that particular subfield. To the people involved, there are big conceptual differences between neural networks, connection machines, etc., but to me they all just seem like methods.

      Perhaps this is the same impression that outsiders to statistics feel when they see arguments between Bayesians and non-Bayesians, or disputes about causal reasoning, or other things we argue about on this blog: internal disputes that don’t look like much from the outside.

      • Rahul says:

        Bob’s title & subsequent comments sure doesn’t make it seem that he thinks what Gomes wrote was reasonable.

      • Martha says:

        Andrew,

        Re “Perhaps this is the same impression that outsiders to statistics …”

        An interesting comment. I would guess that this is indeed sometimes the case, but that sometimes the opposite happens: that outsiders see the arguments as huge “battles,” when we might see them as working things out, trying understand, clarify, etc.

        • Andrew says:

          Martha:

          To me it’s all about understanding, clarifying, etc. But others perceive it as battles. For example, in my correspondence with those ovulation-and-clothing researchers, I kept trying to offer suggestions but they kept responding as if I were attacking them. I just think it’s sad but I keep trying. Perhaps worse were the ovulation-and-voting researchers, who didn’t even try to engage with the criticism at all. The point is just that some conflict can be perceived by one party as “working things out” and by the other party as a “battle.”

          Just to be clear: by saying this, I’m not saying that I’m an innocent peacemaker and these people just want a fight. Rather, I’m saying that the exact same interaction can be perceived in two different ways by the two parties involved. It’s not that my perception is right and theirs is wrong, or that their perception is right and mine is wrong, but rather that there’s no stable description of what’s happening.

          I had a similarly difficult encounter with political scientist Larry Bartels; see here. That particular back-and-forth bothers me so much (I’m still confused by what Larry is thinking on this one) that I’ll be returning to it in a future post.

          • Martha says:

            Andrew:

            I see the differences in perception that you refer to as, in large part, a consequence of human variation in values and customs. I have known people who believe that one should never criticize, and others who make a distinction between criticizing and critiquing (although there may be variation among the latter group on just where the distinction lies), and people who believe that criticism should only be delivered in certain ways (e.g., “Wouldn’t it be better to …” is OK, but “I think that’s wrong” isn’t — whereas I’d prefer the latter to the former if I’m the recipient of the criticism.)

            Many years ago, I decided to put some of my time where my mouth was and, instead of just complaining about the quality of incoming students, volunteer to teach courses for future teachers. I quickly realized that the human variability of opinions on criticism had to be addressed. As difficult as that was, I think it was easier in the context of a class for teachers (first, because the students were planning to be teachers, so the case that they needed to know their subject matter well couldn’t be denied, and second, because giving criticism could be a subject of class discussion, where students could see the spectrum of different opinions among their classmates) than it is in the context of blogs, particularly when they are interdisciplinary.

            One upshot for the current thread is that one can lose credibility when one is an “outsider” to the “culture” of the person one is trying to engage with. If I may (respectfully, I hope) make a suggestion to you in particular, I suspect that one way you sometimes lose credibility with some of those with whom you try to engage is your tendency to call papers you are criticizing “crappy.”

Leave a Reply