Skip to content
 

Bird fight! (Kroodsma vs. Podos)

Donald Kroodsma writes:

Birdsong biologists interested in sexual selection and honest signalling have repeatedly reported confirmation, over more than a decade, of the biological significance of a scatterplot between trill rate and frequency bandwidth. This ‘performance hypothesis’ proposes that the closer a song plots to an upper bound on the graph, the more difficult the song is to sing, and the more difficult the song the higher quality the singer, so that song quality honestly reveals male quality. In reviewing the confirming literature, however, I can find no support for this performance hypothesis.

OK, that sounds jargony, so let me make it clear: when Kroodsma says he “can find no support for this performance hypothesis,” what he’s really saying is that a sub-literature in the animal behavior literature is in error. Rip it up and start over.

How did everyone get it wrong? Kroodsma continues:

I will argue here that the scatter in the graph for songbirds is better explained by social factors and song learning. When songbirds learn their songs from each other, multiple males in a neighbourhood will sing the same song type. The need to conform to the local dialect of song types guides a male to learn a typical example of each song type for that population, not to take a memorized song and diminish or exaggerate it in trill rate or frequency bandwidth to honestly demonstrate his relative prowess. . . . There is no consistent, reliable information in the song performance measures that can be used to evaluate a singing male.

Damn.

But the other side is not going down without a fight. Jeffrey Podos responds:

Kroodsma [in the above-linked article] has critiqued ‘the performance hypothesis’, which posits that two song attributes, trill rate and frequency bandwidth, provide reliable indicators of singer quality and are used as such in mate or rival assessment. . . .

I address these critiques in turn, offering the following counterpoints: (1) the reviewed literature actually reveals substantial plasticity in song learning, leaving room for birds to tailor songs to their own performance capacities; (2) reasonable scenarios, largely untested, remain to explain how songs of repertoire species could convey information about singer quality; and (3) the playback studies critiqued actually enable direct, reasonable inferences about the function of vocal performance variations, because they directly contrast birds’ responses to low- versus high-performance stimuli.

Where did the critics go wrong? Podos continues:

My analyses also reveal numerous shortcomings with Kroodsma’s arguments, including an inaccurate portrayal throughout of publications under review, logic that is thus rendered questionable and reliance on original data sets that are incomplete and thus inconclusive.

I have not read either paper because it just all seems so technical. I suppose with some effort I could untangle this one, but I don’t feel like putting in the effort right now.

Any ornithologists in the house?

Fun fact: Both authors in this discussion had the same academic affiliation of Department of Biology, University of Massachusetts, Amherst. Podos is a professor there, and Kroodsma is a retired professor. Either way, the story is compelling: youngster does shoddy research and the retired prof blows the whistle, or cranky old man can’t handle new methods. In some general sense, I’ve been on both sides of this debate: Sometimes I criticize what I see as flashy research with empty claims, otherwise I’m frustrated that traditionalists will seem to find any excuse not to take a new method seriously.

P.S. A google search turned up this review from 2005 of a book on the science of birdsong. In the review, Bernard Lohr writes:

The contributors do not shy away from controversy. Donald Kroodsma, for example, issues a challenge to those who suggest large song repertoires are a consequence of sexual selection. Kroodsma remains unconvinced that existing direct experimental data demonstrate female choice for larger repertoires in a natural context. Although his criticism is general, he selects—as he did in an earlier critique of song playback designs—studies of other eminent birdsong biologists as specific examples. Because those researchers are more than capable of defending their conclusions and viewpoints, an interesting and vigorous debate is sure to ensue.

That was 12 years ago, and the issue doesn’t yet seem to have been resolved. Strike one against the story that science is self-correcting.

P.P.S. Also relevant is this article, Response to Kroodsma’s critique of banded wren song performance research, by S. L. Vehrencamp, S. R. de Kort, and A. E. Illes.

11 Comments

  1. Anoneuoid says:

    Something is off about how people are writing about this topic. What does it mean to “repeatedly report confirmation of the biological significance of a scatterplot”?

    • jrc says:

      I thought we covered that. Something about elephant parameters.

      http://andrewgelman.com/2017/07/22/a-stunned-dyson/

    • Don Kroodsma says:

      Please allow me to try to clarify (you’d need more information to make sense of the passage). What I’m trying to say is that dozens of research papers have confirmed the biological significance of something (revealed in the scatterplot graph) that is entirely false. It’s the garden of forking paths, the p-hacking, the do-whatever-it-takes-to-get a p of 0.05 that makes a good story publishable. Then the next paper confirms what is now known to be true, and the next, and so on, with no one questioning the process or the results (until now).

      Hundreds of laudatory citations for these stories have solidified the place of these mistruths in birdsong biology. The authors have achieved “fame, fortune, and acclaim” (e.g., Podos is President of the Animal Behavior Society, as were two of his mentors before him; quote from Andrew’s “Winds”). Trouble is, none of the publications are true. It’s all rather inconvenient, and embarrassing, to say the least.

      • Anoneuoid says:

        Thanks,
        So it sounds like you believe there are some reproducible results (the relationship shown in this scatterplot), but they have been interpreted incorrectly. Same as if you test how quickly mice get a food reward in a maze and believe you are testing memory rather than motivation (hunger, etc).

  2. I think these discussions can be helpful, even if they last 12 or more years and sometimes even if they are acrimonious. Thus I disagree with the last sentence of the post– that this situation is strike one for the self-correcting nature of science. The interchange being discussed here is precisely what self-correction looks like, up close. Now, all readers of Animal Behaviour and similar venues will be able to make up our own minds on the relevant issues. Resolution of a scientific dispute does not mean that one of the two (or more) disputants have changed each others’ minds. We fallible scientists are not always able to accomplish that, unfortunately. Sometimes, as the post mentions, there are psychological or other personal things going on that only partly relate to the science involved, and don’t we all tend to hold too tightly to our views, and sometimes get angry when we should just get serious? So those of us in a dispute are sometimes hobbled by our own humanness. It’s the rest of the field, and the future of the field, where resolution is accomplished.

  3. mightysparrow says:

    I’m an ornithologist with an interest in birdsong. I read Kroodsma’s critique and Podos’a response, and found the critique very interesting and persuasive, the response much less so (it seemed to focus on peripheral issues without really engaging Kroodsma’s main points). I’m also familiar with a couple of the papers that Kroodsma’s criticizes, and found them to be good examples of authors walking through the garden of forking paths to reach a desired conclusion. When it comes to the particular “performance hypothesis” that Kroodsma targets, I find myself leaning toward the rip it up and start over camp.

  4. Don Kroodsma says:

    MORE WIND

    [yes, no, yes, no . . . do I press “Submit Comment,” or not . . .yes, no. . . yes (gulp) wins]

    It’s me. The cranky old fart. Set in his ways. Can’t handle new methods . . . OR, the wise old whistle blower exposing shoddy research. But you’ll never find out which from the ornithologists in the house, at least not from those who sign their comments, as illustrated by the Lahti post and as predicted by me in a private email to Andrew. Lahti essentially says “I know birdsong, but you have to read all the material and decide for yourself; I’m not going to help you by offering my opinion.” Unstated is “I don’t dare,” and I completely understand.

    I didn’t become cranky as I aged; I was already cranky as a youngster, and braced for the consequences. Almost 30 years ago I challenged birdsong colleagues on pseudoreplication, what seems like a tame issue now; I was immediately challenged in print by Searcy, who would later become a semi-mentor to Podos. I addressed other malpractices in our field of avian bioacoustics as well, pushing for better science.

    And there were CONSEQUENCES. But after rediscovering the joy of life on a bicycle journey from Virginia to Oregon during the summer of 2003 (Listening to a Continent Sing, Princeton Univ. Press), I left academics at age 57, to begin a new life writing about the magic of birdsong for real people (see http://donaldkroodsma.com/). I’m 71 now. It’s been a wonderful, refreshing ride through life since then, but after a 10-year blissful hiatus, three years ago I got sucked into the old battles once again, by now with three generations of the same crowd; the battle is even more intense this time around, because I continue to care deeply about what birds actually do, and I resent deeply all the shoddy science and falsehoods permeating the literature. Maybe, I thought (naively, of course), if I’m even more blunt (and cranky) this time around I could stamp out some of this nonsense.

    In a private email to Andrew, I had predicted that few, if any, experts on birdsong would openly step forward with an opinion, simply because they couldn’t afford to. Lahti, early in his career, depends on his lifeline to his postdoctoral advisor Podos, a member of a fecund, prolific, influential group from Duke (mentor Nowicki, with close colleague Searcy). It would be professional suicide for Lahti to destroy his lifeline and say Kroodsma is right, and he knows better than to say Podos is right, because he knows that “the winds have changed,” and they’re getting stronger (driving rain too, of course, rivers rising, you name it). No one who is still in the business of studying birdsong (i.e., relying on grant money, jobs, publications, promotions, etc.) can afford to take sides openly.

    In an email to me, Andrew said it was ok if ornithologists wanted to post comments anonymously. That seems gutless, but it’s the only way to move comments beyond Lahti’s vague response. The “mightysparrow” has now weighed in, anonymously. Good. But who is this person who smartly chimes in anonymously, you have to wonder? Must be a pawn of mine, one would suspect. One thing is clear: The sparrow appreciates Andrew’s blogs.

    Perhaps I am out of line by offering any opinions on this topic, as I should let others sort it out independently. But, I go to Andrew’s “Winds” again, to the last paragraph, and see how Fiske was encouraged to respond. This is an open dialogue, not terrorism, and Lord knows how many open dialogues I have attempted to have with Podos over the last three years and been denied (n = 8.0, to be exact, all documented in the criminal harassment item below). Maybe this is the open forum where he and I can exchange ideas about what constitutes appropriate scientific and ethical conduct, a conversation to be held at everyone else’s expense, or amusement, or edification. Feathers might fly. Let’s see.

    * * * * * * * *

    THE PROBLEMS WITH PODOS ET AL. AND THEIR RESEARCH ARE LEGION

    Let me list a few of the problems (quotes from Andrew’s “Winds are blowing” blog), in more candid terms than were allowed by editors of the Animal Behavior journal (where it was important that no one’s feelings were hurt). In my opinion . . .

    1. Podos et al. wholeheartedly adopt the “ . . .find-statistical-signficance-any-way-you-can-and-declare-victory paradigm.”

    2. Podos et al. follow “ . . . what I’ve sometimes called the research incumbency rule: that, once an article is published in some approved venue, it should be taken as truth . . .”

    3. Podos et al. exemplify “ . . . the deadly combination of weak theory being supported almost entirely by statistically significant results which themselves are the product of uncontrolled researcher degrees of freedom.”

    4. Podos et al. have “ . . . huge, obvious multiple comparisons problems. . .”

    5. Podos et al. reveal the “ . . . connection between scientific fraud, sloppiness, and plain old incompetence . . . ” Indeed, here’s an email I wrote to Podos on 8 October 2014: “I have no idea what is in your head . . . Only two possibilities come to mind: 1) You truly believe you are doing fine research and learning about bird song. . . . 2) Research and publishing are a game not to be taken too seriously, and it’s no big deal if what you write has no semblance of truth, no big deal that you dupe the vast majority of readers into believing things you know not to be true.“ The first is incompetence, the second fraud, the results of either process indistinguishable from the other in the literature. I received no answer.

    6. Podos et al. illustrate the following, that “the real ‘conclusion of the paper’ doesn’t depend on any of its details—all that matters is that there’s something, somewhere, that has p less than .05, because that’s enough to make publishable, promotable claims about ‘the pervasiveness and persistence of . . .’ whatever . . . they want to publish that day. When the authors protest that none of the errors really matter, it makes you realize that, in these projects, the data hardly matter at all.”

    7. Podos et al. use “ . . . the paradigm of the open-ended theory, of publication in top journals and promotion in the popular and business press, based on ‘p less than .05’ results obtained using abundant researcher degrees of freedom. It’s the paradigm of the theory that . . . is ‘more vampirical than empirical—unable to be killed by mere data’.”

    8. Podos et al. reveal a cultural transmission of research and publication techniques, across three generations, with “collaborators and former students . . . [and mentors all using] . . . similar research styles, favoring flexible hypotheses, proof-by-statistical-significance, and an unserious attitude toward criticism.”

    9. Podos and his colleagues “ . . . followed a certain path which has given them fame, fortune, and acclaim. Question the path, and you question the legitimacy of all that came from it. And that can’t be pleasant . . . [We all] . . . spend our professional lives building up a small fortune of coin in the form of publications and citations, and it’s painful to see that devalued.”

    10. Podos et al. do not acknowledge the simplest of alternative explanations for their data, and instead point out how consistent their data are with their chosen explanation, which not coincidentally happens to be their performance paradigm. “ . . . the data hardly matter . . .”

    11. Podos operates in extreme secrecy, refusing to communicate, hiding . . . from what? (See items 1 and 2 below for examples.) This secretive behavior is forbidden by NSF for Podos’ federally funded research, and just downright unethical according to the publishing guidelines of the Animal Behavior Society, of which Podos himself is President.

    12. And more . . . but I rest, lest someone might think I protest too much and am too WINDY!

    * * * * * * * *

    CRANKINESS DEFINED

    I’d hate to have anyone underappreciate how cranky I am (but perhaps keep open the possibility that I could be both cranky and wise). You can read all about the details at http://donaldkroodsma.com/?page_id=1596. Here’s some evidence:

    1. I’m so cranky that I’ve dared to ask questions about the research of Podos and his students, only to be stonewalled and then threatened with criminal harassment charges by the University of Massachusetts police if I ask one more question of this other group of birdsong biologists in my own Biology Department. Furthermore, the police have informed me that I must tell correspondents worldwide that none of them are allowed to talk to Podos either. (This is a novel approach to trying to silence critics. I suggest rereading the section about “collaborators” and “adversaries” in the Winds blog.)

    2. I’m cranky enough that I asked Biology Letters to retract Goodwin and Podos (2014), and an open exchange was about to ensue; then, however, a confidential communication arrived from the Dean of the UMass graduate school, submitted by Podos. Top-secret. I am not allowed to know the contents. End of discussion. Biology Letters says, sorry, but “per university rules” the contents of the letter cannot be revealed. Good-bye. But there’s one small problem: the dean (John McCarthy, Vice Provost for Graduate Education and Dean of Graduate School, and Distinguished Professor of Linguistics—I love titles) has no idea who wrote this letter that supposedly had come from him. End of story, as he was not interested in finding out the source. . . . “Journals and authors [and university administrators] often apply massive resistance to bury criticisms.”—quote from Gelman’s Winds.

    3. Do you know anyone who’s cranky enough to ask a scientific organization that they retract a best student paper award, on the grounds that it was entirely false, because slick marketing and glitz had won the day? . . . My argument was the same as that advanced by Gelman in Winds: “Fiske expresses concerns for the careers of her friends, careers that may have been damaged by public airing of their research mistakes. Just remember that, for each of these people, there may well be three other young researchers who were doing careful, serious work but then didn’t get picked for a plum job or promotion because it was too hard to compete with other candidates who did sloppy but flashy work . . .”

    4. Any of you cranky (and tireless) enough to take what you perceive as misconduct to University Administrators, those who are charged with maintaining the quality of science and integrity of investigators at the university? To departmental chairs, multiple deans, several vice provosts, the chancellor and deputy chancellor and vice chancellors for this and that, and to the very top of the campus hierarchy, to the Provost & Sr. Vice Chancellor for Academic Affairs? Nothing. All this is perfectly acceptable behavior for scientists at our institution. Go UMass! “No surprise,” says NSF to me: “Universities are big business, and their number one priority is to protect their own. Get used to it.”

    5. I’m sufficiently cranky that, over ethical considerations, I resigned as a Fellow from my favorite scientific society, and then suggested to the Officers of the Society that they might want to ask President Podos himself to resign his position, thus sending a strong message of zero tolerance for scientific and ethical misconduct to everyone, especially those beginning their careers and wondering how to “get ahead.”

    * * * * * * * *

    All the gory details of these shenanigans and more are documented on my website, http://donaldkroodsma.com/?page_id=1596, documented there in part because I needed to have a full record of everything should I be taken to court for criminal harassment, but documented there mostly in disbelief—no one could make this stuff up. In the end, and I do hope I am near the end of my efforts, I inevitably ask myself how close I’ve come to stamping out these kinds of behaviors among birdsong biologists. Was it worth the effort and agony over the last three years? I have my doubts.

    One clarification, written to me by a referee of my published account in Animal Behavior after seeing the Bird Fight! post: “I think he [Gelman] may misunderstand some of the sociology of the Kroodsma-Podos battle, though, as a full professor and president of the Animal Behavior Society does not quite qualify as an upstart ‘youngster’. Podos seems more like another stock character in Gelman’s world, the established scientist who can’t handle the possibility that he might have made a mistake. See http://andrewgelman.com/2016/09/21/what-has-happened-down-here-is-the-winds-have-changed/, . . . as much of what he describes seems to match up with your experience.”

    Back in 2004, Podos might have been called an “upstart youngster” in my Biology Department when I scolded him with the following message in a review of his manuscript: “Science is the search for truth regardless of how good the story is, whereas ‘marketing or advertising’ is the search for a good story regardless of the truth.” After my 10-year hiatus from academics, I returned in shock to see how successful this marketing had been. Since that review of his manuscript during late 2004, Podos has not spoken to me or communicated with me in any way, except via the UMass Police, even though we have ties to the same academic department and both live in the same small New England town.

    A blunt summary: Podos et al. should “ . . . step back and think that maybe almost everything they’ve been doing for years is all a mistake . . . that’s a big jump to take. Indeed, they’ll probably never take it. All the incentives fall in the other direction” (from Winds, of course).

    The movie rights to this sorry saga? They’re all mine. I’m hoping Redford will play me.

    Enough said on my part. I welcome an open dialogue from anyone, especially Podos.

    –Don Kroodsma

Leave a Reply