Convincing Evidence

Keith O’Rourke and I wrote an article that begins:

Textbooks on statistics emphasize care and precision, via concepts such as reliability and validity in measurement, random sampling and treatment assignment in data collection, and causal identification and bias in estimation. But how do researchers decide what to believe and what to trust when choosing which statistical methods to use? How do they decide the credibility of methods? Statisticians and statistical practitioners seem to rely on a sense of anecdotal evidence based on personal experience and on the attitudes of trusted colleagues. Authorship, reputation, and past experience are thus central to decisions about statistical procedures.

It’s for a volume on theoretical or methodological research on authorship, functional roles, reputation, and credibility in social media, edited by Sorin Matei and Elisa Bertino.

4 thoughts on “Convincing Evidence

  1. Completely off topic, but this reminded me, do you have any idea when that COPSS 50th anniversary volume will be published? A number of people have released their papers online (you, Larry Wasserman, Grace Wahba, are there others?), but I have heard nothing more about it since early this year.

  2. On a purely technical point, I’m trying to make sense of this sentence from page 1:
    “Gigerenzer and colleagues have done some for deciding to use percentages versus natural frequencies for better understanding of analysis when those doing the analysis are medical students or faculty.”
    Should “done some” mean “done so”, or is there a word missing after “some”, or have I just not understood something? If “done some” refers to “fitting”, as in “psychometricians who fit item response models” in the previous sentence, then maybe it should say “done some such fitting”.

  3. I am surprised that the word “utility” does not appear in the article. Do you think that there are some ideas from decision theory and/or micro-economics that might be relevant to these questions?

Comments are closed.