Skip to content
Archive of posts filed under the Decision Theory category.

Inauthentic leadership? Development and validation of methods-based criticism

Thomas Basbøll writes: I need some help with a critique of a paper that is part of the apparently growing retraction scandal in leadership studies. Here’s Retraction Watch. The paper I want to look at is here: “Authentic Leadership: Development and Validation of a Theory-Based Measure” By F. O. Walumbwa, B. J. Avolio, W. L. […]

Economists betting on replication

Mark Patterson writes: A bunch of folks are collaborating on a project to replicate 18 experimental studies published in prominent Econ journals (mostly American Economic Review, a few Quarterly Journal of Economics). This is already pretty exciting, but the really cool bit is they’re opening a market (with real money) to predict which studies will […]

On deck this week

Mon: Discreteland and Continuousland Tues: “There are many studies showing . . .” Wed: An Excel add-in for regression analysis Thurs: Unreplicable Fri: Economists betting on replication Sat: Inauthentic leadership? Development and validation of methods-based criticism Sun: “Physical Models of Living Systems”

Humility needed in decision-making

Brian MacGillivray and Nick Pidgeon write: Daniel Gilbert maintains that people generally make bad decisions on risk issues, and suggests that communication strategies and education programmes would help (Nature 474, 275–277; 2011). This version of the deficit model pervades policy-making and branches of the social sciences. In this model, conflicts between expert and public perceptions […]

On deck this week

Mon: God is in every leaf of every probability puzzle Tues: Where does Mister P draw the line? Wed: Recently in the sister blog Thurs: Humility needed in decision-making Fri: “Why should anyone believe that? Why does it make sense to model a series of astronomical events as though they were spins of a roulette […]

On deck this week

Mon: Hey, what’s up with that x-axis?? Tues: A question about race based stratification Wed: Our new column in the Daily Beast Thurs: Irwin Shaw: “I might mistrust intellectuals, but I’d mistrust nonintellectuals even more.” Fri: An amusing window into folk genetics Sat: “Faith means belief in something concerning which doubt is theoretically possible.” — […]

“When more data steer us wrong: replications with the wrong dependent measure perpetuate erroneous conclusions”

Evan Heit sent in this article with Caren Rotello and Chad Dubé: There is a replication crisis in science, to which psychological research has not been immune: Many effects have proven uncomfortably difficult to reproduce. Although the reliability of data is a serious concern, we argue that there is a deeper and more insidious problem […]

In which a complete stranger offers me a bet

Piotr Mitros wrote to Deb and me: I read, with pleasure, your article about the impossibility of biasing a coin. I’m curious as to whether researchers believe what they write. Would you be willing to place some form of iterated bet? For example: I provide a two-sided coin and a table. The table looks like […]

Born-open data

Jeff Rouder writes: Although many researchers agree that scientific data should be open to scrutiny to ferret out poor analyses and outright fraud, most raw data sets are not available on demand. There are many reasons researchers do not open their data, and one is technical. It is often time consuming to prepare and archive […]

On deck this week

Mon: Because there is no observable certainty other than the existence of thought Tues: Michael LaCour in 20 years Wed: Born-open data Thurs: You can crush us, you can bruise us, yes, even shoot us, but oh—not a pie chart! Fri: In which a complete stranger offers me a bet Sat: Statistics Be Sun: “When […]

“The psychologists are getting a hard time for doing what they do, whereas people doing real harm to society are happily roaming around like free range chicken”

Shravan Vasishth writes: At least people like Amy Cuddy are just doing bullshit research that’s harmless (after all, raising your arms up high before an interview is unlikely to hurt society much). But check out this MIT “Professor” explaining the “statistically significant” autism-vaccine “connection”: She even takes a notorious, fraudulent, and retracted Lancet article as […]

On deck this week

Mon: “The psychologists are getting a hard time for doing what they do, whereas people doing real harm to society are happily roaming around like free range chicken” Tues: The posterior distribution of the likelihood ratio as a summary of evidence Wed: “Best Linear Unbiased Prediction” is exactly like the Holy Roman Empire Thurs: Applied […]

Should you get the blood transfusion?

Gur Huberman writes: Apropos Ethics & Logistic Regression, the piece you wrote with Madigan: In late 2001 I [Gur] broke my femur trying to rollerblade with my daughter. (No IQ award for that.) I had surgery and my recovery was slow. Every time I tried to get on crutches I’d collapse and faint. Diagnosis: Anemia. […]

A quick one

Fabio Rojas asks: Should I do Bonferroni adjustments? Pros? Cons? Do you have a blog post on this? Most social scientists don’t seem to be aware of this issue. My short answer is that if you’re fitting mutlilevel models, I don’t think you need multiple comparisons adjustments; see here.

Cross-validation != magic

In a post entitled “A subtle way to over-fit,” John Cook writes: If you train a model on a set of data, it should fit that data well. The hope, however, is that it will fit a new set of data well. So in machine learning and statistics, people split their data into two parts. […]

On deck this week

Mon: All the things that don’t make it into the news Tues: Cross-validation != magic Wed: Of buggy whips and moral hazards; or, Sympathy for the Aapor Thurs: Low-power pose Fri: Should you get the blood transfusion? Sat: “History is the prediction of the present” Sun: What to do to train to apply statistical models […]

“With that assurance, a scientist can report his or her work to the public, and the public can trust the work.”

Dan Wright writes: Given your healthy skepticism of findings/conclusions from post-peer-reviewed papers, I thought I would forward the following from Institute of Educational Sciences. Here is a sample quote: Simply put, peer review is a method by which scientists who are experts in a particular field examine another scientist’s work to verify that it makes […]

On deck this week

Mon: An inundation of significance tests Tues: Stock, flow, and two smoking regressions Wed: What’s the worst joke you’ve ever heard? Thurs: Cracked.com > Huffington Post, Wall Street Journal, New York Times Fri: Measurement is part of design Sat: “17 Baby Names You Didn’t Know Were Totally Made Up” Sun: What to do to train […]

Creativity is the ability to see relationships where none exist

Brent Goldfarb and Andrew King, in a paper to appear in the journal Strategic Management, write: In a recent issue of this journal, Bettis (2012) reports a conversation with a graduate student who forthrightly announced that he had been trained by faculty to “search for asterisks”. The student explained that he sifted through large databases […]

Can talk therapy halve the rate of cancer recurrence? How to think about the statistical significance of this finding? Is it just another example of the garden of forking paths?

James Coyne (who we last encountered in the sad story of Ellen Langer) writes: I’m writing to you now about another matter about which I hope you will offer an opinion. Here is a critique of a study, as well as the original study that claimed to find an effect of group psychotherapy on time […]