From August 1990. It was in the form of a note sent to all the people in the statistics group of Bell Labs, where I’d worked that summer. To all: Here’s the abstract of the work I’ve done this summer. It’s stored in the file, /fs5/gelman/abstract.bell, and copies of the Figures 1-3 are on Trevor’s [...]
Archive of posts filed under the Bayesian Statistics category.
Understanding simulations in terms of predictive inference?
David Hogg writes: My (now deceased) collaborator and guru in all things inference, Sam Roweis, used to emphasize to me that we should evaluate models in the data space — not the parameter space — because models are always effectively “effective” and not really, fundamentally true. Or, in other words, models should be compared in [...]
Non-Bayesian analysis of Bayesian agents?
Econometrician and statistician Dale Poirier writes: 24 years ago (1988, Journal of Economics Perspectives) I [Poirier] noted cognitive dissonance among some economists who treat the agents in their theoretical framework as Bayesians, but then analyze the data (even in the same paper!) as a frequentist. Recently, I have found similar cases in cognitive science. I [...]
Believe your models (up to the point that you abandon them)
In a discussion of his variant of the write-a-thousand-words-a-day strategy (as he puts it, “a system for the production of academic results in writing”), Thomas Basbøll writes: Believe the claims you are making. That is, confine yourself to making claims you believe. I always emphasize this when I [Basbøll] define knowledge as “justified, true belief”. [...]
Demystifying Blup
In our recent thread on computing hierarchical models with big datasets, someone brought up Blup. I thought it might be worth explaining what Blup is and how it relates to hierarchical models. Blup stands for Best Linear Unbiased Prediction, but in my terminology it’s just hierarchical modeling. Let me break it down: – “Best” doesn’t [...]
“Not only defended but also applied”: The perceived absurdity of Bayesian inference
Updated version of my paper with Xian: The missionary zeal of many Bayesians of old has been matched, in the other direction, by an attitude among some theoreticians that Bayesian methods are absurd—not merely misguided but obviously wrong in principle. We consider several examples, beginning with Feller’s classic text on probability theory and continuing with [...]
The Naval Research Lab
I worked at the U.S. Naval Research Laboratory for four summers during high school and college. I spent much of my time writing a computer program to do thermal analysis for an experiment that we put on the space shuttle. The facility I developed with the finite-element method came in handy in my job at [...]
How things sound to us, versus how they sound to others
Hykel Hosni noticed this bit from the Lindley Prize page of the Society for Bayesan Analysis: Lindley became a great missionary for the Bayesian gospel. The atmosphere of the Bayesian revival is captured in a comment by Rivett on Lindley’s move to University College London and the premier chair of statistics in Britain: “it was [...]
17 groups, 6 group-level predictors: What to do?
Yi-Chun Ou writes:
More philosophy of Bayes
Konrad Scheffler writes: I was interested by your paper “Induction and deduction in Bayesian data analysis” and was wondering if you would entertain a few questions:
Continuous variables in Bayesian networks
Antti Rasinen writes: I’m a former undergrad machine learning student and a current software engineer with a Bayesian hobby. Today my two worlds collided. I ask for some enlightenment. On your blog you’ve repeatedly advocated continuous distributions with Bayesian models. Today I read this article by Ricky Ho, who writes: The strength of Bayesian network [...]
Whassup with deviance having a high posterior correlation with a parameter in the model?
Jean Richardson writes:
Modeling group-level predictors in a multilevel regression
Trey Causey writes: Do you have suggestions as to model selection strategies akin to Bayesian model averaging for multilevel models when level-2 inputs are of substantive interest? I [Causey] have seen plenty of R packages and procedures for non-multilevel models, and tried the glmulti package but found that it did not perform well with more [...]
As a Bayesian I want scientists to report their data non-Bayesianly
Philipp Doebler writes:
Gelman on Hennig on Gelman on Bayes
Deborah Mayo pointed me to this discussion by Christian Hennig of my recent article on Induction and Deduction in Bayesian Data Analysis. A couple days ago I responded to comments by Mayo, Stephen Senn, and Larry Wasserman. I will respond to Hennig by pulling out paragraphs from his discussion and then replying. Hennig: for me [...]
95% intervals that I don’t believe, because they’re from a flat prior I don’t believe
Arnaud Trolle (no relation) writes: I have a question about the interpretation of (non-)overlapping of 95% credibility intervals. In a Bayesian ANOVA (a within-subjects one), I computed 95% credibility intervals about the main effects of a factor. I’d like to compare two by two the main effects across the different conditions of the factor. Can [...]
Coming to agreement on philosophy of statistics
Deborah Mayo collected some reactions to my recent article, Induction and Deduction in Bayesian Data Analysis. I’m pleased that that everybody (philosopher Mayo, applied statistician Stephen Senn, and theoretical statistician Larry Wasserman) is so positive about my article and that nobody’s defending the sort of hard-core inductivism that’s featured on the Bayesian inference wikipedia page. [...]
Inference = data + model
A recent article on global warming reminded me of the difficulty of letting the data speak. William Nordhaus shows the following graph:
Any available cookbooks on Bayesian designs?
Robert Bell writes: I [Bell] just finished “The Emperor of All Maladies” from 2010. One of the sections is captioned with the “In God we trust, all others bring data” quote which is supposedly from Deming. If you haven’t read it, I think you might like it even though the topic is pretty morbid (i.e. [...]
Untangling the Jeffreys-Lindley paradox
Ryan Ickert writes: I was wondering if you’d seen this post, by a particle physicist with some degree of influence. Dr. Dorigo works at CERN and Fermilab. The penultimate paragraph is: From the above expression, the Frequentist researcher concludes that the tracker is indeed biased, and rejects the null hypothesis H0, since there is a [...]
Philosophy: Pointer to Salmon
Larry Brownstein writes: I read your article on induction and deduction and your comments on Deborah Mayo’s approach and thought you might find the following useful in this discussion. It is Wesley Salmon’s Reality and Rationality (2005). Here he argues that Bayesian inferential procedures can replace the hypothetical-deductive method aka the Hempel-Oppenheim theory of explanation. [...]
Philosophy of Bayesian statistics: my reactions to Wasserman
Continuing with my discussion of the articles in the special issue of the journal Rationality, Markets and Morals on the philosophy of Bayesian statistics: Larry Wasserman, “Low Assumptions, High Dimensions”: This article was refreshing to me because it was so different from anything I’ve seen before. Larry works in a statistics department and I work [...]
Adding an error model to a deterministic model
Daniel Lakeland asks, “Where do likelihoods come from?” He describes a class of problems where you have a deterministic dynamic model that you want to fit to data. The data won’t fit perfectly so, if you want to do Bayesian inference, you need to introduce an error model. This looks a little bit different from [...]
Philosophy of Bayesian statistics: my reactions to Hendry
Continuing with my discussion here and here of the articles in the special issue of the journal Rationality, Markets and Morals on the philosophy of Bayesian statistics: David Hendry, “Empirical Economic Model Discovery and Theory Evaluation”: Hendry presents a wide-ranging overview of scientific learning, with an interesting comparison of physical with social sciences. (For some [...]
Bayesian model-building by pure thought: Some principles and examples
This is one of my favorite papers: In applications, statistical models are often restricted to what produces reasonable estimates based on the data at hand. In many cases, however, the principles that allow a model to be restricted can be derived theoretically, in the absence of any data and with minimal applied context. We illustrate [...]