In an email I sent to a colleague who’s writing about lasso and Bayesian regression for R users: The one thing you might want to add, to fit with your pragmatic perspective, is to point out that these different methods are optimal under different assumptions about the data. However, these assumptions are never true (even [...]
Archive of posts filed under the Bayesian Statistics category.
Plain old everyday Bayesianism!
Sam Behseta writes: There is a report by Martin Tingley and Peter Huybers in Nature on the unprecedented high temperatures at northern latitudes (Russia, Greenland, etc). What is more interesting is the authors are have used a straightforward hierarchical Bayes model, and for the first time (as far as I can remember) the results are [...]
More on Bayesian model selection in high-dimensional settings
David Rossell writes: A friend pointed out that you were having an interesting philosophical discussion on my paper with Val Johnson [on Bayesian model selection in high-dimensional settings]. I agree with the view that in almost all practical situations the true model is not in the set under consideration. Still, asking a model choice procedure [...]
A mess with which I am comfortable
Having established that survey weighting is a mess, I should also acknowledge that, by this standard, regression modeling is also a mess, involving many arbitrary choices of variable selection, transformations and modeling of interaction. Nonetheless, regression modeling is a mess with which I am comfortable and, perhaps more relevant to the discussion, can be extended [...]
Stan 1.3.0 and RStan 1.3.0 Ready for Action
The Stan Development Team is happy to announce that Stan 1.3.0 and RStan 1.3.0 are available for download. Follow the links on: Stan home page: http://mc-stan.org/ Please let us know if you have problems updating. Here’s the full set of release notes. v1.3.0 (12 April 2013) ====================================================================== Enhancements ———————————- Modeling Language * forward sampling (random [...]
X on JLP
Christian Robert writes on the Jeffreys-Lindley paradox. I have nothing to add to this beyond my recent comments: To me, the Lindley paradox falls apart because of its noninformative prior distribution on the parameter of interest. If you really think there’s a high probability the parameter is nearly exactly zero, I don’t see the point [...]
When is there “hidden structure in data” to be discovered?
Michael Collins sent along the following announcement for a talk: Fast learning algorithms for discovering the hidden structure in data Daniel Hsu, Microsoft Research 11am, Wednesday April 10th, Interschool lab, 7th floor CEPSR, Columbia University A major challenge in machine learning is to reliably and automatically discover hidden structure in data with minimal human intervention. [...]
Hierarchical array priors for ANOVA decompositions
Alexander Volfovsky and Peter Hoff write: ANOVA decompositions are a standard method for describing and estimating heterogeneity among the means of a response variable across levels of multiple categorical factors. In such a decomposition, the complete set of main effects and interaction terms can be viewed as a collection of vectors, matrices and arrays that [...]
Another Feller theory
My paper with Christian Robert, “Not Only Defended But Also Applied”: The Perceived Absurdity of Bayesian Inference, was recently published in The American Statistician, along with discussions by Steve Fienberg, Steve Stigler, Deborah Mayo, and Wesley Johnson, and our rejoinder, The Anti-Bayesian Moment and Its Passing. These articles revolved around the question of why the [...]
“Two Dogmas of Strong Objective Bayesianism”
Prasanta Bandyopadhyay and Gordon Brittan write: We introduce a distinction, unnoticed in the literature, between four varieties of objective Bayesianism. What we call ‘strong objective Bayesianism’ is characterized by two claims, that all scientific inference is ‘logical’ and that, given the same background information two agents will ascribe a unique probability to their priors. We [...]
The harm done by tests of significance
After seeing this recent discussion, Ezra Hauer sent along an article of his from the journal Accident Analysis and Prevention, describing three examples from accident research in which null hypothesis significance testing led researchers astray. Hauer writes: The problem is clear. Researchers obtain real data which, while noisy, time and again point in a certain [...]
Stan at Google this Thurs and at Berkeley this Fri noon
Michael Betancourt will be speaking at Google and at the University of California, Berkeley. The Google talk is closed to outsiders (but if you work at Google, you should go!); the Berkeley talk is open to all: Friday March 22, 12:10 pm, Evans Hall 1011. Title of talk: Stan: Practical Bayesian Inference with Hamiltonian Monte [...]
Tibshirani announces new research result: A significance test for the lasso
Lasso and me For a long time I was wrong about lasso. Lasso (“least absolute shrinkage and selection operator”) is a regularization procedure that shrinks regression coefficients toward zero, and in its basic form is equivalent to maximum penalized likelihood estimation with a penalty function that is proportional to the sum of the absolute values [...]
Everyone’s trading bias for variance at some point, it’s just done at different places in the analyses
Some things I respect When it comes to meta-models of statistics, here are two philosophies that I respect: 1. (My) Bayesian approach, which I associate with E. T. Jaynes, in which you construct models with strong assumptions, ride your models hard, check their fit to data, and then scrap them and improve them as necessary. [...]
Misunderstanding the p-value
The New York Times has a feature in its Tuesday science section, Take a Number, to which I occasionally contribute (see here and here). Today’s column, by Nicholas Balakar, is in error. The column begins: When medical researchers report their findings, they need to know whether their result is a real effect of what they [...]
My problem with the Lindley paradox
From a couple years ago but still relevant, I think: To me, the Lindley paradox falls apart because of its noninformative prior distribution on the parameter of interest. If you really think there’s a high probability the parameter is nearly exactly zero, I don’t see the point of the model saying that you have no [...]
Stan 1.2.0 and RStan 1.2.0
Stan 1.2.0 and RStan 1.2.0 are now available for download. See: http://mc-stan.org/ Here are the highlights. Full Mass Matrix Estimation during Warmup Yuanjun Gao, a first-year grad student here at Columbia (!), built a regularized mass-matrix estimator. This helps for posteriors with high correlation among parameters and varying scales. We’re still testing this ourselves, so [...]
Stan in L.A. this Wed 3:30pm
Michael Betancourt will be speaking at UCLA: The location for refreshment is in room 51-254 CHS at 3:00 PM. The place for the seminar is at CHS 33-105A at 3:30pm – 4:30pm, Wed 6 Mar. ["CHS" stands for Center for Health Sciences, the building of the UCLA schools of medicine and public health. Here's a [...]
PyStan!
Stan is written in C++ and can be run from the command line and from R. We’d like for Python users to be able to run Stan as well. If anyone is interested in doing this, please let us know and we’d be happy to work with you on it. Stan, like Python, is completely [...]
An AI can build and try out statistical models using an open-ended generative grammar
David Duvenaud writes: I’ve been following your recent discussions about how an AI could do statistics [see also here]. I was especially excited about your suggestion for new statistical methods using “a language-like approach to recursively creating new models from a specified list of distributions and transformations, and an automatic approach to checking model fit.” [...]
Zero Dark Thirty and Bayes’ theorem
A moviegoing colleague writes: I just watched the movie Zero Dark Thirty about the hunt for Osama Bin Laden. What struck me about it was: (1) Bayes theorem underlies the whole movie; (2) CIA top brass do not know Bayes theorem (at least as portrayed in the movie). Obviously one does not need to know [...]
Wacky priors can work well?
Dave Judkins writes: I would love to see a blog entry on this article, Bayesian Model Selection in High-Dimensional Settings, by Valen Johnson and David Rossell. The simulation results are very encouraging although the choice of colors for some of the graphics is unfortunate. Unless I am colorblind in some way that I am unaware [...]
Toward a framework for automatic model building
Patrick Caldon writes:
Why waste time philosophizing?
I’ll answer the above question after first sharing some background and history on the the philosophy of Bayesian statistics, which appeared at the end of our rejoinder to the discussion to which I linked the other day: When we were beginning our statistical educations, the word ‘Bayesian’ conveyed membership in an obscure cult. Statisticians who [...]
P-values and statistical practice
From my new article in the journal Epidemiology: Sander Greenland and Charles Poole accept that P values are here to stay but recognize that some of their most common interpretations have problems. The casual view of the P value as posterior probability of the truth of the null hypothesis is false and not even close [...]