Radford Neal’s blog

Radford’s a leading researcher in statistical computing. He started a new blog. Radford writes:

Many of my technical posts will point out flaws in research, methods, and tools that are commonly used. Such negative comments are essential to the scientific enterprise, being the social counterpart of the crucial role of self-criticism in individual research. I find that one of the main challenges in supervising PhD students is getting them to constantly ask whether their results, or the results of others, might be wrong. The aim in research is to discard bad ideas quickly, and with minimal effort. For this a blog is much more efficient than formal publication.

I hope he posts some positive things too. I mean, our blog here could be all Kanazawa all the time, and that would be fun for awhile, but generally it’s much more of a challenge to make a positive contribution than a negative contribution, so I hope Radford applies his blogging talents, as he applies his research talents, to this area.

P.S. Here’s my favorite Radford Neal quote. It’s from his Ph.D. thesis:

Sometimes a simple model will outperform a more complex model . . . Nevertheless, I believe that deliberately limiting the complexity of the model is not fruitful when the problem is evidently complex. Instead, if a simple model is found that outperforms some particular complex model, the appropriate response is to define a different complex model that captures whatever aspect of the problem led to the simple model performing well.

4 thoughts on “Radford Neal’s blog

  1. I dunno the intuition provided by Hastie gives a reason that won't be true:
    when there is more things to estimate in your model you have more variability in the estimates.

    The intuition says that even though a process underlying what you are modelling might be complex; if it is *well* approximated by a simple model albeit it in a biased manner than take the simple model bias and skip the variability.

    I am not really an expert–just to a student with apparently to conflicting 'catch-all statements'. Though to be fair Hastie doesn't say anything as cras as 'take the bias'. He just points out that degrees of freedom is why a realistic model is not always better than a simple model.

  2. I'm curious as to what Radford's example is of

    "the appropriate response is to define a different complex model that captures whatever aspect of the problem led to the simple model performing well."

    Thanks for the link; I'll go ask him.

Comments are closed.