Tom Ball writes:
Here’s another query to add to the stats backlog…Minimum Description Length (MDL). I’m attaching a 2002 Psych Rev paper on same. Basically, it’s an approach to model selection that replaces goodness of fit with generalizability or complexity.
Would be great to get your response to this approach.
My reply:
I’ve heard about the minimum description length principle for a long time but have never really understood it. So I have nothing to say! Anyone who has anything useful to say on the topic, feel free to add in the comments.
The rest of you might wonder why I posted this. I just thought it would be good for you to have some sense of the boundaries of my knowledge.
The link to the pdf seems broken.
Yeah, we still have some bugs in the blog transition. The link will appear soon, I hope.
Minimizing description length is essentially like Bayesian MAP.
The idea of MDL is to incorporate the model parameter's encoding length along with the actual data being compressed using the model into the measure of quality (in compression length). Arithmetic coders encode data with -log p_data bits if p_data is the predicted probability of the data item – so -log_2 of likelihood equals bit length. Encoding the parameters essentially corresponds to -log p_parameters under the prior.
Well, your correspondent wrote that it's an approach to model selection, so presumably you abhor it for the same reasons you abhor Bayes factors and model posterior probabilities…