Someone asked me about the distinction between bias and noise and I sent him some links. Then I thought this might interest some of you too, so here it is:
Here’s a recent paper on election polling where we try to be explicit about what is bias and what is variance:
And here are some other things I’ve written on the topic:
– The bias-variance tradeoff
– Everyone’s trading bias for variance at some point, it’s just done at different places in the analyses
– There’s No Such Thing As Unbiased Estimation. And It’s a Good Thing, Too.
– Balancing bias and variance in the design of behavioral studies
Finally, here’s the sense in which variance and bias can’t quite be distinguished:
– An error term can be mathematically expressed as “variance” but if it only happens once or twice, it functions as “bias” in your experiment.
– Conversely, bias can vary. An experimental protocol could be positively biased one day and negatively biased another day or in another scenario.
P.S. These two posts are also relevant:
– How do you think about the values in a confidence interval?
(The question was “Are all values within the 95% CI equally likely (probable), or are the values at the “tails” of the 95% CI less likely than those in the middle of the CI closer to the point estimate?”
And my answer was: In general, No and It depends.)
– Why it doesn’t make sense in general to form confidence intervals by inverting hypothesis tests