No conf intervals? No problem (if you got replication).

This came up in a research discussion the other day. Someone had produced some estimates, and there was a question: where are the conf intervals. I said that if you have replication and you graph the estimates that were produced, then you don’t really need conf intervals (or, for that matter, p-values). The idea is that the display of the different estimates (produced from different years, or different groups, or different scenarios, or whatever) gives you the relevant scale of variation. In general, this is even better than confidence intervals in that the variation is visually clear and less assumption-based.

What I’m saying is, use the secret weapon.

11 thoughts on “No conf intervals? No problem (if you got replication).

  1. I strongly agree.

    But an exception to the rule could be what if this was a meta-analysis, in particular point estimates extracted from published papers. Then I might want to include the CI’s, for a non-traditional reason; if all the CI’s had lower bounds close to 0, I would think for sure I have an upwardly biased estimate in averaging across the point estimates.

    I’m not an expert in the field of meta-analysis. Does anyone know if they have methods that attempt to reduce the upward bias generated by the p = 0.05 filter?

Leave a Reply to Andrew Cancel reply

Your email address will not be published. Required fields are marked *