I often include references to CS Peirce in my comments. Some might think way too often. However, this whole post will be trying to extract some morsels of insight from some of his later work. With the hope that it will enable applying statistics more thoughtfully. Now, making sense of Peirce, that is getting him right, is an onerous task. The huge volume of his work is comprised of mainly unfinished or unpublished manuscripts, unsuccessful grant applications, lecture notes (some where his arm was twisted to dumb them down for the students) and a selection of published articles. And those mainly in popular as opposed to scholarly journals. His work also evolved and changed (and became publicly available at different points in time). Given that, here I will just try to extract some insight that won’t really be able to stand on authority. But then, authority based claims are the weakest of claims anyhow. However, consider this post risky reading. As my adviser in Oxford used to say “You have been warned”.

Peirce’s primary focus in his career was on logic. Until late in his career, he considered ethics and aesthetics to be largely frivolous topics. Then around 1900 he saw them as absolutely necessary to understanding logic. His thinking was that you first need to decide what you value above all, second how one should deliberately act to best obtain what you value and third how you should best represent what you plan to act upon prior to acting in the world. The “how one should best represent” to profitably advance inquiry being logic. So Aesthetics -> Ethics -> Logic.

Peirce took aesthetics to be the topic of what you should value above all regardless of any ulterior purposes. The ultimate thing to value above all he took to be the grasping of reality as being reasonable. That is being construed in ways seen to be capable of being understood rather than being mysterious. Even though we have no direct access to it. He took ethics to be the topic of deliberate controlled acting to achieve desired goals (the ultimate goal being reasonableness). He took logic to be the topic of deliberate controlled thinking or representation (which entails unending re-representation). To think about reality we need to somehow represent it that is not too wrong and re-represent it in equivalent ways – avoiding making it even more wrong.

The latter being a more common concern of logic being about truth preservation. But before and continually as we act on reality, we need to assess if our representation is too wrong to allow acting without being frustrated by reality. That is, delaying deliberate acceptance until we are satisfied we can’t do better. The first he thought of as being abduction, the second deduction and the third induction – the three topics of his total concept of logic. That is represent, re-represent faithfully and critically check what profitably should be doubted. Doubt, according to Peirce, being an art which has to be acquired with difficulty.

If aesthetics is taken as the theory of the objectively admirable without ulterior reasons – as ends simply as they present themselves – in statistics it might be the grasping of the real uncertainties of learning about that reality that is beyond us from planned and unplanned observations. If ethics is taken as the theory of deliberate self controlled conduct – as ends simply as they relate to actions and efforts – in statistics the list of virtues given here does seem a very good start. It would be here where the grasping would or would not be resolved into quantification. If logic is taken as the theory of deliberate thinking aimed at getting purposeful representations – as ends in regard to representation – in statistics it might be the discerning, modifying and tentatively keeping of joint probability models as representations of the realities that could have produced the observations we have.

Using this perspective, I attempt here to re-represent purposeful Bayesian inference to hopefully bringing its reasonableness into a clearer view.

In purposeful (or some say pragmatic) Bayesian approaches in statistical (in contrast to classical subjective or objective Bayesian approaches) there are always three intermingled aspects.

First (speculative inference) (1) choosing a (probabilistic) representation of how unknown quantities were set or came about (aka a prior), (2) a (probabilistic) representation of how the data in hand came about or were generated (aka data generating model or likelihood) and (3) how these two representations connect for a joint representation of an empirical happening to be interpreted more generally.

Second (quantitative inference),(1) revising the first representation (the prior) in light of the data to get the implied representation given by the joint representation _and_ the data (aka the posterior via Bayes theorem), (2) (conceptually) generating what data could result from such an implied representation (aka posterior predictive).

Third (evaluative inference), (1) choosing/guessing how the first and second steps might have been wrong and in light of this (2a) assessing the reasonableness of the data generating model to have generated the data in hand (checking for model data conflict) and then (2b) the reasonableness of the prior to have generated the parameters now most supported in the posterior (aka checking for prior data conflict), (3a) choosing what aspects of the joint representation might be made less wrong and how, (3b) working through the implications and how those fit with the data in hand and possibly past experience and finally (3c) deciding whether to settle on the joint representation as is and its implications for now or starting again at the first step.

All in all,

speculative inference -> quantitative inference -> evaluative inference or

abduction -> deduction -> induction -> or

First -> Second -> Third

Over and over again, endlessly.

Peirce: A Guide for the Perplexed. Cornelis de Waal – https://www.amazon.com/Peirce-Guide-Perplexed-Guides/dp/1847065163