Nathaniel Egwu writes:
I am a PhD student working on machine learning using artificial neural networks . . . Do you have some recent publications related to how one can construct priors depending on the type of input data available for training? I intend to construct a prior distribution for a given trade-off parameter of my non model obtained through training a neural network. At this stage, my argument is due to the fact that Bayesian nonparameteric estimation offers some insight on how to proceed on this problem.
As I’ve been writing here for awhile, I’ve been interested in weakly informative priors. But I have little experience with nonparametric models. Perhaps Aki Vehtari or David Dunson or some other expert on these models can discuss how to set them up with weakly informative priors? This sounds like it could be important to me.
I don’t think there is much difference in how think about weakly informative priors for nonparametric models. For example, Andrew’s ideas about weakly informative priors for weights in logistic models can be used for nonparametric models as long as you know which hyperparameters control the magnitude of the latent function and thus affect the scale of the a priori predictive distribution. Radford Neal’s 1994 book contains useful material about what kind of effects hyperpriors of neural networks have on a priori assumptions in function space.