Skip to content
Archive of entries posted by

When LOO and other cross-validation approaches are valid

Introduction Zacco asked in Stan discourse whether leave-one-out (LOO) cross-validation is valid for phylogenetic models. He also referred to Dan’s excellent blog post which mentioned iid assumption. Instead of iid it would be better to talk about exchangeability assumption, but I (Aki) got a bit lost in my discourse answer (so don’t bother to go […]

Parsimonious principle vs integration over all uncertainties

tl;dr If you have bad models, bad priors or bad inference choose the simplest possible model. If you have good models, good priors, good inference, use the most elaborate model for predictions. To make interpretation easier you may use a smaller model with similar predictive performance as the most elaborate model. Merijn Mestdagh emailed me […]

Comments on Limitations of Bayesian Leave-One-Out Cross-Validation for Model Selection

There is a recent pre-print Limitations of Bayesian Leave-One-Out Cross-Validation for Model Selection by Quentin Gronau and Eric-Jan Wagenmakers. Wagenmakers asked for comments and so here are my comments. Short version: They report a known limitation of LOO when it’s used in a non-recommended way for model selection. They report that their experiments show that […]

Aki’s favorite scientific books (so far)

A month ago I (Aki) started a series of tweets about “scientific books which have had big influence on me…”. They are partially in time order, but I can’t remember the exact order. I may have forgotten some, and some stretched the original idea, but I can recommend all of them. I have collected all […]

The curse of dimensionality and finite vs. asymptotic convergence results

Related to our (Aki, Andrew, Jonah) Pareto smoothed importance sampling paper I (Aki) received a few times a comment that why bother with Pareto smoothing as you can always choose the proposal distribution so that importance ratios are bounded and then central limit theorem holds. The curse of dimensionality here is that the papers they […]

Stacking and multiverse

It’s a coincidence that there is another multiverse posting today. Recently Tim Disher asked in Stan discussion forum a question “Multiverse analysis – concatenating posteriors?” Tim refers to a paper “Increasing Transparency Through a Multiverse Analysis” by Sara Steegen, Francis Tuerlinckx, Andrew Gelman, and Wolf Vanpaemel. The abstract says Empirical research inevitably includes constructing a […]

StanCon 2018 Helsinki, 29-31 August 2018

Photo (c) Visit Helsinki / Jussi Hellsten StanCon 2018 Asilomar was so much fun that we are organizing StanCon 2018 Helsinki August 29-31, 2018 at Aalto University, Helsinki, Finland (location chosen using antithetic sampling). Full information is available at StanCon 2018 Helsinki website Summary of the information What: One day of tutorials and two days […]

Custom Distribution Solutions

I (Aki) recently made a case study that demonstrates how to implement user defined probability functions in Stan language (case study, git repo). As an example I use the generalized Pareto distribution (GPD) to model extreme values of geomagnetic storm data from the World Data Center for Geomagnetism. Stan has had support for user defined […]

Postdoc in Finland and NY to work on probabilistic inference and Stan!

I (Aki) got 2 year funding to hire a postdoc to work on validation of probabilistic inference approaches and model selection in Stan. Work would be done with Stan team in Aalto, Helsinki and Columbia, New York. We probably have PhD positions, too. The funding is part of the joint project with Antti Honkela and […]

Tenure-Track or Tenured Prof. in Machine Learning in Aalto, Finland

This job advertisement for a position in Aalto, Finland, is by Aki We are looking for a professor to either further strengthen our strong research fields, with keywords including statistical machine learning, probabilistic modelling, Bayesian inference, kernel methods, computational statistics, or complementing them with deep learning. Collaboration with other fields is welcome, with local opportunities […]

Stacking, pseudo-BMA, and AIC type weights for combining Bayesian predictive distributions

This post is by Aki. We have often been asked in the Stan user forum how to do model combination for Stan models. Bayesian model averaging (BMA) by computing marginal likelihoods is challenging in theory and even more challenging in practice using only the MCMC samples obtained from the full model posteriors. Some users have […]

Research fellow, postdoc, and PhD positions in probabilistic modeling and machine learning in Finland

Probabilistic modeling and machine learning are strong in Finland. Now is your opportunity to join us in this cool country! There are several postdoc and research fellow positions open in probabilistic machine learning in Aalto University and University of Helsinki (deadline Marh 19). Some of the topics are related also to probabilistic programming and Stan […]

Practical Bayesian model evaluation in Stan and rstanarm using leave-one-out cross-validation

Our (Aki, Andrew and Jonah) paper Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC was recently published in Statistics and Computing. In the paper we show why it’s better to use LOO instead of WAIC for model evaluation how to compute LOO quickly and reliably using the full posterior sample how Pareto smoothing importance […]

Tenure Track Professor in Machine Learning, Aalto University, Finland

Posted by Aki. I promise that next time I’ll post something else than a job advertisement, but before that here’s another great opportunity to join Aalto Univeristy where I work, too. “We are looking for a professor to either further strengthen our strong research fields, with keywords including statistical machine learning, probabilistic modelling, Bayesian inference, […]

Several postdoc positions in probabilistic modeling and machine learning in Aalto, Helsinki

This post is by Aki In addition to the postdoc position I advertised recently, now Aalto University and University of Helsinki have 20 more open postdoc and research fellow positions. Many of the positions are in probabilistic models and machine learning. You could work with me (I’m also part of HIIT), but I can also […]

R demos for BDA3

Last year we published some Matlab/Octave and Python demos for BDA3. During the summer my student Markus Paasiniemi ported these demos to R. New R BDA3 demos are now available in github. We hope these are helpful for someone. They are now just R code, although R Markdown would be cool. Btw. we are expecting […]

Postdoc in Finland with Aki

I’m looking for a postdoc to work with me at Aalto University, Finland. The person hired will participate in research on Gaussian processes, functional constraints, big data, approximative Bayesian inference, model selection and assessment, deep learning, and survival analysis models (e.g. cardiovascular diseases and cancer). Methods will be implemented mostly in GPy and Stan. The […]

Postdoctoral Researcher and Research Fellow positions in Computer Science in Helsinki, Finland

There are several PostDoc positions open in Aalto University and University of Helsinki related to statistical modeling, Bayesian inference, probabilistic programming (including Stan) and machine learning. There is also possibility to collaborate with me :) See a detailed list of the research areas and the full call text. The deadline is April 1, 2016. My […]

Stan – The Bayesian Data Scientist’s Best Friend

My friend Juuso Parkkinen has interesting Stan related blog, which is worth following. The above cool animation is from today’s post discussing the updated results of using Stan to model apartment prices in Finland. Few weeks ago Juuso also blogged about a probabilistic programming seminar in Finland with a title Stan – The Bayesian Data […]

Phd positions in Probabilistic Machine Learning at #AaltoPML group Finland

There are PhD positions in our Probabilistic Machine Learning group at Aalto, Finland, and altogether 15 positions in Helsinki ICT network. Apply here The most interesting topic in the call is supervised by Prof. Samuel Kaski at AaltoPML (and you may collaborate with me too :) We are looking for PhD candidates interested in probabilistic […]