Skip to content
 

Stan Weekly Roundup, 21 July 2017

It was another productive week in Stan land. The big news is that

  • Jonathan Auerbach, Tim Jones, Susanna Makela, Swupnil Sahai, and Robin Winstanley won first place in a New York City competition for predicting elementary school enrollment. Jonathan told me, “I heard 192 entered, and there were 5 finalists….Of course, we used Stan (RStan specifically). … Thought it might be Stan news worthy.”

    I’d say that’s newsworthy. Jon also provided a link to the “challenge” page, a New York City government sponsored “call for innovations”: Enhancing School Zoning Efforts by Predicting Population Change.

    They took home a US$20K paycheck for their efforts!

    Stan’s seeing quite a lot of use these days among demographers and others looking to predict forward from time series data. Jonathan’s been very active using government data sets (see his StanCon 2017 presentation with Rob Trangucci, Twelve Cities: Does lowering speed limits save pedestrian lives?). Hopefully they’ll share their code—maybe they already have. I really like to see this combination of government data and careful statistical analysis.

In other big news coming up soon,

In other news,

  • Andrew Gelman‘s been driving a lot of rethinking of our interfaces because he’s revising his and Jennifer Hill’s regression book (the revision will be two books). Specifically, he’s thinking a lot about workflow and how we can manage model expansion by going from fixed to modeled parameters. Right now, the process is onerous in that you have to move data variables into the parameters block and keep updating their priors. Andrew wants to be able to do this from the outside, but Michael Betancourt and I both expressed a lot of skepticism in terms of it breaking a lot of our fundamental abstractions (like a Stan program defining the density that’s fit!). More to come on this hot topic. Any ideas on how to manage developing a model would be appreciated. This goes back to the very first thing Matt Hoffman, Michael Malecki and I worked on with Andrew when he hired us before we’d conceived Stan. You’d think we’d have better advice on this after all this time. I’ve seen people do everything from use the C++ preprocessor to write elaborate program generation code in R.

  • Breck Baldwin has been working on governance and we’re converging on a workable model that we’ll share with everyone soon. The goal’s to make the governance clear and less of a smoke-filled room job by those of us who happen to go to lunch after the weekly meetings.

  • Jonah Gabry is taking on the ggplot2-ification of the new regression book and trying to roll everything into a combination of RStanArm and BayesPlot. No word yet if the rest of the tidyverse is to follow. Andrew said, “I’ll see what Jonah comes up with” or something to that effect.

  • Jonah has alos been working on priors for multilevel regression and poststratification with Yajuan Si (former postdoc of Andrew’s, now at U. Wisconsin); the trick is doing somethign reasonable when you have lots of interactions.

  • Ben Goodrich has been working on the next release of RStanArm. It’s beginning to garner a lot of contributions. Remember that the point has been to convert a lot of common point estimation packages to Bayesian versions and supply them with familiar R interfaces. Ben’s particularly been working on survival analysis lately.

  • Our high school student intern (don’t know if I can mention names online—the rest of our developers are adults!) is working on applying the Cook-Gelman-Rubin metric to evaluating various models. We’re doing much more of this method and it needs a less verbose and more descriptive name!

  • Mitzi Morris submitted a pull request for the Stan repo to add line numbers to error messages involving compound declare-and-define statements.
  • A joint effort by Mitzi Morris, Dan Simpson, Imad Ali, and Miguel A Martinez Beneito has led to convergence of models and fits among BUGS, Stan, and INLA for intrinsic conditional autorgression (ICAR) models. Imad’s building the result in RStanArm and has figured out how to extend the loo (leave one out cross-validation) package to deal with spatial models. Look for it in a Stan package near you soon. Mitzi’s working on the case study, which has been updated in the example-models repo.

  • Charles Margossian knocked off a paper on the mixed ODE solver he’s been working on, with a longer paper promised that goes through all the code details. Not sure if that’s on arXiv or not. He’s also been training Bill Gillespie to code in C++, which is great news for the project since Charles has to contend with his first year of grad school next year (whereas Bill’s facing a pleasant retirement of tracking down memory leaks). Charles is also working on getting the algebraic and fixed state solvers integrated into Torsten before the fall.

  • Krzysztof Sakrejda has a new paper out motivating a custom density he wrote in Stan for tracking dealys in diseseas incidence in a countrywide analysis for Thailand. I’m not sure where he put it.

  • Yuling Yao is revising the stacking paper (for a kind of improved model averaging). I believe this is going into the loo package (or is maybe already there). So much going on with Stan I can’t keep up!

  • Yuling also revised the simulated tempering paper, which is some custom code on top of Stan to fit models with limited multimodality. There was some discussion about realistic examples with limited multimodality and we hope to have a suite of them to go along with the paper.

  • Sebastian Weber, Bob Carpenter, and Micahel Betancourt, and Charles Margossian had a long and productive meeting to design the MPI interface. It’s very tricky trying to find an interface that’ll fit into the Stan language and let you ship off data once and reuse it. I think we’re almost there. The design discussion’s on Discourse.

  • Rob Trangucci is finishing the GP paper (after revising the manual chapter) with Dan Simpson, Aki Vehtari, and Michael Betancourt.

I’m sure I’m missing a lot of goings on, especially among people not at our weekly meetings. If you know of things that should be on this list, please let me know.

12 Comments

  1. It’s not out yet! There’ll be a draft posted in a few weeks.

  2. Aki Vehtari says:

    I missed most of the weekly meeting as I was teaching Stan for two days in València International Bayesian Analysis Summer School http://vabar.es/events/vibass/ . Before my part, students had had two day introduction to Bayesian inference, students were great and it was pleasure to teach them. ShinyStan received several praises from students, and it made the teaching also easier as it was so easy for students to check all the necessary diagnostics. It’s great how ShinyStan and diagnostics are improving all the time! Many students were also happy to hear better alternative to Akaike weights for model averaging :)

  3. Roy says:

    You write:

    “Any ideas on how to manage developing a model would be appreciated”.

    I have only played with Stan a little, so I am not certain I understand totally what the discussion is about. But just wondering if you have looked at what McElreath does in the “rethinking” package. He has some programs that make it in some sense easier to write out the desired model, which than gets translated to Stan, which can also be saved and edited if it is just as you would like.

    Again, this may be far from what the concern is, but just in case,,,,

    • Something along those lines for a fixed set of models could work. In some sense that’s also what RStanArm is doing—making it easier to experiment with a range of Stan models that’d be cumbersome to type out and manage. I was more looking for an in-Stan solution, though, rather than a new add-on language. Though that may be what it takes in the end.

      • Wayne says:

        My sense is that Stan is the “assembler language” built on top of the “machine language” of MCMC, and we will want to build a higher-level language on top of Stan.

        I believe `brms` is a great illustration of this. With a single function (brm), and a single “language” (an extended R formula and families), brms lets me fit a huge variety of regression models such as:

        Simple regression: y ~ x1 + x2, family=normal

        Regression with monotonic predictor: y ~ mo (x1) + x2, family=

        Logistic regression: y ~ x1 + x2, family=bernoulli

        Ordinal model with category-specific predictor and group-level intercept: y ~ cs (x1) + (cs(1) | g), family=acat

        GAM-style regression: y ~ s(x1) + x2, family=

        Censored logistic regression: y | cens (c1) ~ x1 + x2, family=

        Mixed-Effects regression: y ~ x1 + (1 + x1 | g1), family=

        Survival model: y | cens (c1) ~ x1 + x2, family=weibull

        Survival model with frailty: y | cens (c1) ~ x1 + x2 + (1 + x1 | g1), family=weibull

        Regression with errors-in-variables: y ~ me (x1, sdx), family=

        Mixture model: y ~ 1, mu1 ~ x, mu2 ~ z, family = mixture (gaussian, gaussian)

        Zero-inflated model, prediction zero-inflated (zi) part: y ~ x * z + (1 + x |ID1| g), zi ~ x + (1 |ID1| g)

        I’ll stop here, but I think this shows how I can build complex models or change model types piece-by-piece, which I believe is a critical component of the task you’re describing. I’ve stopped showing examples, but brms can handle nonlinear models, multivariate models, Conditional Autoregressive (CAR) models, zero-inflated and hurdle models, GAM, GAMM, survival, and a whole host of models, with a single formula language that is translated into Stan, rather than a dozen different frequentist-style functions that happen to use Stan under the hood. I really think the `rstanarm` approach might be a bridge from frequentist to Bayesian approaches or it might be a trap that makes a high-level unified approach more difficult to achieve.

        • Matt says:

          I really think the `rstanarm` approach might be a bridge from frequentist to Bayesian approaches or it might be a trap that makes a high-level unified approach more difficult to achieve.

          I cautiously cast my vote for “trap”. I have been using brms as well, and I think it is much more in the spirit of generalization that makes Bayesian modeling so powerful. I also don’t think the difficulty translating code from [some r function] to an interface like brms is high enough to justify the loss in expressiveness from a less generic interface like rstanarm.

          Some API developers call this the floor/ceiling tradeoff: I think the floor (ease of getting started) of rstanarm is only slightly lower than brms, but the ceiling (expressiveness, longterm usability for an expert, etc) of brms is much higher. The similarity in floor but difference in ceiling makes brms preferable.

          • Wayne says:

            Agreed. Another advantage of brms’ approach is that the Stan code is much like Stan code that you or I might generate. The rstanarm code is very sophisticated and highly optimized, but it’s not something you might copy/paste and then build on.

            • Andrew says:

              Wayne:

              One key aspect of Stan’s design is that it is open and transparent and easy to build upon; thus we can have rstanarm, brms, StataStan, and all sorts of other ways to access Stan’s power and flexibility.

              • Wayne says:

                Agreed. I’m just suggesting that at the strategic level we’re talking about here the rstanarm approach — which regularly appears in the Stan updates — may be fragmenting rather than unifying and flexible. Some people are more comfortable switching to something that is named and works almost exactly as their old tool does. But that approach also keeps the silo’d mentality rather than encouraging a systemic, modular approach that Stan can support.

                If we are going to build a higher-level system on Stan that has Stan-like flexibility but works at a higher level of abstraction and is modular, brms offers a compelling vision. It has chosen to broaden and deepen R’s formula language, but other variations are possible.

              • Wayne says:

                Wish I could edit my comment. I want to emphasize that I’m not criticizing rstanarm or suggesting it should not continue. Just saying that I don’t think it should be the foundation or inspiration for a higher-level-of-abstraction tool built on Stan that I think you’re talking about in this week’s Stan status. I just feel that brms and it’s flexible formula-based approach should inspire your thinking.

  4. Ben Goodrich says:

    Didn’t we agree in each of the last two years that you should be able to specify the lower bound equal to the upper bound to provisionally fix a parameter to a value and Stan would skip the corresponding change of variables?

  5. Bryan says:

    Is there a description of the approach used to win the NYC population prediction challenge?

Leave a Reply