This is Eric.
Brian Parbhu took over the Bayesian Data Analysis meetup (formerly NYC Stan Users Group), and he is running an event in NYC this Friday, during which Brian Ward is going to discuss BridgeStan, the new in-memory interface for Stan. You can register here.
P.S. This is the presentation Brian gave at the Meetup.
This post is by Eric.
On August 15, I will be teaching this two-week course offered by the PRIISM center at NYU. The initial plan was to offer it to the NYU students entering the A3SR MS Program, but we are opening it up to a wider audience. In case you don’t like clicking on things, here is a short blurb:
This course aims to prepare students for the Applied Statistics for Social Science Research program at NYU. We will cover basic programming using the R language, including data manipulation and graphical displays; some key ideas from Calculus, including differentiation, integration, and optimization; an introduction to Linear Algebra, including vector and matrix arithmetic, determinants, and eigenvalues and eigenvectors; some core concepts in Probability including random variables, discrete and continuous distributions, and expectations; and a few simple regression examples.
This is a paid class, but Jenniffer Hill, who runs the program, tells me that department scholarships are available based on program and student needs.
If you would like to take a course, we ask that you fill out a short survey here. (If you need financial assistance, please indicate it under “Is there anything else you’d like to share with us?” survey question.) You can register here. We are planning to offer it in-person at NYU and online via Zoom.
Warning: This is my first time teaching this class, so I am not sure how much material we will be able to cover. We will have to gauge that as we go.
If you have taught something like this before and have suggestions for me, please leave those in the comments.
This post is by Eric.
On Thursday, 23 June, Duco Veen will stop by to discuss his work on prior elicitation. You can register here.
Abstract
Duco will discuss how expert knowledge can be captured and formulated as prior information for Bayesian analyses. This process is also called expert elicitation. He will highlight some ways to improve the quality of the expert elicitation and provide case examples of work he did with his colleagues. Additionally, Duco will discuss how the captured expert knowledge can be contrasted with evidence provided by traditional data collection methods or other prior information. This can be done, for instance, for quality control or training purposes where experts reflect on their provided information.
Duco Veen is an Assistant Professor at the Department of Global Health situated at the Julius Center for Health Sciences and Primary Care of the University Medical Center Utrecht. In that capacity, he is involved in the COVID-RED, AI for Health, and Trials@Home projects. In addition, he is appointed as Extraordinary Professor at the Optentia Research Programme of North-West University, South Africa. Duco works on the development of ShinyStan and has been elected as a member of the Stan Governing Body.
This post is by Eric.
On Wednesday, Juho Timonen from Aalto University is stopping by to tell us about his work. You can register here.
Juho will present what he thinks is an ideal modular design for statistical modeling software, based on his experiences as a user, developer, and researcher. He will look at the structure of existing software, focusing on high-level user interfaces that use Stan as their inference engine. Juho will use the longitudinal Gaussian process modeling package lgpr and its future improvements to demonstrate different concepts and challenges related to software design and development.
Juho Timonen completed his Bachelor’s degree majoring in Mathematics and Systems Analysis at Aalto University, Finland. His first research project was in game theory, and in his Bachelor’s thesis, Juho studied mixed-strategy equilibria in repeated games. He went on to do his Master’s degree at Aalto, majoring in Applied Mathematics. Juho joined the Computational Systems Biology group in the CS department at Aalto, where he got involved in probabilistic modeling and did his Master’s thesis on ODE modeling of gene regulatory networks. Juho continued as a Doctoral student in the same group, where his research is focused on probabilistic modeling.
This post is by Eric.
On Thursday, Maria Skoularidou from the University of Cambridge, Biostatistics Unit is stopping by to talk to us about generating high-quality synthetic data. You can register here.
You can read the whole paper here.
Abstract
Modeling the probability distribution of rows in tabular data and generating realistic synthetic data is a non-trivial task. Tabular data usually contain a mix of discrete and continuous columns (the most representing examples of such data are Electronic Health Records). Continuous columns may have multiple modes whereas discrete columns are sometimes imbalanced making their modeling difficult. Existing statistical and deep neural network models fail to properly model this type of data. We design CTGAN, which uses a conditional generator to address these challenges. To aid in comparison, we designed a benchmark with 7 simulated and 8 real datasets including several datasets often used to evaluate Bayesian networks. CTGAN outperformed Bayesian networks on most of the real datasets whereas other deep learning methods did not.
About the speaker
Maria Skoularidou holds a 4-year diploma in Computer Science and a 2-year MSc in Statistical Science, both from Athens University of Economics and Business, and is currently a final year PhD student at the University of Cambridge, MRC-Biostatistics Unit supervised by Professor Sylvia Richardso. Her thesis is focused on probabilistic machine learning and more precisely, on using generative modeling in healthcare.
Her fields of interest lie in the underpinnings, theory, and methodology of machine intelligence, Bayesian inference, theoretical computer science, neuroscience, and information theory.
Maria is also the Founder and Chair of {Dis}Ability in AI and founder and secretary of Women in Data Science and Statistics (RSS).
This post is by Eric.
On Friday, Elizabeth Wolkovich from the University of British Columbia is stopping by to talk to us about her work. You can register here.
Abstract
Climate change is having large impacts on natural and agricultural systems around the globe. Mitigating the worst consequences requires models that mechanistically predict changes. Towards that goal, the Temporal Ecology Lab works on models to better predict the most reported biological impact – shifts in phenology, the timing of recurring life history events such as leafout, and flowering. Here I review three major areas of research where Bayesian inference has been critical to my lab’s insights and advances: declining plant sensitivity to warming temperatures over time and space, mismatches between critical species interactions (for example, plants and pollinators), and shifting winegrowing regions with warming.
About the speaker
Elizabeth Wolkovich is an Associate Professor and Canada Research Chair at the University of British Columbia where she runs the Temporal Ecology Lab. Her research focuses on understanding how climate change shapes plants and plant communities, with a focus on shifts in the timing of seasonal development. She is particularly interested in how climate change will affect different winegrape varieties, and how shifting varieties may help growers adapt to warming.
This post is by Eric.
Next Thursday, Rok Češnovar is stopping by to talk to us about recent developments in and around Stan. You can register here.
Abstract
We will present the current state of the Stan ecosystem, highlight some of the advances that improved the performance of Stan in the past few years, and discuss what is still to come in the not-so-distant future. You will hear about the core modules of Stan, how they all fit together, and how the various interfaces bring that core to life in different ways to make your Stan models run. If your models have been compiling and running faster recently and you were wondering why, we will present the improvements that were made in Stan over the last few years that sped up those pesky gradient evaluations. If you feel that your models are still running slowly, we will provide a few tips on how to identify the computational (non-modeling) reasons why that may be the case, and what are some of the things you can do to speed up computation.
About the speaker
Rok Češnovar is a Stan developer and a PhD student at the University of Ljubljana who is interested in making Bayesian inference faster using parallelism on GPUs and multi-core CPUs. He joined the Stan development team while working on adding GPU support with the team of prof. Štrumbelj at UL, but quickly became interested in almost all aspects of Stan. In addition to working on adding the GPU support, he has co-authored the cmdstanr R package, added profiling for Stan, and implemented the language side of reduce_sum and the new ODE interface. He has also been managing and coordinating Stan & CmdStan releases for the last two years.
The video is now available here.
This post is by Eric.
Tomorrow, we will be hosting Lester Mackey from Microsoft Research. You can register here.
Abstract
This talk will introduce two new tools for summarizing a probability distribution more effectively than independent sampling or standard Markov chain Monte Carlo thinning:
These tools are especially well-suited for tasks that incur substantial downstream computation costs per summary point like organ and tissue modeling in which each simulation consumes 1000s of CPU hours.
About the speaker
Lester Mackey is a Principal Researcher at Microsoft Research, where he develops machine learning methods, models, and theory for large-scale learning tasks driven by applications from healthcare, climate forecasting, and the social good. Lester moved to Microsoft from Stanford University, where he was an assistant professor of Statistics and (by courtesy) of Computer Science. He earned his Ph.D. in Computer Science and MA in Statistics from UC Berkeley and his BSE in Computer Science from Princeton University. He co-organized the second place team in the Netflix Prize competition for collaborative filtering, won the Prize4Life ALS disease progression prediction challenge, won prizes for temperature and precipitation forecasting in the yearlong real-time Subseasonal Climate Forecast Rodeo, and received best paper and best student paper awards from the ACM Conference on Programming Language Design and Implementation and the International Conference on Machine Learning.
This post is by Eric.
We are resuming our Webinar series this Thursday with Uri Shalit from Technion. You can register here.
Abstract
A plethora of new methods for estimating patient-level causal effects have been proposed recently, focusing on what is technically known as (high-dimensional) conditional average effects (CATE). The intended use of many of these methods is to inform human decision-makers about the probable outcomes of possible actions, for example, clinicians choosing among different medications for a patient. For such high-stakes decisions, it is crucial for any algorithm to responsibly convey a measure of uncertainty about its output, in order to enable informed decision making on the side of the human and to avoid catastrophic errors.
We will discuss recent work where we present new methods for conveying uncertainty in CATE estimation stemming from several distinct sources: (i) finite data (ii) covariate shift (iii) violations of the overlap assumption (iv) violation of the no-hidden confounders assumption. We show how these measures of uncertainty can be used to responsibly decide when to defer decisions to experts and avoid unwarranted errors.
This is joint work with Andrew Jesson, Sören Mindermann, and Yarin Gal of Oxford University.
About the speaker
Uri Shalit is an Assistant Professor in the Faculty of Industrial Engineering and Management at Technion University. He received his Ph.D. in Machine Learning and Neural Computation from the Hebrew University in 2015. Prior to joining Technion, Uri was a postdoctoral researcher at NYU working with prof. David Sontag.
Uri’s research is currently focused on three subjects. The first is applying machine learning to the field of healthcare, especially in terms of providing physicians with decision support tools based on big health data. The second subject Uri is interested in is the intersection of machine learning and causal inference, especially the problem of learning individual-level effects. Finally, Uri is working on bringing ideas from causal inference into the field of machine learning, focusing on problems in robust learning, transfer learning, and interpretability.
The video is available here.
This post is by Eric.
This Wednesday, at 11:30 am ET, Elea Feit is stopping by to talk to us about her recent work on Conjoint models fit using GPs. You can register here.
Abstract
Choice-based conjoint analysis is a widely-used technique for assessing consumer preferences. By observing how customers choose between alternatives with varying attributes, consumers’ preferences for the attributes can be inferred. When one alternative is chosen over the others, we know that the decision-maker perceived this option to have higher utility compared to the unchosen options. In addition to observing the choice that a customer makes, we can also observe the response time for each task. Building on extant literature, we propose a Gaussian Process model that relates response time to four features of the choice task (question number, alternative difference, alternative attractiveness, and attribute difference). We discuss the nonlinear relationships between these four features and response time and show that incorporating response time into the choice model provides us with a better understanding of individual preferences and improves our ability to predict choices.
About the speaker
Elea Feit is an Associate Professor of Marketing at Drexel University. Prior to joining Drexel, she spent most of her career at the boundary between academia and industry, including positions at General Motors Research, The Modellers, and Wharton Customer Analytics. Her work is inspired by the decision problems that marketers face and she has published research on using randomized experiments to measure advertising incrementality and using conjoint analysis to design new products. Methodologically, she is a Bayesian with expertise in hierarchical models, experimental design, missing data, data fusion, and decision theory. She is also the co-author of R for Marketing Research and Analytics. More at eleafeit.com.
Video recording is available here.
This post is by Eric.
This Thursday, at 12 pm ET, Jessica Hullman is stopping by to talk to us about theories of inference for data interactions. You can register here.
Abstract
Research and development in computer science and statistics have produced increasingly sophisticated software interfaces for interactive and exploratory analysis, optimized for easy pattern finding and data exposure. But design philosophies that emphasize exploration over other phases of analysis risk confusing a need for flexibility with a conclusion that exploratory visual analysis is inherently “model free” and cannot be formalized. I will motivate how without a grounding in theories of human statistical inference, research in exploratory visual analysis can lead to contradictory interface objectives and representations of uncertainty that can discourage users from drawing valid inferences. I will discuss how the concept of a model check in a Bayesian statistical framework unites exploratory and confirmatory analysis, and how this understanding relates to other proposed theories of graphical inference. Viewing interactive analysis as driven by model checks suggests new directions for software and empirical research around exploratory and visual analysis, as well as important questions about what class of problems visual analysis is suited to answer.
About the speaker
Jessica Hullman is an Associate Professor of Computer Science at Northwestern University. Her research looks at how to design, evaluate, coordinate, and theorize representations for data-driven decision making. She co-directs the Midwest Uncertainty Collective, an interdisciplinary group of researchers working on topics in visualization, uncertainty communication and human-in-the-loop data analysis, with Matt Kay. Jessica is the recipient of a Microsoft Faculty Fellowship, NSF CAREER Award, and multiple best papers at top visualization and human-computer interaction conferences.
The video is available here.
This post is by Eric.
This Wednesday, at 12 pm ET, Tamara Broderick is stopping by to talk to us about pairwise interactions in high dimensions. You can register here.
Abstract
Discovering interaction effects on a response of interest is a fundamental problem in medicine, economics, and many other disciplines. In theory, Bayesian methods for discovering pairwise interactions enjoy many benefits such as coherent uncertainty quantification, the ability to incorporate background knowledge, and desirable shrinkage properties. In practice, however, Bayesian methods are often computationally intractable for problems of even moderate dimension p. Our key insight is that many hierarchical models of practical interest admit a particular Gaussian process (GP) representation; the GP allows us to capture the posterior with a vector of O(p) kernel hyper-parameters rather than O(p^2) interactions and main effects. With the implicit representation, we can run Markov chain Monte Carlo (MCMC) over model hyper-parameters in time and memory linear in p per iteration. We focus on sparsity-inducing models; on datasets with a variety of covariate behaviors, we show that our method: (1) reduces runtime by orders of magnitude over naive applications of MCMC, (2) provides lower Type I and Type II error relative to state-of-the-art LASSO-based approaches, and (3) offers improved computational scaling in high dimensions relative to existing Bayesian and LASSO-based approaches.
About the speaker
Tamara Broderick is an Associate Professor in the Department of Electrical Engineering and Computer Science at MIT. She is a member of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), the MIT Statistics and Data Science Center, and the Institute for Data, Systems, and Society (IDSS). She completed her Ph.D. in Statistics at the University of California, Berkeley in 2014.
The video is available here.
This post is by Eric.
This Wednesday, at 12 pm ET, Paul Bürkner is stopping by to talk to us about brms. You can register here.
Abstract
The talk will be about Bayesian multilevel models and their implementation in R using the package brms. We will start with a short introduction to multilevel modeling and to Bayesian statistics in general followed by an introduction to Stan, which is a flexible language for fitting open-ended Bayesian models. We will then explain how to access Stan using the standard R formula syntax via the brms package. The package supports a wide range of response distributions and modeling options such as splines, autocorrelation, and censoring all in a multilevel context. A lot of post-processing and plotting methods are implemented as well. Some examples from Psychology and Medicine will be discussed.
About the speaker
Paul Bürkner is a statistician currently working as a Junior Research Group Leader at the Cluster of Excellence SimTech at the University of Stuttgart (Germany). He is the author of the R package brms and a member of the Stan Development Team. Previously, he studied Psychology and Mathematics at the Universities of Münster and Hagen (Germany) and did his PhD in Münster on optimal design and Bayesian data analysis. He has also worked as a Postdoctoral researcher at the Department of Computer Science at Aalto University (Finland).
The video is available here and the slides are available here.
This post is by Eric.
This Wednesday, at 12 pm ET, Aki Vehtari is stopping by to talk to us about Bayesian workflow. You can register here.
Abstract
We will discuss some parts of the Bayesian workflow with a focus on the need and justification for an iterative process. The talk is partly based on a review paper by Gelman, Vehtari, Simpson, Margossian, Carpenter, Yao, Kennedy, Gabry, Bürkner, and Modrák with the following abstract: “The Bayesian approach to data analysis provides a powerful way to handle uncertainty in all observations, model parameters, and model structure using probability theory. Probabilistic programming languages make it easier to specify and fit Bayesian models, but this still leaves us with many options regarding constructing, evaluating, and using these models, along with many remaining challenges in computation. Using Bayesian inference to solve real-world problems requires not only statistical skills, subject matter knowledge, and programming, but also awareness of the decisions made in the process of data analysis. All of these aspects can be understood as part of a tangled workflow of applied Bayesian statistics. Beyond inference, the workflow also includes iterative model building, model checking, validation and troubleshooting of computational problems, model understanding, and model comparison. We review all these aspects of workflow in the context of several examples, keeping in mind that applied research can involve fitting many models for any given problem, even if only a subset of them are relevant once the analysis is over.” The pre-print is available here.
The video is available here.
This post is by Eric.
This Wednesday, at 12 pm ET, Charles Margossian is stopping by to talk to us about solving ODEs using Bayesian methods. You can register here.
If you want to get a feel for the types of issues he will be discussing, take a look at his (and Andrew’s) recent case study: “Bayesian Model of Planetary Motion: exploring ideas for a modeling workflow when dealing with ordinary differential equations and multimodality.”
Abstract
Many scientific models rely on differential equation-based likelihoods. Some unique challenges arise when fitting such models with full Bayesian inference. Indeed as our algorithm (e.g. Markov chain Monte Carlo) explores the parameter space, we must solve, not one, but a range of ODEs whose behaviors can change dramatically with different parameter values. I’ll present two examples where this phenomenon occurs: a classical mechanics problem where the speed of the solver differs for different parameter values; and a pharmacology example, wherein the ODE behaves in a stiff manner during the warmup phase but becomes non-stiff during the sampling phase. We’ll then have a candid discussion about the difficulties that arise when developing a modeling workflow with ODE-based models and brainstorm some ideas on how to move forward.
The video is now available here.
This post is by Eric.
This Wednesday, at 12 pm ET, Kristian Brock is stopping by to talk to us about functional uniform priors for dose-response models. You can register here.
Abstract
Dose-response modeling frequently employs non-linear regression. Functional uniform priors are distributions that can be derived for parameters that convey approximate uniformity over the range of function shapes generated by the model. They provide a stark alternative to regular uniform priors, which in the non-linear setting can provide potentially undue influence on the estimated functional form. Using methods introduced by Bornkamp, we provide full analytical derivations of functional uniform priors for a range of non-linear dose-response models. We then examine the numerical performance of these two types of prior, and analogous maximum likelihood models, in a simulation study. We also investigate the incidence of several markers that question the adequacy of model fit, in simulated and real phase I clinical trial datasets.
We show that mean absolute errors of response estimates are smaller when using functional uniform priors instead of regular uniform priors. This effect was seen in all simulation scenarios at all sample sizes. Irrespective of the decreases in errors, biases tended to be slightly larger under functional uniform priors. When the analysis model and data generating model matched, functional uniform priors yielded very close to nominal coverage probabilities, even at the smallest sample size. This was not true for models that used regular uniform priors. Markers of model fit inadequacy using real and simulated datasets were much more common in maximum likelihood models, with parameters being estimated near the boundary being a notable problem. Functional uniform priors provide a general improvement over regular uniform priors in Bayesian dose-response modeling and should be preferred. The substantial mathematical work required to derive the priors is abrogated by the results derived in this research.
This post is by Eric.
A few months ago we started running monthly webinars focusing on Bayes and uncertainty. Next week, we will be hosting Arman Oganisian, a 5th-year biostatistics PhD candidate at the University of Pennsylvania and Associate Fellow at the Leonard Davis Institute for Health Economics. His research focuses on developing Bayesian nonparametric methods for solving complicated estimation problems that arise in causal inference. His application areas of interest include health economics and, more recently, cancer therapies.
Abstract
Bayesian nonparametrics combines the flexibility often associated with machine learning with principled uncertainty quantification required for inference. Popular priors in this class include Gaussian Processes, Bayesian Additive Regression Trees, Chinese Restaurant Processes, and more. But what exactly are “nonparametric” priors? How can we compute posteriors under such priors? And how can we use them for flexible modeling? This talk will explore these questions by introducing nonparametric Bayes at a conceptual level and walking through a few common priors, with a particular focus on the Dirichlet Process prior for regression.
If this sounds interesting to you, please join us this Wednesday, 18 November at 12 noon ET.
P.S. Last month we had Matthew Kay from Northwestern University discussing his research on visualizing and communicating uncertainty. Here is the link to the video.
This post is by Eric.
We have a number of Stan related events in the pipeline. On 22 Nov, Ben Goodrich and I will be holding a free webinar called Introduction to Bayesian Computation Using the rstanarm R Package.
Here is the abstract:
The goal of the rstanarm package is to make it easier to use Bayesian estimation for most common regression models via Stan while preserving the traditional syntax that is used for specifying models in R and R packages like lme4. In this webinar, Ben Goodrich, one of the developers of rstanarm, will introduce the most salient features of the package.
To demonstrate these features, we will fit a model to loan repayments data from Lending Club and show why, in order to make rational decisions for loan approval or interest rate determination, we need a full posterior distribution as opposed to point predictions available in non-Bayesian statistical software.
As part of the upcoming StanCon 2017, we will be teaching a number of classes on Bayesian inference and statistical modeling. Here is the lineup:
For Stan users and readers of this blog, please use the code “stanusers” to get a 10% discount.
We hope to see many of you online and in person.
This post is by Eric.
We are starting a series of free webinars about Stan, Bayesian inference, decision theory, and model building. The first webinar will be held on Tuesday, October 25 at 11:00 AM EDT. You can register here.
Stan is a free and open-source probabilistic programming language and Bayesian inference engine. In this talk, we will demonstrate the use of Stan for some small problems in sports ranking, nonlinear regression, mixture modeling, and decision analysis, to illustrate the general idea that Bayesian data analysis involves model building, model fitting, and model checking. One of our major motivations in building Stan is to efficiently fit complex models to data, and Stan has indeed been used for this purpose in social, biological, and physical sciences, engineering, and business. The purpose of the present webinar is to demonstrate using simple examples how one can directly specify and fit models in Stan and make logical decisions under uncertainty.
—-
Update: a video recording of the webinar is now available here.
Bob Carpenter, Daniel Lee, and Michael Betancourt will be teaching the 3-day class starting on 19 September in Paris. Following is the outline for the course:
Day 1
Introduction to Bayesian statistics
- Likelihood / sampling distributions
- Priors, Posteriors via Bayes’s rule
- Posterior expectations and quantiles
- Events as expectations of indicator functions
Introduction to Stan
- Basic data types
- Variable declarations
- Constrained parameters and transforms to unconstrained
- Program blocks and execution
- Derived quantities
- Built-in functions and operators
- Statements: sampling, assignment, loops, conditionals, blocks
- How to use Stan within R with RStan
Hands-on examples
Day 2
ODE and PK/PD Modeling
- Parameters and data to ODEs
- Non-stiff ODE solver
- Stiff ODE solver
- Control parameters and tolerances
- Coupled ODE systems for sensitivities
- Elimination half-lifes
Inference with Markov chain Monte Carlo
- Monte Carlo methods and plug-in inference
- Markov chain Monte Carlo
- Convergence diagnostics, R-hat, effective sample size
- Effective sample size vs. number of iterations
- Plug-in posterior expectations and quantiles
- Event probability calculations
Hands-on examples
Day 3
Additional Topics in PK/PD Modeliong
- Bolus and infusion dosing
- Lag time and absorption models
- Linear versus Michaelis/Menten elimination
- Hierarchical models for patient-level effects
- Transit compartment models and time lags
- Multi-compartment models and varying time scales
- Joint PK/PD modeling: Bayes vs. “cut”
- Meta-analysis
- Formulating informative priors
- Clinical trial simulations and power calculations
Stan programming techniques
- Reproducible research practices
- Probabilistic programming principles
- Generated quantities for inference
- Data simulation and model checking
- Posterior predictive checks
- Cross-validation and predictive calibration
- Variable transforms for sampling efficiency
- Multiple indexing and range slicing
- Marginalizing discrete parameters
- Handling missing data
- Ragged and aparse data structures
- Identifiability and problematic posteriors
- Weakly informative priors
If you are in Europe in September, please come and join us. Thanks to Julie Bertrand and France Mentré from Université Paris Diderot for helping us organize the course.
You can register here.