Skip to content

What do you do to visualize uncertainty?

Howard Wainer writes:

What do you do to visualize uncertainty?
Do you only use static methods (e.g. error bounds)?
Or do you also make use of dynamic means (e.g. have the display vary over time proportional to the error, so you don’t know exactly where the top of the bar is, since it moves while you’re watching)?

Have you any thoughts on this topic?
I assume that since a Bayesian generates a posterior dist’n the output should not be point but rather a dist’n; and you being the most prolific Bayesian I know that you’ve got three or four old papers that you’ve written on it.

OK, sure, when you put it that way, my collaborators and I do have a few papers on the topic:

Visualization in Bayesian data analysis

Visualizing distributions of covariance matrices

Multiple imputation for model checking: completed-data plots with missing and latent data

A Bayesian formulation of exploratory data analysis and goodness-of-fit testing

All maps of parameter estimates are misleading

But I don’t really have much else to say right now. Dynamic graphics seem like a good idea but I’ve never programmed them myself. In many settings it will work to display point estimates, but sometimes this can create big problems (as discussed in some of the above-linked papers) because Bayesian point estimates will tend to be too smooth—less variable—compared to the variation in the underlying parameters being modeled.

So I’m kicking this one out to the commenters to see if they can offer some useful suggestions.

They know my email but they don’t know me

This came (unsolicited) in the inbox today (actually, two months ago; we’re on a delay, as you’re probably aware), subject line “From PWC – animations of CEO opinions for 2014″:

Good afternoon,

I wanted to see if the data my colleague David sent to you was of any interest. I have attached here additional animated Gifs from PwC’s CEO survey. Let me know if you would be interested in featuring these pieces or in a guest post by PwC.

Best,
**

** on behalf of **

Attached were two infographics which you can bet I’m not including here.

P.S. Just to be clear: I don’t think unsolicited emails are so horrible; I myself send emails to strangers all the time. Nor am I offended by the content. I just think it’s funny that there are people out there who think I’m interesting in publishing animated chartjunk.

More bad news for the buggy-whip manufacturers

In a news article regarding difficulties in using panel surveys to measure the unemployment rate, David Leonhardt writes:

The main factor is technology. It’s a major cause of today’s response-rate problems – but it’s also the solution.

For decades, survey research has revolved around the telephone, and it’s worked very well. But Americans’ relationship with their phones has radically changed. It’s no surprise that survey research will have to as well. . . .

In the future, we are unlikely to live in a country in which information is scant. We are certain to live in one in which information is collected in different ways. The transition is under way, and the federal government is among those institutions that will need to adapt.

Let’s hope that the American Association for Public Opinion Research can adapt too.

PostcardTorringtonCTRacingBuggyCirca1910

On deck this week

Mon: More bad news for the buggy-whip manufacturers

Tues: They know my email but they don’t know me

Wed: What do you do to visualize uncertainty?

Thurs: Sokal: “science is not merely a bag of clever tricks . . . Rather, the natural sciences are nothing more or less than one particular application — albeit an unusually successful one — of a more general rationalist worldview”

Fri: Question about data mining bias in finance

Sat: Estimating discontinuity in slope of a response function

Sun: I can’t think of a good title for this one.

Six quotes from Kaiser Fung

He just ordered a translation from Diederik Stapel

Fernando Martel Garcia writes:

So I am applying for a DC driver’s license and needed a translation of my Spanish license to show to the DMV. I go to http://www.onehourtranslation.com/ and as I prepare to pay I see a familiar face in the bottom banner:

Stapel

It appears Stapel is one of their “over 15,000 dedicated professional translators” (or maybe they put his picture there unauthorized). Either way now worried I may get a made up/plagiarized translation.

There are worse ways for a multilingual person to make a living . . . .

Perhaps they could get Bruno Frey to do some translations too. He’d only have to do it once, then he could just copy it over and over and over.

What is the purpose of a poem?

OK, let’s take a break from blogging about economics. OK, I haven’t actually been blogging so much about econ lately, but it just happens that I’m writing this on 19 July, a day after poking a stick into the hornet’s nest by posting “Differences between econometrics and statistics: From varying treatment effects to utilities, economists seem to like models that are fixed in stone, while statisticians tend to be more comfortable with variation” (which in turn was on auto-post as I’d written it a couple months earlier).

As is often the case, I’m on the blog to procrastinate: in this case, my colleagues and I are preparing a new course and there’s tons of important work to be done. I’m getting tired of reading comments on economics and empiricism and so I scooted over to Basbøll’s blog and kicked off a brief comment thread about the academic entertainer Slavoj Zizek. At first I was going to post and continue that discussion here, but I don’t give too poops about Slavoj Zizek, so I followed Basbøll’s blogroll link to “Stupid Motivational Tricks” and right away found something interesting.

The something interesting that I found was a post by Jonathan Mayhew about someone who’s the poet laureate of North Carolina. I had no idea that an individual state would have a poet laureate but it seems like a good idea, a quite reasonable nearly cost-free thing to do, indeed it would be cool to have all sorts of official state art. In reading the post I was mildly irritated by Mayhew’s use of “NC” as a generic replacement for “North Carolina.” The abbreviation is fine in some contexts but I founn it a bit jarring to read, “The literary community of NC . . .” On the other hand, it’s just a goddam blog so I don’t know what I’m supposed to be expecting.

But I’m getting completely off the point here. What happened is that Mayhew quoted a couple of mediocre passages from poems by two of North Carolina’s poet laureates (apparently they just had a changing of the guard).

Mayhew’s reactions gave me some thoughts of my own regarding the purpose of poetry. I’ll first copy what he wrote and then give my reflections.

Mayhew quotes from the previous laureate:

“Joan and I were in Raleigh together
for the first time to take the tour
for new vista volunteers
at North Carolina’s Central Prison…”

and then shares his reaction:

Ouch. It’s fine to use seemingly plain language, etc… but no rhythm, nothing going on in the language. This kind of writing just causes physical pain to me.

Then he quotes from the recent laureate:

“I’m grateful for my car, he says,
voice raspy with hard living.
Tossed on the seat, a briefcase
covered with union stickers,
stuffed with unemployment forms,
want ads, old utility bills,
birth certificate, school application
papers for the skinny ten-year-old
sitting beside him who loves baseball…”

This he characterizes as worse than the first poem (“not much worse,” though), but I don’t quite understand where this ranking is coming from, given that he follows up with, “More is going on in her language, actually. It’s not exactly good, but it’s salvageable, with some concreteness there at least.”

I assume that we can all agree, though, that it’s hard to judge either poem, or either poet, by these short excerpts. Both excerpts radiate mediocrity but of course a bit of mediocrity can do the job in the context of a larger message. I’m pretty sure that, for almost any major poet, you could without much difficulty find passages that, if shown to me in isolation, would not sparkle and could indeed look a bit like hackwork. I mean, sure, “voice raspy with hard living” sounds cliched, but who among us does not grab a cliche from time to time. For all we know from this excerpt, the use of the cliche is part of the point in establishing the narrator’s voice.

OK, let me be clear here. I’m not trying to get all contrarian on you and praise these two poets. I have no problem giving Mayhew the benefit of the doubt, I’ll assume he read a bit by each of them and with these excerpts is giving something of a true sense of these poems’ style and content. So I will accept (until convinced otherwise) that these poets are indeed mediocre.

What is the purpose of a poem?

And this brings us to today’s topic. The thing that bothers me about Mayhew’s post (even though I have a feeling I’d agree with him 100% about the strengths and weaknesses of these poems, and I wouldn’t be surprised if we share many tastes about and attitudes toward literature) is the implicit attitude that I see there, which I feel I’ve seen in other discussions of poetry, which is that the purpose of a poem is to be wonderful.

Huh? “The purpose of a poem is to be wonderful.” That seems like a reasonable statement, no? Who could disagree with that?

To see my problem with this statement (which, to be fair, Mayhew never said, but which I see as implicit in his post), consider the related question, “What is the purpose of a novel?” Or, for that matter, what is the purpose of a research article? Or what is the purpose of a song?

My point is that I think it’s a bad attitude to think that the purpose of a poem is to be wonderful. It’s insulting to poetry to give it such a narrow range. A poem is a sort of song without music and, as such can have many different purposes.

OK, procrastination successful. An hour spent, now time for bed.

mysterious shiny things

(Disclaimer: I’m new to Shiny, and blog posts, but I know something about geography.)  In the Shiny gallery, take a look at 2001 versus 2002. Something funny happens to Switzerland (and other European countries), in terms of the legend, it moves from Europe to the Middle East. Also, the legend color scheme switches.

 

2001_Shiny

2002_Shiny

 

To reproduce it yourself: download ui.R, server.R, and healthexp.Rds

Have a folder called “App-Health-Exp” in your working directory, with ui.R and server.R in the “App-Health-Exp” folder. Have the dataset healthexp.Rds in your working directory.
Then run this code:

if (!require(devtools))
  install.packages("devtools")
devtools::install_github("jcheng5/googleCharts")

install.packages("dplyr")
install.packages("shiny")

library(shiny)
library(googleCharts)
library(dplyr)

data = readRDS("healthexp.Rds")
head(data)

# Problem isn't the data, it seems that Switzerland is in Europe 
# in both 2001 and 2002:
data[data$Year == 2001 & data$Country == "Switzerland",]
data[data$Year == 2002 & data$Country == "Switzerland",]

runApp("App-Health-Exp")

Anyone know what is happening?

Bayesian Cognitive Modeling  Examples Ported to Stan

Bayesian Cognitive Modeling book cover

There’s a new intro to Bayes in town.

This book’s a wonderful introduction to applied Bayesian modeling. But don’t take my word for it — you can download and read the first two parts of the book (hundreds of pages including the bibliography) for free from the book’s home page (linked in the citation above). One of my favorite parts of the book is the collection of interesting and instructive example models coded in BUGS and JAGS (also available from the home page). As a computer scientist, I prefer reading code to narrative!

In both spirit and form, the book’s similar to Lunn, Jackson, Best, Thomas, and Spiegelhalter’s BUGS Book, which wraps their seminal set of example models up in textbook form. It’s also similar in spirit to Kruschke’s Doing Bayesian Data Analysis, especially in its focus on applied cognitive psychology examples.

Bayesian Cognitive Modeling Examples Now in Stan!

One of Lee and Wagenmaker’s colleagues, Martin Šmíra, has been porting the example models to Stan and the first batch is already available in the new Stan example model repository (hosted on GitHub):

Many of the models involve discrete parameters in the BUGS formulation which need to be marginalized out in the Stan models. The Stan 2.5 manual is adding a whole new chapter with some non-trivial marginalizations (change point models, CJS mark-recapture models, and categorical diagnostic accuracy models).

Expect the rest soon! And feel free to jump on the Stan users group to discuss the models and how they’ve been coded.

Warning: The models are embedded as strings in R code. We’re looking for a volunteer to pull the models out of the R code and generate data for them in a standalone file that could be used in PyStan or CmdStan.

Your Models Next?

If you’d like to contribute Stan models to our example repo, the README at the bottom of the front page of the GitHub repository linked above contains information on what we’d like to get. We only need open-source distribution rights — authors retain copyright for all their work on Stan. Contact us either via e-mail or via the Stan users group.

One-tailed or two-tailed

Unknown

This image of a two-tailed lizard (from here, I can’t find the name of the person who took the picture) never fails to amuse me.

But let us get to the question at hand . . .

Richard Rasiej writes:

I’m currently teaching a summer session course in Elementary Statistics. The text that I was given to use is Triola’s Elementary Statistics, 12th ed.

Let me quote a problem on inference from two proportions:

11. Is Echinacea Effective for Colds? Rhino viruses typically cause common colds. In a test of the effectiveness of echinacea, 40 of the 45 subjects treated with echinacea developed rhinovirus infections. In a placebo group, 88 of the 103 subjects developed rhinovirus infections (based on data from “An Evaluation of Echinacea Angustifolia in Experimental Rhinovirus Infections,” by Turner et. al., New England Journal of Medicine, Vol. 353, No. 4). We want to use a 0.05 significance level to test the claim that echinacea has an effect on rhinovirus infection.

The answer in the back of the teacher’s edition sets up the hypothesis test as H0: p1 = p2, H1: p1 <> (not equal to) p2, gives a test statistic of z = 0.57, uses critical values of +/- 1.96, and gives a P-value of .5686.

I was having a hard time explaining the rationale for the book’s approach to my students. My thinking was that since there is no point in claiming that echinacea has an effect on the common cold unless you think it helps, we should be doing a one-tailed test with H0: p1 = p2, H1: p1 < p2. We would still fail to reject the null hypothesis, but with a P-value of .2843.

Or, is what I am missing that, if you are testing the claim that something has an effect you want to also test the possibility that the effect is the opposite of what you’d normally want (e.g. this herb is bad for you, or inhaling smoke is good for you, etc.)?

Any advice you could give me on how best to parse this problem for my students would be greatly appreciated. I already feel very nervous stating, in effect, “well, that’s not the way I would do it.”

My reply:

The quick answer is that maybe echinacea is bad for you! Really though the example is pretty silly, as one can simply compare 40/45 and 88/103 and look at the sampling variability of the proportions. I don’t see that the hypothesis test and p-value add anything.

This doesn’t sound like much, but, amazingly enough, Rasiej replied later that day:

I guess I was led astray by the lead-in to the problem, which seemed to imply that there was a benefit. Obviously it’s better to read the claim carefully and take it literally. So, “test the claim that echinacea has an effect” is two-tailed since ANY effect, beneficial or not, would be significant.

That said, I do agree with you that the example is silly, given the data in the problem.

Thanks again for your insights. They helped in my class today.

Perhaps (maybe I should say “probably”) he was just being polite, but I prefer to think that even a brief reply can convey some useful understanding. Also I think it’s a good general message to take what people say literally. This is not a message that David Brooks likes to hear, I think, but it is, to me, an essential aspect of statistical thinking.

P.S. Perhaps I should stress that in my response above I wasn’t saying that confidence intervals are some kind of wonderful automatic replacement for p-values. I was just saying that, in this particular case, it seems to me that you’d want a summary of the information provided by the experiment, and that this summary is best provided by the estimated proportions and their standard errors. To set if up in a p-value context would seem to imply that you’re planning on making a decision about echinacea based on this single experiment, but that wouldn’t make sense at all! No need to jump the gun and go all the way to a decision statement; it seems enough to just summarize the information in the data.