Yes, you can learn a lot from N=1, as long as you have some auxiliary information.

The other day I was talking with a friend who’s planning to vote for Andrew Cuomo in the primary. What about Cynthia Nixon? My friend wasn’t even *considering* voting for her. Now, my friend is, I think, in the left half of registered Democrats in the state. Not far left, but I’d guess left of the median. If Nixon didn’t even have a chance with this voter, there’s no way she can come close in the primary election. She’s gonna get slaughtered.

A survey with N=1! And not even a random sample. How could we possibly learn anything useful from that? We have a few things in our favor:

– Auxiliary information on the survey respondent. We have some sense of his left-right ideology, relative to the general primary electorate.

– An informative measure of the respondent’s attitude. He didn’t just answer a yes/no question about his vote intention; he told me that he wasn’t even considering voting for her.

– A model of opinions and voting: Uniform partisan swing. We assume that, from election to election, voters move only a small random amount on the left-right scale, relative to the other voters.

– Assumption of random sampling, conditional on auxiliary information: My friend is not a random sample of New York Democrats, but I’m implicitly considering him as representative of New York Democrats at his particular point in left-right ideology.

Substantive information + informative data + model + assumption. Put these together and you can learn a lot.

I could be wrong, of course, and I haven’t tried to attach an uncertainty to my prediction. But this is what I’m going with, from my N=1 survey.

I’m not sure I buy the assumption of random sampling, since it seems like, conditioned on the auxiliary information provided, the probability of any individual voting for Cuomo is 1. I’m not even necessarily convinced it’s > 0.5. There could easily be another confounder at work, from some specific policy of Nixon’s this voter dislikes / policy of Cuomo’s this voter likes, to something out of the 2016 Democratic presidential primary playbook of “well I just don’t like her.”

Shannon:

No, the auxiliary information here is

notthat my friend planned to vote for Cuomo. The auxiliary information is that my friend is a registered Democrat, a bit to the left of the median for Democratic primary voters.What would you say had Nixon won?

Garnett:

Had Nixon won, I would’ve said that something was wrong with my model. That’s how we learn, by making inferences and then reassessing when they don’t work out.

I wonder how many N=1 studies wind up in the file drawer?

Or how many N=1 trials become N=2,3,… trials when the results are disappointing.

Although I suppose in this case that would entail staging more elections.

Or I guess I could interview more of my friends . . .

Is there publication bias here? Cuomo was a heavy favorite–that might even be understating it–and I’m assuming you were aware of that. If your friend had come out as strongly in favor of Nixon, would you have framed it for yourself as strong evidence that Nixon was going to win? Or would it, even before the outcome was known, have forced you to tilt your head and re-examine your model?

Jeff:

Good point!

Would it have been important to know the reasons your friend had for his choice?

In the second half of your post bullet-pointing the information in your favor, you left out what I thought was the most important piece of information from the first half, namely the certainty of your friend’s preference. “My friend wasn’t even considering voting for her.” I have an underlying model in which almost every (politically aware) voter has a sense of the preferences of ideologically similar voters and pundits in their orbit, and if the voter is aware of enough ideologically similar people who disagree with him then that will give him pause. That your friend was so certain suggests that his personal non-randomized poll of people like him didn’t turn up a lot of Nixon support. Under my model, with his degree of certainty you have access to the implied extreme results of his own non-random poll of the segment of the electorate that’s like him.

Z:

You’re right! I was meaning to put this on the list, and I just forgot while writing the post. I’ll amend.

https://en.wikipedia.org/wiki/Franchise_(short_story)