Um . . . no, “if the general election were held today” is not a particularly interesting question. The polls can move a lot in 5 months. Remember President Dukakis? See here (from our 1993 paper):
The triangles on the right side of each plot are the actual election outcomes, and the little arrows on each graph show the dates of the Democratic and Republican conventions in each year. As you can see, polls this early are in many cases not even close to the outcome.
I’m sure that Dr. Tyson means well, and I’m a big fan of Nova, but, really, he should talk with some political scientists before glibly writing about politics and concluding, “The political analysts need to take it from here.” We’ve taken it pretty far already, dude! Tyson has every right to speculate about politics–I wouldn’t claim that you need some sort of political science affiliation as a “union card” to do political science research–but it would make sense to ask around a bit, right? I mean, if a couple of political scientists wrote a paper on astrophysics in a journal called Mathematical and Computer Modeling . . . well, before trumpeting it in the New York Times I’d first go up to the 10th floor and ask my friend David, the astronomer, whether it’s for real.
Thank you for posting this. To look at the electoral college through the lens of state level opinion polls now without this caveat is far-fetched at best. I've been looking at (and posting) an electoral college analysis since late March and just yesterday did a postmortem of how much things have changed between that point and now. Obama averaged a loss of nearly 1.5 points per state while Clinton averaged a gain of just more than 0.5 points in each state. The bottom line is that a lot can, and will, change.
"This conclusion comes not from wishful thinking … The authors … are not political scientists. They are astrophysicists. And one of the tasks of scientists is to clarify the apparent complexity of the universe by using the language of mathematics."
AAAaaaaaaggghhhh!!!!!!
"Here’s what they discovered: in swing states, the median result of all the polls conducted in the weeks prior to an election is an especially effective predictor of which candidate will win that election — even in states where the polls consistently fall within the margin of error."
Should this somehow be surprising? What am I missing?
I think this is par-for the course for the NYTimes op-ed page. Someone there could have thought about sending this over to a statistically oriented political scientist for a brief once-over.
Just a couple weeks ago they published another op-ed from a supposed expert on how Obama is a potential apostate in the Muslim world — an editorial that the public editor found had very little support with most Islamic scholars, no support in opinion polls, and made very sketchy arguments.
I just read the article in the Times and decided to check your blog immediately. It's ridiculous. You'd think that astrophysicists would have developed some high tech math to make sense of polls, maybe, but no — they just mined some data and found one variable (probably in the top thirty but not the top ten obvious ones to try) that happened to explain the results well in one election in a dozen or however many swing states. I haven't read the article — did they really, seriously not test this "out of sample" on the 2000 and 1996 and 1992 elections?
I love how "The journal Mathematical and Computer Modeling" is name-dropped to make a simple, obvious, and obviously flawed method seem sexy and complicated.
And for them to say that this "proves" Obama would lose Ohio if the election were held today, because 2/3 polls say so! I mean, trying to come up with an argument against that is like trying to come up with an argument for why the Earth isn't flat. It's almost too ridiculous to rebut.
The Times should be running op-eds by poblano (Nate Silver), the proprietor of FiveThirtyEight.com, instead. He has a well-explained and reasonable methodology (he doesn't claim it's perfect or publishable in the MaCM) for running simulations of electoral vote outcomes, and his analysis adds a lot more value than taking medians of polls.
I mean, if a couple of political scientists wrote a paper on astrophysics in a journal called Mathematical and Computer Modeling . . . well, before trumpeting it in the New York Times I'd first go up to the 10th floor and ask my friend David, the astronomer, whether it's for real.
Cosma Shalizi wrote a great blog post on this topic in 2005.
Susan,
I see what Cosma is saying but I don't think it applies to the stuff discussed in the NYT op-ed linked to above. These guys seem to be just grabbing some polls and doing the simplest possible thing from there.
I hope you wrote a letter to the Times. I saw it, got upset, thought of your 1993 piece, but got distracted before I could shoot off a glib letter.
It's like these guys never heard of Truman v. Dewey.
I'm extremely surprised you didn't comment on the accompanying graphs!
Surely they are candidates for worst collection of graphs ever? (start from the absence of adequate description of what they show, then briefly comment on the existence of a pie chart & a bar chart, and then take it from there…)
I think the graphs are an illustration and not actually representative of any of the data the article discusses. I rather liked it. If you look at the whole image you'll notice the shape of the charts spell out "CLINTON".
Leo Jung is an illustrator who often contributes sciency looking pictures to NYT articles like this…
http://www.nytimes.com/2007/01/28/magazine/28nutr…
Well, don't tar all Astronomers/Astrophysicists because NDT isn't a very sophisticated political observer. Those of us who DO Astronomy and Astrophysics have, for the most part, known that he's not very sophisticated in our field either! After all, he's one of the ringleaders of the demote Pluto bandwagon and a very, very poor CANDIDATE for a successor to Carl Sagan as a popularizer of science.
Ooops… Tom P you are right, I feel like an idiot! I blame my small screen..
The illustration is indeed rather nice.
A very common over interpretation of polling results occurs when commentators make too much of a small yet very disturbing percentage. After the New Yorker cover came out many commentators cited a poll indicating that 12% of Americans think Obama took his oath of office on the Koran.
I suspect that of that 12%, many were not telling the truth. Many who don’t like Obama and know it is not true, might nevertheless see answering the poll that way as a way of spreading the rumor. Even some Obama supporters might have reason to lie on this question as the poll results are used to press the argument that Obama is a victim of voter ignorance.
Lies usually do not affect poll results too much. On most questions liars are about equally divided on both sides of the question and if the respondents are divided 50-50 to begin with it all washes out (although the National Election Survey reports voter turnout in the 70% range, when the actual vote counts indicate that turnout is closer to 55%).
But on questions where just one side has reason to lie and the real percentage is very, the results can be very misleading.
I suspect that at least 5% of Americans will chose the absolutely most ridiculous answer offered to any question.
Unfortunately, no one has ever conducted a poll asking respondents whether they tell the truth when they respond to polls. And I’m not sure whether you could trust those numbers anyway.
Gary Klass
[email protected]