This one from 1995 (with D. Stephen Voss and Gary King) was fun. For our “Why are American Presidential election campaign polls so variable when votes are so predictable?” project a few years earlier, Gary and I had analyzed individual-level survey responses from 60 pre-election polls that had been conducted by several different polling organizations. We wanted do know exactly what went on in these surveys but it was hard to learn anything much at all from the codebooks or the official descriptions of the polls. So Voss, a student of Gary’s who had experience as a journalist, contacted the polling organizations and got them to cough up lots of information, which we reported in this paper, along with some simple analyses indicating the effects of weighting. All the surveys had serious nonresponse issues, of course, but some dealt with them during the data-collection process while others did more of the adjustment using weights.
By the way, the paper has a (small) error. The two outlying “h” points in Figure 1b are a mistake. I can’t remember what we did wrong, but I do remember catching the mistake, I think it was before publication but too late for the journal to fix the error. The actual weighted results for the Harris polls are not noticeably different from those of the other surveys at those dates.
Polling has changed in the past twenty years, but I think this paper is still valuable, partly in giving a sense of the many different ways that polling organizations can attempt to get a representative sample, and partly as a convenient way to shoot down the conventional textbook idea of survey weights as inverse selection probabilities. (Remember, survey weighting is a mess.)