The “shy Trump voter” meta-question: Why is an erroneous theory so popular?

In trying to make sense of the 2016 election and its polling, people keep bringing up the idea of the “shy Trump voters”—those people who supported Trump for president but didn’t want to admit this to pollsters.

I’ve never thought this idea made much sense—I mean, sure, there are some people who don’t accurately report their political leanings on a poll, but I have no reason to believe that Trump supporters were more likely than Clinton supporters to misreport in this way. Why bother responding to a poll in the first place if you’re not going to state your opinion and let your vote be counted, as it were?

We also have data casting doubt on the importance of “shy Trump voters”; see the third paragraph of Section 5 of this paper, which I’m repeating here for convenience:

But that’s not what I want to talk about here. Rather, I want to consider the meta-question: Why is the “shy Trump voter” story so popular, given that (a) it doesn’t make a lot of sense, and (b) it doesn’t fit the data? I don’t have any great answers, just that it’s an accessible explanation—lying to pollsters is a simpler story than differential survey nonresponse—and also it has something in it to appeal to pundits in the center and the left, as helping explain how they missed this big phenomenon, that an unpopular candidate lacking traditional qualifications received nearly half the vote, and the story also appeals to people on the right as it suggests that all sorts of people who say they oppose Trump, actually don’t.

Again, I don’t claim to offer any definitive answer to the meta-question. I do think it’s worth asking. <cheap shot>Perhaps this will all be resolved in a PPNAS paper soon, with all conclusions validated by “p less than 0.05” on the training data, as usual.</cheap shot>

37 thoughts on “The “shy Trump voter” meta-question: Why is an erroneous theory so popular?

  1. Interesting. But, I am a little confused by one point. Would not “differential non-response” be a possible outcome of the existence of “shy Trump voters”?

    Bob.

  2. Bob:

    Yes, differential nonresponse could be considered shyness. Usually when I’ve seen the “shy Trump voter” theory, the implication has been that these people were responding to the survey but just not admitting that they wanted to vote for Trump. To the extent that was happening, it would be very hard to adjust for using the survey data. We’re saying that these people were just not responding to the survey at all, at certain times. That sort of differential nonresponse is much easier to adjust for.

  3. I am always surprised by a foundational assumption of a measurement method of human thought processes that includes:

    **People do not lie in this particular situation**

    This is surprising because some percentage of people otherwise seem very willing to lie in virtually all other situations.

    • To expand just a bit. Psychologists commonly argue that indeed people may lie but the lying occurs on both ends of the question and therefore has no appreciable effect on parameter estimates. The problem I see with this position is that the default is to make a strong claim where weak knowledge exists. It would be more accurate to say, “We don’t know whether dissembling has effected the estimates.” The common method in psychology is to use some social desirability measure to adjust scores under the assumption that the deception occurs in the direction of trying to make oneself look better in the eyes of others. The problem with this assumption is that it assumes a consistent tendency that is not affected by situational effects.

      • I would turn that umbrella upside down and put the concept of reality versus its distortion at the apex and broadly divide into intentional (i.e., self-presentation, lying) and unintentional (i.e., self-deception, cognitive dissonance).

        • That’s excellent. However there are many perceptions of reality that can translate into different concepts. Again the theoretical basis for a specific concept may not contain the same theory forming epistemological constituents as another concept.

          I’m on the fence as to whether lying is unintentional b/c that demarcation of intentional/unintentional is too stark for my taste. Cognitive dissonance, as an umbrella, therefore is intuitively appealing. I would equate ‘cognitive dissonance’ to ‘rationalization’ which can contain any combination of the categories you list

  4. It’s not just the US in 2016, the Shy “x” hypothesis comes up after pretty much every polling miss. I think it’s because it’s a lot more intuitive than questions of representativeness (which seem to be the cause in a high share of polling misses). Here’s the section on shy voters from a review article we wrote on polling misses:

    Given the secrecy of the ballot, it is difficult to prove whether people lie about how they will vote. There is some evidence to suggest it happens on at least some occasions. In the US, the ‘Bradley effect’ whereby polls overstated support for African-American politicians (due to white respondents being reluctant to say they would vote for a different candidate) appears to have been true in the 1980s and early 1990s, though it has since disappeared (Hopkins 2015). As with the ‘late swing’ excuse, ‘shy’ voters are more often accused than guilty (Durand, Blais, and Vachon 2002; Coppock 2017; Kennedy et al. 2017; Durand, Blais, and Vachon 2001; Mellon and Prosser 2017a; Sturgis et al. 2016).

    ‘Shy Tories’ – supporters of the Conservative Party in the UK – are the canonical explanation for the 1992 polling miss in Britain. There is some evidence that people who did not disclose their vote intention prior to the 1992 election but reported voting in recontact surveys were disproportionally Conservative (Jowell et al. 1993b), though whether this represents ‘shy’ voters or late deciders swinging in one direction is unclear, and the extent to which people lied to pollsters has been disputed (Crewe 1993; Worcester 1996).

    Shy Tories were suggested to be behind the 2015 polling miss (Singh 2015), but looking at differential vote by non-disclosers during the campaign, the geographic distribution of Conservative support and vote intention reporting, a question ordering experiment, and modelling the likely vote choices of non-disclosers, Mellon and Prosser (2017a) found no evidence of a Shy Tory phenomenon, a conclusion later confirmed by the British Polling Council inquiry into the polling miss (Sturgis et al. 2016). Some researchers also suggested that people lying to pollsters about their likelihood of turning out to vote was a factor in the polling error (Whiteley 2016). Examining this question, however, Mellon and Prosser (2017a) found that although people over-reported their likelihood of voting and having voted, over-reporting was uncorrelated with vote choice, meaning the errors in vote share estimates cancelled out.

    ‘Shy Trump’ supporters were also suggested to be behind Trump’s unexpected victory (Anderson 2016). However, there is no evidence supporting this. Using a list experiment – a technique designed to elicit accurate reporting of socially undesirable behaviour – Alexander Coppock (2017) found no evidence of Shy Trump voters. Likewise, the AAPOR report into the 2016 US polls compared presidential vote intentions to Republican support in down ballot races and found nothing to suggest a Shy Trump effect (Kennedy et al. 2017).

  5. The hypothesis does come from the Shy Tories idea and the so-called Bradley Effect. We did a mode study at Pew to see if there was evidence of socially desirable reporting comparing phone and web responses on a range of measures of support for Trump, the theory being that people are more likely to give socially undesirable answers when the survey is self-administered. We didn’t find any evidence of a difference at all. That report is here:

    http://www.pewresearch.org/2017/03/31/are-telephone-polls-understating-support-for-trump/

    I think that this originates with the fact that for most of the people proposing this hypothesis, voting for trump IS socially undesirable and they think people SHOULD have been embarrassed to say so to an interviewer. But it’s not socially undesirable to support Trump if you are, in fact, a Trump supporter. This is a really difficult aspect of survey research. It’s hard to get into the mindset of people who don’t think like you.

    • Precisely what I was thinking. Given the divide between camps in the last election, it seems more parsimonious to ground the “shy X” effect in the prejudices of the researchers. I’m no expert, but a quick google search didn’t show any “shy X” being applied to left-wing parties, presumably because the Academy (on the whole) tends to lean slightly to the left.

      I had a professor once who constantly warned against the pathologising of voters, of any stripe. As soon as you turn a personal value judgement into a defect, you compromise your ability to empathise. Without honest attempts at empathy, you’re almost guaranteed to be measuring the wrong things and draw erroneous inferences. (In my opinion, anyway.) You correctly observe that it’s hard to get into the mindset of someone who doesn’t think like you. I’d go one further, based on my limited experience. Academics don’t want to get into the mindset of others. To the extent that they try, they tend to do so in a way that reflects more on their self-identity as intellectuals rather than a genuine engagement. There are more than a few shades of Orientalism here.

        • Andrew,

          Very good point, and perhaps I’m a little removed from the intricacies of US politics and who falls into what camp. In my defence, I absorb what I can from a friend of mine who is studying psephology, specifically the question of “how did they get it so wrong”. To me, it appears like it’s an inherently ‘liberal’ view, used to interrogate the moral failings of those who vote for the right. Mind you, I just found references to “shy Corbynites”, so I feel like my afternoon will be spent broadening my horizons. Regardless, the theory does seem rooted in an assumption that one group – academics, Republican leaders, the Clinton campaign, #NeverTrumpers, the media – knows what is ‘right’, and therefore what another group ‘ought’ to be ashamed of. It then follows that the theory has little power to explain the data, as it tells you more about the prejudices of the group conducting the research than the behaviours of the group under investigation.

          Additionally, I stand by my comments regarding academics. It is far easier to blame an undesirable outcome on the supposed failings of a group seen as socially beneath oneself, than to turn the lens around and search within. That’s not an academy-specific problem, but a human one. The reason I single out academics is that we are notionally aware of this, and occupy an office that ought to strive to be above it. It astounds me when I hear specialists on Indigenous politics, the history of social policy, social discourse, etc. uncritically talk about Trump voters in terms that – if we described Indigenous Australians or 19C working-class Londoners in the same way – would lead them to point out the role of history and circumstance, exploitation etc. I point to academics because the same language keeps cropping up, only to be recognised as erroneous and self-serving decades later. (Social Darwinism, anyone? Standards of civilisation? Hysteria? End of History? IQ and the Wealth of Nations?)

          I’d like to think we can break the cycle and start trying to build theories that, from the perspective of 100 years hence, might not look quite so inane/narcissistic. But if there’s one thing we learn from history…

        • Alex:

          I think the “shy Trump voter” story was popular among many Trump supporters because it gave them a way to think that Trump was more popular than he appeared in the polls.

        • Andrew,

          Interesting. I’ve never heard that. From my limited contact with various Trump supporters (the Alt-Right, Tea Partiers, echoed narratives in the Australian extreme right), it seemed there was more of a conspiratorial angle, that the “mainstream media” was working hand-in-glove with Clinton and omnipresent ‘globalists’ to hide how popular Trump is. I’ve only ever seen “shy X” invoked in the context of polling failures, and normally with the tone of ‘how did this happen to us?’. Trump supporters, to the best of my knowledge, invoked the “moral majority” justification combined with “and you’re distorting the truth by attacking anyone who has the guts to admit they like Trump!!!”. To me, this fits in with more of a victimisation/conspiracy narrative, but if you were to point to this as Trump supporters endorsing a ‘shy X’ defence, I guess I could concede on this point.

        • Alex:

          I did a quick search and found this quote from an article from 4 Oct 2016 in the Financial Times:

          As polls have turned against him, Donald Trump’s allies have speculated that there is a shy tribe of supporters who are reluctant to declare themselves and whose invisibility in the polls would mean Hillary Clinton’s advantage is being overstated.

          The problem for the New York businessman? Pollsters have looked far and wide and deep into their methodology and they say they have struggled to unearth evidence of the shy Trump voter theory advocated among others by Kellyanne Conway, Mr Trump’s campaign manager.

        • Andrew –

          I’m curious why you think the “shy Trump voters” myth was anywhere near as popular on the left as on the right. I have seen it show up on the right almost uniformly – but haven’t any serious attempt to evaluate associations between ideology and belief in this myth.

          Here is a recent example of how the myth persists:

          https://www.washingtonpost.com/opinions/a-blue-wave-is-predicted-for-the-midterms-im-not-convinced/2018/11/01/15b5963a-d63f-11e8-a10f-b51546b10756_story.html?outputType=comment&utm_term=.4a0dd1c2356f

      • I also find it a bit ironic that your critique of researchers attempting to get into the mindset of survey respondents involves getting into the mindset of the researcher.

        • Could you elaborate? I don’t feel I’m criticising people for trying to get inside the minds of another. My criticism is that they don’t consistently do so. The line for that distinction seems to be “when it suits my identity as a cosmopolitan liberal intellectual”.

          This is from a low-impact journal, but this is the sort of thing I hear an awful lot.
          https://www.tandfonline.com/doi/full/10.1080/07393148.2017.1378295

          I’m in Australia. The academics around me do not have a direct political stake in the Trump presidency. Many of them are actively intellectually engaged with the issues that are still plaguing Indigenous culture based on the paternalism of ‘settler culture’, and the misrepresentations that were established during the colonial encounter. But talk about Trump voters, and you’ll hear the kind of language that characterised early Australian discourse on ‘the Aboriginie’…you know, from the time before we decided that we should probably consider Indigenous Australians to be humans. (1967, for those playing at home.) What could justify that kind of hostility to people they’ve never met? The answer that I’ve come up with is identity salience. If you’ve got a better theory though, preferably one with harder data than my own (which admittedly is not a high bar), then I’m all ears. I personally don’t study social or political psychology.

    • While this is an interesting study, the findings rest on the assumption that the mode delivery would change the tendency to distort one’s true feelings. It might, but it might not. I certainly don’t find the study a convincing final word on the subject.

      • After reading some of the linked research on response distortion, I must retract my initial response. The study is a pretty strong argument against social desirability as an explanation for Trump’s job approval.

  6. The theory is popular among pollsters because it highlights the difficulty and uncertainty of their job. “What can we do if a mass of people starts lying to us?” (Well, ahem, be less confident.)

    It’s popular with the losers (establishment politicians) because it portrays their opponent’s supporters as shifty, ashamed, dishonest, anti-social. Which dovetails with their story about the candidate and his movement.

    What’s not to like?

    • I think the theory is popular among all groups. Members of tribe A hate heretics even worse than members of tribe not-A, and the theory confirms their belief that they must be ever-vigilant and that increasingly tougher loyalty oaths are needed. Members of the Excluded Middle tribe perceive confirmation of their belief that something is deeply amiss and that a third way must be found. Members of tribe not-A see it as confirmation of their belief that their rebel alliance is hidden yet strong and clever. Pollsters like it because it diverts attention from their unwarranted predictions and failure to discuss/comprehend uncertainty. Of course I’ve no evidence for any of those four assertions; they’re just parts of a story I made up to show that post hoc theories with apparently impressive (or so I imagine) explanatory power ought to be treated accordingly; and not simply judged according to their “loveliness” and thereby accorded causal status thank to IBE.

  7. If I was trying to get a handle on anything to do with Trump’s elction (the thing itself or polling accuracy), I’d first want to get to the bottom of why it was that all the places I’d heard of, from all the big cities in Florida to Salt Lake City voted Hilary, and everywhere I hadn’t heard of voted Trump!

  8. there are some people who don’t accurately report their political leanings on a poll, but I have no reason to believe that Trump supporters were more likely than Clinton supporters to misreport in this way

    I think that you do have a reason. But it has nothing to do with “shyness.”

    Trump supporters may have been more likely than Clinton supporters to see media in general, and pollsters in particular, as their enemies. (Consider the large partisan gaps in trust of the media during the run-up to the election.) That animosity toward media and pollsters would have given Trump supporters incentive to misreport their vote intentions — not because they were embarrassed by their intentions, but because they didn’t want to help the pollsters. This animosity could also help to explain why Trump outperformed the polls most in those states in which he was already popular.

    I doubt that animosity explains most of the problem with state-level polling in 2016. But it may explain some of it. At the least, it should be plausible that animosity toward pollsters and the desire to deceive them was concentrated among Trump supporters, rather than uniform throughout the electorate.

    • John:

      I think animosity to pollsters can explain differential nonresponse. Once someone responds to a poll, he or she has a strong motivation to declare support sincerely, as it feels like a vote for one’s preferred candidate. And, as you remember, one of Trump’s constant themes throughout the primaries was how he was leading in the polls.

  9. I vote we go full-on conspiracy theory. Enough pollsters filled out enough forms on their own (because polling is hard work!) to bias the results, and they leaned toward Hilary because everybody knew she couldn’t lose so that would be the end of it.

  10. “Why bother responding to a poll in the first place if you’re not going to state your opinion and let your vote be counted, as it were?”

    I have several friends who openly admit to lying in polls in “rational” ways. In Australia, voting is compulsory, so declaring that you are an undecided voter who cares about issue x, will bring more attention to issue x from the political parties looking to target their campaigns to undecided voters. Another friend said he would vote for the conservative party (a lie) in a poll because he believed the left party had no chance of winning his electorate and should direct their resources elsewhere.

  11. I think the recurrence of the ‘shy voter’ theory could be attributed to occam’s razor. It’s the easy solution. Respondents must have lied. I don’t buy the ‘shy voter’ theory becomes it makes no sense why a respondent would lie to a pollster.

  12. I first learned of the “shy Trump voter” in the ‘90s, from friends who would later become Trump’s base. Even then, they were disaffected, they believed that their vote didn’t matter, they distrusted media and believed that their views and lifestyle were under assault by the liberal media and the political establishments. They would happily lie about their political views in any public or “official” forum, like an opinion poll, in order to “fly under the radar” and protect their conservative tribe from further persecution until such time as they could unexpectedly emerge victorious.

    For them, it’s an oddly rational choice.

    I have assumed that *this* is the source of the “shy Trump voter” theme, and not some rationalizations by pollsters or the media. That such people exist does not preclude they possibility that they’re not why the polls missed in 2016.

  13. With robot calling, push polling is so commonplace that most of us I suspect have been harassed by such calls. Personally I tend to reject all polling calls immediately. It sounds as if there is some unease about widespread opposition to polling as such creating a systemic bias. And this fear is latched onto in an effort to “explain” the Trump election.

    The thing is, it is entirely unclear to me why anyone thought the election wouldn’t hinge on turnout. Even opinion polling that tries to identify likely voters has the problem that turnout doesn’t get measured until the election. And for that, I have no idea why anyone expected the same kind of turnout from African American voters for Clinton as for Obama. And, just as much to the point, although the current crop of economic statistics were well nigh universally interpreted as showing the good times were rolling, as near as I can tell, economic statistics are not only difficult conceptually, but the government has repeatedly changed them to serve political ends. That is, when Clinton campaigned on Obama’s economic record, she was putting her worst foot forward. The voters punish the candidate and party presiding over the bad times, it seems. Clinton clearly tried to turn the election into a referendum on social issues, and won the popular vote. But lower turnout and the idiocy of the Electoral College did the rest.

    But the real moral of the story here it seems to me is that insight into statistics is directly useful in understanding economic statistics. Perhaps this would be an interesting topic to address? For example, is shadowstats.com fraudulent statistics? Or merely cracked? Or?

  14. Hi Andrew,
    I’ve done some work with Jonathon Schuldt and Julius Lagodny that argues that there were what we call “hidden” Trump supporters: (https://doi.org/10.1515/spp-2017-0003 / http://peterenns.org/sites/peterenns.org/files/pdf/SPP.2017.HiddenTrumpSupporters.pdf.

    Social desirability bias is one potential mechanism, but we also posit a 2nd potential mechanism. We write, “we might expect that the extremely negative news associated with Trump [e.g. his mocking a disabled reporter or boasting that he groped women against their will (Carmon 2016; Graham 2016)] would create strong “top of the head” (Taylor and Fiske 1978; Zaller 1992) considerations among Trump supporters. If so, these considerations may have led some who would eventually vote for Trump to tell pollsters that they were undecided or voting for a third party.” I think this argument fits well with your work on campaign poll variation and the fundamentals.

    Our evidence comes from state surveys in early October (we find that Trump underperformed relative to Republican senate candidates) and from two national surveys we conducted in early October. We use a proxy for vote intentions to estimate vote preference among those who did not directly express a vote intention for Clinton or Trump. With this proxy, we find that the group not directly expressing a vote intention leaned heavily toward Trump, and with this proxy our national and state-level estimates (based on MRP) from October come much closer to the final outcome.

    Although we hadn’t seen your paper with Julia Azari (which we’re very sorry to have missed), in our footnote 8 we address the argument about Trump’s over-performance in states where respondents would seemingly be most likely to express support for Trump. We also discuss how our findings relate to the AAPOR study and the lack of difference across modes noted above.

    Of course, as we mention in our conclusion, we understand that our paper is just one part of the ongoing conversation. And none of this answers your meta-question, because most people promoting the shy Trump voter theory probably have not seen our research!

  15. Obviously a bit late….

    Can someone explain to the the reasoning why there would be a shy voter effect in state polls that wouldn’t manifest on the national polls? I suppose there might be factors that would exacerbate the effect in certain key states but don’t show up in voting on a national scale. What would they be?

Leave a Reply to steven johnson Cancel reply

Your email address will not be published. Required fields are marked *