Does “status threat” explain the 2016 presidential vote?

Steve Morgan writes:

The April 2018 article of Diana Mutz, Status Threat, Not Economic Hardship, Explains the 2016 Presidential Vote, was published in the Proceedings of the National Academy of Sciences and contradicts prior sociological research on the 2016 election. Mutz’s article received widespread media coverage because of the strength of its primary conclusion, declaimed in its title. The current article is a critical reanalysis of the models offered by Mutz, using the data files released along with her article.

The title of Morgan’s article is, “Fake News: Status Threat Does Not Explain the 2016 Presidential Vote.”

What happened? According to Morgan:

Material interests and status threat are deeply entangled in her cross-sectional data and, together, do not enable a definitive analysis of their relative importance. . . . Her panel-data model of votes, which she represents as a fixed-effect logit model, is, in fact, a generic pooled logit model.

And, the punch line:

In contrast, the sociological literature has offered more careful interpretations, and as such provides a more credible interpretation of the 2016 election.

Mutz, like me, is a political scientist. Morgan is a sociologist.

So, what we have here is:
1. A technical statistical dispute
2 . . . about the 2016 election
3 . . . in PNAS.

Lots to talk about. I’ll discuss each in turn.

But, before going on, let me just point out one thing I really like about this discussion, which is that, for once, it doesn’t involve scientific misconduct, and it doesn’t involve anyone digging in and not admitting ambiguity. Mutz made a strong claim in a published paper, and Morgan’s disputing it on technical grounds. That’s how it should go. I’m interested to see how Mutz replies. She may well argue that Morgan is correct, that the data are more ambiguous than implied in her article, but that she favors her particular explanation on other grounds. Or she might perform further analyses that strengthen her original claim. In any case, I hope she can share her (anonymized) raw data and analyses.

1. The technical statistical dispute

In his article, Morgan writes:

A first wave of sociological research on the 2016 presidential election has now been published, and a prominent theme of this research is the appeal of Trump’s campaign to white, working-class voters. Analyses of Obama-to-Trump voters, along with the spatial distribution of votes cast, are both consistent with the claim that white, working-class voters represented the crucial block of supporters who delivered the electoral college victory to Trump . . . The overall conclusion of this first wave of sociological research would appear to be that we have much more work to do in order to understand why so many white voters supported Trump. And, although we may never be able to definitively decompose the sources of their support, three primary motives deserve further scrutiny: material interests, white nativism, and the appeal of the Trump persona. At the same time, it remains to be determined how much of the animus toward his competitor – Hillary Clinton – was crucial to his success . . .

From a statistical perspective, Morgan is saying that there’s collinearity between three dimensions of variation among white voters: (a) lack of education, (b) nativism, and (c) opposition to Clinton. He and Mutz both focus on (a) and (b), so I will too. The quick summary is that Mutz is saying it’s (b) not (a), while Morgan is saying that we can’t disentangle (a) from (b).

Mutz’s argument derives from an analysis of a survey where people were interviewed during the 2012 and 2016 campaigns. She writes:

It could be either, but this study shows that the degree to which you have been personally affected had absolutely no change between 2012 and 2016. It’s a very small percentage of people who feel they have been personally affected negatively. It’s not that people aren’t being hurt, but it wasn’t those people who were drawn to support Trump. When you look at trade attitudes, they aren’t what you’d expect: It’s not whether they were in an industry where you were likely to be helped or hurt by trade. It’s also driven by racial attitudes and nationalistic attitudes—to what extent do you want to be an isolationist country? Trade is not an economic issue in terms of how the public thinks about it.

Morgan analyzes three of Mutz’s claims. Here’s Morgan:

Question 1: Did voters change their positions on trade and immigration between 2012 and 2016, and were they informed enough to recognize that Trump’s positions were much different than Romney’s, in comparison to Clinton’s and Obama’s?

Answer: Voters did change their positions on trade and immigration, but only by a small amount. They were also informed enough to recognize that the positions of Trump and Clinton were very different from each other on these issues, and also in comparison to the positions of Romney and Obama four years prior.

On this question, Mutz’s results are well supported under reanalysis, and they are a unique and valuable addition to the literature . . .

Question 2: Can the relative appeal of Trump to white voters with lower levels of education be attributed to status threat rather than their material interests?

Answer: No. . . .

Question 3: Do repeated measures of voters’ attitudes and policy priorities, collected in October of 2012 and 2016, demonstrate that status threat is a sufficiently complete explanation of Trump’s 2016 victory?

Answer: No. . . .

The key points of dispute, clearly, are questions 2 and 3.

Question 2 is all about the analysis of Mutz’s cross-sectional dataset, a pre-election poll from 2016. Mutz presents an analysis predicting Trump support (measured by relative feeling thermometer responses or vote intention), given various sets of predictors:
A: education (an indicator for “not having a college degree”)
B: five demographic items and party identification
C: four economic items (three of which are about the respondent’s personal economic situation and one of which is an attitude question on taxes and spending)
D: eight items on status threat (four of which represent personal beliefs and four of which are issue attitudes).

For each of feeling thermometer and vote intention, Mutz regresses on A+B, A+B+C, and A+B+C+D. What she finds is that the predictive power of the education indicator is high when regressing on A+B, remains high when regressing on A+B+C, but decreases mostly to zero when regressing on A+B+C+D. She concludes that the predictive power of education is not explained away by economics, but is largely explained away by status threat. Hence the conclusion that it is status threat, not economics, that explains the Trump shift among low-education whites.

Morgan challenges Mutz’s analysis in four ways. First, he would prefer not to include party identification as a predictor in the regression because he considers it to be an intermediate outcome: there are some voters who will react to Clinton and Trump and change their affiliation. My response is: sure, this can happen, but my impression from lookin gat such data for many years is that party ID is pretty stable, even during an election campaign, and it predicts a lot. So I’m OK with Mutz’s decision to treat it as a sort of demographic variable. Morgan does his analyses both ways, with and without party ID, so I’ll just look at his results that include this variable. Second, Morgan would prefer to just restrict the analysis to white respondents, rather than including ethnicity as a regression predictor. I agree with him on this one. There were important shifts in the nonwhite vote, but much of that is, I assume due to Barack Obama not being on the ticket, and this is not what’s being discussed here. So, for this discussion, I think the analyses should be restricted to whites. Morgan’s third point has to do with Mutz’s characterization of the regression predictors. He relabels Mutz’s economic indicators as “material interests.” That’s no big deal given that in her paper Mutz labeled them as measures of “being left behind with respect to personal financial wellbeing,” which sounds a lot like “material interests.” But Morgan also labels several of Mutz’s “status threat” variables as pertaining “material interests” and “foreign policy.” When he does the series of regressions, Morgan finds that the material interest and foreign policy variables explain away almost all the predictive power of education (that is, the coefficient of “no college education” goes to nearly zero after controlling for these other predictors), and that the remaining status threat variables don’t reduce the education coefficient any further. Thus, repeating Mutz’s analysis but including the predictors in a different order leads to a completely different conclusion. I find Morgan’s batching to be as convincing as Mutz’s, and considering that Morgan is making a weaker statement—all he’s aiming to show is that the data are ambiguous regarding the material-interest-or-status-threat-question, I think he’s done enough to make that case. Morgan’s fourth argument is that, in any case, this sort of analysis—looking at how a regression coefficient changes when throwing in more predictors—can’t tell us much about causality without a bunch of strong assumptions that have not been justified here. I agree: I always find this sort of analysis hard to understand. In this case, Morgan doesn’t really have to push on this fourth point because his reanalysis suggests issues with Mutz’s regressions on their own terms.

Question 3 is the panel-data analysis, about which Mutz writes:

Because the goal is understanding what changed from 2012 to 2016 to facilitate greater support for Trump in 2016 than Mitt Romney in 2012, I estimate the effects of time-varying independent variables to determine whether changes in the independent variables produce changes in candidate choice without needing to fully specify a model including all possible influences on candidate preference. Significant coefficients thus represent evidence that change in an independent variable corresponds to change in the dependent variable at the individual level.

Morgan doesn’t buy this claim, and neither do I. At least, not in general.

Also this:

To examine whether heightened issue salience accounts for changing preferences, I include in the models measures of respondents’ pre-Trump opinions on these measures interacted with a dichotomous wave variable. These independent variable by wave interactions should be significant to the extent that the salience of these issue opinions was increased by the 2016 campaign, so that they weighed more heavily in individual vote choice in 2016 than in 2012. For example, if those who shifted toward Trump in 2016 were people who already opposed trade in 2012 and Trump simply exploited those preexisting views for electoral advantage, this would be confirmed by a significant interaction between SDO and wave.

I dunno. This to me puts a big burden on the regression model that you’re fitting. I think I’d rather see these sorts of comparisons directly, than fit a big regression model and start looking at statistically significant coefficients.

And this:

To the extent that changes in issue salience are responsible for changing presidential preferences, the interaction coefficients should be significant; to the extent that changing public opinions and/or changing candidate positions also account for changing presidential vote preferences, the coefficients corresponding to these independent variables will be significant.

Again, I’m skeptical. The z-score, or p-value, or statistical significance of a coefficient is a noisy random variable. It’s just wrong to equate statistical significance with reality. It’s wrong in that the posited story can be wrong and you can still get statistical significance (through a combination of model misspecification and noise), and in that the posited story can be right and you can still not get statistical significance (again, model misspecification and noise). I know this is how lots of people do social science, but it’s not how statistics works.

That said, this does not mean that Mutz is wrong in her conclusions. Although it is incorrect to associate statistical significance with a casual explanation, it’s also the case that Mutz is an expert in political psychology, and I have every reason to believe that her actual data analysis will be reasonable. I have not tried to follow what she did in detail. I read her article and Morgan’s critique, and what I got out of this is that there are lots of ways to use regression to analyze opinion change in this panel survey, and different analyses lead to different conclusions, which again seems to support Morgan’s claim that we can’t at this time separate these different stories of the white voters who switched from Obama to Trump. Again, Mutz’s story could be correct but it’s not so clearly implied by the data.

2. The 2016 election

Mutz writes that “change in financial wellbeing had little impact on candidate preference. Instead, changing preferences were related to changes in the party’s positions on issues related to American global dominance and the rise of a majority–minority America,” but I find Morgan’s alternative analyses convincing, so for now I’d go with Morgan’s statement that available data don’t allow us to separate these different stories.

My own article (with Julia Azari) on the 2016 election is here, but we focus much more on geography than demography, so our paper isn’t particularly relevant to the discussion here.

All I’ll say on the substance here is that different voters have different motivations, and that individual voters can have multiple motivations on vote choice. As we all know, most voters went with their party ID in 2016 as in the past; hence it makes sense each year to explain what motivated the voters who went for the other party. Given the multiple reasons to vote in either direction (or, for that matter, to abstain from voting), I think we have to go beyond looking for single explanations for the behaviors of millions of voters.

3. PNAS

Morgan wrote, “Mutz’s article received widespread media coverage because of the strength of its primary conclusion.” I don’t think this was the whole story. I think the key reason Mutz’s article received widespread media coverage was because it appeared in PNAS. And here’s the paradox: PNAS seems to be considered by journalists to be a top scientific outlet—better than, say, the American Political Science Review or the American Journal of Sociology—even though, at least when it comes to social sciences, I think it’s edited a lot more capriciously than these subject-matter journals. (Recall air rage, himmicanes, and many others.) It’s not that everything published in PNAS is wrong—far from it!—but I wish the news media would spend a bit less attention on PNAS press releases and a bit more time on papers in social science journals.

One problem I have with PNAS, beyond all issues of what papers get accepted and publicized, is what might be called PNAS style. Actually, I don’t think it’s necessarily the style for physical and biological science papers in PNAS, maybe it’s just for social science. Anyway, the style is to make big claims. Even if, as an author, you don’t want to hype your claims, you pretty much have no choice—if you want your social science paper to appear in Science, Nature, or PNAS. And, given that publication in these “tabloids” is likely to give you work a lot more influence, it makes sense to go for it. The institutional incentives favor exaggeration.

I hope that Morgan’s article, along with discussions by Mutz and others, is published in the APSR and gets as much, or more, media attention as the PNAS paper that motivated it.

19 thoughts on “Does “status threat” explain the 2016 presidential vote?

  1. Great post, Andrew. Thanks for engaging with this so quickly. I read Morgan’s online draft last night, but it helps to have your perspective.

  2. “Anyway, the style is to make big claims. Even if, as an author, you don’t want to hype your claims, you pretty much have no choice—if you want your social science paper to appear in Science, Nature, or PNAS. And, given that publication in these “tabloids” is likely to give you work a lot more influence, it makes sense to go for it. The institutional incentives favor exaggeration.”

    I used to think these were all sign of the system being broken, but perhaps they are all signs of the system working exactly as it should.

    Perhaps all these folks chosing to engage with, and/or be part of, the different stages/parts of this system just do their part to make the system work as is intended.

    Perhaps the system is just not about science anymore, but about trying to get attention and influence…..

    • “Perhaps the system is just not about science anymore, but about trying to get attention and influence…..”

      In light of this, i’ve always been fascinated by, what i view as, the anti-scientific idea behind journal impact factor.

      Even when publishing my 1st paper, i was never concerned where to publish because i reasoned what matters was that it got published so people could find it. I didn’t matter to me where. I never understood this whole trying to publish your stuff in “the best” journals.

      Later on i encountered some papers discussing the many things wrong with impact factors, but what really shocked me was that they seem to be calculated using only the most recent 2 years (see https://en.wikipedia.org/wiki/Impact_factor).

      I am not sure if this is always the case, but using only the most recent 2 years already seems highly illogical to me from a scientific perpective. If anything, shouldn’t science be about what holds up, and what is unfluential, over time?

      If anything, i reason impact factors should be calculated using no. of citations and no. of papers of all published papers/issues per journal, not just to most recent 2 years. I think you can calculate this just as easy as 2-year traditional impact factors. If so, does anybody know if this has been done?

  3. On a related note, it is my understanding that the vote share of the incumbent party normally declines after 8 years in office. If so, that implies that there are always a fairly substantial number of people who switch their vote from the incumbent part to the non-incumbent party. Was that percentage actually unusually high in 2016, or was it perhaps unusually high in PA, WI, etc? If not, is there really a unique phenomenon that needs explanation?

    • I don’t follow the news, but i am amused with the so-called “fake news” discussions that are apparently going on if i understood thing correctly.

      I wondered how it differs from other “fake” stuff like manipulating scientific findings (e.g. tobacco industry influence of scientific papers related to smoking; https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1497700/)

      The wikipedia page about “fake news” states the following (https://en.wikipedia.org/wiki/Fake_news):

      “Fake news is a type of yellow journalism or propaganda that consists of deliberate misinformation or hoaxes spread via traditional print and broadcast news media or online social media.[1][2] This false information is mainly distributed by social media, but is periodically circulated through mainstream media.[3] Fake news is written and published with the intent to mislead in order to damage an agency, entity, or person, and/or gain financially or politically,[4][5][6] often using sensationalist, dishonest, or outright fabricated headlines to increase readership, online sharing, and Internet click revenue. In the latter case, it is similar to sensational online “clickbait” headlines and relies on advertising revenue generated from this activity, regardless of the veracity of the published stories.[4]”

      So, if i compare this to the tobacco industry influencing the general public via scientific papers, and subsequently the media, isn’t this tobacco industry influencing scientific findings concerning smoking also an example of “fake news” (via “fake science”)?

      • Ah, here is 1 reason i don’t follow the news (because it usually depresses/angers/saddens me) and 1 possible example of “fake science/news” probably having trickled down via the press, into the classrooms and the minds of teachers and subsequently young children, and now back to the press again:

        http://www.bbc.com/news/av/education-44076279/pupils-sitting-sats-taught-power-pose-to-improve-focus

        Let’s hope they will also be taught critical thinking, logic, rational thought, etc.

        +1 though for the kid who said “i’m not really a test person, i am more like a relax person” (i feel you dude)

        and +1 for the kid who went all in and added the batman-mask :)

  4. The authors of this article should try some machine-learning (ML) algorithms, such as xgboost. Then look at the feature importance plot. What’s most important? Also, look at a single regression tree, but do not make it too deep or too shallow. Useful interactions are fairly easy to spot from a regression tree. Let the ML algorithm suggest the linear model form. In addition, try some K-fold cross validation with and without different variables in xgboost. (Contrary to some received wisdom, noise variables often degrade ML algorithmic performance.) Do predictions improve or not? After all that, go back to the linear model with ML-inspired variables, including suggested interactions. Garden of forking paths, sure. But description is not the entire point of this retrospective study. Many want a good predictive model for 2018/2020.

    • Casey:

      I think you’ve fallen for some tech hype. Your proposed nonlinear prediction models will have all the same problems of interpretation as the linear prediction models discussed by Morgan.

    • The authors of this article should try some machine-learning (ML) algorithms, such as xgboost. Then look at the feature importance plot. What’s most important?…try some K-fold cross validation with and without different variables in xgboost.

      I think you’ll find that if you change the features then the “feature importance” will usually also change a lot, which will make you question the premise that you are learning much from the “importance”. Eg, you can take out the most “important” feature and see it has little effect on predictive skill.

      This is the same thing as the changing coefficients in arbitrary linear models, only more obvious since CARTs are so much better at extracting info from the web of correlation. In general I think trying to interpret the parameters of a model that has no theoretical basis is going to mislead more than help. You can still use them for predictions though.

  5. “From a statistical perspective, Morgan is saying that there’s collinearity between three dimensions of variation among white voters: (a) lack of education, (b) nativism, and (c) opposition to Clinton. He and Mutz both focus on (a) and (b), so I will too. The quick summary is that Mutz is saying it’s (b) not (a), while Morgan is saying that we can’t disentangle (a) from (b).”

    I’m a little confused by the suggestion that status threat has to be operating on a nativist level and NOT an education level. Intuitively it would seem to me that there has been a huge potential status decrease in a lot of the lower rungs of the education spectrum. Is it possible that status threat is a problem, but that it might not be primarily nativist?

    • Saying there is collinearity between the three factors does not imply or even suggest that “status threat has to be operating on a nativist level,” nor that status threat is primarily nativist. Collinearity simply says that in the data used, the three factors exhibit a collineear relationship — so that, in the data given, modeling with any two of the factors makes the third redundant. (Sometimes collinearity can result from a causal relationship, but it can exist without one.)

  6. “Morgan also labels several of Mutz’s ‘status threat’ variables as pertaining [to] ‘material interests’ and ‘foreign policy.’ … I find Morgan’s batching to be as convincing as Mutz’s.”

    It depends on the questions one is asking. If one is asking whether the shift to Trump was partly about “economic” rather than “status” issues, Morgan’s reclassification is defensible.

    But a key point of Mutz’s study is to answer a different question: whether people who have **personally** experienced economic hardship or a deteriorating economic/financial situation were more likely to vote for Trump, cet. par. In this context, Morgan’s reclassification misses a key distinction that Mutz made, between “personal economic hardship” variables (household income, looking for work, and personal finances) and opinions on issues (foreign trade, immigration, policy toward China, and state of US economy). Actual effects and opinions are 2 different things.

    She made this focus clear in her paper, and in the Slate interview, where she says that foreign trade “could be either” an economic issue or about status anxiety, “but this study shows that the degree to which you have been **personally affected** had absolutely no change between 2012 and 2016” (my emphasis).

    FWIW, I’m neither a sociologist nor a political scientist, but an economist.

  7. I won’t go item for item but I looked at some of Mutz’s questions. She is grouped a bunch of abstract questions under the category “status anxiety”. I am not sure about the content reliability and validity of the questions. Also I don’t think people understand the idea of foreign policy versus trade.

  8. Decolonization movements, the whole Ghandi and whatnot, can be described as caused by the “status anxiety” . Who is to say that white people should not be the ruling class of Uganda? Everyone agrees that Africa needs to be more diverse. China is clearly not diverse enough. Diversity is strength. African nativists in Tanzania say that it is “their country”, that white man should go home. Those Black nationalists.

  9. Excellent post.

    The point about PNAS is particularly relevant: my guess is that a journal like APSR would have produced many of Morgan’s points through its reviewing, and that the editors would have enforced them. One of those two steps evidently did not happen at PNAS, to the loss of the journal (I’ve heard jokes about soc. sci. published in PNAS, and they were not nice jokes).

  10. Re: ‘Again, I’m skeptical. The z-score, or p-value, or statistical significance of a coefficient is a noisy random variable. It’s just wrong to equate statistical significance with reality. It’s wrong in that the posited story can be wrong and you can still get statistical significance (through a combination of model misspecification and noise), and in that the posited story can be right and you can still not get statistical significance (again, model misspecification and noise). I know this is how lots of people do social science, but it’s not how statistics works.’

    I agree with Andrew on this.

Leave a Reply to Andrew Cancel reply

Your email address will not be published. Required fields are marked *