Does “status threat” explain the 2016 presidential vote? Diana Mutz replies to criticism.

A couple months ago we reported on an article by sociologist Steve Morgan, criticizing a published paper by political scientist Diana Mutz.

Mutz’s original article was called, “Status Threat, Not Economic Hardship, Explains the 2016 Presidential Vote,” and Morgan’s reply is called, “Status Threat, Material Interests, and the 2016 Presidential Vote” (it originally had the more provocative title, “Fake News: Status Threat Does Not Explain the 2016 Presidential Vote”).

Mutz wrote a long and detailed response, with lots of which will appear in the same journal as Morgan’s article. I have not had a chance to read all of this in detail, but just speaking generally I am happy with how this exchange is going, in that both these researchers are (a) getting into the details, and (b) connecting these data issues with the larger political questions under studied. Yes, both sides are somewhat annoyed at this point, but that’s fine: both Diana and Steve are acting professionally, and I think all this discussion will help in further research in this area.

In addition to her formal response, Mutz also had some responses to our blog post, which I can share right now. Here’s Mutz:

Before adding my own [Mutz’s] reactions, first some corrections to Gelman’s description of my study are in order. In the cross-sectional analysis, Gelman describes four blocks, A, B, C and D, when in reality there were only three: A) demographics, including education, B) seven (not four) items indicating a basis for retrospective and prospective concern about personal financial conditions, and 3) eight status threat indicators tapping both racial and global threats to dominant status.

Second, Gelman describes the analysis as gradually adding additional blocks of variables, and changing only the order in which they are entered. This is inaccurate. I include the first block of demographics including education, and then I add either B) the personal finance indicators, or C) the status threat indicators. The analysis never includes all of them at once, nor do I change the order in which they are entered. What I altered in the analyses in Table S5 was not the order of entering the variables, but which additional variables were added to the basic model—economic variables or status threat.

Third, this is also described as a causal analysis, which was not the point of using the cross-sectional data. The panel data are much better for those purposes. The question I was attempting to answer using cross-sectional data is why education was so strongly related to voting for Trump in 2016. The “left behind” interpretation was based on the assumption that education represented the effect of economic self-interest among working class people with lower incomes/education. Income was not as good a predictor of preferring Trump, but this interpretation of the relationship persisted nonetheless.

Because the relationship with education disappears when status threat variables are included, but changes negligibly when economic variables are included, I conclude that education’s strong relationship to Trump preference is because those with low education are also higher in status threat.

Of course, there may be other variables that also could erase the relationship between Trump voting and education. But thus far, the only analyses that have been able to eliminate education’s impact have done so with the same kind of status threat indicators I have used here, that is, indicators tied to racial attitudes (e.g., Sides, Tesler and Vavreck 2018; Schaffner, MacWilliams & Tatishe 2017).

And yes, PNAS insists that all articles have a single declarative sentence as their title. No subtitles, no colons, etc. So I was asked to change my title before publication. Likewise, because most election interpretations stress the importance of priming or activation of existing opinions over attitude change, I was asked by reviewers to include tests of priming along with the panel models showing that opinion change over time is related to changing candidate support. Including these interactions doesn’t change anything about the results as Morgan undoubtedly knows, but combining the two analyses takes up less space in a journal that allows a maximum of 10 pages.

Gelman also says he doesn’t buy my claim that estimating a fixed effects model with time-varying independent variables and time-varying dependent variable “represents evidence that change in an independent variable corresponds to change in the dependent variable at the individual level.” My statement is simply a restatement of what fixed effects does. If the objection is to the idea that one does not need a fully specified model as one would in an analysis that models between-subject variance, that is the whole point of using fixed effects approach—less risk of omitted variable bias (see Vaisey and Miles 2017). Some assumptions of the model cannot be tested without three or more waves, but it is nonetheless the best that one can do with available data, with far less risk of omitted variable bias than other approaches.

Finally, while I admit that perhaps I’m old-fashioned, I don’t consider Twitter the ideal venue for a scholarly discussion. [I agree! — AG] Unlike the usual academic practice when one critiques another’s work, Morgan did not share his article with me, but posted a link via Twitter where he characterized it as “a frontal assault on someone else’s article,” a designation that did not make me especially eager to engage in dialogue with him. That approach, combined with his original article title, “Fake News,” made me suspect this was yet another troll. Professor Morgan has apparently chosen to forego the mutual respect that is customary when questioning others’ scholarly work, and has instead taken to Trump-like tactics in both his mode of communication as well as his tone.

I certainly hope this is not “science at its best.” As someone who has spent a lot of time studying the impact of incivility in discourse, I know that while it is extremely useful for attracting audiences, incivility has unfortunate consequences for serious and productive dialogue (Mutz 2007, 2015). Nonetheless, I greatly appreciate this opportunity to respond to the substantive claims in Morgan’s critique.

Morgan also criticizes the University of Pennsylvania for writing a press release based on the article and for distributing the article to the press in advance. The distribution process for PNAS articles is controlled entirely by PNAS. Morgan is apparently unaware that it is standard PNAS procedure to embargo publications up to a specific date, and to release them to the press in advance via Eurekalert. Thus journalists have access before even the author or the author’s university is allowed to have a copy of the final publication. This practice has nothing to do with one’s university.

Morgan further criticizes the PNAS for the title it has chosen for its journal, arguing that it is a “journal with a title that implies that its contents are first presented in front of a body of the country’s leading scientists.” I am not sure why he would have this impression, but unlike Morgan’s article which appeared and was publicized via twitter, the PNAS does at least have a review process before a paper is publicly released. Morgan spends three pages of his manuscript criticizing press coverage of the article, which to me seemed an extremely unusual focus for an academic critique. While I sympathize with the lack of control academics have over media coverage, this is hardly new. And I question whether it is preferable that policymakers and the general public have no access or exposure to academic findings. The press performs this important service.

My remaining comments, especially those pertaining to Morgan’s reclassification of my indicators, are included in the attached document.

19 thoughts on “Does “status threat” explain the 2016 presidential vote? Diana Mutz replies to criticism.

  1. RE:

    ‘Finally, while I admit that perhaps I’m old-fashioned, I don’t consider Twitter the ideal venue for a scholarly discussion. [I agree! — AG] Unlike the usual academic practice when one critiques another’s work, Morgan did not share his article with me, but posted a link via Twitter where he characterized it as “a frontal assault on someone else’s article,” a designation that did not make me especially eager to engage in dialogue with him. That approach, combined with his original article title, “Fake News,” made me suspect this was yet another troll. Professor Morgan has apparently chosen to forego the mutual respect that is customary when questioning others’ scholarly work, and has instead taken to Trump-like tactics in both his mode of communication as well as his tone.
    —-

    I’ve learnt over the years that one has to make each and every debate and encounter as meaningful and rich as possible. Stick to the merits. And interject an insult if it is witty and captures the weakness of the argument well.

    I am often stunned when out and about with exceptionally smart people how personality conflicts and rivalries shape knowledge itself in pretty annoying ways. An example, I use to tell my Dad to not engage in putting others down. It was a process of acculturation that took years to result in what I considered a dysfunctional environment. It is one reason I shunned an academic career. On the other hand, I loved hanging out with academics. They kinda pampered me, hoping I’d relent and join their ranks. I though felt as if I had been attending university since the age of 8.

  2. >My statement is simply a restatement of what fixed effects does…

    Sigh…

    What a fixed effects model does is it models the way that population averages change conditional on the value of observed covariates… End of story. Changes in the population average need not correspond to any particular change in individuals.

    • Daniel:

      From my earlier post, here is the quote from Mutz:

      Because the goal is understanding what changed from 2012 to 2016 to facilitate greater support for Trump in 2016 than Mitt Romney in 2012, I estimate the effects of time-varying independent variables to determine whether changes in the independent variables produce changes in candidate choice without needing to fully specify a model including all possible influences on candidate preference. Significant coefficients thus represent evidence that change in an independent variable corresponds to change in the dependent variable at the individual level.

      I had two problems with the above quote. First, the analysis is correlational not causal. That doesn’t mean that I think causal inference is impossible here, but it seems a bit strong for Mutz to say that her analysis can “determine whether . . .” Second, I’m not happy with the identification of “significant coefficients” as evidence: (a) there’s the the usual problems with statistical significance, and (b) except in rare exceptions such as factorial experiments, it’s tricky to try to give causal interpretations to multiple coefficients at once.

      You bring up a third issue, which is the interpretation of averages as individual-level changes. That doesn’t bother me so much, in that I can take Mutz’s “change . . . at the individual level” as “average change . . . at the individual level.”

    • The exasperated sigh seems unnecessary and condescending. For all the problems with peer review, it does seem to typically improve civility.

  3. Diana:

    Thanks again for your comments. You’re right in your first point above that I misread which variables were going into your regression analyses. I think I messed up by reading table S5 of your supplementary material and table 2 of Morgan’s paper at the same time. Your regression did have seven economic variables, not four (I’d only counted the four economic variables that were survey questions, not the three that were geographic aggregates). And, I’d written that you regressed on A+B, A+B+C, and A+B+C+D, but it was really A+B, A+B+C, and A+B+D. (I’m considering A and B as different blocks, even though they’re always together in the regression, because you are giving them different interpretations: you’re focusing on the coefficient of A, including various other variables at the same time.)

  4. “Because the goal is understanding what changed from 2012 to 2016 to facilitate greater support for Trump in 2016 than Mitt Romney in 2012…”

    Perhaps it’s just my treacherous memory, but I seem to remember Trump getting enormous amounts of free publicity in 2016 that he didn’t get in 2012. Although he was never an official candidate, Trump’s Birtherism got a skeptical treatment (again, perhaps it’s just my treacherous memory?) that left him reported as marginal, when reported at all. He got the kind of treatment Sanders got in the news. And his opponent Clinton in 2016 received enormous negative publicity on supposed scandals which were never quite objectively justified to even be scandals in the everyday sense of the word. Certainly Trump’s business dealings had very little scrutiny. His past relationship with Roy Cohn, or his public activities in the Central Park case were more or less buried.

    It is perhaps foolish of me, but for Diana Mutz to neglect testing the alternative hypothesis that the real change in Trump’s support was that from the business community, first the mass media, then the advertising purchasers in the mass media. And that the this change in support was the primary factor in Trump being a serious candidate at all, not the fringe figure he was in 2012. And the huge amounts of free publicity was directly instrumental in his campaign’s success. I’m not at all convinced Mutz is asking a sensible question at all, regardless of how formally correct her statistics are.

      • Then I am lost as to whether Mutz was assuming that Romney’s vote was the Republican base and that Trump was just another Republican candidate whose extra vote must be attributed to some sea change in the souls of the population. That strikes me as defiance of the facts. Or whether Mutz was assuming that although Trump was perceived as a hostile takeover of the Republican Party, it was still appropriate to compare Romney vs. Trump. That strikes me as illogical.

        The objection that the owners of media and the major purchasers of media advertising gave billions in publicity to Trump while giving relentless negative publicity to his opposition applies to any difference in Republican votes in 2012 as well…except that coverage of an incumbent President differs. That’s why a Reagan could spout incredible nonsense throughout his two terms with no damage to his reputation. I’ve forgotten already how Mutz adjusted for loss of incumbency advantage for the Democratic candidate. Or how she adjusted for changes in African-American turnout in 2016, as opposed to 2008 and 2012.

        The notions that Trump actually won the election (as opposed to the Electoral College,) and that Trump’s true supporters are the rabble in the streets rather than the executives in the suites strike me as political commitments, not reasonable scientific hypotheses. Jason Brennan’s presentation of Mutz highlights this, I think.

        • Can you give examples of the “mass media” organizations that you believe supported Donald Trump during the 2016 elections? Also, do you also believe the supported him as well?

        • Free publicity morphs into support in this thoroughly neutral call for scientific evidence. Nonetheless, the old saw that there is no such thing as bad publicity is true enough to be relevant. I am required to have a full-time job keeping documentation, complete with access to university library and academic journals, LexisNexis, a video archive and who knows what before I can speak? Again, nonetheless, I say that when the major news networks interrupt programming for a Trump news conference, they are promoting him as a serious candidate in a way they didn’t support, say, Sanders. And when emails and Benghazi and Clinton cash are repeated over and over and over, while Trump’s entire shady career goes unexamined, I say that’s support as well.

          Even trying to be charitable to this kind of nonsense, the best thing I can come up with is that Hollywood celebrities strongly tend to not like Trump. Since no prophet is with honor in his home town, how can this be surprising? Hollywood knows Trump the TV performer, he’s one of them but with a big head that puts the most extravagant diva to shame. Of course they don’t like him. As is, I can only wonder, “Diana, is that you?”

        • Originally you made it seem that you believed the “support” was on purpose. Now it sounds more like you believe it was incidental, ie the media couldn’t help themselves because Trump leads to more viewers. Which is it?

          Also I’m still not sure about which organizations you are referring to as “mass media”.

        • Mass media means things like NBC, CBS, ABC, Fox.

          Treating Sanders like a viable candidate because of the intensity of an unexpected support would have made the Democratic primary season more of a horse race, raising ratings, hence ad revenue. And story after story about Trump’s shady business dealings would have been scandal-mongering, raising ratings, hence ad revenue. The theory that Trump’s massive publicity was somehow forced strikes me as nonsense. Lots of publicity for the socialist Sanders would have riled advertisers, I think. And I think realistic Trump investigation would have been called character assassination by advertisers. And that’s assuming that the men in the boardrooms of mass media didn’t already find Trump’s disdain for politicians waving their winning votes as if mere popular election gave them legitimacy to make trouble for hard-working businessmen kind of attractive.

          It sounds as if you think only blatant adulation counts as support. When the owner of a venue lets someone in, that is support for the entrants credentials as the loyal opposition, if nothing else. But it’s usually more, I think.

  5. As an outsider to the field of political science, it’s really hard for me to see the value of this type of research. Is it falsifiable? Does it have meaningful implication on policy?

    A regression analysis of cross sectional observational data with a number of mutually correlated predictors at best give you some tentative direction in where to look for causal relationship. With inclusion of predictors whose classifications are ambiguous at best, the interpretation of the associations becomes difficult to say the least. I just don’t see how opinions on trade and isolationism or even immigration are only driven by status threat. Even “status threat” itself can not be separated from the concern over long term economic interest.

    The lack of a compelling theory (as far as I know) also makes it hard to assess the model. Are all important variables included and properly measured? The economic interest variables certainly seem too narrow. 2012 was at the beginning of a difficult recovery. Doing better personally in 2016 relative to 2012 doesn’t mean a lot if one was still worse off than 10 years ago. What about interaction effects? Interactions between many of these terms certainly seem likely. There are also the issues with predictors that probably should be intermediate outcomes.

    • Yyw:

      Just speaking generally, without specific reference to these two papers, I think description can be valuable even without a direct causal story or theory. For example, Thomas Frank had lots of theories about Kansas voting Republican based on various things happening since the 1960s, and I think Frank’s theorizing can be tempered by looking at some data that shows Kansas being solidly Republican since the 1930s. Similarly, I think we can learn a bit by looking at how richer and poorer people have voted over the years, even without having a full theory. Clear and comprehensive descriptive work can be valuable.

  6. This is just another case of torturing the data to fit the pre-ordained narrative. There isn’t a real justification for using positions on trade, immigration, and China as proxies for status threat; sure, there’s a correlation, but they can also be explained by plenty of other factors. Now, social dominance orientation (SDO) is in fact a proxy for such, but it doesn’t yield nearly as impressive results in explaining the election.

  7. Not that anyone is reading this, but, for posterity’s sake, this thread ended without any resolution, partly because I chose not to respond to it at the time. Instead, I decided to write a reply only in the journal (see links below). I did not see value in responding to the more personal comments in Mutz’s comments above, about my supposed “Trump-like” tactics, “incivility,” and and so on.

    Nonetheless, after some time has passed, it seems useful to now write a bit (and also provide the links below). For the record, I exchanged a number of emails with Diana Mutz after I downloaded her data and before I wrote my manuscript (i.e., while I was trying to understand what she had done, and why her data were incomplete, from 4/26 to 5/1). I then wrote up my manuscript and submitted it to Socius. I did not send it to her for a few reasons. First, I suggested her as a reviewer to the journal, and I am quite confident that she was a reviewer (although one cannot know for sure). Second, her article was receiving a lot of media attention, and I felt it needed to be countered publicly without delay. I assumed that sending my manuscript to her would slow down my ability to share it with others as publicly as I think it needed to be. Third, she never asked to see any possible manuscripts I would write. I learned what I needed to learn from her in our email exchange, and I did not believe that I was going to get any more useful reaction to my argument if I had given her the full manuscript after I completed it. Instead, I just suggested her as a reviewer at the journal so that she could identify any errors. After peer review, very little in the manuscript changed because none of the reviewers found any errors. The editors did ask that I soften the language in the writeup in a few places, but they wanted to publish it in full. After the twitter reaction I received to the “fake news” title, which I thought was a bit more ironic than others read it, I was the one who decided to change the title for the published version. I saw no point in giving people an opening to attempt to trash the core of the argument by trashing the title. (In hindsight, I should have recognize that at first, and just started the title with “Incorrect News” or “False News,” which would have been enough of irony without causing as much apparent offense.)

    All that being said, I did expect a disagreement in interpretation between me and Mutz, and that is further confirmed in the exchange published below. Here are all the links (to open access, undated) manuscripts:

    Mutz PNAS article:
    https://www.pnas.org/content/115/19/E4330
    pdf version here:
    https://www.pnas.org/content/pnas/115/19/E4330.full.pdf

    Morgan Socius article:
    https://journals.sagepub.com/doi/10.1177/2378023118788217
    pdf version here:
    https://journals.sagepub.com/doi/pdf/10.1177/2378023118788217

    Mutz comment on Morgan:
    https://journals.sagepub.com/doi/full/10.1177/2378023118808619
    pdf version here:
    https://journals.sagepub.com/doi/pdf/10.1177/2378023118808619

    Morgan reply to Mutz:
    https://journals.sagepub.com/doi/full/10.1177/2378023118811502
    pdf version here:
    https://journals.sagepub.com/doi/pdf/10.1177/2378023118811502

    P.S. For amusement: Mutz has provided a bit of personal history in her published comment, about her birth state of Indiana and whatnot. It is amusing, although I did not think it valuable to respond in kind within the pages of Socius. A blog seems a bit better, and again for posterity’s sake. I was also born in Indiana, and in fact my best friend in the 1st grade, before I left the state, was Pat Knight. To me, his father, Bobby Knight was a pleasant and generally kind man (who used to give us candy bars, etc.). He turned out to be a Trump supporter, and probably his son too. So, in addition to not appreciating his full basketball personality when I was 6 years old, I also did not see his potential to find a demagogue inspiring. We moved to Ohio, where I received a lot of education in swing-state politics and Midwestern resentment of coastal elites. And, I know what too many journalists don’t seem to recognize: Places like Youngstown are not rural. They are depopulating, for structural reasons, and with consequences for state and national politics.

    • Steve:

      I was not born in Indiana, but I grew up in suburban Maryland (about 10 miles from Brett Kavanaugh, but I went to a public school in a different part of the county and so had zero social overlap with his group). One of the kids in my school was Chuck Driesell, son of Lefty Driesell, former basketball coach at the University of Maryland. Lefty is unfortunately now most famous for his association with Len Bias, but Chuck and I were in high school before that all happened, when Lefty was famous for being a good coach who couldn’t ever quite beat North Carolina. Once I remember the Terps decided to beat Dean Smith at his own game by doing the four corners offense themselves. But it didn’t work; I think Maryland ended up losing 21-18 or some other ping-pong-like score. Chuck was in my economics class. I have no idea if he’s now a Trump supporter. I guess it’s possible. One of the other kids in that econ class was an outspoken conservative, one of the few Reagan supporters of our group of friends back in 1980. Chuck grew up and became a basketball coach; the other kid grew up and became an economist.

Leave a Reply to Kyle Cancel reply

Your email address will not be published. Required fields are marked *