Nick and Nate and Mark on Leicester and Trump

Just following up on our post the other day on retrospective evaluations of probabilistic predictions:

For more on Leicester City, see Nick Goff on Why did bookmakers lose on Leicester? and What price SHOULD Leicester have been? (forwarded to me by commenter Iggy).

For more on Trump, see Nate Silver on How I Acted Like A Pundit And Screwed Up On Donald Trump and Mark Palko making a lot of these points, back in September, 2015, on how journalists were getting things wrong and not updating their models given the evidence in front of them.

Which is closely related to the “Why did bookmakers lose on Leicester?, because it seems that their big losses came not from those 5000-1 preseason odds but from bets during the early part of the season, when Leicester was winning and winning but the bookies were lowering the odds only gradually, not fully accounting for the information provided by these games.

I’ll also use this post as an occasion to repeat my idea that a good general framework for predicting events that only occur once is to embed them in a larger system, which can be done in two ways: (a) by considering the event in question as one of many in a reference set (other sports championships, other national elections), and (b) considering precursor events (won-lost record, score differentials, vote differentials). Either way requires assumptions (“models”) but that’s what you gotta do.

11 thoughts on “Nick and Nate and Mark on Leicester and Trump

  1. I think Nate’s been clear about the fact that he’s trying to use your points a) and b) for Trump, but it seems he placed too much reliance on the fact that there hasn’t been a Trump-like success before and any comparable previous instances in the universe of political nominees seemed to flame out before they got close to this stage. I think the important question is when to start giving a lot of weight to the potential that you are really witnessing a low-odds event and stop giving as much weight to the historical reference points. I’m not saying Nate did everything right, but there’s not much data for the primary system, so it’s hard to get it right.

    • Right – Taleb makes a great point about the mistakes of induction that modelers will always have because they’re learning from only a few draws of one of potentially many models. But this is really exacerbated by the training set / prior selection being too focused on what they take to be analogous histories (same country, same market, etc.). The world is big and history is long, and sometimes what would appear to be a surprise (to an election forecaster in the USA) would appear to be relatively common (to someone with a more global view, a student of populism, say). In the forecasting we do (predicting cashflow volatility for small-lenders in frontier markets), we use information from thousands of other lenders from around the world to inform our priors of the base rate of mass defaults etc. I’d rather appear to be less certain in most futures than too certain in a few.

      I had an idea for dealing with when to ignore history. The first two sections here outlines it: https://github.com/khakieconomics/Thesis_work/blob/master/Thesis_final_Savage_James.pdf

      • And I’m sure the other exacerbating factor was that it made the 538 crew feel cool and smart when they were able to state that their data-driven (theoretically) forecast was better than what mainstream pundits and the media were predicting. I think they got emotionally attached to their model because of that. Probably less of a problem in small-lender markets than in politics, but I don’t think you can ignore the fact that there’s some inertia when people get attached to their results.

  2. > not fully accounting for the information provided by these games.

    not fully accounting (including checking the model to dump or modify when appropriate)

  3. I thought the interesting bit in the Leicester links was the very last where he says maybe it should have been 1 in 29 or so odds. The Leicester championship drew so much attention because the 5000 to 1 odds were absurd and reflected the mortmain, the dead hand of old Premier League models. That bookies lost because they didn’t shift odds quickly enough isn’t necessarily meaningful because the bet is on winning the League not playing well and they didn’t have it sewn up until almost the end. But by then they were locked in to a certain amount of money on Leicester at uncomfortable odds. That’s what happens when long shots come in. The other neat thing to me is you can fiddle with how you rated the teams but you are shooting in the dark without more information: is this a true shift in the Premier League, not toward Leicester but toward a more balanced league. That needs more data.

  4. It’s helpful to take a look at where conventional wisdom was among data journalists back in August.

    http://fivethirtyeight.com/datalab/podcast-totally-subjective-presidential-odds-early-august-edition/

    https://espnfivethirtyeight.files.wordpress.com/2015/08/oddspic-e1438804785711.jpg?w=575&h=396

    Look at the numbers for Bush/Walker/Rubio. I suspect you would have seen something similar if you had asked Nate Cohn or polled the Mischiefs of Faction.

  5. Nate Silver should have looked at recent elections in other countries: immigration policy, Trump’s distinctive issue, has been a huge factor in foreign elections over the last couple of years: usually favoring the right, but every now and then the left (e.g., Canada).

    It’s a little bit like the Thatcher-Reagan wave from 1979 onward that eventually brought down the Soviet Union, although it’s more complicated because the current realignment is orthogonal to the old left-right polarization, with the new poles being globalism vs. localism.

    • Steve:

      Could be, but I’m skeptical about immigration issues as being much of an explanation for Trump’s success, given that not many Americans, when surveyed, characterize immigration as the “most important problem facing this country today.” See this from Gallup, for example: 8% mention immigration or illegal aliens, compared to 13-16% who mention dissatisfaction with government, and 27-40% who mention some sort of economics problem.

      • The lack of enforcement of immigration laws isn’t a reason dissatisfaction with government due to economics problems?

        In 2015, many of the biggest stories in the world — such as the two massacres in Paris and Merkel’s million Muslim mob — have been related to immigration. Not surprisingly, immigration has been a major issue in a host of elections around the world, with substantial impact on the results in multiple countries.

        There’s a global wave going on, so why not notice that the Trump phenomenon is part of it?

Leave a Reply to Mark Palko Cancel reply

Your email address will not be published. Required fields are marked *