Skip to content

Did Colombia really vote no in that peace referendum?

Mike Spagat and Neil Johnson write:

The official line is that the “no” vote won the referendum in Colombia. The internationally lauded peace treaty with the FARC guerillas was rejected . . .

But did “no” actually win?

The numbers divide four ways, rather than just two “no” and “yes” answers: 6,431,376 against the treaty, 6,377,482 in favour, 86,243 unmarked ballots, and 170,946 nullified ballots.

The referendum process itself was without doubt transparent and fair . . . But there were nonetheless several inevitable sources of statistical error in the counting process that could have swamped the razor-thin victory margin of 53,894.

Spagat and Johnson first point out errors in ballot counting:

As a large research literature has made clear, we can reasonably assume that even well-rested people would have made mistakes with between 0.5% and 1% of the ballots. On this estimate, about 65,000-130,000 votes would have been unintentionally misclassified. It means the number of innocent counting errors could easily be substantially larger than the 53,894 yes-no difference.

But this would only make a difference if the miscounts were biased, not if the mistakes were happening at random.

Spagat and Johnson then write:

Second, there were 170,946 nullified votes. As the photo below shows, the ballots were so simple that it’s hard to imagine how there could be so many invalid ones.


Of course, it is correct to toss out any ballot with both “yes” and “no” marked, but it seems surprising that so many ballots were apparently spoiled this badly. Again, the consistency of the counters’ decisions could have been decisive.


Then there are the blank or nullified ballots. It’s quite possible that many of the 86,243 unmarked ballots were simply marked very lightly, meaning their votes did not register in the tired eyes of well-meaning volunteer counters – and the nearly 270,000 voters whose ballots were rejected as blank or null must have had some voting intention that they somehow failed to express.

So these ballots represent 270,000 voting failures, more than four times the victory margin. Even if many of these ballots were reasonably well-classified, this figure is an enormous red flag.

What about biases? Spagat and Johnson write:

We know of no evidence of cheating, and Colombia is to be lauded for the seriousness of its referendum process, but the distinction between intentional and unintentional misclassification by individual counters can occasionally become blurred in practice.

As a study that asked voters to judge “ambiguous” ballots demonstrated, those doing the counting can be driven by unconscious biases. . . .

In total, therefore, the result presents as many as 400,000 opportunities for classification mistakes. That’s before counting any systematic human behaviours not listed above. This represents a numerical uncertainty that swamps the victory margin of 53,894.

They summarize:

None of the above analysis proves that most voters on Sunday supported the peace treaty, but there’s an immense difference between declaring that “no” won and declaring the result inconclusive. This referendum has momentous implications, not just for the Colombian people but also for the many national governments and the United Nations that supported the peace deal. It is very sad that a country’s future may be dictated by an inaccurate declaration (one readily amplified by international media).

Interesting. I don’t know anything about Colombia myself, so all I can say is there’s a lot to think about here.


  1. Chris G says:

    I’m not much of a football fan but one-time coach of the local team, Bill Parcells, once said (roughly) “Losing a game 7-6 to a really good team doesn’t mean ‘You’re almost as good as the really good team.’ it means ‘You can’t win close games.'” I agree. Absent evidence of fraud, the question shouldn’t be “Would we have won if not for bad luck?” but “Why the did only half the voters support the peace treaty?” That’s the elephant in the room. The country has been at war for how long and voters rejected the peace referendum? How bad did the terms have to be for voters to reject it? Or is it that there’s so much animosity of one side towards the other that allowing the conflict to continue is more desirable than ending it? I want some insight into what’s going on there. Concern over +/-0.1% of the vote seems quaint by comparison.

    • Mike Spagat says:


      You are certainly right that there is a huge looming question. Why did so many people rise up and vote against the peace treaty?

      On the face of it this might seem completely insane. I think that “no” voters did make a critical error (in the conventional, not statistical sense) but voting “no” wasn’t just crazy. There is actually a cottage industry springing up on the internet explaining their reasons. Of the many choices I’d single out this article for a briefing:

      In my opinion all roads lead to anger and the desire for payback. On this subject this book is great:

      It’s worth conjuring up an image of where South Africa would be now if Nelson Mandela had spent his years in prison planning how he was going to annihilate his tormentors as soon as he got the chance.

      • Chris G says:

        > In my opinion all roads lead to anger and the desire for payback.

        To me that seems the crux of it. How would a 50%+1 vote have affected anger and desire for payback relative to the outcome the got.

        > It’s worth conjuring up an image of where South Africa would be now if Nelson Mandela had spent his years in prison planning how he was going to annihilate his tormentors as soon as he got the chance.

        A grim thought. It seems like a Colombian version of the Truth and Reconciliation Commission would be a good thing.

    • Dzhaughn says:

      The problem here is naive dependence on simple majorities. Even if the referendum had passed by 100,000 votes, the correct response is to not conclude the treaty with FARC. Simple majorities are not stable under perturbations.

      For another example, Brexit should have required (say) a 60/40 vote. For a third, entrance to the EU should have required 60/40, making Brexit moot. If there would still be an EU, it would have institutions that are more robust, although with a smaller scope.

  2. Jonathan says:

    By their logic, we should not consider conclusive any close vote involving a large count. So we have no decisions unless the various sources of error are less than the margin of victory. That’s workable, right? We just don’t have winners or losers and treat everything as a tie if it’s close because that would make things so much more governable, right? You lose a vote and declare victory because the winning margin was less than the possible error in the count so you actually won if you can get away with declaring victory and taking power because that is how democracy works, right?

  3. Rahul says:

    Their interpretation of blank & nullified ballots seems strange. They are not all errors. I think some people consciously do it because they don’t want to make a choice but still exercise their right to vote.

    At least it reduces the chance of fraud votes.

  4. mark k says:

    I think “this wasn’t a win, it was inconclusive” is the wrong viewpoint to bring. It’s viewing the votes cast as the whole point of the exercise.

    The main justification for a vote is to make a decision based on the will of the people. So that’s the point. “Votes honestly counted and tallied” is the observable parties have agreed upon to measure it. Worrying about another proxy, “votes cast,” when you can’t measure it seems silly.

    In a vote this close I’ll agree the “votes counted and tallied” can’t really distinguish the people’s will, but that’s a problem larger than counting error. Hold the same vote the next day and even with perfect counting you might get a different result. You still need some rule to reach the decision, and absent gross incompetence, fraud, or systematic bias, votes tallied seems as good a heuristic as any. It may not be much better than flipping a coin, but without more information it’s still a little more likely that the side that finished ahead in the best count actually got more votes.

  5. Rahul says:

    Naive question: So why are so many elections so closely contested? Or is that just a perception. Any data on (say) recent national-scale elections in the world what percent end up being what one would call “close”?

    Is it a resource optimization issue? i.e. If I really win an election with a landslide margin it means I wasted too many resources (money, people, risks, crimes etc. ) into winning it when I could have cut down on my campaign efforts had I been more prudent?

    Historically, are election outcomes getting closer to call over the years? Is there a trend?

    Is this because we can predict things better so we can be certain quicker whether we are winning and hence cut back on the excess resources?

  6. Xi'an says:

    You do not know anything about Columbia? I thought you were a professor there…

  7. Cris says:

    Of the total voting population, 40% went to the ballot box and of that, 20% wanted the deal. That deal had no support whatsoever, and even if it would have passed by your slim margin, it would have been doomed to failure.

  8. Jon says:

    One would have to assume that the population of excluded ballots has heavily slanted towards yes for them to change the vote. i.e. that about 225,000 out of 400,000 were really “yes” and the rest no. This is far more than you could get from random noise.

    For counting errors to be a factor, again there would need to be bias. A 1% error rate on 13M ballots implies 130,000 miscounts. If these miscounts are completely random they would swing the spread between no and yes votes something under 2000 ballots (I am being a little sloppy here, but it is not close to the 53,0000.

    A more likely source of bias is factors affecting participation in various regions. Some commentators have noted that the storms discouraged people from going to the polls near the coastal regions.

Leave a Reply