
Nick Stevenson directed me to a recent op-ed in the Raleigh News & Observer, where political science professor Andrew Reynolds wrote:
In 2005, in the midst of a career of traveling around the world to help set up elections in some of the most challenging places on earth . . . my Danish colleague, Jorgen Elklit, and I designed the first comprehensive method for evaluating the quality of elections around the world. . . . In 2012 Elklit and I worked with Pippa Norris of Harvard University, who used the system as the cornerstone of the Electoral Integrity Project. Since then the EIP has measured 213 elections in 153 countries and is widely agreed to be the most accurate method for evaluating how free and fair and democratic elections are across time and place. . . .
So far so good. But then comes the punchline:
In the just released EIP report, North Carolina’s overall electoral integrity score of 58/100 for the 2016 election places us alongside authoritarian states and pseudo-democracies like Cuba, Indonesia and Sierra Leone. If it were a nation state, North Carolina would rank right in the middle of the global league table – a deeply flawed, partly free democracy that is only slightly ahead of the failed democracies that constitute much of the developing world.
I searched on the web and could not find a copy of the just released EIP report but I did come across this page which lists all 50 states plus DC.
North Carolina is not even the lowest-ranked state! Alabama, Michigan, Ohio, Georgia, Rhode Island, Pennsylvania, South Carolina, Mississippi, Oklahoma, Tennessee, Wisconsin, and Arizona are lower.
Hmmm. Whassup with that?
Here’s the international map from The Year in Elections, 2014, by Pippa Norris, Ferran Martinez i Coma, and Max Grömping:

There’s North Korea in yellow, one of the countries with “moderate” electoral integrity. Indeed, go to the chart and they list North Korea as #65 out of 127 countries. The poor saps in Bulgaria and Romania are ranked #90 and 92, respectively. Clearly what they need is a dose of Kim Jong-il.
Let’s see what this measure actually is. From the report:
The survey asks experts to evaluate elections using 49 indicators, grouped into eleven categories reflecting the whole electoral cycle. Using a comprehensive instrument, listed at the end of the report, experts assess whether each national parliamentary and presidential contest meets international standards during the pre-election period, the campaign, polling day and its aftermath. The overall PEI index is constructed by summing the 49 separate indicators for each election and for each country. . . .
Around forty domestic and international experts were consulted about each election, with requests to participate sent to a total of 4,970 experts, producing an overall mean response rate of 29%. The rolling survey results presented in this report are drawn from the views of 1,429 election experts.
OK, let’s check what the experts said about North Korea; it’s on page 9 of the report:
Electoral laws 53
Electoral procedures 73
District boundaries 73
Voter registration 83
Party and candidate registration 54
Media coverage 78
Campaign finance 84
Voting process 53
Vote count 74
Results 80
Electoral authorities 60
Each of these is on a 0-100 scale with 100 being good. So, you got it, North Korea is above 50 in every category on the scale.
Who did they get to fill out this survey? Walter Duranty?
OK, let’s look more carefully. In this table, the response rate for North Korea is given as 6%. And the report said they consulted about 40 “domestic and international experts” for each election. Hmmm . . . 6% of 40 is 2.4, so maybe they got 3 respondents for North Korea, 2 of whom were Stalinists.
That 2014 report mentioned above gave North Korea a rating of 65.3 out of 100 and Cuba a rating of 65.6. Both these numbers are higher than at least 27 of the 50 U.S. states in 2016, according to the savants at the Electoral Integrity Project.
Political science, indeed.
How’s North Korea been doing lately? Stevenson writes:
North Korea is in The Year in Elections 2014 but was quietly removed from The Year in Elections 2015. It’s not a matter of the 2014 elections not being in the 2015 timeframe either – diagram 5 of The Year in Elections 2015 says ‘PEI Index 2012-2015’ and North Korea was in Diagram 1 of The Year in Elections 2014, PEI Index 2012-2014. They have North Korea in gray in the later world map as ‘Not yet covered’. On p. 73 of The Year in Elections 2015 they list their criteria for inclusion in the survey (no microstates, no Taiwan, etc) but don’t explain why PRK_09032014_L1 has suddenly gone missing.
Perhaps North Korea was too embarrassing for them?
In his email to me, Stevenson wrote:
This is terrible research that I [Stevenson] think has the potential to do real damage in the real world with their absurdly high scores for fake elections in places like Oman, Kuwait, Rwanda, and Cuba. Suppose Oman’s government arrests an opposition politician or cracks down on a peaceful demonstration and the EU and US ambassadors protest. What if the Omani government argues that according to Harvard University’s measure which is “widely agreed to be the most accurate method for evaluating how free and fair and democratic elections are across time and place”, Oman is in much better shape than many EU countries and US states and that they should get their own houses in order before criticizing others? The EIP is just as likely to serve as a freebie to repressive governments that somehow fluke a high score as it is to spur the repeal of Wisconsin’s ID law.
If Reynolds, Norris, etc., don’t like what the North Carolina legislature has been doing, fine. It could even be unconstitutional, I have no sense of such things. And I agree with the general point that there are degrees of electoral integrity or democracy or whatever. Vote suppression is not the same thing as an one-party state and any number-juggling that suggests that is just silly, but, sure, put together enough restrictions and gerrymandering and ex post facto laws and so on, and that can add up.
Electoral integrity is an important issue, and it’s worth studying. In a sensible way.
What went wrong here? It all seems like an unstable combination of political ideology, academic self-promotion, credulous journalism, and plain old incompetence. Kinda like this similar thing from a few years ago with the so-called Human Development Index.
P.S. I googled *reynolds north carolina democracy* to see how much exposure this story got, and I found links to Democracy Now, Vox, Slate, Daily Caller, Common Dreams, American Thinker, Qz.com, MSNBC, Huffington Post, Think Progress, The Week, Grist.com . . . basically a lot of obscure outlets. I write for Slate and Vox, so I was sorry to see them pick this one up.
But the good news is that the usual suspects such as ABC, NBC, CBS, CNN, NPR, BBC, NYT didn’t fall for it. I give these core media outlets such a hard time when they screw up, and they deserve our respect when they don’t take the bait on this sort of juicy, but bogus, story.
P.P.S. See here for more from Pippa Norris.