Skip to content

Gigo update (“electoral integrity project”)

Someone sent me this note:

I read your takedown of the EIP on Slate and then your original blog post and the P. Norris response. I wanted to offer a couple of points.

First, as you can see below, I was asked to be one of the ‘experts.’ I declined. I think we all can see the kind of bias introduced into the sample of experts when the sampling frame is a list of email addresses of elections scholars and participation is based on self selection.

Second, and probably more fundamentally, the data are generated by expert responses to a 12 minute survey (quick!). But the issues surveyed are pretty significant. Is media coverage adequate? Is the vote counted fairly? How am I supposed to assess this for an entire state (much less an entire country)? Even if I am paying close attention (and I’m not, since most of my time is spent, you know, trying to get things published…), do they really expect my range of attention to cover ** media markets and ** counties? It’s not like I have a range of sources across the state that I could call upon to give me information on these topics from areas far from ** – if I did, I’d be a journalist.

Finally, notice the date of the request – two weeks after Election Day. Even if I could gather all the necessary information to make my survey responses valid, I would have had to be primed to look for these things not only before Election Day, but also before the campaign even took place. Asking me to recall information after the fact when I wasn’t necessarily looking for it at the time is, well, bad.

I thought this effort sounded fishy when the request to participate landed in my inbox, and with your assessment and the Norris response I now have even less confidence in these data. I appreciate Norris’s effort to engage in a public dialogue on this and (in general) her efforts to get us to think about how to assess electoral integrity, but I more appreciate your efforts to point out the methodological issues and to keep journalists from running away with “findings” based on faulty social science.


Prof. **
Dept. of Political Science
** University

Here’s the email that my correspondent received:

From: Electoral Integrity Project <**@**>
Sent: Tuesday, November 22, 2016 10:41 AM
To: **
Subject: Harvard University seeks your expertise on electoral integrity in **

Dr. **
Political Science
** University

Dear Dr. **,

How do we know when elections meet international standards and principles – and when they fail?

Given your knowledge and expertise we are interested in learning your views about how the US presidential election on 8 November 2016 was conducted in **.

The survey usually takes around 12 minutes to complete. Your answers are anonymous and all replies will be treated with the strictest confidence. You can participate in the survey or decline to do so by clicking below.

I would like to participate. Your unique reference number is: *****
I would like to decline the opportunity to participate.

The aim of the survey is to gather comprehensive, impartial, and reliable information which can compare the quality of all national elections held worldwide. So far the research has monitord contests in over 100 countries. The study is conducted by an independent team of scholars based in Australia, Europe and the United States. The data is made widely available to the user community and released every year. You are welcome to contact us at **@** for any further information.

You can read the relevant Participation Information Statement, which provides more information about this study. Completing the survey is an indication that you have read and understand the Participation Information Statement.

We realize that you have numerous demands on your time and we greatly appreciate you collaboration and help in this project.

Professor Pippa Norris
(Harvard University and the University of Sydney)

The Electoral Integrity Project
Department of Government and International Relations,
University of Sydney
Sydney, NSW 2006

Professor Pippa Norris (Harvard University and the University of Sydney)
Professor Jørgen Elklit (Aarhus University)
Professor Andrew Reynolds (University of North Carolina, Chapel Hill)
Professor Jeffrey Karp (University of Exeter)
Project Manager: Dr. **
Survey Manager: Mr. **
Research assistant: Ms **

Email: **@**

It’s always good to know where your data came from. I think Norris has done a great service with the World Values Survey but I’m more skeptical about the Electoral Integrity Project.

P.S. Joseph Cummins writes, “my girlfriend now wishes to externalize her love for our neighborhood’s most boss street cat onto you.” I suspect this cat has strong feelings about electoral integrity.


  1. christian says:

    Note that Pippa Norris has not helped collect data for the world values survey, although she uses the data

  2. Dear Christian,

    Thank you very much for your remark. On behalf of the World Values Survey Association (WVSA), we are delighted to inform you that WVS is a non-commercial academic research project. We deliberately made our data available in free access for broad public on the Internet. There are several hundred thousand of social scientists, policymakers, teachers, students, journalists etc. who use our data in their research and other activities on the daily basis – despite they did not “[help] collect data for the world values survey” as you said. Hence, usage of WVS data is absolutely free and does not require involvement into the WVS activities.

    Secondly, Pippa Norris is a member of the WVSA governing board since many years. In wave 6 Pippa Norris served as the WVS Association’s Treasurer and helped raise several hundred thousand dollars which were used to conduct WVS survey in around 15 countries in the Middle East and Sub-Saharan Africa. If you have any further questions regarding the WVS activities, please, do not hesitate to contact the WVSA Secretariat. Kindly note that we can not provide any comment regarding EIP project as it is not linked with the WVS.

    Best wishes,
    WVSA Secretariat

Leave a Reply