Question 4 of my final exam for Design and Analysis of Sample Surveys

4. Researchers have found that survey respondents overreport church attendance. Thus, naive estimates from surveys overstate the percentage of Americans who attend church regularly. Does this have a large impact on estimates of time trends in religious attendance?

Solution to question 3

From yesterday:

3. We discussed in class the best currently available method for estimating the proportion of military servicemembers who are gay. What is that method? (Recall the problems with the direct approach: there is no simple way to survey servicemembers at random, nor is it likely that they would answer such a question honestly.)

Solution: I was talking about the work of Gary Gates, combining an estimate of the percentage of gays in the population with an estimate of the probability that someone is in the military, given that he or she is gay.

11 thoughts on “Question 4 of my final exam for Design and Analysis of Sample Surveys

  1. This depends on whether or not the level of overreporting has changed over time. My guess would be that overreporting has increased, but this is an empirical question that I assume someone has looked into.

  2. Depends on what you think drives the overreporting. If overreporting is driven by societal expectations of attendance, then it may shrink as attendance falls, which would lead to an overstatement of the time-trend.

  3. The proportion that are less than honest will be related to the proportion attending (social pressures, shame etc.) So yes.

  4. Even if a constant proportion of non-churchgoers report going to church, the fraction of the population that these non-churchgoers make up changes over time. So yes, it will have an impact, although it could be large or small.

  5. My first reaction is to say that it should make no difference to the overall trend. It may look as though the figure has dropped from 80% to 70% over a set period of time, but if the reality is that it’s dropped from 70% to 60% during that time, it doesn’t matter.

    But now that I think about it, I realise that it’s more complicated than that. There will probably be a tipping point (or, more realistically, a series of tipping points in each sub-demographic) where people no longer feel the need to pretend that they’re going to Church, and this will be linked very closely to how many people are actually going. If 70% of the public are going to Church, then there’s a strong incentive for the other 20% to hint that they are as well. If 30% of the public are going, that incentive becomes much weaker for the non-Churchgoers.

    This means that if Churchgoing fell from 70% to 20% over a long time frame at a consistent rate, it may well look as though the fall started off gently, then got much faster and then slowed down again even though the truth of the matter is that it had remained steady.

    In other words, the over-reporting should have no impact on the DIRECTION of the trend, but may have a significant impact on the MAGNITUDE of the trend.

  6. Pingback: Question 5 of my final exam for Design and Analysis of Sample Surveys « Statistical Modeling, Causal Inference, and Social Science

Comments are closed.