Question 23 of my final exam for Design and Analysis of Sample Surveys

23. Suppose you are conducting a survey in which people are asked about their health behaviors (how often they wash their hands, how often they go to the doctor, etc.). There is a concern that different interviewers will get different sorts of responses—that is, there may be important interviewer effects. Describe (in two sentences) how you could estimate the interviewer effects within your survey. Can the interviewer effects create problems of reliability of the survey responses? Explain (in one sentence). Can the interviewer effects create problems of validity of the survey responses? Explain (in one sentence).

Solution to question 22

From yesterday:

22. A supermarket chain has 100 equally-sized stores. It is desired to estimate the proportion of vegetables that spoil before being sold. Three stores are selected at random and are checked: the percent of spoiled vegetables are 3%, 5%, and 10% in the three stores. Give an estimate and standard error for the percentage of spoiled vegetables for the entire chain.

Solution: Estimate is (.03+.05+.10)/3=.06. Std err is sqrt(1-3/100)*sqrt(1/3)*s, where s=sqrt((1/(3-1))*((.03-.06)^2+(.05-.06)^2+(.10-.06)^2))=.036. Thus, se=sqrt(1-3/100)*sqrt(1/3)*.036=.02.

Almost all the students got the estimate right, but most choked on the standard error and worked with sqrt(.06*(1-.06)) rather than correctly working with the cluster-level variation.

2 thoughts on “Question 23 of my final exam for Design and Analysis of Sample Surveys

  1. Could use intra-interviewer correlation to detect the effect, I guess. Reliability is a less concern than validity since survey is usually well-structured, but the interviewer effect will create problem to both validity and reliability. So, train them, lol.

  2. Would you really have taken off marks if the student had left off multiplying by sqrt(1-3/100) = 0.985? If you are concerned with this level of precision wouldn’t you define s as
    s = sqrt((1/(3-1))*((.03-.06)^2+(.05-.06)^2+(.10-.06)^2))*sqrt((100-3)/(100-1))
    ?

Comments are closed.