ASA Connect

 View Only
Back to eGroups
Expand all | Collapse all

Census Bureau says a citizenship question would likely not reduce response rates

  • 1.  Census Bureau says a citizenship question would likely not reduce response rates

    Posted 11-08-2019 08:54
    The Census Bureau tested the response rate for questionnaires with and without the citizenship question. The result was no significant difference.

    "The major finding of the test was that there was no difference in self-response rates between forms with and forms without a citizenship question" from

    https://www.census.gov/newsroom/blogs/random-samplings/2019/10/2019_census_testpre.html

    ------------------------------
    Terry Meyer
    ------------------------------


  • 2.  RE: Census Bureau says a citizenship question would likely not reduce response rates

    Posted 11-11-2019 08:11
    Note that according to the footnote following the sentence that Terry Meyer quoted, the words "no difference" should be taken to mean 'no significant difference with a 10% significance level'.  The actual difference in response rates was 0.5% (52.0% for the group without the citizenship question and 51.5% for the group with the citizenship question).  This experiment was conducted to evaluate how many people the census bureau would need to hire to complete the work needed when people don't respond to their initial automated contact efforts through the mail.  Thus, their conclusion was that there would be no difference in required staffing levels - not that the citizenship question would not affect response rates.  Indeed the question of response rates to live interviewers was not even examined - since this was completely about response to mailings.

    ------------------------------
    Dennis Pearl
    Professor
    Penn State University
    ------------------------------

    Original Message


  • 3.  RE: Census Bureau says a citizenship question would likely not reduce response rates

    Posted 11-16-2019 12:08
    I'm perplexed about how such a large sample size wouldn't make this difference in response rates statistically significant.  (GASP!  I used those words...)  If you split the sample into two groups of 240,000 and have one with a success proportion of 0.515 and the other with 0.52, then using a simple two-proportion test results in an incredibly small p-value.  On the face of it, it doesn't seem like you'd really have to use any more complicated of a test than a two-proportion test, since it seems like they are just comparing two success rates.

    I'm not a trained statistician, but I've been teaching introductory and upper-level statistics for almost 20 years.  So am I missing something here??

    I would grant you that this could be a setting where "statistical significance" doesn't necessarily mean "practical significance".  But the blog/article is saying that the results were not significant at the 0.10 level, so that is speaking to statistical significance.

    ------------------------------
    Mark Mills
    Professor of Mathematics
    Central College
    ------------------------------

    Original Message


  • 4.  RE: Census Bureau says a citizenship question would likely not reduce response rates

    Posted 11-11-2019 12:15
    There was quite a bit more content to the "result" than this.  In part, from the report's conclusions: " ... we are not able to measure the impact of a citizenship question for the completeness and accuracy of 2020 Census overall. Furthermore, the results of the 2019 Census Test will not trigger a major change in our communications campaign strategy, which was built on prior research that indicates that self-response differs across communities, and that some populations may be fearful about participating in the census."

    The report was clear in stating the limited objective of the study: " ... experiment designed to inform hiring levels for the Nonresponse Followup operation, as well as offer insight for the integrated partnership and communication campaign."

    ------------------------------
    Berne Howard
    Bench Mark Heuristics, LLC
    ------------------------------

    Original Message


  • 5.  RE: Census Bureau says a citizenship question would likely not reduce response rates

    Posted 11-12-2019 18:00
    What is being referred to here as "the report"  is, I think, the October 31st Census Bureau blog with preliminary results from the 2019 Census Test.  It should be noted that the Census Bureau submitted a detailed analysis plan as part of its OMB clearance request and that most of the analyses proposed have not yet been released.  One of those analyses that was proposed by the Census Bureau was that there would be comparison of response rates to a census with or without the citizenship question in census tracts with high proportions of non-citizens vs. those in in census tracts with low proportions of non-citizens. Those findings--and others--have not yet been published--a shame because they might provide useful insights to guide messaging and enumerator deployment.

    ASA members should also look at the excellent blog post by Rob Santos exploring the possibility that there was bias in the sample of those who responded to the Census 2019 Test.  I haven't seen information on the survey response rate overall or for sub-populations (although I may have missed that....)

    ------------------------------
    Edward Kissam
    ------------------------------

    Original Message


  • 6.  RE: Census Bureau says a citizenship question would likely not reduce response rates

    Posted 11-11-2019 14:39
    Good that someone ran an experiment and we have data vs opinion.

    ------------------------------
    Sidney Young
    Retired
    ------------------------------

    Original Message


  • 7.  RE: Census Bureau says a citizenship question would likely not reduce response rates

    Posted 11-13-2019 09:10
    Ignoring the politics of both the original post and the comments, I am baffled by the fact that barely 50% of people typically respond to the initial Census survey, regardless of the questions being asked.  I have seen surveys on all sorts of topics of arguable importance that garnered better response rates than that. This is the one and only survey in this country that is mandated by the Constitution. And apparently only ~50% of the populace can be bothered.  That seems to be a pretty sad commentary on the level of civics education in this country and/or the level of trust (distrust) the government has engendered in the populace.

    ------------------------------
    David Metcalf
    Research Statistician
    Henry Ford Allegiance Health
    ------------------------------

    Original Message


  • 8.  RE: Census Bureau says a citizenship question would likely not reduce response rates

    Posted 11-13-2019 09:47
    50+% is actually very good for self-response.  There are LOTS of surveys that don't get anywhere near that for TOTAL response.  The total response to the Decennial Census has in the past ended up being well over 90% once non-response followup is completed.

    ------------------------------
    Michael Ikeda
    ------------------------------

    Original Message


  • 9.  RE: Census Bureau says a citizenship question would likely not reduce response rates

    Posted 11-14-2019 07:59
    ​I was totally unaware the response rate was so dismal on the census.  This would make the whole exercise worthless, as the bias is unacceptable.  I am skeptical of the second line remedies. Why are non-responders given a free ride?  They should face stiff fines, as the census is such a nationally important tool. 

    When I was 18, I spent the summer as an enumerator in Montreal for the Canadian Census.  I ultimately got 100% of my assigned frame to respond.  The area I worked in included English, French (no problem) and Italian (big problem). Fortunately, a 7 year old boy I encountered at one Italian home helped me translate my questions.  His salary was paid in candy bars.

    Sometimes, old-fashioned works best.


    ------------------------------
    Jon Shuster
    ------------------------------

    Original Message


  • 10.  RE: Census Bureau says a citizenship question would likely not reduce response rates

    Posted 11-15-2019 15:56
    Some figures from 2010.

    Mail participation rate (mail returns/households receiving mail questionnaires) was about 72% going in to nonresponse followup. Approximately an additional 2% of households mailed in responses later making the final mail participation rate about 74%.

    https://www.census.gov/newsroom/releases/archives/2010_census/cb10-cn81.html

    About 1.6% of household persons were in households providing only a census count.  Approximately 0.4% of household persons were in households where the count needed to be imputed (includes households where unit status also needed to be imputed).  (In the second paragraph of the section on "Components of Coverage").

    https://www.census.gov/newsroom/releases/archives/2010_census/cb12-95.html

    ------------------------------
    Michael Ikeda
    ------------------------------

    Original Message


  • 11.  RE: Census Bureau says a citizenship question would likely not reduce response rates

    Posted 11-15-2019 09:57

    All the non-responders must be university professors and statisticians! I don't even know what a physical mailbox is anymore; Amazon.com delivers to my door and texts me when it arrives.

    Anyways, how would you model the probably of response? In the article, the plot of mailings (1st through 5th ) vs. cumulative response rate looks kinda like a geometric distribution. However we don't know which mailing the subjects are responding to per se. Would a zero-inflated geometric distribution be an appropriate model? Or something else??



    ------------------------------
    Matthew Robinson
    ------------------------------

    Original Message


Related Content