Results on Data Quality for Newly Voluntary Canadian Long-Form Census Coming In

By Steve Pierson posted 05-16-2013 13:36


[This blog entry was posted in May, 2013 and I'm adding links to related news stories as I become aware of them. Otherwise text of blog entry has not changed.]

Proponents of a mandatory American Community Survey (ACS) have watched closely Canada's experience since converting their mandatory 2011 Census long-form to the voluntary National Household Survey. We learned more than a year ago that, as expected, their response rates decreased and costs went up but we had to wait until last week, May 8, to learn about the data quality.

As one would have predicted the small-area data suffered the most. Former Chief Statistician of Canada Munir A. Sheikh I think summarized it most succinctly in his article for The Globe and Mail, Canada has lost its census anchor,

The NHS has released data for just 75.3 per cent of the Canada-wide 4,567 census subdivisions, compared to the 2006 long-form census rate of 96.6 per cent. The data not released were of particularly poor quality. For Saskatchewan, at the low end, the release rate is just 57.4 per cent.
Sheikh also agreed with Statistics Canada’s view that the survey “will produce usable and useful data that will meet the needs of many users … It will not, however, provide a level of quality that would have been achieved through a mandatory long-form census.”

Another article in The Globe and Mail, Experts debate how much National Household Survey statistics count, put it this way:
At the 30,000-foot level it will be difficult to spot the difference, but as users drill down to smaller communities or small geographical areas they will likely discover the findings are fuzzier, or even missing.

“This will not have the detail or the precision of the traditional long-form census,” said Ian McKinnon, chair of the National Statistics Council, which advises Canada’s chief statistician. “For small groups and small areas, it will be harder to get a clear view of Canada.”
Sheikh and others are also cautioning about the comparison of the 2011 NHS data to the 2006 Census long-from.

To be specific on the response rate for the 2011 NHS, it was 69% according to this Statistics Canada website, down from the expected mandatory response rate of 94%.  In terms of cost, The 2011 Census Program budget over 7 years was $630M. $30M million in additional funding was provided for the shift to voluntary to cover 2 fiscal years. This took the total budget to $660M (although it was possible some of the additional funding would be returned.) 

Those of us communicating with the U.S. Congress about the merits of a mandatory ACS caution about decreased response rates, increased costs, and uncertain data quality, especially for small areas. Canada's experience with the voluntary NHS backs these points up. Given the current efforts in the U.S. to make the ACS voluntary or to eliminate it altogether, it is imperative to communicate to Members of Congress the importance of these data to i) economic growth and job creations; ii) smarter, more efficient government; and iii) saving taxpayer money. To read more about U.S. ACS developments, the following three blog entries chronicle Congress's recent efforts to make the ACS voluntary: Below is a selection of articles regarding the NHS data release, some expressing counterpoints within the article or as the point of the column. Note that some of the articles appeared before May 8, the NHS data release date. 

See also:

See other ASA Science Policy blog entries. For ASA science policy updates, follow @ASA_SciPol on Twitter. For more on ASA science policy, see