[This blog entry was posted in May, 2013 and I'm adding links to related news stories as I become aware of them. Otherwise text of blog entry has not changed.]
Proponents of a mandatory American Community Survey (ACS) have watched closely Canada's experience since converting their mandatory 2011 Census long-form to the voluntary National Household Survey. We learned more than a year ago that, as expected, their response rates decreased and costs went up but we had to wait until last week, May 8, to learn about the data quality.
As one would have predicted the small-area data suffered the most.
Former Chief Statistician of Canada Munir A. Sheikh I think summarized it most succinctly in his article for The Globe and Mail, Canada has lost its census anchor,
The NHS has released data for just 75.3 per cent of the Canada-wide 4,567 census subdivisions, compared to the 2006 long-form census rate of 96.6 per cent. The data not released were of particularly poor quality. For Saskatchewan, at the low end, the release rate is just 57.4 per cent.
Sheikh also agreed with
Statistics Canada’s view that the survey “will produce usable and useful data that will meet the needs of many users … It will not, however, provide a level of quality that would have been achieved through a mandatory long-form census.”
Another article in The Globe and Mail,
Experts debate how much National Household Survey statistics count, put it this way:
At the 30,000-foot level it will be difficult to spot the difference, but as users drill down to smaller communities or small geographical areas they will likely discover the findings are fuzzier, or even missing.
“This will not have the detail or the precision of the traditional long-form census,” said Ian McKinnon, chair of the National Statistics Council, which advises Canada’s chief statistician. “For small groups and small areas, it will be harder to get a clear view of Canada.”
Sheikh and others are also cautioning about the comparison of the 2011 NHS data to the 2006 Census long-from.
To be specific on the response rate for the 2011 NHS, it was 69% according to this
Statistics Canada website, down from the expected mandatory response rate of 94%. In terms of cost, The 2011 Census Program budget over 7 years was $630M. $30M million in additional funding was provided for the shift to voluntary to cover 2 fiscal years. This took the total budget to $660M (although it was possible some of the additional funding would be returned.)
Those of us communicating with the U.S. Congress about the merits of a mandatory ACS caution about decreased response rates, increased costs, and uncertain data quality, especially for small areas. Canada's experience with the voluntary NHS backs these points up. Given the current efforts in the U.S. to make the ACS voluntary or to eliminate it altogether, it is imperative to communicate to Members of Congress the importance of these data to i) economic growth and job creations; ii) smarter, more efficient government; and iii) saving taxpayer money. To read more about U.S. ACS developments, the following three blog entries chronicle Congress's recent efforts to make the ACS voluntary:
Below is a selection of articles regarding the NHS data release, some expressing counterpoints within the article or as the point of the column. Note that some of the articles appeared before May 8, the NHS data release date.
- Let’s move beyond census debates. The world is rich with data, Philip Cross, May 8, The Globe and Mail.
- Loss of long-form census blurs snap shot of Canadians, Michael Den Tandt, May 8, Calgary Herald.
- Skepticism surrounds household survey results, May 14, YorkRegion.com
- Tory heartland hit the hardest with loss of long-form census data, May 8, Times Colonist.
- Compare voluntary survey with mandatory census at own risk: Statcan, Steve Rennie, May 8, Times Colonist.
- Census replacement costs more, gets less info, Mia Rabson, May 13, Winnipeg Free Press
- Data from census replacement to detail Canada's immigration patterns, May 7, CTV News
- National Household Survey That Replaced Long-Form Census Is Unreliable: Experts, Sunny Freeman, May 6, Huffington Post.
- Province's top statistician prods feds to hold 2016 census, Mia Rabson, May 17, Brandon Sun.
See also:
- O, Canada! More Lessons From North of the Border, Terri Ann Lowenthal's May 29 blog entry for the Census Project.
- "Statscan chief fires back at survey critics," Tavia Grant, June 24, Globe and Mail.
- "Statistics Canada clients see ‘gaps’ in latest survey analysis," Mike de Souza, June 28, Leader Post.
- "Statscan's National Household Survey Is Not a 'Small' Data Problem," Maripierre Isabelle, July 4, Huffington Post.
- "Census changes a problem for city’s planners," David Reevely, July 8, Ottawa Citizen.
- "The Statistical Society Of Canada Comments On The National Household Survey," June 26, 2013 SSC press release.
- "StatsCan delays household survey release over data problem," August 13, The Canadian Press, CBC News.
- "Why Canada still needs census in age of data mining," Shenaz Kermalli, August 15, CBC News.
- "Voluntary census already damaging reliability of statistics, harm is ‘cumulative’," Laura Ryckewaert, August 26, The Hill Times online.
- "We’re all paying extra for Ottawa’s census decision," Paul Schneidereit, August 27, The Chronicle Herald.
- "National Household Survey: Statistics Canada says errors won’t harm its reputation," Jason Fekete, September 11, The Vancouver Sun.
- "The ‘birthday paradox’ and why polls can’t tell us everything," David Parkinson, September 17, Globe and Mail.
- "Household survey or long-form census? Tories to choose (again) for 2016," Anna Mehler Paperny and Rebecca Lindell, September 17, Global News.
- "Census picture far from perfect," Mary Agnes Welch, September 23, Brandon Sun
- "Bring back compulsory census," Editorial, September 24, Winnipeg Free Press.
- "Inferior info impacts Gladstone," September 24, Brandon Sun.
- "A number is never just a number: Pseudo census," Hennessy's Index, October 1, Rabble.CA.
- "Canadian income data 'is garbage' without census, experts say," Tavia Grant, October 4, Globe and Mail.
- "NDP calls on Conservatives to reinstate the long-form census," press release, New Democratic Party of Canada.
- "Cheers & Jeers," Editorial, October 7, The Telegram.
- National Household Survey: Here’s what we know about what we don’t know, Patrick Cain and Anna Mehler Paperny, October 14, Global News.
- Statscan’s National Household Survey missed people on social assistance. Here’s why that’s a problem, Patrick Cain and Anna Mehler Paperny, October 21, Global News.
- Despite bad reviews, Tories sticking with voluntary census for 2016, CTV News, April 15, 2014.
- Records shed light on why National Household Survey release was postponed, Jason Fekete, Ottawa Citizenk May 5, 2014.
- Internal survey blasts feds for missing database: long-form census, Dean Beeby, The Vancouver Sun, May 10, 2014.
- Less data available without long-form census, AG confirms, Jessica Hume, Sun News, May 6, 2014.
- Policy making suffering in Canada without the long-form census, Paul Jacobsen, Globe and Mail. November 5, 2014.
- Former Chief Statistician, MPs urge Harper to bring back long-form census, Janice Dickson, Vancouver Sun, November 6, 2014.
- End of long-form census has cut deep into Surrey services, critics say, Patrick Blennerhassett, Business Vancouver, December, 15, 2014.
- MP Ted Hsu calls for reinstatement of Long-Form Census, 9/22/14
- 1/29/14: Advocates for Canada long-form census launch Evidence for Democracy
- Damage from cancelled census as bad as feared, researchers say, Tavia Grant, Glob and Mail, January 29, 2015.
- Why Canada should reinstate long-form census: Scharper, Stephen Bede Scharper, Toronto Star, 2/1/15
- Nurses back bill to reinstate the long-form census, SooToday.com Staff, 1/31/15
- Simcoe North MP says return of long-form census 'not going to happen', Patrick Bales, The Orillia Packet & Times, 2/2/15
- Census Debate Back In House Of Commons With Duelling Bills, Jennifer Ditchburn, The Canadian Press, Huffington Post, 2/2/15
- Data crimes and misdemeanors: Why child care, too, needs the long-form census, Martha Friendly, rabble.com, 2/2/15
- Ted Hsu's Bill To Bring Back Long-Form Census Doesn't Pass Vote, Canadian Press, Huffington Post, 2/4/15.
- ‘As Canadians we should have good data': Why Ted Hsu tilts at census windmills, Anna Mehler Paperny, Global News, 2/4/15.
- Long-form census is needed for good decision-making: Editorial, Toronto Star Editorial, 2/5/15.
- The new long-form survey, Patrick Grady, Financial Post, 2/9/15.
- Revive the census, Editorial, Montreal Telegram, 2/11/15.
- Mayor wants return of long-form census: Information helps with planning. By Charles Hamilton, The Starphoenix, With Canadian Press Files. Leader-Post 2/11/15
- Did losing the long-form census weaken Canada’s jobs data?, Kevin Milligan, MacLean's, 2/16/14.
- Consider move to long survey, The Star Phoenix, 2/18/15.
- Statistics Canada chief pledges to fill jobs, wages data gap, Tavia Grant, The Globe and Mail, 2/25/15.
- The Tragedy of Canada's Census, Aarian Marshall, City Lab, 2/26/15.
- Statistics Canada chief defends data as quality concerns persist, Tavia Grant, The Globe and Mail, 3/11/15
- What happened when Canada stopped counting its numbers, Leyland Cecco, 4/2/15.
- Canadian research study hits snag over lousy population data, Tavia Grant, The Globe and Mail, 5/7/2015.
- A last chance to save the census, Editorial, National Post, 5/13/15.
- What We Don’t Know About Canada Might Hurt Us, Ben Casselman, FiveThirtyEight, 8/11/2015.
- Data On Canada Is Drying Up Since The Nation Scrapped Its Mandatory Long-Form Census, Alexander Howard, Huffington Post, 8/18/15.
- Long-form census could be reinstated for 2016, experts say, Tavia Grant, The Globe and Mail, 10/21/15.
- Trudeau urged to reinstate long-form census, Katie Bueckert, Yahoo! News, 10/22/15.
- Return of the long-form census excites the experts, Marco Chown Oved, Toronto Star, 10/23/15.
- Stats Can waiting to possibly reinstate long-form census, Jason Fekete, Ottawa Citizen, 10/27/15.
- Quickly restore credible survey, Editorial, Saskatoon Star Phoenix, 10/31/15.
- Liberals to restore mandatory long-form census, Bruce Campion-Smith, Toronto Star, 11/2/15.
- Return of long-form census a win for science, rationality, Paul Schneidereit, Halifax Chronicle Herald, 11/3/15.
- Canada expected to reinstate mandatory census, Brian Owens, ScienceInsider, 11/5/15.
- Canada’s long-form census is back for 2016, Bruce Campion-Smith, Toronto Star, 11/5/15.
- Official Government statement, 11/5/15.
See other ASA Science Policy blog entries. For ASA science policy updates, follow @ASA_SciPol on Twitter. For more on ASA science policy, see http://amstat.org/policy/.