In addition to being posted on ASA Connect, this message is being sent to the Government Statistics, Health Policy Statistics, Social Statistics, Statistics in Epidemiology, and Teaching Statistics in the Health Sciences Sections, as well as the Washington Statistical Society.
Below is a link to a long letter I sent on October 8, 2015 to ASA leadership, including the Chairs of the Scientific and Public Affairs Advisory Committee and the Committee on Law and Justice Statistics. I’ve since sent the letter all members of those two committees.
http://jpscanlan.com/images/Letter_to_American_Statistical_Association_Oct._8,_2015_.pdf
The letter recommends that ASA do two things, one of which involves a long-term undertaking and one of which can be done almost immediately. First, the letter recommends that ASA form a committee to address the unsoundness of analyses of demographic and other differences in outcome rates that fail to consider the ways measures employed in such analyses tend to be systematically affected by the frequency of an outcome (including, among other patterns, that whereby the rarer an outcome the greater tends to be the relative difference in experiencing it and the smaller tends to be the relative difference in avoiding it). This subject is something I have recently treated at length in references 1 to 3, as well as in the methods workshops listed in note 3 of the letter. This subject of the letter has implications regarding a wide range of ASA member activities.
Second, the letter recommends that in the immediate future ASA formally advise arms of the US government that reducing the frequency of an outcome tends to increase, not reduce, relative differences in rates of experiencing it (as well as the proportion groups most susceptible to the outcome make up of persons experiencing it). By way of background, for more than 20 years the federal government has been encouraging lenders to relax lending standards, and for at least several years the government has been encouraging public schools to relax discipline standards, in order to reduce relative racial differences in adverse borrower and discipline outcomes (or, in the case of discipline, to reduce the proportion racial minorities and other disadvantaged groups make up of persons disciplined). But, while relaxing standards and thereby reducing the frequency of adverse outcomes tends to reduce relative differences in rates of avoiding the outcomes, it tend to increase relative differences in rates of experiencing the outcomes (as well as increase the proportions disadvantaged groups make up of persons experiencing the outcomes). Also, the Department of Justice’s March 2015 report on the racial impact of police and court practices in Ferguson, Missouri was based on the mistaken premise that reducing the frequency of adverse interactions between the police/courts and the population would reduce the proportion African Americans make up of persons experiencing those outcomes. Other investigations of the racial impact of police and court procedures that the DOJ is currently conducting in communities across the country are presumably based on the same mistaken premise.
The perverse consequences of the government’s mistaken view as to the consequence of reducing the frequency of an adverse outcome were the subject of my Amstat News Statistician’s View column three years ago.[4] It is also the subject of a recent letter to the Chief Data Scientist of the Office of Science and Technology Policy, which urged him to cause the government immediately to cease leading the public and entities covered by civil rights laws to believe that reducing the frequency of an outcome tends to reduce relative differences in rates of experiencing the outcome. But only action by an entity like ASA is likely to cause the government (including the Office of Science and Technology Policy) to recognize that so many government policies are based on a statistical belief that is the opposite of reality.
The two subjects of the letter are closely connected, among other reasons, because the government’s mistaken belief that reducing the frequency of an adverse outcome will tend to reduce relative differences in experiencing it seems to be shared by a significant part of the research community. Whatever ASA decides to do regarding these recommendations, I hope that all ASA members will recognize the importance that researchers and policy makers understand the ways measures of differences between outcome rates tend to be affected by the frequency of an outcome.
- “Race and Mortality Revisited,” Society (July/Aug. 2014)http://jpscanlan.com/images/Race_and_Mortality_Revisited.pdf
- “Measuring Health and Healthcare Disparities,” Proceedings of Federal Committee on Statistical Methodology 2013 Research Conference (March 2014)http://jpscanlan.com/images/2013_Fed_Comm_on_Stat_Meth_paper.pdf
- Letter to Agency for Healthcare Research and Quality (July 1, 2015)http://jpscanlan.com/images/Letter_to_Agency_for_Healthcare_Research_and_Quality_July_1,_2015_.pdf
- “Misunderstanding of Statistics Leads to Misguided Law Enforcement Policies,” Amstat News (Dec. 2012)http://magazine.amstat.org/blog/2012/12/01/misguided-law-enforcement/
- Letter to DJ Patil, Chief Data Scientist of the Office of Science and Technology Policy (Sept. 8, 2015)http://jpscanlan.com/images/Letter_to_DJ_Patil,_Chief_Data_Scientist_Sept._8,_2015_.pdf
------------------------------
James Scanlan
James P. Scanlan Attorney At Law
------------------------------