In recent months I have posted here a number of letters to institutions or organizations regarding implications of failure to understand the ways standard measures of differences between outcome rates tend to be affected by the frequency of an outcome. One of the more recent is an October 8, 2015 letter to American Statistical Association itself.[1] Among other things, those letters discuss the near universal failure to understand the pattern whereby the rarer an outcome the greater tends to be the relative difference in experiencing and the smaller tends to be the relative difference in avoiding it. One implication of that failure is that the government encourages lenders and public schools to reduce adverse borrower and discipline outcomes while mistakenly believing that doing will tend to reduce, rather than increase, relative differences in adverse borrower and discipline outcomes (something I discussed in the Amstat News column listed as reference 2) which is why the October 8 letter urges ASA to explain to the government that the government’s understanding is the opposite of reality. See Section B of the letter (at 36-40).
One manifestation of the above-described pattern of relative differences is that relative differences in adverse outcomes tend to be larger, while relative differences in the corresponding favorable outcomes tend to be smaller, in populations/settings where the adverse outcomes are comparatively rare – e.g., persons with high education or high income, the young, British civil servants, highly-qualified job applicants, the countries of Norway and Sweden or the states of Minnesota and Massachusetts – than in populations/settings where the outcomes are comparatively common. See the ASA letter at 9 and the reference 3, which discusses the issues with regard to Norway, Sweden. and Minnesota, but in which Massachusetts could as well be substituted for Minnesota.
Item 4 is a letter sent today to the Boston Lawyers’ Committee for Civil Rights and Economic Justice discussing the failure of the organization’s 2014 study of demographic differences in discipline rates in Massachusetts and the nation to recognize that relative differences in discipline rates will tend to be larger in Massachusetts than nationally simply because discipline rates are so much lower in Massachusetts than nationally. The study is part of the Lawyers' Committee’s ongoing efforts to cause general reductions in suspensions and expulsions based on the belief that doing so will tend to reduce relative demographic differences in discipline rates. But the type of analysis of differences in discipline rates by the Boston Lawyers' Committee is no different from that one could expect from any like organization, any government agency, and most universities. Item 5 discusses a putative class action lawsuit brought in 1995 by the Washington, DC Lawyers' Committee based on a study that ranked lenders on the basis of the size of relative differences in mortgage rejection rates without understanding that lenders with lenient lending criteria would tend to show larger relative differences in mortgage rejection rates than other lenders. The understanding of these issues today is little different from what it was in 1995.
- Letter to American Statistical Association (Oct. 8, 2015)http://jpscanlan.com/images/Letter_to_American_Statistical_Association_Oct._8,_2015_.pdf
- “Misunderstanding of Statistics Leads to Misguided Law Enforcement Policies,” Amstat News (Dec. 2012)http://magazine.amstat.org/blog/2012/12/01/misguided-law-enforcement/
- “It’s easy to misunderstand gaps and mistake good fortune for a crisis,” Minneapolis StarTribune (Feb. 8, 2014)http://www.startribune.com/opinion/commentaries/244080771.html
- Letter to Boston Lawyers’ Committee for Civil Rights and Economic Justice (Nov. 12, 2015)http://jpscanlan.com/images/Letter_to_Boston_Lawyers_Committee_Nov._12,_2015_.pdf
- “When Statistics Lie,” Legal Times (Jan. 1, 1996)http://jpscanlan.com/images/When_Statistics_Lie.pdf
------------------------------
James Scanlan
James P. Scanlan Attorney At Law
------------------------------