ASA Connect

 View Only
Expand all | Collapse all

Crooked graph

  • 1.  Crooked graph

    Posted 09-29-2015 19:00

     

    Many of my/our fellow statisticians may have seen the deliberately distorting graph presented at the congressional hearing about Planned Parenthood.  If you haven't seen it yet you can find it at

     

    http://www.msnbc.com/msnbc/congressman-chaffetz-misleading-graph-smear-planned-parenthood

     

    The abuses of axes suggest, among many other egregious falsehoods, that 935,573 < 289,750. (For those who teach statistics, show the graph to your class and ask how many distortions they can identify.)

     

    This is a challenge to our Association:  Do we take a public stand (e.g., letters from our ED, our President, and our Board, to Congress and to major newspapers) or do we sit complacently on our tight little alphas, afraid to commit a Type I error?  Theory tells us:  the more we shrink our alpha, the weaker our power.

     

    George Cobb

    Former ASA VP



  • 2.  RE: Crooked graph

    Posted 09-30-2015 04:08

    Surely professional statisticians should challenge unprofessional use (or abuse) of statistics, just as professional engineers should challenge unprofessional engineering?

    ------------------------------
    Andrew Sharp
    International Air Rail Organisation/TSUG
    ------------------------------




  • 3.  RE: Crooked graph

    Posted 09-30-2015 08:24

    Yes, the graph is misleading.  They have a point, but distort it.  Two side by side bar graphs would show the trend just as well.  In addition, if they were to the same scale they would show the relative volume of each procedure.  The only problem still left is that we are talking about two very different types of procedures.  An abortion is very different from a diagnostic screening.  I am not sure that this is relevant, but I thought I would mention it.


    LOUIS W. GIOKAS

    Student

    M. S. in Applied Statistics

    DePaul University

    Chicago, IL 60614

    Phone: +1-630-596-6019






  • 4.  RE: Crooked graph

    Posted 10-01-2015 10:33

    As Trevor Noah pointed out on last night's Daily Show, the graph also implies that there is a correlation between the number of cancer screenings performed and the number of abortions, showing a relabeled version of the graph suggesting that the number of Best Exotic Marigold Hotel movies is correlated to the decrease in Arctic ice ("Damn you, Dame Judy Dench!").

    http://www.cc.com/full-episodes/h42n5j/the-daily-show-with-trevor-noah-september-30--2015---chris-christie-season-21-ep-21003

    >>Kathy

    ------------------------------
    Katherine Godfrey
    ------------------------------




  • 5.  RE: Crooked graph

    Posted 10-01-2015 15:28

    Regarding the “crooked graph”, maybe it’s me , but I’ve clicked on the link provided by Dr. Cobb twice & I see two videos, one of a congressman talking & another some news interview of a woman arguing pro-choice, but I don’t see this GRAPH everyone keeps referring to, although I’ve clicked numerous points on the videos. Can someone clarify? Thanks.  

    Joseph J. Locascio, Ph.D.,

    Assistant Professor of Neurology,

    Harvard Medical School,

    and Bio-Statistician,
    Memory and Movement Disorders Units, 
    Massachusetts Alzheimer's Disease Research Center,
    Neurology Dept.,
    Massachusetts General Hospital (MGH),
    Warren Building,
    Office # 806,
    55 Fruit Street,
    Boston, Massachusetts 02114
    Phone: (617) 724-7192
    Email: JLocascio@partners.org

    ------------------------------
    Joseph Locascio
    ------------------------------




  • 6.  RE: Crooked graph

    Posted 10-01-2015 18:42

    For more discussion on this graph, see http://andrewgelman.com/2015/10/01/jason-chaffetz-is-the-garo-yepremian-of-the-u-s-house-of-representatives-and-i-dont-mean-that-in-a-good-way/

    ------------------------------
    Martha Smith
    University of Texas
    ------------------------------




  • 7.  RE: Crooked graph

    Posted 09-30-2015 10:52
    Maybe they should have used percentages instead of numbers.... ;-)

    Sandra R. Chandler
    (aka "Prof. Sandee")
    Instructor of Mathematics
    Department of Mathematics
    Kennesaw State University
    275 Kennesaw State University Rd. NW, MD 1102
    Kennesaw, Georgia  USA

    schand17@kennesaw.edu 
    -------
    "Mathematics is the Alphabet with which God has Written the Universe." -- Galileo Galilei (1564-1642)
    "Mathematics is Music for the Mind; Music is Mathematics for the Soul" -- Anonymous
    "Math is Our Friend!" -- Prof. Sandee :-) 







  • 8.  RE: Crooked graph

    Posted 09-30-2015 12:49
    My colleague Michael Lavine rightly points out the need to be careful so as not to make ourselves a target:   If you follow the link in my previous post, you get a full-screen version of the graph, with the four points labeled with numbers.  In calling the graph deceptive, we should acknowledge that the graph includes the numbers, and we do not in any way mean to suggest that ASA thinks that American voters believe that 9 < 2.  Rather, our concern is that presentations of data should not create a misleading impression.

    Although the version of the graph on the web is full-screen and shows the numbers for the four vertices, in the hearing the graph was shown briefly and from a distance, so that the actual numbers were almost impossible to read,and the visual effect of the steeply crossing lines was  what stood out.

    Bottom line:  The graph was constructed and used in a way that is clearly deceptive, but is not dishonest in the strictest sense.

    This bears on the larger issue of when ASA should take a public position.  Although I've argued in the past that we have been too cautious, too reluctant to take a stand, I also think Michael Lavine is right to point out that we need to continue to be cautious.

    George









  • 9.  RE: Crooked graph

    Posted 09-30-2015 17:44

    I agree with Dr. Cobb that statisticians should not shy away from challenging misleading uses of statistics, and many members have mentioned compilations of such.  We should recognize, however, that these misuses occur on both sides of the political spectrum, and that calling out those only those that occur on one side can be misleading in and of itself.

    ------------------------------
    Morris Olitsky
    Statistician
    USDA
    ------------------------------




  • 10.  RE: Crooked graph

    Posted 10-01-2015 03:25

    The joker that made graph has been called out in public forums and on tv. 

    I can think of dozens of larger issues dealing with the use and abuse of statistics in the sciences.

    On the other hand, most, meaning 90%+ of all scientists still believe you can only change one thing at a time during an experiment. For anyone on this forum, try reading through some of the "latest and greatest" articles in "top notch" journals. How often did the authors make errors in their interpretations of their data. As a former chemistry student, I had to read dozens of articles. I still haven't found a journal article from the American Chemical Society where I thought the authors used the correct method of analysis. I've seen plenty of papers where the authors used multiple t-tests or created several simple linear regressions from a data set. These types of issues go unnoticed by the scientists because they don't know any better. (If you ask them, they know what they are doing and don't need your help.)  

    Reproduciblity and Reliability have been issues in academic science for a long time. Yet, I don't hear ANYTHING coming from statisticians and data scientists about how to fix these issues. Publishing an Op-Ed piece in a stats journal is about as useful to the situation as writing it on a bathroom stall wall. The people that need to hear the message never see it. Those that do read the message have heard it all before. Perhaps statisticians and leaders in the ASA should stand up and write Op-Eds for the big journal factories, ACS, Nature, Springer, etc. We know how to fix this mess. We need to let other know!

    To us, Design of Experiments is a branch of statistics that goes back at least 150 years. We used split plots and factorial designs before the invention of the t-test. Yet, talking with most scientists you'd never know they exist. For those of you in academic positions, take a walk through the hallways of the sciences departments. Look at the posters, proudly displayed upon the walls. Ask yourself, "If my student submitted that analysis for that data, what grade would I give them?" I wouldn't go much more than a D+ most of the time. But I, as a student, cannot tell Dr Dimwit they did their analysis wrong. Cuz, I just a student. And as such, it's my job to sit down and get to work. Not to mention, Dr Dimwit and their colleagues have been "doing it that way" for decades.       

    If the ASA is going to take a stand against bad stats, ANSWER THESE OP-EDS!!!!!! Go on Science Friday and discuss how Designed Experiments can beat OFAT methods every time. Write about how to use Plackett Burman Designs and Definitive Screening Designs and how they will increase productivity and decrease the chances of these types of errors! Put those articles in non-stats journals. Explain how a Box-Behnken Design of a Mixture Design can be used in science labs! Do something! Preaching to the choir won't fill your seats.  

    Weak statistical standards implicated in scientific irreproducibility
    http://www.nature.com/news/weak-statistical-standards-implicated-in-scientific-irreproducibility-1.14131

    Animal studies produce many false positives
    http://www.nature.com/news/animal-studies-produce-many-false-positives-1.13385

    Policy: NIH plans to enhance reproducibility
    http://www.nature.com/news/policy-nih-plans-to-enhance-reproducibility-1.14586

    Number crunch
    http://www.nature.com/news/number-crunch-1.14692

    Scientific method: Statistical errors
    http://www.nature.com/news/scientific-method-statistical-errors-1.14700

    Uncertainty on trial
    http://www.nature.com/news/uncertainty-on-trial-1.13868

    ------------------------------
    Andrew Ekstrom
    ------------------------------




  • 11.  RE: Crooked graph

    Posted 10-02-2015 08:19

    No. The joker that made the graph is (to my knowledge) completely unknown. We know which organization made the graph – an organization called "Americans United for Life". We have no way to know whether this organization has any statisticians in their employ. Congressman Chaffetz reused someone's bad work – and is still rightly called out for doing that – but he didn't make the graph.

     

    Others in this thread have mentioned that some professionals like lawyers and doctors can remove people from their discipline for malpractice. The point doesn't translate to statistics. The public doesn't care if we put on a graph "created by a certified statistician, license number XXXX" that someone else could verify. They'll consume the bad product, whether it's endorsed (or openly decried) by ASA or other statisticians. This is the culture we need to find a way to change, but how to make others care?

     

     






  • 12.  RE: Crooked graph

    Posted 10-01-2015 10:10

    Physicians loudly decry malpractice. Skilled trades organizations publicly protest when the government fails to enforce professional standards on the quality of work. The Better Business Bureau publishes the names of businesses with deceptive practices.

    It is incumbent upon us to stand for professional standards. While many people will have different views on practice in general, I trust we can all agree to deplore our science used with a clear and willful intent to deceive in a context where testimony is given.

    This patently false and misleading graph, presented by a member of the US House of Representatives in a congressional hearing, demands a response from the statistical community. For my part, I must wonder if this would constitute perjury, were it not for the fact it was used against a witness by a Congressperson. (N.B: I am not an attorney and have no legal training of any sort.) If an attorney must face legal consequences for lying to a judge, why should not a Congressperson face a contempt citation for lying to a Congressional committee?

    If we deplore professional misconduct, how then shall we declaim it? That is the material question. 


    ------------------------------
    David J Corliss, PhD
    Speaking only for myself and not any company, university or other organization
    ------------------------------




  • 13.  RE: Crooked graph

    Posted 10-01-2015 10:13

    Exercise caution. The actual problem here is that the ordinates have no scale. Were right and left ordinates to have appropriate value levels labeled, the fact that two trends are graphed on the same graph is acceptable -- as long as it is pointed out that each line is drawn on its respective scale.

    ------------------------------
    Donald McMahon
    Director of Statistics & Data Management
    Columbia University Medical Center
    ------------------------------




  • 14.  RE: Crooked graph

    Posted 10-02-2015 12:35

    There is definitely a need, imho, to call out poor statistics used to advanced public policy - regardless of which side of the debate is using the statistics.  Picking examples on an ad hoc basis will inevitably lead to accusations of partisanship.  

    Perhaps there should be something along the lines of annual StIgNoble prizes given to the best examples of bad statistics used over the year in public policy debates.  That way, the partisanship issues could be balanced.  


    ------------------------------
    Timothy Sliter
    Associate Professor
    University of Texas Southwestern Medical Center
    -----------------------------




  • 15.  RE: Crooked graph

    Posted 10-06-2015 12:37


    PolitiFact.  A fact checking site gave this graph and its use by others a 'pants on fire' rating.  

    Chart shown at Planned Parenthood hearing is misleading and 'ethically wrong'

    @politifact remove preview
    Chart shown at Planned Parenthood hearing is misleading and 'ethically wrong'
    Congress may have avoided a government shutdown related to Planned Parenthood funding - at least so far - but the controversy over the women's health organization looks like it's here to stay. Rep. Jason Chaffetz, R-Utah, who chairs the House Oversight and Government Reform Committee, sparred with Planned Parenthood president Cecile Richards during a high-profile hearing on Sept.
    View this on @politifact >


    ------------------------------
    Jeffrey Smith
    U.S. Army Research Laboratory
    ------------------------------




  • 16.  RE: Crooked graph

    Posted 10-12-2015 10:34

    Thanks to all of you who have commented on this topic. Thanks also to George for initiating the conversation and for writing this excellent piece for Stats.org: http://www.stats.org/the-graph-that-launched-a-thousand-news-stories/

    As a reminder on ASA's involvement with Stats.org—one of David Morganstein's presidential initiatives—please see these Amstat News pieces: http://magazine.amstat.org/blog/2015/01/01/asa-stats-org-partner/ and http://magazine.amstat.org/blog/2015/03/01/broadening-our-communication-through-collaboration/

    Best,

    Steve


    ------------------------------
    Steve Pierson
    Director of Science Policy
    American Statistical Association
    ------------------------------