ASA Connect

 View Only

  • 1.  How to talk about religion without offending your students

    Posted 05-05-2022 15:01
    I attended an interested forum "Indoctrination vs. Education: A Conversation on the Appropriate Role of the Higher Ed Professor" which was very interesting, but there were so many questions and comments that they never got around to answering my question. So I thought I'd pose it here because it relates how we discuss controversial topics in Statistics.

    "I have a question about how to properly discuss examples of research that involve religious beliefs. More specifically, there were a series of research studies around the start of the millenium where patients in a hospital were randomly assigned to either to receive prayers from a stranger or to a control group. These studies raise interesting questions about informed consent (most of these prayer studies did not require informed consent) and scientific plausibility (your perspective about what is plausible depends on whether you are an atheist or not). They even raise questions about one-sided versus two-sided hypotheses (should you entertain the possibility that prayer could be harmful instead of helpful). I want to raise these issues without seeming like I am criticizing people based on their particular theology. What are some of the pitfalls that I need to look out for when I talk about these studies?"

    I'm interested in what you all think.

    ------------------------------
    Stephen Simon, blog.pmean.com
    Independent Statistical Consultant
    ------------------------------


  • 2.  RE: How to talk about religion without offending your students

    Posted 05-06-2022 07:25

    Stephen Simon raises a number of important issues regarding statistical methodology applied to controversial subjects. I would like to suggest a process for addressing questions such as these. He speaks of religion but the same process could be consider4ed for any controversial or highly politicized issue such as climate change or the outcome of certain elections in the United States. 

    First, we need to eliminate unscientific studies. Stephen Simon raises a number of serious problems in studies on religion he has seen. The first thing we much do is eliminate all work that is not conducted in a scientific manner and following accepted best practices. The so-called studies mentioned in the original post should never be mentioned at all not addressed in any way except to demonstrate them as pseudoscience. The problems Simon raises about these studies means they are not reproducible and therefore have no place in scientific inquiry. 

    The next step is to address remaining reports (if any) in a scientific manner using statistical best practices, We're all scientists here, so I don't need to say any more, except to recall the ASA holds an annual Conference on Statistical Practice dedicated to exactly what statistical best practices actually are. It is worth mentioning that studies on effects of religion on the physical world are, by definition, supernatural and therefore not reproducible by human agency. These are not and cannot be science! Inherently irreproducible questions cannot be even considered in a scientific context! This principle is known as Non-Overlapping Magisteria. (Full disclosure: I am myself a Lutheran minister as well as a scientist, as likely to be found behind a lectern in a lecture hall as pulpit. This makes me very familiar with the boundaries inherent to each. Putative religious effects on the physical world are by definition supernatural and therefore cannot be reproduced by human agency. This makes it literally impossible to address these questions in a scientific context.)

    There is an additional consideration I would like to raise. I gained this perspective from judging science fairs, many with my local ASA chapter. It's important to participate in judging science fairs for a number of reasons...one of the most important is that is gives us a valuable window into the perception of our profession by the general public. One common and very damaging misconception is that student are often told a hypothesis is something the scientist intends to "prove" and the outset of the investigation. This reflects how the general pub lic  believe (and, tragically, are often taught to believe) that scientific work consists of confirming a priori conclusions. We can repeat Andrew Lang's joke about the person who "uses statistics as a drunken man uses lampposts-for support rather than illumination" - but it's a lot less funny when realize this is what the general public really things we do most of the time. All of us need to worked together to drive a stake through the heart of this monster that never seems to die. 

    One last thing. Many people have a vested interest in pseudoscience, When a person refuses to accept carefully vetted scientific evidence - whether it be about, say, evolution or a round earth or human-caused climate change or the outcome of thoroughly validated elections results or some other question - once a person declares data and evidence mean nothing to them, just walk away. Demonstrate their irrationality once and then leave. Nothing will be gained from arguing people who have a vested interest in being wrong and we have more productive avenues for our efforts. 

    To sum up, my suggested process for addressing controversial questions including putative effects of divine intervention has four steps:

    1. Get rid of any pseudoscience

    2. Treat the remainder (if any) using statistical best practices

    3. Remember the general public often misunderstands what we do. We often need to explain what accepted statistical best practices really are.  

    4. Don't feed the trolls. 



    ------------------------------
    David J Corliss, PhD
    Director, Peace-Work www.peace-work.org
    davidjcorliss@peace-work.org
    ------------------------------

    Original Message


  • 3.  RE: How to talk about religion without offending your students

    Posted 05-06-2022 11:10

    I think I'd better comment here. 

    I think I'd better start with saying that the way you phrase the question is problematic. Compare the question "How can I study the natives without getting them all riled up?" with the question "How can I study indigenous peoples ethically and respectfully?" The first suggests one is dealing with easily offended irrational sub-beings that one has to mollify in order to get what one wants out of them. The second suggests one is dealing with human beings like oneself that one has an internal ethical obligation to treat with respect. 


    Second, if an intervention is worth studying, it is worth treating as having ethical content, as having ontological reality for ethical purposes. That means that, like any intervention, ethical study requiress informed consent. Indeed ethical equipoise requires this. If one thinks informed consent isn't required, that logically implies one is certain there is no effect. And it is scientifically unethical to waste resources conducting a study when one is certain of the outcome in advance. Moreover, a level of certainty sufficent to assure informed consent is not ethically required would also be sufficient to unconsciously bias the investigators. 


    Of course informed consent requires people to know they are being studied, and this risks Hawthorne and placebo effects. But Hawthorne and placebo effects are simply realities of conducting a study ethically. They have to be addresed by things like randomization and blinding. Just because we are scientists doesn't give us power or right to impose our will on others without their consent. And people aren't mere subjects that we are merely concerned with mollifying, with not offending and not riling up so they won't interfere with our doing what we want with them. They are human beings who deserve to be treated, that we ethically ought to want to treat, with respect. 



    ------------------------------
    Jonathan Siegel
    Director Clinical Statistics
    ------------------------------

    Original Message


  • 4.  RE: How to talk about religion without offending your students

    Posted 05-08-2022 13:51

    Stephen, thank you for a provocative and relevant question. Here are some thoughts:

    A statistician could simply to decline to get involved, because there's too much risk of misinterpretation or misrepresentation. Imagine headlines like "Statistician proves prayer works!" or "Statistician denounces believers!"

    If you do get involved you might have occasion to point out that, like the evangelical Christian, Francis Collins, who headed the National Institutes of Health for over ten years, a statistician, whether or not a religious believer, can be faithful to his or her role as scientist.

    Experimentation seeks to discover or confirm factual truths, but it has other uses. Pharma companies run clinical trials to establish efficacy and safety, but also to generate buzz among prescribers about a new product (seeding trial, Ref. 1). Which is the principal motive of the experimenters: to understand reality or to promote beliefs?

    The statistician should make explicit the principles of the scientific method that guide planning a new study or interpreting completed studies. Doing this in writing will help protect the statistician, even if it doesn't stop others from publishing misinformation about the study, if the beliefs turn out not to be scientifically supported. Some principles to consider are:

    "Put the hypothesis in jeopardy." (2) Those hypothesizing that prayer improves health outcomes have a burden to prove the case. The initial assumption of statistical hypothesis testing is that their hypothesis is false, and sufficient data can overturn this assumption. Starting with the 'null hypothesis' in this way has been the norm for at least a hundred years. It is not because statisticians are biased against prayer but because we are trained to produce credible findings.

    Another principle is that scientific results require replication. Should a first study favor prayer, a second study usually would be needed to confirm. "Usually" but not always, as in a clinical study that stops prematurely because of overwhelming efficacy. In this case there is very strong evidence in favor of the alternative hypothesis and, importantly, a procedure established before the study starts on how to assess the evidence during the study (as well as at the end of the study). Replication and strength of evidence are things to consider in planning a prayer study or in retrospective analysis.

    Of course, other principles apply, like blinding and control groups.

    The experimenters may ask whether a written statement about principles is applied to all studies. I think a frank 'no' is an appropriate response. It would be unusual to have it for a benign topic, say a comparative study of car seat upholstery. Religion has always been a sensitive topic, and now it's politically charged, even an excuse for violent reaction. A written statement serves to intellectually and professionally protect the proponents of prayer who organized a study, the statisticians and others involved.

    If study results are misleadingly reported, the statisticians should be free to counter. Among other things, non-disclosure agreements must either be avoided or written explicitly to allow counter-reporting by statisticians.

    Lots more for me or others to say, but I'll stop here.

    References

    1. https://en.wikipedia.org/wiki/Seeding_trial
    2. I think George Box said this but I couldn't find the quote. In his ASA Presidential Address, he does mention jeopardy in this context. See JASA, March 1979.


    ------------------------------
    Dick Bittman
    ------------------------------

    Original Message


  • 5.  RE: How to talk about religion without offending your students

    Posted 05-10-2022 18:44

    I recognize in my previous reply I was focusing too much on my concern about lack of informed consent, which I understand was not the intended main topic of the post, 

    I honestly don't think statisticsl methods are going to be especially useful to addressing these sorts of questions, and it might be helpful to talk about why. think it might be useful to talk about strictly secular topics where it's clear assumptions behind standard statistical methods break down. This can occur in various ways when subjects that are actually elements of an interacting system are erroneously assumed to be independent. 


    Consider an experiment assessing whether voting affects the outcome of elections. A seemingly straightforward way to do that (ar least in a thought experiment) would be to obtain some standard social science experiment level number of subjects, say 100, find out their preferred candidate, randomize them 1:1, and get half of them to not vote or even switch their vote. (It's a thought experiment, so let's imagine we can verify they did this.) Is voting more likely to result in ones candidate winming than not voting? Will switching make a difference? If we're dealing with an election with a large voter base and the election isn't a cliffhanger election, it won't. The experiment will show that voting, and gor that matter switching votes, made no difference at all to the outcome. But should we conclude that voting has no efficacy, that who one votes for and whether one votes at all doesn't matter?

    The problem here is that votes don't operate individually, they operate as a system. All votes in excess of the ones needed to create the majority (or plurality) can be in some sense regarded as extraneous. Any sample obtained in an election absent a cliffhanger margin will consist entirely of extraneous votes. Dropping or switching them won't make any difference. But we nonetheless cannot make the completely natural-seeming inference from the sample to the population and conclude the entire population's votes are all extraneous, and hence voting in general makes no difference and nobody's vote affects the election outcome. The reason is that votes in an election are not independent. The outcome of each voter's vote depends on how other people voted. This dependence defeats the independence assumption underlying the inference. 


    After discussing these sorts of examples, it might be useful to discuss their relevance to sorts of assumptions relgious traditions tend to make. If one sees the world as an interdependent system, simple inference becomes harder. 


    W. Edwards Deming was a religious man, and it was perhaps because he was a religious man that he thought keenly about thr implications of interdependent systems. His work focused a great deal on the follies of attempting to apply independent-subjects inference methods to highly dependent context. And this outlook might also have influenced the way he applied the conceptual pragmatism approach of CI Lewis to problems of statistical inference. tradtook the implications of conceptual pragmatism, that knowledge occurs with respect to a purpose, to perhaps its logical extent. He argued that we can never construct a true random number generator. Not only can we never know if any finite sequence is absolutely random or not, but nothing in the finite macro world humans deal with is ever "truly" random. All we can determine is whether it's random enough for some particular purpose or to some degree of probability. One correlary of that was his well-known skepticism of the use of goodness of fit tests. Draw a large enough sample size, and you can always get a goodness of fitness test to fail. Perfect distributions no more exist than planets with perfectly ellipical orbits. The debate over p-values continues to raise some of these issues. 


    But the interesting thing about Deming, perhaps the useful thing for this purpose, is that he managed to be both a deeply religuous man and a highly skeptical one, someone who perhaps sometimes thought of his peers as simpletons who clung to simple assumptions because they found them easier to work with (or aesthetically pleasing). Perhaps this tendency is a highly universal human trait that everybody, if we're going to be honest about it, shares and expresses in some aspect of their life or other. Scientific progress has tended to lead to seeing the world as much more complex than we thought it was, and has made problems of inference much harder. The last couple of decades has required questioning the validity of convenient and aesthetically pleasing assumptions, and the process is accelerating. Statistics as a field has sometimes had entrenched traditions that have had (and are having) to be overcome to make progress. The interesting thing about Deming was his association of awareness of compexity and interdependence, and his criticism and skepticism of inherented scientific traditions, with a religious mindset. It was perhaps his religious mindset that made interdependent systems conceptually easier, despite the math being much harder if not intractable.


    And in some respects, perhaps his acceptance of personal religious traditions and assumptions made his skepticism of traditions and assumptions in his scientific and professional work come easier. 



    ------------------------------
    Jonathan Siegel
    Director Clinical Statistics
    ------------------------------

    Original Message


Related Content