Discussion: View Thread

Georgette's mention of "statistically significant" calls another point to mind:  whenever possible I use the term "statistically detectable" in place of "statistically significant."  It conveys what the hypothesis test outcome means:  the data indicate the nonzero effect is not spurious.  And nothing more than that.

However, when communicating outside of our organization, I feel compelled to say "statistically significant" because it is the convention, and deviating from convention in communications can lead to confusion.  If the statistics community organizes to address issues of research irreproducibility, is there any chance we could also push for a replacement for "statistically significant?"

I also will not be attending JSM.

------------------------------
James Garrett
Sr. Assoc. Dir. of Biostatistics
Novartis
------------------------------


Original Message:
Sent: 07-16-2015 13:11
From: Georgette Asherman
Subject: Origins of current p-value discussion

Throughout my education as a scientist and as a statistician, I've never heard the phase "statistically significant", not once.  It was not until I started working in a biology lab that even heard the phase, "statistically significant". It comes from two parts; one is from the column labeled “significance” that is to the right of the p-value in the most outputs of statistical software, and the other comes from the need to distinguish, “yes the null hypothesis can be reject” (statistically significance) from, “yes this difference actually matters for the organism (or population)” which is called biological significance. I was taught to, and much prefer to talk about degrees of evidence. Using phases like, “little to no evidence”, “weak evidence”, and “strong evidence”. I will be attending JSM this year and I am hoping I can participate in the round table.

------------------------------
Joe Swintek
Statistician
Badger Technical Services
------------------------------


Original Message:
Sent: 07-16-2015 14:18
From: James Garrett
Subject: Origins of current p-value discussion

Georgette's mention of "statistically significant" calls another point to mind:  whenever possible I use the term "statistically detectable" in place of "statistically significant."  It conveys what the hypothesis test outcome means:  the data indicate the nonzero effect is not spurious.  And nothing more than that.

However, when communicating outside of our organization, I feel compelled to say "statistically significant" because it is the convention, and deviating from convention in communications can lead to confusion.  If the statistics community organizes to address issues of research irreproducibility, is there any chance we could also push for a replacement for "statistically significant?"

I also will not be attending JSM.

------------------------------
James Garrett
Sr. Assoc. Dir. of Biostatistics
Novartis
------------------------------


Original Message:
Sent: 07-16-2015 13:11
From: Georgette Asherman
Subject: Origins of current p-value discussion

Joe,

I like to discuss whether or not results are "clinically meaningful" or "biologically relevant".  I'm with you on the overuse of "significant". But it does not solve the problem of non-statisticians misusing p-values or their desire to have a singular number that tells them they have something publishable.  I also like to ask my scientific colleagues if they can tell a compelling story around their findings.  This usually make them do more literature research to answer the questions "is this finding clinically relevant?"

------------------------------
Susan Spruill
Statistical Consultant
------------------------------


Original Message:
Sent: 07-16-2015 17:59
From: Joe Swintek
Subject: Origins of current p-value discussion

Throughout my education as a scientist and as a statistician, I've never heard the phase "statistically significant", not once.  It was not until I started working in a biology lab that even heard the phase, "statistically significant". It comes from two parts; one is from the column labeled “significance” that is to the right of the p-value in the most outputs of statistical software, and the other comes from the need to distinguish, “yes the null hypothesis can be reject” (statistically significance) from, “yes this difference actually matters for the organism (or population)” which is called biological significance. I was taught to, and much prefer to talk about degrees of evidence. Using phases like, “little to no evidence”, “weak evidence”, and “strong evidence”. I will be attending JSM this year and I am hoping I can participate in the round table.

------------------------------
Joe Swintek
Statistician
Badger Technical Services
------------------------------


Original Message:
Sent: 07-16-2015 14:18
From: James Garrett
Subject: Origins of current p-value discussion

Georgette's mention of "statistically significant" calls another point to mind:  whenever possible I use the term "statistically detectable" in place of "statistically significant."  It conveys what the hypothesis test outcome means:  the data indicate the nonzero effect is not spurious.  And nothing more than that.

However, when communicating outside of our organization, I feel compelled to say "statistically significant" because it is the convention, and deviating from convention in communications can lead to confusion.  If the statistics community organizes to address issues of research irreproducibility, is there any chance we could also push for a replacement for "statistically significant?"

I also will not be attending JSM.

------------------------------
James Garrett
Sr. Assoc. Dir. of Biostatistics
Novartis
------------------------------


Original Message:
Sent: 07-16-2015 13:11
From: Georgette Asherman
Subject: Origins of current p-value discussion

First off, the problems with p-values long predate Gelman - I like Gelman a lot, but this goes back at least to Meehl - see this article and probably before.

Second, are we sure that banning p values is bad?  My favorite professor in graduate school (Herman Friedman) used to say "Stop p-ing on the research!" There may be cases where  p = XXXX provides useful information, but they are rare.

------------------------------
Peter Flom
------------------------------


Original Message:
Sent: 07-17-2015 11:44
From: Susan Spruill
Subject: Origins of current p-value discussion

Joe,

I like to discuss whether or not results are "clinically meaningful" or "biologically relevant".  I'm with you on the overuse of "significant". But it does not solve the problem of non-statisticians misusing p-values or their desire to have a singular number that tells them they have something publishable.  I also like to ask my scientific colleagues if they can tell a compelling story around their findings.  This usually make them do more literature research to answer the questions "is this finding clinically relevant?"

------------------------------
Susan Spruill
Statistical Consultant
------------------------------


Original Message:
Sent: 07-16-2015 17:59
From: Joe Swintek
Subject: Origins of current p-value discussion

Throughout my education as a scientist and as a statistician, I've never heard the phase "statistically significant", not once.  It was not until I started working in a biology lab that even heard the phase, "statistically significant". It comes from two parts; one is from the column labeled “significance” that is to the right of the p-value in the most outputs of statistical software, and the other comes from the need to distinguish, “yes the null hypothesis can be reject” (statistically significance) from, “yes this difference actually matters for the organism (or population)” which is called biological significance. I was taught to, and much prefer to talk about degrees of evidence. Using phases like, “little to no evidence”, “weak evidence”, and “strong evidence”. I will be attending JSM this year and I am hoping I can participate in the round table.

------------------------------
Joe Swintek
Statistician
Badger Technical Services
------------------------------


Original Message:
Sent: 07-16-2015 14:18
From: James Garrett
Subject: Origins of current p-value discussion

Georgette's mention of "statistically significant" calls another point to mind:  whenever possible I use the term "statistically detectable" in place of "statistically significant."  It conveys what the hypothesis test outcome means:  the data indicate the nonzero effect is not spurious.  And nothing more than that.

However, when communicating outside of our organization, I feel compelled to say "statistically significant" because it is the convention, and deviating from convention in communications can lead to confusion.  If the statistics community organizes to address issues of research irreproducibility, is there any chance we could also push for a replacement for "statistically significant?"

I also will not be attending JSM.

------------------------------
James Garrett
Sr. Assoc. Dir. of Biostatistics
Novartis
------------------------------


Original Message:
Sent: 07-16-2015 13:11
From: Georgette Asherman
Subject: Origins of current p-value discussion

The benefits of p-values accrue from ready, widespread knowledge of a common statistical concept.

The shortcomings of p-values are shortcomings in the communication of more specific information.

As statisticians our job is to be precise but understandable, and this is an art as much as a science:

"We have left undone those things which we ought to have done;

And we have done those things that we ought not to have done"

- vs -

"Type I and II errors were committed"

------------------------------
John Dawson
Assistant Professor
Texas Tech University
------------------------------


Original Message:
Sent: 07-23-2015 14:29
From: Charles Coleman
Subject: Origins of current p-value discussion

Perhaps Statistics only has itself to blame? As John points the p-value has somehow become ubiquitous. It has become the de facto requirement for a discovery or theory to be considered believable. In someways this is a victory for Statistics in that we have convinced almost every scientific field that a pretty theory is not enough; theory must be supported by data and that data must have enough weight to be considered plausible. Of course we know the p-value is only one part of a practice. And that without a rigorous design that controls for confounding variables and spurious correlations, or a clearly defined scope of inference or a standard of ethics that recognizes the dangers in data snooping and multiple comparison, the p-value is meaningless.

A metaphor: it's as if we've sold the world on religion, but the masses missed all that stuff the importance of deep contemplation and spiritual inquiry and only take this message away on Sunday: "It doesn't matter what you do, so long as you ask for forgiveness for your sins."

------------------------------
Dalton Hance
Environmental Statistician

Anchor QEA LLC
------------------------------


Original Message:
Sent: 07-23-2015 14:54
From: John Dawson
Subject: Origins of current p-value discussion

The benefits of p-values accrue from ready, widespread knowledge of a common statistical concept.

The shortcomings of p-values are shortcomings in the communication of more specific information.

As statisticians our job is to be precise but understandable, and this is an art as much as a science:

"We have left undone those things which we ought to have done;

And we have done those things that we ought not to have done"

- vs -

"Type I and II errors were committed"

------------------------------
John Dawson
Assistant Professor
Texas Tech University
------------------------------


Original Message:
Sent: 07-23-2015 14:29
From: Charles Coleman
Subject: Origins of current p-value discussion

If statistics is the religion we're sold to the masses, then our current p-value practices do not reflect original dogma. Rather, they are a corruption of the teachings of our patron saint:

Personally, the writer prefers to set a low standard of significance at the 5 percent point … A scientific fact should be regarded as experimentally established only if a properly designed experiment rarely fails to give this level of significance
- R. A. Fisher, Statistical Methods for Research Workers, 1926

One instance of p < 0.05 was never intended to be the final word in any investigation.

------------------------------
John Dawson
Assistant Professor
Texas Tech University
------------------------------


Original Message:
Sent: 07-23-2015 19:15
From: Dalton Hance
Subject: Origins of current p-value discussion

Perhaps Statistics only has itself to blame? As John points the p-value has somehow become ubiquitous. It has become the de facto requirement for a discovery or theory to be considered believable. In someways this is a victory for Statistics in that we have convinced almost every scientific field that a pretty theory is not enough; theory must be supported by data and that data must have enough weight to be considered plausible. Of course we know the p-value is only one part of a practice. And that without a rigorous design that controls for confounding variables and spurious correlations, or a clearly defined scope of inference or a standard of ethics that recognizes the dangers in data snooping and multiple comparison, the p-value is meaningless.

A metaphor: it's as if we've sold the world on religion, but the masses missed all that stuff the importance of deep contemplation and spiritual inquiry and only take this message away on Sunday: "It doesn't matter what you do, so long as you ask for forgiveness for your sins."

------------------------------
Dalton Hance
Environmental Statistician

Anchor QEA LLC
------------------------------


Original Message:
Sent: 07-23-2015 14:54
From: John Dawson
Subject: Origins of current p-value discussion

The benefits of p-values accrue from ready, widespread knowledge of a common statistical concept.

The shortcomings of p-values are shortcomings in the communication of more specific information.

As statisticians our job is to be precise but understandable, and this is an art as much as a science:

"We have left undone those things which we ought to have done;

And we have done those things that we ought not to have done"

- vs -

"Type I and II errors were committed"

------------------------------
John Dawson
Assistant Professor
Texas Tech University
------------------------------


Original Message:
Sent: 07-23-2015 14:29
From: Charles Coleman
Subject: Origins of current p-value discussion

If statistics is the religion we're sold to the masses, then our current p-value practices do not reflect original dogma. Rather, they are a corruption of the teachings of our patron saint:

Personally, the writer prefers to set a low standard of significance at the 5 percent point … A scientific fact should be regarded as experimentally established only if a properly designed experiment rarely fails to give this level of significance
- R. A. Fisher, Statistical Methods for Research Workers, 1926

One instance of p < 0.05 was never intended to be the final word in any investigation.

------------------------------
John Dawson
Assistant Professor
Texas Tech University
------------------------------

Perhaps Statistics only has itself to blame? As John points the p-value has somehow become ubiquitous. It has become the de facto requirement for a discovery or theory to be considered believable. In someways this is a victory for Statistics in that we have convinced almost every scientific field that a pretty theory is not enough; theory must be supported by data and that data must have enough weight to be considered plausible. Of course we know the p-value is only one part of a practice. And that without a rigorous design that controls for confounding variables and spurious correlations, or a clearly defined scope of inference or a standard of ethics that recognizes the dangers in data snooping and multiple comparison, the p-value is meaningless.

A metaphor: it's as if we've sold the world on religion, but the masses missed all that stuff the importance of deep contemplation and spiritual inquiry and only take this message away on Sunday: "It doesn't matter what you do, so long as you ask for forgiveness for your sins."

------------------------------
Dalton Hance
Environmental Statistician

Anchor QEA LLC
------------------------------

The theme of JSM 2016 is “The Extraordinary Power of Statistics.”

Can this current discussion about the use and abuse of the p-value be a session "With great power comes great responsibility" ;-)  I know Spiderman said this, but he can't have been the first to say it, or the last.

------------------------------
Jean Recta
Mathematical Statistician
FDA/Center for Veterinary Medicine
------------------------------


Original Message:
Sent: 07-23-2015 19:15
From: Dalton Hance
Subject: Origins of current p-value discussion

Perhaps Statistics only has itself to blame? As John points the p-value has somehow become ubiquitous. It has become the de facto requirement for a discovery or theory to be considered believable. In someways this is a victory for Statistics in that we have convinced almost every scientific field that a pretty theory is not enough; theory must be supported by data and that data must have enough weight to be considered plausible. Of course we know the p-value is only one part of a practice. And that without a rigorous design that controls for confounding variables and spurious correlations, or a clearly defined scope of inference or a standard of ethics that recognizes the dangers in data snooping and multiple comparison, the p-value is meaningless.

A metaphor: it's as if we've sold the world on religion, but the masses missed all that stuff the importance of deep contemplation and spiritual inquiry and only take this message away on Sunday: "It doesn't matter what you do, so long as you ask for forgiveness for your sins."

------------------------------
Dalton Hance
Environmental Statistician

Anchor QEA LLC
------------------------------

It wouldn't be a bad idea to finally establish a consistent p-value convention. Half the time. statisticians announce a low p-value denotes a high probability of a real effect; half the time, they announce a high p-value to denote a high probability of a real effect. We really need to stop using the same term to denote p and 1-p when we discuss p-values.

------------------------------
Nelson Lipshutz
Regulatory Research Corp.
------------------------------


Original Message:
Sent: 07-23-2015 14:29
From: Charles Coleman
Subject: Origins of current p-value discussion

I have never seen anyone use a high p-value to denote a high probability of a real effect.  Not once.

If anyone actually does that, it would, of course, lead to huge confusion.

------------------------------
Peter Flom
------------------------------


Original Message:
Sent: 07-23-2015 18:04
From: Nelson Lipshutz
Subject: Origins of current p-value discussion

It wouldn't be a bad idea to finally establish a consistent p-value convention. Half the time. statisticians announce a low p-value denotes a high probability of a real effect; half the time, they announce a high p-value to denote a high probability of a real effect. We really need to stop using the same term to denote p and 1-p when we discuss p-values.

------------------------------
Nelson Lipshutz
Regulatory Research Corp.
------------------------------


Original Message:
Sent: 07-23-2015 14:29
From: Charles Coleman
Subject: Origins of current p-value discussion

Sent: 07-23-2015 18:04
From: Nelson Lipshutz

It wouldn't be a bad idea to finally establish a consistent p-value convention. Half the time. statisticians announce a low p-value denotes a high probability of a real effect; half the time, they announce a high p-value to denote a high probability of a real effect. We really need to stop using the same term to denote p and 1-p when we discuss p-values.


Sent: 07-23-2015 14:29
From: Charles Coleman