John, you have hit the nail precisely on its head. My graduate work in statistics was done at Iowa State using a later edition of a Fisher' text. The magic words are 'only,' 'rarely' and 'experiment' in my estimation. The use of p values found once and/or not in controlled experiments is misleading. The lack of replicable results seen in the literature is well documented.
Further isues arise when claims of 'scientific consensus' are made about complex model results as substitutes for data and experimental results. When those models don't predict actual data, the data definition is changed rather than conclude the model is wrong. Fisher is probably rolling over in his grave at the extravagant claims made about global warming.
------------------------------
R. Latta
Executive Director
YTMBA Research & Consulting
------------------------------
Original Message:
Sent: 07-23-2015 20:52
From: John Dawson
Subject: Origins of current p-value discussion
If statistics is the religion we're sold to the masses, then our current p-value practices do not reflect original dogma. Rather, they are a corruption of the teachings of our patron saint:
Personally, the writer prefers to set a low standard of significance at the 5 percent point … A scientific fact should be regarded as experimentally established only if a properly designed experiment rarely fails to give this level of significance
- R. A. Fisher, Statistical Methods for Research Workers, 1926
One instance of p < 0.05 was never intended to be the final word in any investigation.
------------------------------
John Dawson
Assistant Professor
Texas Tech University
------------------------------
Original Message:
Sent: 07-23-2015 19:15
From: Dalton Hance
Subject: Origins of current p-value discussion
Perhaps Statistics only has itself to blame? As John points the p-value has somehow become ubiquitous. It has become the de facto requirement for a discovery or theory to be considered believable. In someways this is a victory for Statistics in that we have convinced almost every scientific field that a pretty theory is not enough; theory must be supported by data and that data must have enough weight to be considered plausible. Of course we know the p-value is only one part of a practice. And that without a rigorous design that controls for confounding variables and spurious correlations, or a clearly defined scope of inference or a standard of ethics that recognizes the dangers in data snooping and multiple comparison, the p-value is meaningless.
A metaphor: it's as if we've sold the world on religion, but the masses missed all that stuff the importance of deep contemplation and spiritual inquiry and only take this message away on Sunday: "It doesn't matter what you do, so long as you ask for forgiveness for your sins."
------------------------------
Dalton Hance
Environmental Statistician
Anchor QEA LLC
------------------------------
Original Message:
Sent: 07-23-2015 14:54
From: John Dawson
Subject: Origins of current p-value discussion
The benefits of p-values accrue from ready, widespread knowledge of a common statistical concept.
The shortcomings of p-values are shortcomings in the communication of more specific information.
As statisticians our job is to be precise but understandable, and this is an art as much as a science:
"We have left undone those things which we ought to have done;
And we have done those things that we ought not to have done"
- vs -
"Type I and II errors were committed"
------------------------------
John Dawson
Assistant Professor
Texas Tech University
------------------------------
Original Message:
Sent: 07-23-2015 14:29
From: Charles Coleman
Subject: Origins of current p-value discussion
I remember a JSM talk in which the speaker spoke about using False Positives and False Negatives to replace the Type I/Type II terminology. In these terms, the p-value is the probability of a False Positive. That's relatively easy for nonstatisticians to grasp.
Chuck Coleman