Andrew,
Experimental Design is STILL my most favorite concept of statistics. I can not stress enough its importance and like you I fear that the p-value issues have not really addressed the fundamental issues...good experimental design with clear objectives. Doesn't it really come down to managing expectations? Experiments are not supposed to be the Holy Grail, they are supposed to help us weed through the evidence to pick up on things that seems important (correlative, causative) and those that seem to just be random.
------------------------------
Susan Spruill
Statistical Consultant
Original Message:
Sent: 03-31-2016 12:25
From: Andrew Ekstrom
Subject: Following up on the ASA Statement on P-Values and Statistical Significance
I'd be happy if the ASA came out with a statement clarifying the fact Design of Experiments exists. If you look at most of the research done by scientists, they use t-tests and simple ANOVA, not because it's the best, rather because they "know", "You can't change more than one thing at a time during an experiment."
If you do a search using variants of (experimental design using the scientific method), you find hundreds of Web pages dedicated to "You can't change more than one thing at a time. Statistics doesn't allow it." Most of those websites end with .edu!
We all know that is wrong. I see it as a bigger problem than p-values ever could be. Rather than misusing something, most scientists deny it's existance!
------------------------------
Andrew Ekstrom
Statistician, Chemist, HPC Abuser;-)