I think the most common misconception by far is regarding statements about data (whether observed or hypothetical) as if they were statements about the population.
This is the basis of the common mistake known as the "transposition of conditioning," in which P(D | H) is taken to be, or to imply something about, P(H | D), where
H = some hypothesis &
D = data or datum.
For instance, once an experiment has been run, a p-value (p) is the probability of results (Y), in an imaginary repetition of the experiment, at least as extreme as those (x) actually observed, assuming the tested hypothesis H:
P(Y>x | H).
As you can see, this is a probability about data, given a hypothesis.
However, numerous psychological experiments, sampling from both college students & college faculty, have shown that the most common inference made about H, based on p, is that p = P(H | x), which is a probability about a hypothesis, given data. See
Oakes, M. (1986). Statistical inference: A commentary for the social and behavioral sciences. Chichester, UK: Wiley.
Davidoff (1999). Standing statistics right side up. Annals of Internal Medicine (130), 12, 1019-1021.
Another common misconception of the same kind is known as the "confidence trick," in which the estimand, theta, is taken to lie within a given X% confidence interval with X% probability (a statement about a hypothesis given data). This is despite the facts that
- the "X%" portion of the term "X% confidence interval" refers only to the pre-experimental probability of containing theta (again, a statement about the data)
- the frequentist paradigm has no conception of the probability that a given CI contains an estimand
Foster (2014). Confidence Trick: The Interpretation of Confidence Intervals. Canadian Journal of Science, Mathematics and Technology Education, Volume 14, Issue 1.
I could say much more about this, as I have strong feelings concerning the reasons it's so pervasive, but I'd better stop there & assess your interest. -------------------------------------------
Andrew Hartley
Associate Statistical Science Director
PPD, Inc.
-------------------------------------------
Original Message:
Sent: 11-10-2014 09:53
From: Elaine Eisenbeisz
Subject: Statistical Misconceptions
What common statistical misconceptions do you come across?
-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------