If there is limited space in a table, then the *, **, and *** convention is a way to indicate the approximate size of the p-value.
Reporting the actual p-value is preferable however, often with p < 0.001 for small values.
We should avoid reporting "p = 0.034*", and then in the table footer stating that "* p < 0.05". Do we really need a footnote to state that 0.034 < 0.05? Scientists do this because they want to draw attention to significant p-values. But emphasising results is better done with
bold font or by sorting the table by p-value (if possible).
I think the *, **, *** convention does lead to "threshold thinking" and has a certain Orwellian flavour. Much like Newspeak, this convention diminishes people's range of thought. I hope that someone has the courage to smuggle the following into a table footer or figure caption in a manuscript: * good, ** plus good, *** double plus good
:-)
------------------------------
Stanley E. Lazic, PhD
Team Leader, Statistics
AstraZeneca
Cambridge, UK
https://stanlazic.github.io/------------------------------
Original Message:
Sent: 01-20-2017 02:30
From: Tore Wentzel-Larsen
Subject: reporting of p-values in tables
I have on some occasions received comments from editors or reviewers that p-values in tables should be reported by the asterisk notation. This does not seem to be consistent with ASAs recent statement on p-values (http://www.amstat.org/asa/files/pdfs/P-ValueStatement.pdf), particularly not with third principle ("Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold."). This principle is, I think, equally important in tables as in the main text. In many tables, p-values are stated together with confidence intervals. While the whole confidence distribution is a substitute for a p-value, this is not necessarily the case for a single, often 95%, confidence interval. In the asterisk notation p-values are only stated with a rather course categorization, and I have some problems with being associated with such a style of reporting. In some cases this may be necessary due to lack of space in a table, but this is not always the case. This requirement is in some cases based on reference to the APA style; perhaps ASA could contact the American Psychological Association to discuss this matter.
-----
Tore Wentzel-Larsen
Regional Center for Child and Adolescent Mental Health, Eastern and Southern Norway
Norwegian Centre for violence and traumatic stress studies