Jim: that is a fantastic point, and in no way to I intend to suggest that only clinical trials or medical research requires statistical review - that just happens to be my area, and in professional day-to-day chatter with folks involved in various journals, I have periodically gotten this sigh-and-complaint that they would love to have dedicated statistical review for their journals, but they just can't find the people to do so.
I wholly agree that one does not have to be a dedicated applied statistician in a specific area to comment intelligently on whether the analytic approach was sound, but I can foresee some possible hesitancy on the part of journal editors to admit a reviewer that has never worked in that area (even if, as you surmised, we are well-versed in "scientific logic" that applies widely).
If there is interest from ASA leadership in making this something that we, the organization, will do - create a roster of people that have agreed to be contacted in the capacity of "Statistical Reviewer" for journals, with some sort of gatekeeper or steward to make sure that the queries are serious - I am happy to be involved in some capacity.
------------------------------
Andrew D. Althouse, PhD
Supervisor of Statistical Projects
UPMC Heart & Vascular Institute
Presbyterian Hospital, Office C701
Phone: 412-802-6811
Email:
althousead@upmc.eduTwitter: @ADAlthousePhD
------------------------------
Original Message:
Sent: 04-12-2018 14:41
From: James Baldwin
Subject: Call for Prospective Journal Reviewers
It is not just clinical trials and medical research manuscripts that need statistical review at journals. You can name any field: wildlife biology, forestry, hydrology, etc. Many of us already perform reviews for such applied journals (and the scientists we work with) but certainly there are many articles published without any statistical review.
Is it possible that something more formal could be created and managed by ASA to provide a list of such reviewers?
I don't think that extensive knowledge of the applied subject matter is always necessary to find a good statistical reviewer. I claim that we statisticians are generally well-versed in "scientific logic" and even in a subject matter for which we are not intimately familiar we can still determine if the objectives can be addressed with the experimental design, whether the data is adequate (either in kind or in amount), and if the conclusions follow from the results of the analysis. We can also provide constructive suggestions to improve the manuscript. To state the obvious: we provide great benefits. (It's just that few outside of our field seem to know that.)
I don't even think that a complicated analysis would be the trigger point to require a statistician to review a manuscript. The worst scientific offenses are made with means and simple linear regressions.
Anyway…providing such a service would help in our defense of our discipline from everyone who can download R and claim they can perform complete and appropriate statistical analyses.
------------------------------
Jim Baldwin
Station Statistician
USDA-Forest Service
Original Message:
Sent: 04-09-2018 09:31
From: Andrew Althouse
Subject: Call for Prospective Journal Reviewers
ASA Community,
Through a number of different connections - ranging from face-to-face discussion with personal collaborators all the way to "people that I met on Twitter" - I have frequently seen poor methodological quality or lack of statistical rigor excused/explained by journal editors stating that they cannot find enough qualified statistical reviewers to have a dedicated statistical review for each paper submitted to their journal. On our side of the fence, many statistical trainees and faculty seem initially surprised to learn that not every medical paper has a dedicated statistical review - in fact, beneath the highest tier of journals (NEJM, JAMA, Lancet-level journals) my observation suggests that the majority of journals almost NEVER have a dedicated statistical review as part of the process, save for a few that have chosen to emphasize methodological quality over flashy results.
It is my belief that this occurs in part because journal editors simply don't know where to find us. I have volunteered the idea that reaching out "on our turf" - in the ASA forums - would be a productive place for journals that claim they cannot find statistical reviewers to, well, find some!
If you have relevant experience in clinical-outcomes research and would be interested in serving as a statistical reviewer for medical journals, please feel free to contact me and let me know about your interests & analytic expertise, and I can put you in touch with some journal editors who tell me that they can't find enough statistical reviewers!
I would also offer this disclaimer: while it would be nice if we all had the expertise of Frank Harrell, even junior faculty should get involved. So many papers get published with absolutely outrageous things that go unchallenged (things that an attentive student should immediately notice as "awry" even if they can't fully explain why). The journals need high-quality people to serve as editors, but they also need even more of us in the rank-and-file to provide the mass-quantity, basic-level statistical review that would catch the ridiculous. My two cents, anyway.
Thanks!
------------------------------
Andrew D. Althouse, PhD
Supervisor of Statistical Projects
UPMC Heart & Vascular Institute
Presbyterian Hospital, Office C701
Phone: 412-802-6811
Email: althousead@upmc.edu
Twitter: @ADAlthousePhD
------------------------------