ASA Connect

 View Only
Back to discussions
Expand all | Collapse all

Call for Prospective Journal Reviewers

  • 1.  Call for Prospective Journal Reviewers

    Posted 04-09-2018 09:32
    ASA Community,

    Through a number of different connections - ranging from face-to-face discussion with personal collaborators all the way to "people that I met on Twitter" - I have frequently seen poor methodological quality or lack of statistical rigor excused/explained by journal editors stating that they cannot find enough qualified statistical reviewers to have a dedicated statistical review for each paper submitted to their journal.  On our side of the fence, many statistical trainees and faculty seem initially surprised to learn that not every medical paper has a dedicated statistical review - in fact, beneath the highest tier of journals (NEJM, JAMA, Lancet-level journals) my observation suggests that the majority of journals almost NEVER have a dedicated statistical review as part of the process, save for a few that have chosen to emphasize methodological quality over flashy results.

    It is my belief that this occurs in part because journal editors simply don't know where to find us.  I have volunteered the idea that reaching out "on our turf" - in the ASA forums - would be a productive place for journals that claim they cannot find statistical reviewers to, well, find some!

    If you have relevant experience in clinical-outcomes research and would be interested in serving as a statistical reviewer for medical journals, please feel free to contact me and let me know about your interests & analytic expertise, and I can put you in touch with some journal editors who tell me that they can't find enough statistical reviewers!

    I would also offer this disclaimer: while it would be nice if we all had the expertise of Frank Harrell, even junior faculty should get involved.  So many papers get published with absolutely outrageous things that go unchallenged (things that an attentive student should immediately notice as "awry" even if they can't fully explain why).  The journals need high-quality people to serve as editors, but they also need even more of us in the rank-and-file to provide the mass-quantity, basic-level statistical review that would catch the ridiculous.  My two cents, anyway.

    Thanks!

    ------------------------------
    Andrew D. Althouse, PhD
    Supervisor of Statistical Projects
    UPMC Heart & Vascular Institute
    Presbyterian Hospital, Office C701
    Phone: 412-802-6811
    Email: althousead@upmc.edu
    Twitter: @ADAlthousePhD
    ------------------------------


  • 2.  RE: Call for Prospective Journal Reviewers

    Posted 04-10-2018 16:27
    Hi Andrew,
    I think this is a good idea. I serve on the editorial board for the Journal of Clinical Oncology, and find it very rewarding. I would be willing to be on the list with one caveat (that you may already be addressing)...
    Over the last few years I have found myself bombarded with requests to be on the editorial board for various (usually brand new) open access journals. I said yes a few times and greatly regretted it. Specifically, it became very clear very quickly that I was only asked to be on the editorial board because they needed people to submit articles and pay their exorbitant fees. I was on one editorial board for about 2 years and never once had an article review, but was given expectations on how often I should submit articles and asked individually about 50 times to submit an article at a discounted price (which was still ridiculously high). 
    I would therefore suggest that someone (like you) who is the central contact person have some process for when names are or are not volunteered. As long as names were only sent to reputable journals, I think this could be a useful mechanism.
    thanks, Doug

    ------------------------------
    Douglas Landsittel
    Professor of Biomedical Informatics
    University of Pittsburgh
    ------------------------------



  • 3.  RE: Call for Prospective Journal Reviewers

    Posted 04-11-2018 14:36

    Doug,

     

    Thanks for chiming in!  This is an excellent point and something that perhaps I took for granted.  I have avoided the parasitic open-access journals at all costs (not that all "open access" journals are evil, but many amount to nothing more than moneymaking scams with no intention of having true "peer review" – this has been written about elsewhere, the occasional example of a person getting a complete nonsense article published as part of a sting operation to highlight that some journals clearly do not actually "review" articles), and would suggest that all of us do the same. 

     

    Those of you newbies out there that are in graduate school or perhaps have recently started your first job: if the journal is brand new, doesn't really seem to put out "issues" but simply posts articles online every once in a while, and sends you generic form-emails that say "Dear Dr. XXXXXX, we recently read your work "Bingo Bango Bongo Zippity Doo Dah" and would be highly interested in you submitting an article to our esteemed journal for publication in the May issue..." – just run away.  Quickly.  Do not reply, do not submit, do not engage with them.  Just delete the email.

     

    As for the current discussion – I was not trying to 'control' this process at all, I was merely thinking that I would volunteer to connect journal editors that I already know or who contact me through a reputable source (i.e. the "friend of a friend") that have said they are having difficulty finding statistical reviewers with statisticians that have expressed interest in reviewing who may not otherwise get that opportunity (I have received at least fifteen responses so far – thanks so much everyone!)

     

    I would hope to avoid the situation that you describe by never really engaging with the parasitic journals that do this sort of thing.  I'm not connecting anyone unless I can verify the bona fides of both parties (editor and statistician).  But this is a good general reminder for all of us to be on the lookout for suspicious invitations to "join the editorial board" that are really just ways to try and drive business into their journal.

     

    Thanks!

     

    Andrew D. Althouse, PhD

    Supervisor of Statistical Projects

    UPMC Heart & Vascular Institute

    Presbyterian Hospital, Office C701

    Phone: 412-802-6811

    Email: althousead@upmc.edu

    Twitter: @ADAlthousePhD

    Website: Faculty Profile

    Website: HVI-CBC Profile

     

     

     

     



    ------Original Message------

    Hi Andrew,
    I think this is a good idea. I serve on the editorial board for the Journal of Clinical Oncology, and find it very rewarding. I would be willing to be on the list with one caveat (that you may already be addressing)...
    Over the last few years I have found myself bombarded with requests to be on the editorial board for various (usually brand new) open access journals. I said yes a few times and greatly regretted it. Specifically, it became very clear very quickly that I was only asked to be on the editorial board because they needed people to submit articles and pay their exorbitant fees. I was on one editorial board for about 2 years and never once had an article review, but was given expectations on how often I should submit articles and asked individually about 50 times to submit an article at a discounted price (which was still ridiculously high). 
    I would therefore suggest that someone (like you) who is the central contact person have some process for when names are or are not volunteered. As long as names were only sent to reputable journals, I think this could be a useful mechanism.
    thanks, Doug

    ------------------------------
    Douglas Landsittel
    Professor of Biomedical Informatics
    University of Pittsburgh
    ------------------------------


  • 4.  RE: Call for Prospective Journal Reviewers

    Posted 04-12-2018 13:26

    Beall's List of (Potentially) Predatory Publishers can be of some use in trying to discern bad actors in the Open Access game:

    https://beallslist.weebly.com/

    But there has been controversy over it (perhaps mostly generated by the bad actors).

    Best,

    Steve

     

     



    ------Original Message------

    Doug,

     

    Thanks for chiming in!  This is an excellent point and something that perhaps I took for granted.  I have avoided the parasitic open-access journals at all costs (not that all "open access" journals are evil, but many amount to nothing more than moneymaking scams with no intention of having true "peer review" – this has been written about elsewhere, the occasional example of a person getting a complete nonsense article published as part of a sting operation to highlight that some journals clearly do not actually "review" articles), and would suggest that all of us do the same. 

     

    Those of you newbies out there that are in graduate school or perhaps have recently started your first job: if the journal is brand new, doesn't really seem to put out "issues" but simply posts articles online every once in a while, and sends you generic form-emails that say "Dear Dr. XXXXXX, we recently read your work "Bingo Bango Bongo Zippity Doo Dah" and would be highly interested in you submitting an article to our esteemed journal for publication in the May issue..." – just run away.  Quickly.  Do not reply, do not submit, do not engage with them.  Just delete the email.

     

    As for the current discussion – I was not trying to 'control' this process at all, I was merely thinking that I would volunteer to connect journal editors that I already know or who contact me through a reputable source (i.e. the "friend of a friend") that have said they are having difficulty finding statistical reviewers with statisticians that have expressed interest in reviewing who may not otherwise get that opportunity (I have received at least fifteen responses so far – thanks so much everyone!)

     

    I would hope to avoid the situation that you describe by never really engaging with the parasitic journals that do this sort of thing.  I'm not connecting anyone unless I can verify the bona fides of both parties (editor and statistician).  But this is a good general reminder for all of us to be on the lookout for suspicious invitations to "join the editorial board" that are really just ways to try and drive business into their journal.

     

    Thanks!

     

    Andrew D. Althouse, PhD

    Supervisor of Statistical Projects

    UPMC Heart & Vascular Institute

    Presbyterian Hospital, Office C701

    Phone: 412-802-6811

    Email: althousead@upmc.edu

    Twitter: @ADAlthousePhD

    Website: Faculty Profile

    Website: HVI-CBC Profile

     

     

     

     





  • 5.  RE: Call for Prospective Journal Reviewers

    Posted 04-12-2018 17:01
    I have now run into the following situation.
    I sent what I thought was a good paper <g class="gr_ gr_67 gr-alert gr_gramm gr_inline_cards gr_run_anim Grammar multiReplace" id="67" data-gr-id="67">to</g> a mainline journal. It was rejected. Well, that happens. BUT unless I objected, they would forward my paper to their very own open access journal, $3000. Maybe they thought this guy can pay, let's see if he will.
    Free enterprise in action. Mainline journals are learning.

    ------------------------------
    Sidney Young
    Retired
    ------------------------------

    Original Message


  • 6.  RE: Call for Prospective Journal Reviewers

    Posted 04-11-2018 15:07

    Andrew – I am a statistical reviewer for 2 veterinary journals and 1 sports statistics journal and would be willing to review for other journals.   I applaud your initiative to have statisticians serve as reviewers and enjoy my role very much.  Please keep in touch if I can be of any assistance.  Thanks.



    ------Original Message------

    Hi Andrew,
    I think this is a good idea. I serve on the editorial board for the Journal of Clinical Oncology, and find it very rewarding. I would be willing to be on the list with one caveat (that you may already be addressing)...
    Over the last few years I have found myself bombarded with requests to be on the editorial board for various (usually brand new) open access journals. I said yes a few times and greatly regretted it. Specifically, it became very clear very quickly that I was only asked to be on the editorial board because they needed people to submit articles and pay their exorbitant fees. I was on one editorial board for about 2 years and never once had an article review, but was given expectations on how often I should submit articles and asked individually about 50 times to submit an article at a discounted price (which was still ridiculously high). 
    I would therefore suggest that someone (like you) who is the central contact person have some process for when names are or are not volunteered. As long as names were only sent to reputable journals, I think this could be a useful mechanism.
    thanks, Doug

    ------------------------------
    Douglas Landsittel
    Professor of Biomedical Informatics
    University of Pittsburgh
    ------------------------------


  • 7.  RE: Call for Prospective Journal Reviewers

    Posted 04-11-2018 17:48
    I serve several clinical, toxicological, and other journals as a biostatistician for article reviews.  I have worked in those areas for almost 50 years and published a handful of articles as well besides teaching biostatistics in several universities, both here and abroad.  I would be happy to review such articles.  Thanks.

    Ajit K. Thakur, Ph.D.
    Retired Statistician


    ------Original Message------

    Hi Andrew,
    I think this is a good idea. I serve on the editorial board for the Journal of Clinical Oncology, and find it very rewarding. I would be willing to be on the list with one caveat (that you may already be addressing)...
    Over the last few years I have found myself bombarded with requests to be on the editorial board for various (usually brand new) open access journals. I said yes a few times and greatly regretted it. Specifically, it became very clear very quickly that I was only asked to be on the editorial board because they needed people to submit articles and pay their exorbitant fees. I was on one editorial board for about 2 years and never once had an article review, but was given expectations on how often I should submit articles and asked individually about 50 times to submit an article at a discounted price (which was still ridiculously high). 
    I would therefore suggest that someone (like you) who is the central contact person have some process for when names are or are not volunteered. As long as names were only sent to reputable journals, I think this could be a useful mechanism.
    thanks, Doug

    ------------------------------
    Douglas Landsittel
    Professor of Biomedical Informatics
    University of Pittsburgh
    ------------------------------


  • 8.  RE: Call for Prospective Journal Reviewers

    Posted 04-11-2018 17:46
    I joined the Editorial Board last year of a new journal, JCO Clinical Cancer Informatics, which is training up a contingent of new young reviewers through a very interesting process of formal mentoring.

    Experienced reviewers mentor 3 early career reviewers. We will work as a team on 3  manuscripts this year, with each mentee having the opportunity to submit our group work to the Editor in Chief. With n of 1 article completed, I consider the proactive process an enriching concept, and recommend it to others.

    ------------------------------
    Judy-Anne Chapman, P.Stat., PStat (ASA)
    Senior Biostatistician,
    Canadian Cancer Trials Group,
    Queen's University (retired)
    ------------------------------



  • 9.  RE: Call for Prospective Journal Reviewers

    Posted 04-12-2018 09:38
    ​Andrew,

    I think this is a great idea. Why don't you approach the ASA about working with you on this?

    -Jesse

    Jesse D. Troy, PhD, MPH
    Assistant Professor of Pediatrics
    Duke University School of Medicine
    DUMC Box 3850
    Durham, NC 27705
    919-668-2932 (office)
    919-668-7161 (fax)



    ------------------------------
    Jesse Troy
    Assistant Professor
    Duke University
    ------------------------------



  • 10.  RE: Call for Prospective Journal Reviewers

    Posted 04-13-2018 08:48
    ​Here is an idea.  When I was a graduate student, the editor of the American Economic Review, hired me as a mathematical proof reader.   Since the AER had a submission fee, there is money in it for me.  I wonder if medical journals could do likewise; that is, hire statistics graduate students to review the statistics in submitted paper.

    ------------------------------
    Phillip Kott
    RTI International
    ------------------------------

    Original Message


  • 11.  RE: Call for Prospective Journal Reviewers

    Posted 04-12-2018 10:04
    Sadly, most if not all journals and articles need such a reviewer. 

    What types of statistical malfeasance would one look for? If an article uses multiple simple linear regressions instead of one multiple linear regression, would that qualify? More importantly, would the entire statistical community back that up? If some one uses many ANOVA vs a MANOVA does that raise to the same level of malfeasance? 

    If we use the assumption, "What grade would I give my student if they turned this in?" Is anything below an A- not acceptable or do we let them get away with a C? And again, would the statistical community agree? 

    Keep in mind that something that is commonplace in one area of stats is seen as totally misguided by others. Just think about how stats software codes variables in a designed experiment. You subtract the mean from every value in the column and scale it -1 to 1. This gets rid of a lot of the correlation between factors when you use interaction terms and quadratic terms. If I tell a biostatisticIan about that idea, or even just subtracting the mean, most think it is heresy. But, the idea of "centering" or what ever other disiplines is becoming more popular. 

    Before we decide something is right, wrong or questionable, we need to understand what is appropriate and what is possible. 

    If you have any contacts that want help with chemical and  chemical analysis related articles, I'd be glad to help. 



    ------------------------------
    Andrew Ekstrom

    Statistician, Chemist, HPC Abuser;-)
    ------------------------------



  • 12.  RE: Call for Prospective Journal Reviewers

    Posted 04-13-2018 08:02
    Regarding needs for scientific reviewers for additional medical, statistical, and general science journals that have manuscripts in draft form, it definitely benefits organizations such as ASA to recommend persons with previous experience as a reviewer for a peeri reviewed publication.  Since ASA already maintains a database and member directory it makes sense to collaborate with the major publishing houses who are seeking quality reviewers. I agree with the reader who stated that the main importance in reciewer qualifications is someone who can read, write, and think with a critical and analytical mindset.  Most of us have enough statistics training in our arsenal and some have backgrounds in pharmaceuticals or biological sciences. What seems to make a differenceis those people who are sensitive to foreign (non U.S. based investigations) and some cultural and grammatical differences expressed within some ver excellent manuscripts. It is important to do your homework!

    ------------------------------
    Julie Tackett
    ------------------------------

    Original Message


  • 13.  RE: Call for Prospective Journal Reviewers

    Posted 04-12-2018 14:42

    It is not just clinical trials and medical research manuscripts that need statistical review at journals. You can name any field:  wildlife biology, forestry, hydrology, etc.  Many of us already perform reviews for such applied journals (and the scientists we work with) but certainly there are many articles published without any statistical review.

    Is it possible that something more formal could be created and managed by ASA to provide a list of such reviewers?

    I don't think that extensive knowledge of the applied subject matter is always necessary to find a good statistical reviewer. I claim that we statisticians are generally well-versed in "scientific logic" and even in a subject matter for which we are not intimately familiar we can still determine if the objectives can be addressed with the experimental design, whether the data is adequate (either in kind or in amount), and if the conclusions follow from the results of the analysis.  We can also provide constructive suggestions to improve the manuscript.  To state the obvious:  we provide great benefits.  (It's just that few outside of our field seem to know that.)

    I don't even think that a complicated analysis would be the trigger point to require a statistician to review a manuscript. The worst scientific offenses are made with means and simple linear regressions.

    Anyway…providing such a service would help in our defense of our discipline from everyone who can download R and claim they can perform complete and appropriate statistical analyses.



    ------------------------------
    Jim Baldwin
    Station Statistician
    USDA-Forest Service
    ------------------------------



  • 14.  RE: Call for Prospective Journal Reviewers

    Posted 04-12-2018 15:54
    Jim: that is a fantastic point, and in no way to I intend to suggest that only clinical trials or medical research requires statistical review - that just happens to be my area, and in professional day-to-day chatter with folks involved in various journals, I have periodically gotten this sigh-and-complaint that they would love to have dedicated statistical review for their journals, but they just can't find the people to do so.

    I wholly agree that one does not have to be a dedicated applied statistician in a specific area to comment intelligently on whether the analytic approach was sound, but I can foresee some possible hesitancy on the part of journal editors to admit a reviewer that has never worked in that area (even if, as you surmised, we are well-versed in "scientific logic" that applies widely).

    If there is interest from ASA leadership in making this something that we, the organization, will do - create a roster of people that have agreed to be contacted in the capacity of "Statistical Reviewer" for journals, with some sort of gatekeeper or steward to make sure that the queries are serious - I am happy to be involved in some capacity.

    ------------------------------
    Andrew D. Althouse, PhD
    Supervisor of Statistical Projects
    UPMC Heart & Vascular Institute
    Presbyterian Hospital, Office C701
    Phone: 412-802-6811
    Email: althousead@upmc.edu
    Twitter: @ADAlthousePhD
    ------------------------------

    Original Message


  • 15.  RE: Call for Prospective Journal Reviewers

    Posted 04-13-2018 15:14
    I would like to echo Jim's comments, but also add that another problem with poor statistical review is that even when the correct method is used, it is often improperly cited, explained, and/or justified. This spreads false information about methods, leads to difficulty replicating research, doesn't facilitate criticism, and leaves the work of statisticians and other developers of methodology undercited. Let's face it, if the method was originally published in a statistics journal, that's probably not the citation the scientist will try to use, if at all.

    I would be happy to have my name added to a list of statistical reviewers for journals covering ecology, forestry, entomology, and marine and wildlife biology.

    ------------------------------
    Nels G. Johnson
    Mathematical Statistician
    USDA Forest Service
    Pacific Southwest Research Station
    ------------------------------

    Original Message


  • 16.  RE: Call for Prospective Journal Reviewers

    Posted 04-16-2018 16:42
    Andrew -

    Perhaps Stephen Pierson, Director of Science Policy for ASA,
    pierson@amstat.org,
    might be a good contact for having a list of possible reviewers established.

    Cheers - Jim

    ------------------------------
    James Knaub
    Lead Mathematical Statistician
    Retired
    ------------------------------

    Original Message


Related Content