[1/9/17 Update: the document is now final and is updated at http://ww2.amstat.org/misc/StatisticalIssuesInProposals.pdf.]
The ASA Committee on Funded Research (CFR) has been actively working to increase engagement of statisticians in NIH research. With the committee’s urging and guidance, an ASA group—Marie Davidian (NCSU), Liz Stuart (JHU), and Steve Pierson (ASA)—met with senior NIH Center for Scientific Review (CSR) staff in July to discuss how statisticians could be helpful to CSR in its work, particularly as it relates to NIH reproducible research efforts. CSR was very receptive to the offer and this September 2016 Peer Review Notes article on the top statistical issues seen in proposals is the first product of such a meeting. It was because of this meeting we learned of a NIH CSR Feedback Survey revealing the top concern of reviewers was “Having more experienced reviewers especially those with statistical, biostatistical, or clinical expertise.”
To help provide content for the September 2016 Peer Review Notes article, our committee reached out to ASA membership for input on what statistical issues they see in non-statistics proposals. We’re very grateful for the terrific response from ASA membership!
We have synthesized and summarized the input to create a document titled, “Statistical Issues Seen in Non-Statistics Proposals”; this document shares similar content with the NIH Peer Review Notes article but is intended to provide more details and further recommendations. We view this draft as version 1.0. We’re asking the statistical community for two things by October 31: (i) your comments on this document; (ii) short (one-paragraph) descriptions (i.e., case studies) of actual problems (with non-identifying specifics if possible) you have seen in proposals. The purpose of the case studies is to better illustrate statistical issues we’ve identified for a non-statistics audience. We’ll then review the comments and case studies to create a final version of the document (version 2.0).
I’d welcome your comments on the document in the comment space below or in this Google document (where only comments are allowed but not edits.) Case studies can be emailed to Steve Pierson (pierson@amstat.org). Comments can also be sent to him.
Thank you in advance for your help to strengthen this document which we hope will help non-statistics grant applicants strengthen their proposals, ultimately leading to better science.