Dear Consulting Section,
I want to highlight the time-sensitive message from Steve Pierson at ASA about the NIH Request for Information on Measuring and Rewarding Scientific Impact for which comments are due August 19, 2026: see his post on ASA Connect.
If you'd like to comment on a general ASA response, please reach out to Steve Pierson by EOB on Friday, August 14 (tomorrow).
I intend to share some of my own comments as they pertain to my own statistical methodology interests (model selection, post-selection inference, reproducibility & transparency), but also as they pertain to the collaborative nature of our work as quantitative team scientists. For example, a few of my drafted comments are copied below. Please let me know if you have any comments you'd like me to share with Steve, or (better yet) comment on this thread with your thoughts.
Thank you,
Ryan
Chair, ASA Statistical Consulting Section
--- drafted comments on AI-enabled quantitative science, as an example ---
- AI-assisted analysis is a selection procedure. An agent that iterates through dozens or hundreds of specifications and surfaces the one that works is performing model selection at a scale no analysis plan documents and that the investigator often cannot reconstruct. The multiplicity is real, consequential, and unrecorded. This is multiplicity at machine speed, and it will show up in the replication record.
-Recommended indicator: an analytic provenance record. Where generative AI materially contributed to analytic code or decisions, require a short log - model and version, date, nature of the contribution, and the number of alternative specifications generated and examined. This is analogous to declaring how many outcomes were measured.
- Human accountability must remain locatable. A named individual should be able to explain and defend every analytic choice. I would encourage an attestation, parallel to existing authorship and conflict attestations, that AI-generated analytic code was reviewed and is understood by a named quantitative scientist.
------------------------------
Ryan Peterson
Associate Professor
University of Iowa
------------------------------