5/5/2026
Dear ASA Members:
Very readable details of my claims against Statistics in Medicine (SIM) about their mismanagement of a peer review I conducted are in the attached blog. I asked SIM several times in the past month to withdraw the e-publication of Bang-Zhao (and invite a rigorous new version) with a declared firm date of withdrawal, but they declined to do so by my deadline.
I think all ASA members should read about the decision process of the journal editors at SIM with respect to the SIM e-publication by Bang and Zhao linked in the Blog's attachment. I reviewed the first two versions of the paper. I was perhaps an overly author-friendly reviewer. The main element of the paper attempted to provide new methodology for estimation of the incremental cost-effectiveness ratio (ICER) in random-effects meta-analysis of randomized clinical trials. While this is a novel and important topic, invalid methodology was being advanced. I provided a valid approach. The authors' second version retained the invalid methodology and did not use the suggested valid methodology. They did not offer any mathematical justification for the use of their method nor challenge the validity of my suggested methodology. Rather than rejecting the paper at this point, I recommended giving them another chance. The third version was essentially the same flawed paper as the first. Rather than recommending rejection, I declined to review version 3. SIM obtained new reviewers and published this dangerous paper. Use of the authors' methods can lead to erroneous conclusions. For example, the authors' Table 1 Tricco data gives possibly conflicting implications of ICER analyses. The authors' analysis tells you that the experimental intervention is inferred to be cost effective, while the more appropriate ratio estimation method gives a 95% confidence interval that cannot exclude an advantage in ICER of 2,000 units (likely clinically significant) for the control treatment.
This post gets into the importance of editors to respect their unpaid peer-reviewers. Even though I declined to review version 3, they had an ethical obligation to make sure that the authors addressed my many concerns before publishing it. If you are a peer-reviewer who objects to a new e-publication, I suggest the following protocol to ethically raise your concerns.
Step 1: Prepare detailed arguments to the editors as to why this paper should be withdrawn. (Note: unlike retractions, withdrawals are not stains on the authors' reputations.)
Step 2: If the editors make arguments about why they disagree, request a peer-review of your concerns.
Step 3: If you believe the editors failed on both above steps, make your detailed concerns public on ASA Connect and any other ASA blogs you deem necessary.
Going to Step 3 must be a last resort as the authors will have to be publicly embarrassed. In my case, I followed these three steps. The editors were informed about this potential embarrassment, but the potentially dangerous miss-practice by innocent statisticians using Bang-Zhao took higher priority, leading me to step 3.
I know that Professors Bang and Zhao will be disappointed by this blog, but they should understand that I truly went beyond expectations to reach this day. Their SIM editors should have acted differently to properly protect them and champion statistical rigor.
------------------------------
Jonathan Shuster
------------------------------