ASA Connect

5/5/2026

Dear ASA Members:

Very readable details of my claims against Statistics in Medicine (SIM) about their mismanagement of a peer review I conducted are in the attached blog.   I asked SIM several times in the past month to withdraw the e-publication of Bang-Zhao (and invite a rigorous new version) with a declared firm date of withdrawal, but they declined to do so by my deadline.

I think all ASA members should read about the decision process of the journal editors at SIM with respect to the SIM e-publication by Bang and Zhao linked in the Blog's attachment.  I reviewed the first two versions of the paper.  I was perhaps an overly author-friendly reviewer.  The main element of the paper attempted to provide new methodology for estimation of the incremental cost-effectiveness ratio (ICER) in random-effects meta-analysis of randomized clinical trials. While this is a novel and important topic, invalid methodology was being advanced. I provided a valid approach. The authors' second version retained the invalid methodology and did not use the suggested valid methodology. They did not offer any mathematical justification for the use of their method nor challenge the validity of my suggested methodology.  Rather than rejecting the paper at this point, I recommended giving them another chance.  The third version was essentially the same flawed paper as the first. Rather than recommending rejection, I declined to review version 3.  SIM obtained new reviewers and published this dangerous paper.  Use of the authors' methods can lead to erroneous conclusions. For example,  the authors' Table 1 Tricco data gives possibly conflicting implications of ICER analyses. The authors' analysis tells you that the experimental intervention is inferred to be cost effective, while the more appropriate ratio estimation method gives a 95% confidence interval that cannot exclude an advantage in ICER of 2,000 units (likely clinically significant) for the control treatment. 

This post gets into the importance of editors to respect their unpaid peer-reviewers. Even though I declined to review version 3, they had an ethical obligation to make sure that the authors addressed my many concerns before publishing it.  If you are a peer-reviewer who objects to a new e-publication, I suggest the following protocol to ethically raise your concerns. 

Step 1: Prepare detailed arguments to the editors as to why this paper should be withdrawn.  (Note: unlike retractions, withdrawals are not stains on the authors' reputations.)

Step 2: If the editors make arguments about why they disagree, request a peer-review of your concerns.

Step 3: If you believe the editors failed on both above steps, make your detailed concerns public on  ASA Connect and any other ASA blogs you deem necessary. 

Going to Step 3 must be a last resort as the authors will have to be publicly embarrassed.  In my case, I followed these three steps. The editors were informed about this potential embarrassment, but the potentially dangerous miss-practice by innocent statisticians using Bang-Zhao took higher priority, leading me to step 3.

I know that Professors Bang and Zhao will be disappointed by this blog, but they should understand that I truly went beyond expectations to reach this day. Their SIM editors should have acted differently to properly protect them and champion statistical rigor.

I have great respect for Jonathan Shuster and his work. So when he raises a concern about a paper, it it is one I take seriously. However, can it really be reasonable to expect a journal such as Statistics in Medicine to act on a deadline set by a reviewer?. In general, editors do not have to follow instructions set by reviewers. It is their responsibility to come to a decision using the reviews they are given to guide them. Clearly editors can sometime get it wrong and but so can reviewers. Furthermore, reviewers might disagree with each other. I have had several dozen letters to the editors published pointing out what I consider to be errors in papers. I have never thought it reasonable to expect the journal to withdraw the article concerned. In other words, papers that contain flaws are, always will be and should be part of the scientific record. Nothing would be gained, for example, by expunging Galileo's theory of the tides from the scientific record, even though we regard it as wrong.

Surely, a course of action that would be more useful would be for Jonathan Shuster to write a letter to the editor. I hope that the editors of Statistics in Medicine would be receptive to such letter and certainly can give evidence that they have been in the past. The letter, together with the author's reply might be instructive and valuable.

5/5/2026

Dear ASA Members:

Very readable details of my claims against Statistics in Medicine (SIM) about their mismanagement of a peer review I conducted are in the attached blog.   I asked SIM several times in the past month to withdraw the e-publication of Bang-Zhao (and invite a rigorous new version) with a declared firm date of withdrawal, but they declined to do so by my deadline.

I think all ASA members should read about the decision process of the journal editors at SIM with respect to the SIM e-publication by Bang and Zhao linked in the Blog's attachment.  I reviewed the first two versions of the paper.  I was perhaps an overly author-friendly reviewer.  The main element of the paper attempted to provide new methodology for estimation of the incremental cost-effectiveness ratio (ICER) in random-effects meta-analysis of randomized clinical trials. While this is a novel and important topic, invalid methodology was being advanced. I provided a valid approach. The authors' second version retained the invalid methodology and did not use the suggested valid methodology. They did not offer any mathematical justification for the use of their method nor challenge the validity of my suggested methodology.  Rather than rejecting the paper at this point, I recommended giving them another chance.  The third version was essentially the same flawed paper as the first. Rather than recommending rejection, I declined to review version 3.  SIM obtained new reviewers and published this dangerous paper.  Use of the authors' methods can lead to erroneous conclusions. For example,  the authors' Table 1 Tricco data gives possibly conflicting implications of ICER analyses. The authors' analysis tells you that the experimental intervention is inferred to be cost effective, while the more appropriate ratio estimation method gives a 95% confidence interval that cannot exclude an advantage in ICER of 2,000 units (likely clinically significant) for the control treatment. 

This post gets into the importance of editors to respect their unpaid peer-reviewers. Even though I declined to review version 3, they had an ethical obligation to make sure that the authors addressed my many concerns before publishing it.  If you are a peer-reviewer who objects to a new e-publication, I suggest the following protocol to ethically raise your concerns. 

Step 1: Prepare detailed arguments to the editors as to why this paper should be withdrawn.  (Note: unlike retractions, withdrawals are not stains on the authors' reputations.)

Step 2: If the editors make arguments about why they disagree, request a peer-review of your concerns.

Step 3: If you believe the editors failed on both above steps, make your detailed concerns public on  ASA Connect and any other ASA blogs you deem necessary. 

Going to Step 3 must be a last resort as the authors will have to be publicly embarrassed.  In my case, I followed these three steps. The editors were informed about this potential embarrassment, but the potentially dangerous miss-practice by innocent statisticians using Bang-Zhao took higher priority, leading me to step 3.

I know that Professors Bang and Zhao will be disappointed by this blog, but they should understand that I truly went beyond expectations to reach this day. Their SIM editors should have acted differently to properly protect them and champion statistical rigor.

I have great respect for Jonathan Shuster and his work. So when he raises a concern about a paper, it it is one I take seriously. However, can it really be reasonable to expect a journal such as Statistics in Medicine to act on a deadline set by a reviewer?. In general, editors do not have to follow instructions set by reviewers. It is their responsibility to come to a decision using the reviews they are given to guide them. Clearly editors can sometime get it wrong and but so can reviewers. Furthermore, reviewers might disagree with each other. I have had several dozen letters to the editors published pointing out what I consider to be errors in papers. I have never thought it reasonable to expect the journal to withdraw the article concerned. In other words, papers that contain flaws are, always will be and should be part of the scientific record. Nothing would be gained, for example, by expunging Galileo's theory of the tides from the scientific record, even though we regard it as wrong.

Surely, a course of action that would be more useful would be for Jonathan Shuster to write a letter to the editor. I hope that the editors of Statistics in Medicine would be receptive to such letter and certainly can give evidence that they have been in the past. The letter, together with the author's reply might be instructive and valuable.

5/5/2026

Dear ASA Members:

Very readable details of my claims against Statistics in Medicine (SIM) about their mismanagement of a peer review I conducted are in the attached blog.   I asked SIM several times in the past month to withdraw the e-publication of Bang-Zhao (and invite a rigorous new version) with a declared firm date of withdrawal, but they declined to do so by my deadline.

I think all ASA members should read about the decision process of the journal editors at SIM with respect to the SIM e-publication by Bang and Zhao linked in the Blog's attachment.  I reviewed the first two versions of the paper.  I was perhaps an overly author-friendly reviewer.  The main element of the paper attempted to provide new methodology for estimation of the incremental cost-effectiveness ratio (ICER) in random-effects meta-analysis of randomized clinical trials. While this is a novel and important topic, invalid methodology was being advanced. I provided a valid approach. The authors' second version retained the invalid methodology and did not use the suggested valid methodology. They did not offer any mathematical justification for the use of their method nor challenge the validity of my suggested methodology.  Rather than rejecting the paper at this point, I recommended giving them another chance.  The third version was essentially the same flawed paper as the first. Rather than recommending rejection, I declined to review version 3.  SIM obtained new reviewers and published this dangerous paper.  Use of the authors' methods can lead to erroneous conclusions. For example,  the authors' Table 1 Tricco data gives possibly conflicting implications of ICER analyses. The authors' analysis tells you that the experimental intervention is inferred to be cost effective, while the more appropriate ratio estimation method gives a 95% confidence interval that cannot exclude an advantage in ICER of 2,000 units (likely clinically significant) for the control treatment. 

This post gets into the importance of editors to respect their unpaid peer-reviewers. Even though I declined to review version 3, they had an ethical obligation to make sure that the authors addressed my many concerns before publishing it.  If you are a peer-reviewer who objects to a new e-publication, I suggest the following protocol to ethically raise your concerns. 

Step 1: Prepare detailed arguments to the editors as to why this paper should be withdrawn.  (Note: unlike retractions, withdrawals are not stains on the authors' reputations.)

Step 2: If the editors make arguments about why they disagree, request a peer-review of your concerns.

Step 3: If you believe the editors failed on both above steps, make your detailed concerns public on  ASA Connect and any other ASA blogs you deem necessary. 

Going to Step 3 must be a last resort as the authors will have to be publicly embarrassed.  In my case, I followed these three steps. The editors were informed about this potential embarrassment, but the potentially dangerous miss-practice by innocent statisticians using Bang-Zhao took higher priority, leading me to step 3.

I know that Professors Bang and Zhao will be disappointed by this blog, but they should understand that I truly went beyond expectations to reach this day. Their SIM editors should have acted differently to properly protect them and champion statistical rigor.

5/5/2026

Dear ASA Members:

Very readable details of my claims against Statistics in Medicine (SIM) about their mismanagement of a peer review I conducted are in the attached blog.   I asked SIM several times in the past month to withdraw the e-publication of Bang-Zhao (and invite a rigorous new version) with a declared firm date of withdrawal, but they declined to do so by my deadline.

I think all ASA members should read about the decision process of the journal editors at SIM with respect to the SIM e-publication by Bang and Zhao linked in the Blog's attachment.  I reviewed the first two versions of the paper.  I was perhaps an overly author-friendly reviewer.  The main element of the paper attempted to provide new methodology for estimation of the incremental cost-effectiveness ratio (ICER) in random-effects meta-analysis of randomized clinical trials. While this is a novel and important topic, invalid methodology was being advanced. I provided a valid approach. The authors' second version retained the invalid methodology and did not use the suggested valid methodology. They did not offer any mathematical justification for the use of their method nor challenge the validity of my suggested methodology.  Rather than rejecting the paper at this point, I recommended giving them another chance.  The third version was essentially the same flawed paper as the first. Rather than recommending rejection, I declined to review version 3.  SIM obtained new reviewers and published this dangerous paper.  Use of the authors' methods can lead to erroneous conclusions. For example,  the authors' Table 1 Tricco data gives possibly conflicting implications of ICER analyses. The authors' analysis tells you that the experimental intervention is inferred to be cost effective, while the more appropriate ratio estimation method gives a 95% confidence interval that cannot exclude an advantage in ICER of 2,000 units (likely clinically significant) for the control treatment. 

This post gets into the importance of editors to respect their unpaid peer-reviewers. Even though I declined to review version 3, they had an ethical obligation to make sure that the authors addressed my many concerns before publishing it.  If you are a peer-reviewer who objects to a new e-publication, I suggest the following protocol to ethically raise your concerns. 

Step 1: Prepare detailed arguments to the editors as to why this paper should be withdrawn.  (Note: unlike retractions, withdrawals are not stains on the authors' reputations.)

Step 2: If the editors make arguments about why they disagree, request a peer-review of your concerns.

Step 3: If you believe the editors failed on both above steps, make your detailed concerns public on  ASA Connect and any other ASA blogs you deem necessary. 

Going to Step 3 must be a last resort as the authors will have to be publicly embarrassed.  In my case, I followed these three steps. The editors were informed about this potential embarrassment, but the potentially dangerous miss-practice by innocent statisticians using Bang-Zhao took higher priority, leading me to step 3.

I know that Professors Bang and Zhao will be disappointed by this blog, but they should understand that I truly went beyond expectations to reach this day. Their SIM editors should have acted differently to properly protect them and champion statistical rigor.