ASA Connect

When a major discovery is made about statistical practice, it needs to be translated into practical application.  The enclosure is a poster child for this concept, as in two independent ways, it points you to the fact that the current mainstream methods in random-effects meta-analysis of collections of randomized clinical trials (the apex of most evidence pyramids) are not based on sound statistical science. It also provides a distribution-free alternative and user-friendly well-tested software to do the calculations.

Use of the mainstream methods has resulted in adverse public health policy decisions and continued use will likely continue to do this in the future.

My e-mail is shusterj@ufl.edu

Attachment(s)

meta-analysis for forum participants.pdf   459 KB 1 version

Dear distinguished members,

Maybe the attached paper on the pitfall of the conventional meta analyis would be interesting.

https://link.springer.com/article/10.1007/s12561-016-9179-3

Be happy to discuss. Best.

Lee-Jen Wei

p.s. Sorry that i cannot attach the paper due to the copy right issue. 

This is a reply to Dr. Wei's post.

I partly agree with Dr. Wei that not every collection of studies should receive a meta-analysis. But nearly every random-effects meta-analysis of randomized clinical trials of a new intervention is primarily done to assess the average effect size for that intervention.  The questions asked include which treatment is better or which treatment is safer. Random effects allow for a mix of helpful and harmful outcomes.  It is not about showing across the board superiority.

From the link below, read the Inferential framework starting on page 075, lower part of left column. If you believe in overall inferences from randomized clinical trials, where you are assessing the overall balance of good vs. harm, then you believe in the science behind our ratio estimation meta-analysis framework for assessing the overall outcomes in a set of randomized clinical trials.

Heterogeneity is important, but efficacy conclusions apply whether or not substantial heterogeneity is present.  In fact, opting out of a meta-analysis of clinical trials over heterogeneity would contribute to publication bias.

Meta-Analysis 2020: A Dire Alert and a Fix (juniperpublishers.com)

Best wishes,

Jon