Dear Colleagues:
Mainstream random-effects meta-analysis of clinical trials : Stay tuned for a major paper on the subject now in press. When e-published, I will provide a link to the article.
The upcoming article contains the following:
(1) It refutes Michael Borenstein's claims that the mainstream methods, Inverse Variance Weighting (IVW) (a) does not target the unweighted mean of effect sizes in the population and (b) produces an unbiased estimate of its targeted effect size. Since IVW is the default method in this application in the major software packages, this should be a major concern. After all, meta-analysis is at the apex of most evidence pyramids. (See evidence pyramid - Search ) Borenstein is the president of Biostat Inc., which produces the major Comprehensive Meta-Analysis 4.0 and offers short courses. Red flag warning: IVW risks serious bias and targets the unweighted mean. Borenstein was offered the opportunity to respond but did not respond to the invitation.
Reference: "Common mistakes in Meta-Analysis and How to avoid them," pages 44 (calls mistake) and 26 (his model).
(2) Creates a new target population and target parameter where larger studies have more influence. Its asymptotic estimation theory is distribution-free.
(3) Provides readers with research opportunities. You need not be expert in meta-analysis for you or your students to make significant contributions to science.
(4) Paper has a link to validated and user-friendly Excel software and a Users' Guide
(5) In a small survey of highly cited publications of meta-analysis of clinical trials reports, about 10% of meta-analyses reached unsupportable qualitative conclusions and another about 15% reached unsupportable quantitative conclusions.
It is going to take a concerted effort by the ASA Consulting Community to ensure good statistical practices are used in future meta-analysis reports, as well as to go back and redo all meta-analyses that had major impact on public health policy.
------------------------------
Jonathan Shuster
------------------------------