Discussion: View Thread

This message has been cross posted to the following eGroups: Biopharmaceutical Section and Statistical Consulting Section .
-------------------------------------------


-------------------------------------------
Morgan Stewart
-------------------------------------------
I have a client who is planning a non-inferiority trial of a product for which the primary efficacy endpoint is binary. The comparator product has very high efficacy (close to 100%, with lower 95% CI in the mid nineties), and regulators are requiring the trial to test the new product for non-inferiority vs. this specific comparator. Because of the high bar and the binary endpoint, usual methods of calculating sample size yield an impractically large n. Has anyone dealt with this kind of problem and are there any work-arounds to permit a smaller sample size with adequate power? Using a higher level of measurement is not possible in this case.

If you can reveal them, A few more details about your study may be helpful. Have any other similar products been developed that use non-inferiority?

The FDA has a readable, detailed,  guidance on their website about non-inferiority study design and analysis.

While your interest is in  a specific hypothesis of non-inferiority, sometimes you also have to demonstrate "assay sensitivity'.
That is that your new product is non-inferior to standard, and would have 'been better" than placebo. Matters can get more complicated if there is "no placebo" comparator.

Also with a proportion so close to 100% what confidence interval method are  you planning to use? Agresti-coull, Wilson etc?

-------------------------------------------
Chris Barker, Ph.D.
Consultant and
Adjunct Associate Professor of Biostatistics
www,barkerstats.com

---
"In composition you have all the time you want to decide what to say in 15 seconds, in improvisation you have 15 seconds."
-Steve Lacy
-------------------------------------------

Morgan,

I happen to run into an article by Julious and Campbell (2012) Tutorial in biostatistics: sample sizes for parallel group clinical trials with binary data, Stat. Med. 31:2904-2936. Section 3.2 addresses non-inferiority trials.  On Page 2925, it is said that " Sample sizes are not given for anticipated responses greater than 0.90 as for high response rates the Normal approximation used in the sample size calculations may no longer hold. Our recommendation for sample sizes outside of this range would be to estimate the values using alternative methods such as simulation,..". An algorithm then follows. And the result of their simulation is presented in Table XIV.

Suppose the probability of success for the active control is 0.95, power is 90% and a one-sided type I error rate is 2.5%.  Assume that the non-inferiority margin is -0.03. A 0.98 probability of success for the investigational drug corresponds to a sample size of 208 per grp, which is not very high. However, a 0.94 probability of success from the investigational drug leads to a sample size of 2780 per grp. If the non-inferiority margin is -0.05, the two sample drop  to 129 and 717, respectively.  So, the sample size depends on how far the investigational drug differs from the active control and the non-inferiority margin.  

BTW, it is mentioned that Wlison score method is used in the test.  

-------------------------------------------
Qing Kang
Chief Scientist
Statistical Intelligence Group, LLC
-------------------------------------------



Original Message:
Sent: 08-27-2014 18:57
From: Chris Barker
Subject: Sample size for non-inferiority trial

If you can reveal them, A few more details about your study may be helpful. Have any other similar products been developed that use non-inferiority?

The FDA has a readable, detailed,  guidance on their website about non-inferiority study design and analysis.

While your interest is in  a specific hypothesis of non-inferiority, sometimes you also have to demonstrate "assay sensitivity'.
That is that your new product is non-inferior to standard, and would have 'been better" than placebo. Matters can get more complicated if there is "no placebo" comparator.

Also with a proportion so close to 100% what confidence interval method are  you planning to use? Agresti-coull, Wilson etc?

-------------------------------------------
Chris Barker, Ph.D.
Consultant and
Adjunct Associate Professor of Biostatistics
www,barkerstats.com

---
"In composition you have all the time you want to decide what to say in 15 seconds, in improvisation you have 15 seconds."
-Steve Lacy
-------------------------------------------

I don't know what "close to 100%" means to you, but a few years ago I had to deal with a NI trial planning where the comparator had a success rate >99.5%. Thus, a meaningful NI margin needed to be very small (maybe of the same order than the failure rate) and we would have ended up with a huge trial (20,000 - 150,000 from what I remember). The European Agency agreed that an NI trial would not be feasible and pragmatically accepted a single arm trial against historical control. But it was requested that the trial needed to be highly overpowered (I don't remember what was chosen at the end but around 99%). And the treatment would have a chance to be accepted only if the LCL of the success rate was way higher than the historical rate. Unfortunately I can't give you any clinicaltrials.gov details or a publication because finally the trial wasn't realized.

-------------------------------------------
Martin Kappler
statalpha
-------------------------------------------

This is a case where the non-inferiority should be done on a ratio of the new product to the current product.   Percent differences are not useful here.    Usually these ratio problems are for safety, where the goal to show that the Adverse Event rate does not exceed the old product by a percent.    Since this is for efficacy, the non-inferiority boundary will be less than 1.   For example let's say the old product has a success rate of 96%.     Let say you set the lower bound for efficacy at 84%., The ratio is 84/96, i.e. 7/8..   The non-inferiority test could then be used .   The  observed ratio is logged and hypothesis tests and confidence intervals are obtained on the log scale.   This approach is documented in several journal articles and software documentation, including SAS.


-------------------------------------------
Georgette Asherman
-------------------------------------------



Original Message:
Sent: 08-29-2014 03:12
From: Martin Kappler
Subject: Sample size for non-inferiority trial

I don't know what "close to 100%" means to you, but a few years ago I had to deal with a NI trial planning where the comparator had a success rate >99.5%. Thus, a meaningful NI margin needed to be very small (maybe of the same order than the failure rate) and we would have ended up with a huge trial (20,000 - 150,000 from what I remember). The European Agency agreed that an NI trial would not be feasible and pragmatically accepted a single arm trial against historical control. But it was requested that the trial needed to be highly overpowered (I don't remember what was chosen at the end but around 99%). And the treatment would have a chance to be accepted only if the LCL of the success rate was way higher than the historical rate. Unfortunately I can't give you any clinicaltrials.gov details or a publication because finally the trial wasn't realized.

-------------------------------------------
Martin Kappler
statalpha
-------------------------------------------

You note that "the approach is documented in several journal articles" - can you be specific?

-------------------------------------------
Paul Thompson
Director, Methodology and Data Analysis Center
Sanford Research/USD
-------------------------------------------



Original Message:
Sent: 08-29-2014 07:33
From: Georgette Asherman
Subject: Sample size for non-inferiority trial

This is a case where the non-inferiority should be done on a ratio of the new product to the current product.   Percent differences are not useful here.    Usually these ratio problems are for safety, where the goal to show that the Adverse Event rate does not exceed the old product by a percent.    Since this is for efficacy, the non-inferiority boundary will be less than 1.   For example let's say the old product has a success rate of 96%.     Let say you set the lower bound for efficacy at 84%., The ratio is 84/96, i.e. 7/8..   The non-inferiority test could then be used .   The  observed ratio is logged and hypothesis tests and confidence intervals are obtained on the log scale.   This approach is documented in several journal articles and software documentation, including SAS.


-------------------------------------------
Georgette Asherman
-------------------------------------------

It is common in situations to use the "ratio" as non-inferiority when rates are quite small (failure rates).  The diabetes guidance for approval for CV studies talks about achieving a 1.8 ratio or less (CI below) for approval, but then following up with 1.3 or less.  The "delta" issue becomes one of what ratio needs to be achieved.  This can create very large trials because you need events fro powering -- its not uncommon to need 100-120 events observed to start to power with deltas of 2.0 or so -- assuming equality.

We have had success with using both risk ratios and absolute differences in this setting -- very much depends on what you are doing (an absolute difference means if you have tons of exposure you can get success -- makes sense -- rather than ONLY showing a risk ratio).  Is it a device vs. drug, is it riskier, less invasive, 9th drug or device in a class, etc... Each case can create what "proving non-inferiority" means and what it means in that case ...

--Scott

-------------------------------------------
Scott Berry
Berry Consultants
-------------------------------------------



Original Message:
Sent: 08-29-2014 08:54
From: Paul Thompson
Subject: Sample size for non-inferiority trial

You note that "the approach is documented in several journal articles" - can you be specific?

-------------------------------------------
Paul Thompson
Director, Methodology and Data Analysis Center
Sanford Research/USD
-------------------------------------------



Original Message:
Sent: 08-29-2014 07:33
From: Georgette Asherman
Subject: Sample size for non-inferiority trial

This is a case where the non-inferiority should be done on a ratio of the new product to the current product.   Percent differences are not useful here.    Usually these ratio problems are for safety, where the goal to show that the Adverse Event rate does not exceed the old product by a percent.    Since this is for efficacy, the non-inferiority boundary will be less than 1.   For example let's say the old product has a success rate of 96%.     Let say you set the lower bound for efficacy at 84%., The ratio is 84/96, i.e. 7/8..   The non-inferiority test could then be used .   The  observed ratio is logged and hypothesis tests and confidence intervals are obtained on the log scale.   This approach is documented in several journal articles and software documentation, including SAS.


-------------------------------------------
Georgette Asherman
-------------------------------------------

Some references I have found useful:

Farrington and Manning (1990).  Test statistics and sample size formulae for comparative binomial trials with null hypothesis of non-zero risk difference or non-unity relative risk , Statistics in Medicine, Vol 9, 1447-1454.

Newcombe has some papers that review the different methods for calculating CIs for proportion (including Wilson Score method that FDA often uses) - Stat in Med 1998 vol 17.  I also see he now has a book out (2012, CRS/Chapman Hall) that may be very helpful but I have not looked at it.


-------------------------------------------
Kristen Meier, PhD
Myraqa Senior Director
Illumina, Inc
-------------------------------------------



Original Message:
Sent: 08-29-2014 08:54
From: Paul Thompson
Subject: Sample size for non-inferiority trial

You note that "the approach is documented in several journal articles" - can you be specific?

-------------------------------------------
Paul Thompson
Director, Methodology and Data Analysis Center
Sanford Research/USD
-------------------------------------------



Original Message:
Sent: 08-29-2014 07:33
From: Georgette Asherman
Subject: Sample size for non-inferiority trial

This is a case where the non-inferiority should be done on a ratio of the new product to the current product.   Percent differences are not useful here.    Usually these ratio problems are for safety, where the goal to show that the Adverse Event rate does not exceed the old product by a percent.    Since this is for efficacy, the non-inferiority boundary will be less than 1.   For example let's say the old product has a success rate of 96%.     Let say you set the lower bound for efficacy at 84%., The ratio is 84/96, i.e. 7/8..   The non-inferiority test could then be used .   The  observed ratio is logged and hypothesis tests and confidence intervals are obtained on the log scale.   This approach is documented in several journal articles and software documentation, including SAS.


-------------------------------------------
Georgette Asherman
-------------------------------------------

I don't usually remember specific examples from documentation, but this one stuck.    While it deals with the ratio of lognormal means, not proportions,  it is a very good example of how to use a ratio for a non-inferiority test.

From SAS PROC POWER Documentation  

Example 67.4 Noninferiority Test with Lognormal Data

 

http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_power_sect019.htm

I am not sure if PROC POWER is correct for ratios of proportions but I would check it out.

Georgette




-------------------------------------------
Georgette Asherman
-------------------------------------------



Original Message:
Sent: 08-29-2014 09:44
From: Kristen Meier
Subject: Sample size for non-inferiority trial

Some references I have found useful:

Farrington and Manning (1990).  Test statistics and sample size formulae for comparative binomial trials with null hypothesis of non-zero risk difference or non-unity relative risk , Statistics in Medicine, Vol 9, 1447-1454.

Newcombe has some papers that review the different methods for calculating CIs for proportion (including Wilson Score method that FDA often uses) - Stat in Med 1998 vol 17.  I also see he now has a book out (2012, CRS/Chapman Hall) that may be very helpful but I have not looked at it.


-------------------------------------------
Kristen Meier, PhD
Myraqa Senior Director
Illumina, Inc
-------------------------------------------



Original Message:
Sent: 08-29-2014 08:54
From: Paul Thompson
Subject: Sample size for non-inferiority trial

You note that "the approach is documented in several journal articles" - can you be specific?

-------------------------------------------
Paul Thompson
Director, Methodology and Data Analysis Center
Sanford Research/USD
-------------------------------------------

Consider
Tu D: On the use of the ratio or the odds ratio of cure rates in therapeutic equivalence trials
with binary endpoints. J Biopharm Stat 8:263-282. 1998.
and 

Bristol (1999). Clinical Equivalence. Journal of  Biopharmaceutical Statistics , 9(4), 549-561. 

David
-------------------------------------------
David Bristol
Statistical Consulting Services
-------------------------------------------



Original Message:
Sent: 08-29-2014 11:07
From: Georgette Asherman
Subject: Sample size for non-inferiority trial

I don't usually remember specific examples from documentation, but this one stuck.    While it deals with the ratio of lognormal means, not proportions,  it is a very good example of how to use a ratio for a non-inferiority test.

From SAS PROC POWER Documentation  

Example 67.4 Noninferiority Test with Lognormal Data

 

http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_power_sect019.htm

I am not sure if PROC POWER is correct for ratios of proportions but I would check it out.

Georgette




-------------------------------------------
Georgette Asherman
-------------------------------------------



Original Message:
Sent: 08-29-2014 09:44
From: Kristen Meier
Subject: Sample size for non-inferiority trial

Some references I have found useful:

Farrington and Manning (1990).  Test statistics and sample size formulae for comparative binomial trials with null hypothesis of non-zero risk difference or non-unity relative risk , Statistics in Medicine, Vol 9, 1447-1454.

Newcombe has some papers that review the different methods for calculating CIs for proportion (including Wilson Score method that FDA often uses) - Stat in Med 1998 vol 17.  I also see he now has a book out (2012, CRS/Chapman Hall) that may be very helpful but I have not looked at it.


-------------------------------------------
Kristen Meier, PhD
Myraqa Senior Director
Illumina, Inc
-------------------------------------------



Original Message:
Sent: 08-29-2014 08:54
From: Paul Thompson
Subject: Sample size for non-inferiority trial

You note that "the approach is documented in several journal articles" - can you be specific?

-------------------------------------------
Paul Thompson
Director, Methodology and Data Analysis Center
Sanford Research/USD
-------------------------------------------