Discussion: View Thread

This message has been cross posted to the following eGroups: Statistical Consulting Section and Bayesian Statistical Science Section .
-------------------------------------------
Hello All,

Here is something new for me, and I am hoping someone here has had this "problem" and can give some advice or ideas on what to do. 

The PI of one of my clinical trial projects advised me that they are accruing much faster than expected and is considering amending the protocol to allow for accrual of more patients. I would suppose that would be ok to do, leaving the trial open past the original recruitment number.  But I am not sure.

Any references or thoughts would be much appreciated!

Best,

Elaine

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------

Indeed an interesting question. Can you be a little more forthcoming? What was the original sample size? What power calculations were done? How many more cases are to be obtained? What does the IRB and DSMB think of this situation? I obtained more cases in a trial, and was requested to notify the IRB. Has this been done? 

The standard notion is that underpowered trials are unethical. It is the case that massively overpowered trials are similarly unethical, due to waste of resources, displacement of other studies (assuming there is competition) and so forth. Note also that the study might need to justify the additional cases in any manuscript.

-------------------------------------------
Paul Thompson
Director, Methodology and Data Analysis Center
Sanford Research/USD
-------------------------------------------

Hi Paul,

The initial power calculations for the primary endpoint needed about 250 patients accrued over 4 years, with a five year FFF follow-up after all of the patients have had at least one year of follow-up after treatment.  There have been 80 patients accrued now and it has been about 6 months. The PI is thinking they could double the initial patients needed in 4 years. There is also another facility coming on board next month so the accrual is anticipated to be even faster. 

The PI wanted my opinion before they formally went to the IRB etc. My initial thought is, sure, lucky you, just amend the protocol, get the blessing of the IRB and go for it.  However, it might be better to close the trial after the accrual has been met, and just start another trial now or after closing the current trial. But then I am have concerns about median follow-up times if it is closed too early (especially if the other facility is just coming on board)...and there may be other concerns that I haven't thought of, because, well, I haven't had this happen yet! :)

I understand that it really isn't my call on what to do, but the PI is asking what I think, because he isn't sure either. LOL.

What do you think Paul?

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------



Original Message:
Sent: 09-12-2014 18:23
From: Paul Thompson
Subject: Faster than expected accrual to non-inferiority trial

Indeed an interesting question. Can you be a little more forthcoming? What was the original sample size? What power calculations were done? How many more cases are to be obtained? What does the IRB and DSMB think of this situation? I obtained more cases in a trial, and was requested to notify the IRB. Has this been done? 

The standard notion is that underpowered trials are unethical. It is the case that massively overpowered trials are similarly unethical, due to waste of resources, displacement of other studies (assuming there is competition) and so forth. Note also that the study might need to justify the additional cases in any manuscript.

-------------------------------------------
Paul Thompson
Director, Methodology and Data Analysis Center
Sanford Research/USD
-------------------------------------------

And I will make sure to ask the PI if he is fairly certain the rate of accrual will be consistent. What if it just drops off after he amends the protocol? That would not be good...so assuming the rate will continue or increase, I am not positive I am looking at the entire picture.

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------



Original Message:
Sent: 09-12-2014 18:43
From: Elaine Eisenbeisz
Subject: Faster than expected accrual to non-inferiority trial

Hi Paul,

The initial power calculations for the primary endpoint needed about 250 patients accrued over 4 years, with a five year FFF follow-up after all of the patients have had at least one year of follow-up after treatment.  There have been 80 patients accrued now and it has been about 6 months. The PI is thinking they could double the initial patients needed in 4 years. There is also another facility coming on board next month so the accrual is anticipated to be even faster. 

The PI wanted my opinion before they formally went to the IRB etc. My initial thought is, sure, lucky you, just amend the protocol, get the blessing of the IRB and go for it.  However, it might be better to close the trial after the accrual has been met, and just start another trial now or after closing the current trial. But then I am have concerns about median follow-up times if it is closed too early (especially if the other facility is just coming on board)...and there may be other concerns that I haven't thought of, because, well, I haven't had this happen yet! :)

I understand that it really isn't my call on what to do, but the PI is asking what I think, because he isn't sure either. LOL.

What do you think Paul?

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------

OK, so 80 in 6 m is 160 / y, or 640 in 4 yr, which is 2.5x the original sample size. You really need to be clear about that. That's a huge jump, and if you go to a publication on this, you are gonna have to defend this in some manner. What about the notion of an interim analysis once the planned sample size has been reached? Certainly you would want to determine if the original power analysis conditions (variance, effect size, etc) are reflected somewhat in the data. 

But to collect 640 cases in a trial which had a power analysis plan of 250 is generally going to have some push-back.

What do others think?

-------------------------------------------
Paul Thompson
Director, Methodology and Data Analysis Center
Sanford Research/USD
-------------------------------------------



Original Message:
Sent: 09-12-2014 18:45
From: Elaine Eisenbeisz
Subject: Faster than expected accrual to non-inferiority trial

And I will make sure to ask the PI if he is fairly certain the rate of accrual will be consistent. What if it just drops off after he amends the protocol? That would not be good...so assuming the rate will continue or increase, I am not positive I am looking at the entire picture.

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------



Original Message:
Sent: 09-12-2014 18:43
From: Elaine Eisenbeisz
Subject: Faster than expected accrual to non-inferiority trial

Hi Paul,

The initial power calculations for the primary endpoint needed about 250 patients accrued over 4 years, with a five year FFF follow-up after all of the patients have had at least one year of follow-up after treatment.  There have been 80 patients accrued now and it has been about 6 months. The PI is thinking they could double the initial patients needed in 4 years. There is also another facility coming on board next month so the accrual is anticipated to be even faster. 

The PI wanted my opinion before they formally went to the IRB etc. My initial thought is, sure, lucky you, just amend the protocol, get the blessing of the IRB and go for it.  However, it might be better to close the trial after the accrual has been met, and just start another trial now or after closing the current trial. But then I am have concerns about median follow-up times if it is closed too early (especially if the other facility is just coming on board)...and there may be other concerns that I haven't thought of, because, well, I haven't had this happen yet! :)

I understand that it really isn't my call on what to do, but the PI is asking what I think, because he isn't sure either. LOL.

What do you think Paul?

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------

And another thought is to use a rate-limiting technique to recruit only about the number of cases that you orginally planned or some small increase.

-------------------------------------------
Paul Thompson
Director, Methodology and Data Analysis Center
Sanford Research/USD
-------------------------------------------



Original Message:
Sent: 09-12-2014 18:55
From: Paul Thompson
Subject: Faster than expected accrual to non-inferiority trial

OK, so 80 in 6 m is 160 / y, or 640 in 4 yr, which is 2.5x the original sample size. You really need to be clear about that. That's a huge jump, and if you go to a publication on this, you are gonna have to defend this in some manner. What about the notion of an interim analysis once the planned sample size has been reached? Certainly you would want to determine if the original power analysis conditions (variance, effect size, etc) are reflected somewhat in the data. 

But to collect 640 cases in a trial which had a power analysis plan of 250 is generally going to have some push-back.

What do others think?

-------------------------------------------
Paul Thompson
Director, Methodology and Data Analysis Center
Sanford Research/USD
-------------------------------------------



Original Message:
Sent: 09-12-2014 18:45
From: Elaine Eisenbeisz
Subject: Faster than expected accrual to non-inferiority trial

And I will make sure to ask the PI if he is fairly certain the rate of accrual will be consistent. What if it just drops off after he amends the protocol? That would not be good...so assuming the rate will continue or increase, I am not positive I am looking at the entire picture.

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------

It is possible in an overpowered NI trial to have an end result that the test drug is "non-inferior" to standard but also statistically significantly worse than standard. That would be awkward to explain.

-------------------------------------------
Roy Tamura
Associate Professor
University of South Florida
-------------------------------------------



Original Message:
Sent: 9/12/2014 6:55:09 PM
From: paul.thompson@sanfordhealth.org
Subject: RE: Faster than expected accrual to non-inferiority trial

OK, so 80 in 6 m is 160 / y, or 640 in 4 yr, which is 2.5x the original sample size. You really need to be clear about that. That's a huge jump, and if you go to a publication on this, you are gonna have to defend this in some manner. What about the notion of an interim analysis once the planned sample size has been reached? Certainly you would want to determine if the original power analysis conditions (variance, effect size, etc) are reflected somewhat in the data.  But to collect 640 cases in a trial which had a power analysis plan of 250 is generally going to have some push-back. What do others think? ------------------------------------------- Paul Thompson Director, Methodology and Data Analysis Center Sanford Research/USD -------------------------------------------
Original Message: Sent: 09-12-2014 18:45 From: Elaine Eisenbeisz Subject: Faster than expected accrual to non-inferiority trial And I will make sure to ask the PI if he is fairly certain the rate of accrual will be consistent. What if it just drops off after he amends the protocol? That would not be good...so assuming the rate will continue or increase, I am not positive I am looking at the entire picture. ------------------------------------------- Elaine Eisenbeisz Owner and Principal Statistician Omega Statistics -------------------------------------------
Original Message: Sent: 09-12-2014 18:43 From: Elaine Eisenbeisz Subject: Faster than expected accrual to non-inferiority trial Hi Paul, The initial power calculations for the primary endpoint needed about 250 patients accrued over 4 years, with a five year FFF follow-up after all of the patients have had at least one year of follow-up after treatment.  There have been 80 patients accrued now and it has been about 6 months. The PI is thinking they could double the initial patients needed in 4 years. There is also another facility coming on board next month so the accrual is anticipated to be even faster.  The PI wanted my opinion before they formally went to the IRB etc. My initial thought is, sure, lucky you, just amend the protocol, get the blessing of the IRB and go for it.  However, it might be better to close the trial after the accrual has been met, and just start another trial now or after closing the current trial. But then I am have concerns about median follow-up times if it is closed too early (especially if the other facility is just coming on board)...and there may be other concerns that I haven't thought of, because, well, I haven't had this happen yet! :) I understand that it really isn't my call on what to do, but the PI is asking what I think, because he isn't sure either. LOL. What do you think Paul? ------------------------------------------- Elaine Eisenbeisz Owner and Principal Statistician Omega Statistics -------------------------------------------
Original Message: Sent: 09-12-2014 18:23 From: Paul Thompson Subject: Faster than expected accrual to non-inferiority trial Indeed an interesting question. Can you be a little more forthcoming? What was the original sample size? What power calculations were done? How many more cases are to be obtained? What does the IRB and DSMB think of this situation? I obtained more cases in a trial, and was requested to notify the IRB. Has this been done?  The standard notion is that underpowered trials are unethical. It is the case that massively overpowered trials are similarly unethical, due to waste of resources, displacement of other studies (assuming there is competition) and so forth. Note also that the study might need to justify the additional cases in any manuscript. ------------------------------------------- Paul Thompson Director, Methodology and Data Analysis Center Sanford Research/USD -------------------------------------------

Thank you Paul and Ray. I think we are heading toward deciding to open another trial.  Good news all around! Have a great weekend!
Elaine

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------



Original Message:
Sent: 09-12-2014 18:59
From: Roy Tamura
Subject: Faster than expected accrual to non-inferiority trial

It is possible in an overpowered NI trial to have an end result that the test drug is "non-inferior" to standard but also statistically significantly worse than standard. That would be awkward to explain.

-------------------------------------------
Roy Tamura
Associate Professor
University of South Florida
-------------------------------------------

Maybe I read it too quickly, but I don't see your concern. We all know that, sampling being the same, the larger the sample the more accurate the estimates and the higher powered the tests. I can't see how increasing sample size is statistically unethical. Patient ethics is the bailiwick of the IRB, who takes it up with the PI, not the statistician. If there is a patient protection issue and the IRB approves the change, it should be OK with the statistician. As to continuing surveillance, the IRB should be monitoring the PI, who reports all issues and progress on a scheduled basis. The IRB will direct the statistician if a statistical issue arises.

-------------------------------------------
Robert Riffenburgh
Naval Medical Center
-------------------------------------------



Original Message:
Sent: 09-12-2014 18:45
From: Elaine Eisenbeisz
Subject: Faster than expected accrual to non-inferiority trial

And I will make sure to ask the PI if he is fairly certain the rate of accrual will be consistent. What if it just drops off after he amends the protocol? That would not be good...so assuming the rate will continue or increase, I am not positive I am looking at the entire picture.

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------



Original Message:
Sent: 09-12-2014 18:43
From: Elaine Eisenbeisz
Subject: Faster than expected accrual to non-inferiority trial

Hi Paul,

The initial power calculations for the primary endpoint needed about 250 patients accrued over 4 years, with a five year FFF follow-up after all of the patients have had at least one year of follow-up after treatment.  There have been 80 patients accrued now and it has been about 6 months. The PI is thinking they could double the initial patients needed in 4 years. There is also another facility coming on board next month so the accrual is anticipated to be even faster. 

The PI wanted my opinion before they formally went to the IRB etc. My initial thought is, sure, lucky you, just amend the protocol, get the blessing of the IRB and go for it.  However, it might be better to close the trial after the accrual has been met, and just start another trial now or after closing the current trial. But then I am have concerns about median follow-up times if it is closed too early (especially if the other facility is just coming on board)...and there may be other concerns that I haven't thought of, because, well, I haven't had this happen yet! :)

I understand that it really isn't my call on what to do, but the PI is asking what I think, because he isn't sure either. LOL.

What do you think Paul?

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------

If I interpret correctly, there is no fixed follow-up time of patients after their inclusion but all are followed until a certain calendar date regardless of their date of inclusion ("all of the patients have had at least one year of follow-up after treatment" translates to 5 years f-up after the last included patient had received treatment for 1 year). This sort of follow-up is usually done in trials with some kind of time-to-event endpoint. And in this case the number of events is more important than the number of included patients. So you should have a look at your observed event rate and look if this corresponds to what was expected. In my opinion, the decision on an increase in sample size highly depends on what you see there.

@Robert... I slightly disagree. Sample size calculation is done in order to limit the number of patients exposed to the experimental treatment while still satisfying a certain power in order to get a meaningful response out of the trial. The PI might have a strong feeling about the treatment effect and he/she might be ok with a sample size leading to a power of e.g. 99%. In this case the statistician at least should say a word about the number of additionally exposed patients compared to the planned power. However, I agree with you that the IRB in theory should not have such preconceptions and thus should also be concerned about the number of unnecessary exposed patients.

-------------------------------------------
Martin Kappler
Expert Biostatistician
statalpha
-------------------------------------------



Original Message:
Sent: 09-13-2014 14:49
From: Robert Riffenburgh
Subject: Faster than expected accrual to non-inferiority trial

Maybe I read it too quickly, but I don't see your concern. We all know that, sampling being the same, the larger the sample the more accurate the estimates and the higher powered the tests. I can't see how increasing sample size is statistically unethical. Patient ethics is the bailiwick of the IRB, who takes it up with the PI, not the statistician. If there is a patient protection issue and the IRB approves the change, it should be OK with the statistician. As to continuing surveillance, the IRB should be monitoring the PI, who reports all issues and progress on a scheduled basis. The IRB will direct the statistician if a statistical issue arises.

-------------------------------------------
Robert Riffenburgh
Naval Medical Center
-------------------------------------------



Original Message:
Sent: 09-12-2014 18:45
From: Elaine Eisenbeisz
Subject: Faster than expected accrual to non-inferiority trial

And I will make sure to ask the PI if he is fairly certain the rate of accrual will be consistent. What if it just drops off after he amends the protocol? That would not be good...so assuming the rate will continue or increase, I am not positive I am looking at the entire picture.

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------

Martin-- Point taken. Thanks. And another thing neither of us mentioned is sample size dependence of tests. In several tests, perhaps the worst being contingency (chi-square, Fisher's) and correlation tests, you can always find significance if you take a large enough sample (parameters remaining the same).
--Bob

-------------------------------------------
Robert Riffenburgh
Naval Medical Center
-------------------------------------------



Original Message:
Sent: 09-15-2014 05:03
From: Martin Kappler
Subject: Faster than expected accrual to non-inferiority trial

If I interpret correctly, there is no fixed follow-up time of patients after their inclusion but all are followed until a certain calendar date regardless of their date of inclusion ("all of the patients have had at least one year of follow-up after treatment" translates to 5 years f-up after the last included patient had received treatment for 1 year). This sort of follow-up is usually done in trials with some kind of time-to-event endpoint. And in this case the number of events is more important than the number of included patients. So you should have a look at your observed event rate and look if this corresponds to what was expected. In my opinion, the decision on an increase in sample size highly depends on what you see there.

@Robert... I slightly disagree. Sample size calculation is done in order to limit the number of patients exposed to the experimental treatment while still satisfying a certain power in order to get a meaningful response out of the trial. The PI might have a strong feeling about the treatment effect and he/she might be ok with a sample size leading to a power of e.g. 99%. In this case the statistician at least should say a word about the number of additionally exposed patients compared to the planned power. However, I agree with you that the IRB in theory should not have such preconceptions and thus should also be concerned about the number of unnecessary exposed patients.

-------------------------------------------
Martin Kappler
Expert Biostatistician
statalpha
-------------------------------------------



Original Message:
Sent: 09-13-2014 14:49
From: Robert Riffenburgh
Subject: Faster than expected accrual to non-inferiority trial

Maybe I read it too quickly, but I don't see your concern. We all know that, sampling being the same, the larger the sample the more accurate the estimates and the higher powered the tests. I can't see how increasing sample size is statistically unethical. Patient ethics is the bailiwick of the IRB, who takes it up with the PI, not the statistician. If there is a patient protection issue and the IRB approves the change, it should be OK with the statistician. As to continuing surveillance, the IRB should be monitoring the PI, who reports all issues and progress on a scheduled basis. The IRB will direct the statistician if a statistical issue arises.

-------------------------------------------
Robert Riffenburgh
Naval Medical Center
-------------------------------------------



Original Message:
Sent: 09-12-2014 18:45
From: Elaine Eisenbeisz
Subject: Faster than expected accrual to non-inferiority trial

And I will make sure to ask the PI if he is fairly certain the rate of accrual will be consistent. What if it just drops off after he amends the protocol? That would not be good...so assuming the rate will continue or increase, I am not positive I am looking at the entire picture.

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------

What Robert Riffenburgh implies is that the classical null hypothesis is hardly ever true, so in the frequentist world, we should compute
confidence intervals and forget about p-values, which may be the most misinterpreted concept in science. I strongly recommend this paper by Jason Connor:

       Connor, J. T. (2004). The value of a p-valueless paper. Am J Gastroenterol, 99(9):1638-40.
       http://hal.case.edu/~robrien/Connor04The%20value%20of%20a%20p-valueless%20paper.pdf

(That's the best title I've ever seen for a statistics paper.)

If you agree with this, then you must also agree that traditional power analyses lack probative value. We should be studying what the given N might yield in terms of producing a good CI that will directly address the research question. The concept applies perfectly to doing sample size analyses for obtaining good credible/probability intervals in the Bayesian world.

-------------------------------------------
Ralph O'Brien
Professor of Biostatistics (retiring literally today, 9/15/2014; a.k.a I am now on permanent sabbatical)
Case Western Reserve University
-------------------------------------------



Original Message:
Sent: 09-15-2014 11:41
From: Robert Riffenburgh
Subject: Faster than expected accrual to non-inferiority trial

Martin-- Point taken. Thanks. And another thing neither of us mentioned is sample size dependence of tests. In several tests, perhaps the worst being contingency (chi-square, Fisher's) and correlation tests, you can always find significance if you take a large enough sample (parameters remaining the same).
--Bob

-------------------------------------------
Robert Riffenburgh
Naval Medical Center
-------------------------------------------



Original Message:
Sent: 09-15-2014 05:03
From: Martin Kappler
Subject: Faster than expected accrual to non-inferiority trial

If I interpret correctly, there is no fixed follow-up time of patients after their inclusion but all are followed until a certain calendar date regardless of their date of inclusion ("all of the patients have had at least one year of follow-up after treatment" translates to 5 years f-up after the last included patient had received treatment for 1 year). This sort of follow-up is usually done in trials with some kind of time-to-event endpoint. And in this case the number of events is more important than the number of included patients. So you should have a look at your observed event rate and look if this corresponds to what was expected. In my opinion, the decision on an increase in sample size highly depends on what you see there.

@Robert... I slightly disagree. Sample size calculation is done in order to limit the number of patients exposed to the experimental treatment while still satisfying a certain power in order to get a meaningful response out of the trial. The PI might have a strong feeling about the treatment effect and he/she might be ok with a sample size leading to a power of e.g. 99%. In this case the statistician at least should say a word about the number of additionally exposed patients compared to the planned power. However, I agree with you that the IRB in theory should not have such preconceptions and thus should also be concerned about the number of unnecessary exposed patients.

-------------------------------------------
Martin Kappler
Expert Biostatistician
statalpha
-------------------------------------------



Original Message:
Sent: 09-13-2014 14:49
From: Robert Riffenburgh
Subject: Faster than expected accrual to non-inferiority trial

Maybe I read it too quickly, but I don't see your concern. We all know that, sampling being the same, the larger the sample the more accurate the estimates and the higher powered the tests. I can't see how increasing sample size is statistically unethical. Patient ethics is the bailiwick of the IRB, who takes it up with the PI, not the statistician. If there is a patient protection issue and the IRB approves the change, it should be OK with the statistician. As to continuing surveillance, the IRB should be monitoring the PI, who reports all issues and progress on a scheduled basis. The IRB will direct the statistician if a statistical issue arises.

-------------------------------------------
Robert Riffenburgh
Naval Medical Center
-------------------------------------------

Computing CI for NI study is particularly important if that's the inference used for the decision rule.

I also like 
Evans, Mills, and Dawson "The end of the p value?" Br Heart J. Sep 1988; 60(3): 177-180.
But note that it was published in 1988

-------------------------------------------
David Bristol
Statistical Consulting Services
-------------------------------------------



Original Message:
Sent: 09-15-2014 14:59
From: Ralph O'Brien
Subject: Faster than expected accrual to non-inferiority trial

What Robert Riffenburgh implies is that the classical null hypothesis is hardly ever true, so in the frequentist world, we should compute
confidence intervals and forget about p-values, which may be the most misinterpreted concept in science. I strongly recommend this paper by Jason Connor:

       Connor, J. T. (2004). The value of a p-valueless paper. Am J Gastroenterol, 99(9):1638-40.
       http://hal.case.edu/~robrien/Connor04The%20value%20of%20a%20p-valueless%20paper.pdf

(That's the best title I've ever seen for a statistics paper.)

If you agree with this, then you must also agree that traditional power analyses lack probative value. We should be studying what the given N might yield in terms of producing a good CI that will directly address the research question. The concept applies perfectly to doing sample size analyses for obtaining good credible/probability intervals in the Bayesian world.

-------------------------------------------
Ralph O'Brien
Professor of Biostatistics (retiring literally today, 9/15/2014; a.k.a I am now on permanent sabbatical)
Case Western Reserve University
-------------------------------------------



Original Message:
Sent: 09-15-2014 11:41
From: Robert Riffenburgh
Subject: Faster than expected accrual to non-inferiority trial

Martin-- Point taken. Thanks. And another thing neither of us mentioned is sample size dependence of tests. In several tests, perhaps the worst being contingency (chi-square, Fisher's) and correlation tests, you can always find significance if you take a large enough sample (parameters remaining the same).
--Bob

-------------------------------------------
Robert Riffenburgh
Naval Medical Center
-------------------------------------------



Original Message:
Sent: 09-15-2014 05:03
From: Martin Kappler
Subject: Faster than expected accrual to non-inferiority trial

If I interpret correctly, there is no fixed follow-up time of patients after their inclusion but all are followed until a certain calendar date regardless of their date of inclusion ("all of the patients have had at least one year of follow-up after treatment" translates to 5 years f-up after the last included patient had received treatment for 1 year). This sort of follow-up is usually done in trials with some kind of time-to-event endpoint. And in this case the number of events is more important than the number of included patients. So you should have a look at your observed event rate and look if this corresponds to what was expected. In my opinion, the decision on an increase in sample size highly depends on what you see there.

@Robert... I slightly disagree. Sample size calculation is done in order to limit the number of patients exposed to the experimental treatment while still satisfying a certain power in order to get a meaningful response out of the trial. The PI might have a strong feeling about the treatment effect and he/she might be ok with a sample size leading to a power of e.g. 99%. In this case the statistician at least should say a word about the number of additionally exposed patients compared to the planned power. However, I agree with you that the IRB in theory should not have such preconceptions and thus should also be concerned about the number of unnecessary exposed patients.

-------------------------------------------
Martin Kappler
Expert Biostatistician
statalpha
-------------------------------------------



Original Message:
Sent: 09-13-2014 14:49
From: Robert Riffenburgh
Subject: Faster than expected accrual to non-inferiority trial

Maybe I read it too quickly, but I don't see your concern. We all know that, sampling being the same, the larger the sample the more accurate the estimates and the higher powered the tests. I can't see how increasing sample size is statistically unethical. Patient ethics is the bailiwick of the IRB, who takes it up with the PI, not the statistician. If there is a patient protection issue and the IRB approves the change, it should be OK with the statistician. As to continuing surveillance, the IRB should be monitoring the PI, who reports all issues and progress on a scheduled basis. The IRB will direct the statistician if a statistical issue arises.

-------------------------------------------
Robert Riffenburgh
Naval Medical Center
-------------------------------------------

Excellent insight Martin!  Focus on the outcome of interest - and if time then likely events - brilliant deduction "Watson".

The ethical concern to me, is that overpowering a trial could allow you to detect "statistically significant" differences which are too small to be clinically relevant.  Then the patients have been "wasted" and also there is a dilemma when publishing - do you want to publish "trivial" differences?  The other ethical component is time - if the treatment is proven to be effective, then delaying an improved treatment for patients would not be ethical, nor financially prudent.  The money released (trial costs are increased as time increases) could be used to further develop this or another compound.

-------------------------------------------
Janet McDougall
President
McDougall Scientific Ltd
-------------------------------------------



Original Message:
Sent: 09-15-2014 05:03
From: Martin Kappler
Subject: Faster than expected accrual to non-inferiority trial

If I interpret correctly, there is no fixed follow-up time of patients after their inclusion but all are followed until a certain calendar date regardless of their date of inclusion ("all of the patients have had at least one year of follow-up after treatment" translates to 5 years f-up after the last included patient had received treatment for 1 year). This sort of follow-up is usually done in trials with some kind of time-to-event endpoint. And in this case the number of events is more important than the number of included patients. So you should have a look at your observed event rate and look if this corresponds to what was expected. In my opinion, the decision on an increase in sample size highly depends on what you see there.

@Robert... I slightly disagree. Sample size calculation is done in order to limit the number of patients exposed to the experimental treatment while still satisfying a certain power in order to get a meaningful response out of the trial. The PI might have a strong feeling about the treatment effect and he/she might be ok with a sample size leading to a power of e.g. 99%. In this case the statistician at least should say a word about the number of additionally exposed patients compared to the planned power. However, I agree with you that the IRB in theory should not have such preconceptions and thus should also be concerned about the number of unnecessary exposed patients.

-------------------------------------------
Martin Kappler
Expert Biostatistician
statalpha
-------------------------------------------



Original Message:
Sent: 09-13-2014 14:49
From: Robert Riffenburgh
Subject: Faster than expected accrual to non-inferiority trial

Maybe I read it too quickly, but I don't see your concern. We all know that, sampling being the same, the larger the sample the more accurate the estimates and the higher powered the tests. I can't see how increasing sample size is statistically unethical. Patient ethics is the bailiwick of the IRB, who takes it up with the PI, not the statistician. If there is a patient protection issue and the IRB approves the change, it should be OK with the statistician. As to continuing surveillance, the IRB should be monitoring the PI, who reports all issues and progress on a scheduled basis. The IRB will direct the statistician if a statistical issue arises.

-------------------------------------------
Robert Riffenburgh
Naval Medical Center
-------------------------------------------



Original Message:
Sent: 09-12-2014 18:45
From: Elaine Eisenbeisz
Subject: Faster than expected accrual to non-inferiority trial

And I will make sure to ask the PI if he is fairly certain the rate of accrual will be consistent. What if it just drops off after he amends the protocol? That would not be good...so assuming the rate will continue or increase, I am not positive I am looking at the entire picture.

-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------

Yes, you are all correct about sample sizes and power using the frequentist approach. I took that into consideration of course. Just thought that was a given, but others may not have thought about it.  But absolutely you are right! Thank you again.

Elaine


-------------------------------------------
Elaine Eisenbeisz
Owner and Principal Statistician
Omega Statistics
-------------------------------------------



Original Message:
Sent: 09-15-2014 12:19
From: Janet McDougall
Subject: Faster than expected accrual to non-inferiority trial

Excellent insight Martin!  Focus on the outcome of interest - and if time then likely events - brilliant deduction "Watson".

The ethical concern to me, is that overpowering a trial could allow you to detect "statistically significant" differences which are too small to be clinically relevant.  Then the patients have been "wasted" and also there is a dilemma when publishing - do you want to publish "trivial" differences?  The other ethical component is time - if the treatment is proven to be effective, then delaying an improved treatment for patients would not be ethical, nor financially prudent.  The money released (trial costs are increased as time increases) could be used to further develop this or another compound.

-------------------------------------------
Janet McDougall
President
McDougall Scientific Ltd
-------------------------------------------



Original Message:
Sent: 09-15-2014 05:03
From: Martin Kappler
Subject: Faster than expected accrual to non-inferiority trial

If I interpret correctly, there is no fixed follow-up time of patients after their inclusion but all are followed until a certain calendar date regardless of their date of inclusion ("all of the patients have had at least one year of follow-up after treatment" translates to 5 years f-up after the last included patient had received treatment for 1 year). This sort of follow-up is usually done in trials with some kind of time-to-event endpoint. And in this case the number of events is more important than the number of included patients. So you should have a look at your observed event rate and look if this corresponds to what was expected. In my opinion, the decision on an increase in sample size highly depends on what you see there.

@Robert... I slightly disagree. Sample size calculation is done in order to limit the number of patients exposed to the experimental treatment while still satisfying a certain power in order to get a meaningful response out of the trial. The PI might have a strong feeling about the treatment effect and he/she might be ok with a sample size leading to a power of e.g. 99%. In this case the statistician at least should say a word about the number of additionally exposed patients compared to the planned power. However, I agree with you that the IRB in theory should not have such preconceptions and thus should also be concerned about the number of unnecessary exposed patients.

-------------------------------------------
Martin Kappler
Expert Biostatistician
statalpha
-------------------------------------------



Original Message:
Sent: 09-13-2014 14:49
From: Robert Riffenburgh
Subject: Faster than expected accrual to non-inferiority trial

Maybe I read it too quickly, but I don't see your concern. We all know that, sampling being the same, the larger the sample the more accurate the estimates and the higher powered the tests. I can't see how increasing sample size is statistically unethical. Patient ethics is the bailiwick of the IRB, who takes it up with the PI, not the statistician. If there is a patient protection issue and the IRB approves the change, it should be OK with the statistician. As to continuing surveillance, the IRB should be monitoring the PI, who reports all issues and progress on a scheduled basis. The IRB will direct the statistician if a statistical issue arises.

-------------------------------------------
Robert Riffenburgh
Naval Medical Center
-------------------------------------------