A conversation with Ram Tiwari, PhD, Director, Division of Biostatistics, OSB, CDRH, Two Years On
In August 2016 I had an opportunity to talk to Dr. Ram Tiwari, the incoming Director of the Division of Biostatistics, CDRH, as he prepared to take the reins following the retirement of longtime Director Dr. Greg Campbell. Two years later, I sat down with Dr. Tiwari once again to chat about statistics, the medical device industry, FDA CDRH business, and the current view from the Division helm.
As always, the interviewer takes full responsibility for the introduction of any random (or non-random) errors.
On Center Structure
Then (2016):
The CDRH set strategic priorities which included establishing a National Evaluation System for health Technology (NEST), promote a culture of quality, and partner with patients; even potential re-organization to support the Center priority on total product life cycle (TPLC). We want the pre-market and post-market aligned with the TPLC model so everyone can learn from pre-market, post-market, and patient preference to improve the efficiency and quality of medical device evaluation.
Now (2018): Did CDRH end up reorganizing, or did it adapt in other ways to support TPLC? Is the priority still TPLC, or have new directions emerged since 2016?
In order to focus on these strategic priorities, CDRH is undergoing some major organizational changes pending congressional approval. The Office of Surveillance and Biometrics (OSB) will be renamed to the Office of Clinical Evidence and Analysis (OCEA), with only two divisions; Division 2 will be the Division of Biostatistics and Division 1 will be a combined organization of other disciplines, such as epidemiology, post market surveillance, and BIMO (monitoring). I think this new structure could further facilitate interactions between statisticians and other functional groups in OCEA. For example, I have worked with BIOM, on a TPLC project, to develop a risk-based statistical model in site selection for BIOM audit. This methodology has been submitted for publication (Therapeutic Innovation & Regulatory Sciences).
The Division of Biostatistics (DBS) will have seven teams, instead of the current five Branches. Each team will be led by an assistant director and work with the reorganized Office of Device Evaluation (ODE), which will become seven separate Offices of Health Technologies (OHT). The DBS is least affected by the major re-organization at CDRH during the past 6-12 months. We will continue to provide statistical support to submission reviews. The advantage in this new structure is that reviewers and statisticians hopefully can work across teams. This could provide more flexibility and learning opportunities, and share work load among statisticians. Cross-group support is already a unique culture in our division. People work together, help each other. This is why the Center Director made it clear that statisticians will remain as one group, instead of inserting into other functional groups. Because statisticians speak the same language, we interact well, and can help each other. CDER has also recognized this and consolidated their statisticians from separate functional groups into an Office of Biostatistics.
On national evaluation system for health technology
Then:
By the end of year, CDRH is looking forward to the establishment of a National Evaluation System for health Technology (NEST). This is mostly a private-public partnership effort. And it will be a centralized approach to collect Real World Evidence (RWE) that is found in any data sources, including registry data, and aggregating this information could result in more efficient and timely pre-market reviews.
We are also working on developing statistical methodologies to incorporate real world evidence into pre-market evaluation as well as to mine post-market safety signals. We will emphasize on how to use real world evidence in pre-market settings. This whole spectrum—not only a collection of real world data, but statistical methodologies for taking advantage of all this information to design innovative studies from device approval all the way to post-market—is a very important priority for CDRH.
Now: Is the development of NEST on schedule? When is it expected to be on-line? '
What stage is the NEST project at today? How satisfied are you with its uptake by the private sector, and how can industry partners get involved or provide feedback? What are the statistical considerations of using real-world or register data?
Device development is moving fast, it’s hard to catch up. Center Director Dr. Jeff Shuren gave a talk at this year’s FDA/Advamed Medical Devices and Diagnostics Statistical Issues Conference where he pointed out that NEST will continue to be the center’s priority for years to come. In 2016, FDA awarded a cooperative agreement to the Medical Device Innovation Consortium (MDIC) to establish a Coordinating Center for NEST - NESTcc. NESTcc is now up and running and posts regular project updates on its website.
DBS statisticians are working diligently with NESTcc in developing methodologies to analyze and utilize these data, such as benefit-risk analyses, smartphone-based database search applications, and signal detection methods.
Data registry, in general, had some challenges, such as who owns the data and who should be allowed to use it for what purposes. With some uncertainty in these areas, people sometimes become discouraged to provide and use data from data registries. However, the needs of collecting and using the vast amount of data is still there. The problems need to be resolved and we need to move in this direction.
DBS deputy director Lilly Yue gave a very nice talk on ‘the use of real world data (RWD) and real world evidence (RWE)’ (with Greg Campbell’s comments) at the present JSM. She outlined potentials and opportunities in turning RWD into RWE using statistical methods. She also presented areas of ongoing researches and gave examples on the statistician’s role in the new, data rich world. You can get more details on her presentation posted on the MDD’s website.
On therapeutic and diagnostic devices
Then:
There are a lot of challenges in both areas, but perhaps more so in diagnostics. For example, we are looking into the validity of using a single dataset to identify and validate a biomarker. Part of the dataset is used as training data and part of it as test data. This scenario is going to become more and more prevalent under the expedited access pathway (EAP) program.
Now: Two years later, how has the landscape of medical devices changed (if it has)?
Digital health has gained popularity. Wearable and personal devices/applications, such as heart rate monitoring devices/applications and glucose meters, offer health information and even imply diagnostic functions. Not all of them are currently regulated by the FDA; nevertheless, how to ensure the benefits outweigh the risks of a particular device/application is an obvious question needs to be answered.
Another important area is the focus on precision medicine, including the use of companion diagnostic devices on certain oncology drugs. This is a promising area and it is going to grow.
On efficient use of post-market data
Then:
This is a big area, and we don’t have a good sense yet on how to use the post-market data in the device approval process. This would be considered on a case-by-case basis. We have only seen a few cases in biomarker/gene sequencing. It especially makes good sense in companion diagnostics, where oftentimes the drug is already approved before we look into the companion diagnostic device, and post-market data is very useful in the evaluation of safety and effectiveness of the device. In general, it is a good thing and works well with the EAP program. The Center director is very interested in EAP and personally tracks any new EAP submissions.
Now: Is there more clarity on how post-market data can be used to support device approval? Is post-market data closer to either wide-spread acceptance or rejection? Have any companies tried and succeeded?
Still a big area and FDA has made good progress. Our commissioner Scott Gottlieb, in his blog, talked about FDA’s commitment to ‘access and use data collected from all sources’. FDA has released a guidance document on use of electronic health record data in July this year. CDRH is using this real world evidence in device approval and labeling. Constraints and challenges exist: quality of data, missing data, representativeness and applicability (same inclusion/exclusion criteria), bias, etc. Statisticians and epidemiologists both play important roles in evaluating the validity and applicability of RWD; we are working together to achieve the best results. Important considerations when using RWD and RWE in regulatory submissions: Data quality needs to be checked. Propensity score matching is useful; use RWD to augment not only the control but also the experimental arm. Requirements for safety and effectiveness do not change. Do your homework and use common sense, but this is definitely the direction to go.
My philosophy, as with most statisticians, is that real world evidence found in post-market studies is especially useful in supporting clinical evidence, such as for subpopulations or subgroup analyses. EU is also tightening their device approval evaluation process to require more pre- and post-approval clinical evidence.
On presubmission discussions between CDRH and company statisticians
Then:
Many times a presubmission discussion becomes a statistical consultation. It’s not an effective use of the process and it puts a lot of burden on the Division of Biostatistics. Good preparation by the sponsors is the key to a quality review and productive discussion during the presubmission.
Now: Does CDRH have plans to make the presubmission program more efficient and effective? Do you have any words of advice to industry statisticians on what to avoid or focus on in presubmission preparation? (i.e., what in your opinion is the most common mistake made by sponsors?)
Presub is well used by sponsors. We see an increase in numbers of presubs every year. Sometimes companies request presubs before they have the final design of a device or firm ideas on how they want to evaluate it. Some companies use it as an opportunity to get free advice; others rely on FDA statisticians to tell them how to design the studies. They use us as free statistical consultants. Some have to come back again and again to get the useful feedback. That’s part of the reason we see so many presubs and presub supplements. Companies realize the importance of presubs: Effective presub discussion can help in avoiding serious and costly problems in the final PMA or 510K submissions. Because of the amount of presubs, our workload has dramatically increased. Presub is the single most time consuming activity for both clinical and statistical reviewers. There are discussions in the Center on looking for ways to reduce the workload or increase efficiency of presub. One possibility is to provide, maybe by email, only general highlighted comments on critical and new tests, instead of the very detailed reviews on standard, regular analytical tests. Our statisticians spend about 40% of their time reviewing presubs, and our management is wondering why! The most common mistake made by sponsors, in my opinions, is that if you don’t do your homework before a presub, you are not going to get useful feedback on your project, and at the same time add a lot of burden on reviewers.
On benefit-risk analysis
Then:
This is a big area. In CDRH, the approach is somewhat different from CDER. We are trying to bring in a patient preference factor in the approval process. The CDRH has made a lot of progress in this effort: there is a guidance document now and our statistics reviewers are getting trained on how to use patient preference factor in pre- and post-market analysis. This is the direction in which we are moving.
Now: How can Patient Preference Information be considered when designing device evaluation studies? Following the release of CDRH guidance on Patient Preference Information in 2016, CDRH commissioned a patient-centered benefit-risk (PCBR) project to develop a framework for how sponsors could collect and use patient centered benefit-risk information. Can you please give an update on the use of benefit-risk information?
At FDA, we look at benefit-risk (BR) of devices every day. The Center Director is a leader in this area and has made a big impact. He gave a talk about this in his AdvaMed 2018 workshop keynote speech. CDRH had an internal workshop dedicated to this topic and a new guidance document released last year. The real world cases are more complicated. How BR is measured? In individuals? In groups? Benefit and risk are not independent and they are usually not measured on the same scales. All these need new research and statistical methodologies.
I gave a talk in a topic contributed session sponsored by the Medical Devices and Diagnostics Section (MDD) on “Bayesian Approaches for Benefit-Risk Assessment with Examples”, with the collaboration of CDER colleagues, in which we used real examples to demonstrate BR measurements, focusing on the global-benefit-risk score, personalized BR measures, Bayesian modeling for sequential measurements, decision rules, etc. You can get more details on this subject from my presentation, posted on the MDD’s website.
New questions:
What in your view are the most important changes in CDRH’s view of the least-burdensome approach (revised draft guidance), compared to 2002? What impact would you expect the changes to have on study designs and statistical analyses?
Least-burdensome has always been and still is a very important concept for the FDA, and has been implemented in the Center. Besides commonly perceived concept of lessening the burden on sponsors, least-burdensome also means to use the least required amount of information and time to bring a safe and effective device to market. It should be least-burdensome to the sponsor and also to FDA – the reviewers. We want to make the review process as efficient as possible. For example, if a simple design can show safety and effectiveness, why choose a more complex design that could slow down the review process and add unnecessary burden to reviewers? If sponsors want to cut down the review time, they need to help FDA in the review process: avoid unnecessarily complex study designs; provide complete but only relevant documents; clearly report analysis methods used to generate the results; include data and programs that allow verification of the results. Tell us what to focus on. Be creative to make the work more efficient. That’s exactly the least-burdensome principle.
Closing remarks
August 2016:
The Center’s (CDRH) current priorities are: establishing a national evaluation system for health technology, using real-world evidence, maintaining quality, and engaging patients. These are very sound priorities. They will help sponsors to design novel trials. We have a long way to go, but we are moving along. My priority is to meet every statistician. I have a very smart group of knowledgeable and experienced statisticians at CDRH. We are able to bring in good people and we hope to have a very low turnover rate. The Division of Biostatistics is growing fast, especially in the IVD areas because of high demand, such as in companion diagnostic devices.
There are lots of happenings in our division. There are new challenges. I am confident we will meet them with this group of smart and innovative statisticians. I am enjoying my new job.
August 2018:
So how have you enjoyed your first two years as Director of the CDRH Division of Biostatistics? Has your thinking, or market circumstances/policy from on high/etc., changed dramatically on any topic? Have you found anything particularly surprising? What are you proudest accomplishment in the last two years? What’s the view now going forward: what are your top priorities and challenges for the next two (say) years?
I like it.
The Center has moved along with those priorities, meeting and surpassing many goals. Statisticians also made significant contributions in meeting those goals, such as developing methodologies in using real-world data and for risk-benefit analyses.
I really enjoy my work at the DBS. Thanks to my predecessor Greg Campbell, and Deputy Directors Lilly Yue and Yunling Xu, we have a strong, experienced team of nearly 70 statisticians, a large majority with PhDs in statistics, who come from both industry and academia. We have a very low turnover rate of <5% per year (internal moves as well as departures to industry and to academia) and publish over 60 peer-reviewed papers per year, in addition to our regular workload. The culture in this division is very nice. We get along with each other, help each other, and learn from each other.
The biggest surprise? The number of reviews and pre-submissions each statistician has to handle, and the level of detail involved. Sometimes we have to re-analyze the submitted data or provide alternative study designs. That requires a huge amount of work, and we have to complete this in a fixed review time.
Challenges? We have a very competent and smart group. It is always a challenge to keep smart people happy and continuously motivated. We’re trying to maintain a reasonable and balanced workload for biostatisticians and create more promotion opportunities for team members to move into supervisory positions.
Top priorities going forward? With real-world evidence, digital health, and NEST all presenting challenges, we have to keep statisticians on pace with new statistical and technological developments; meet their interest in doing research; and serve the CDRH’s new organization well.
Based on what I’ve learned and the support I’ve had over the past two years, I am confident that we will do well.
By Jeng Mah, Beckman Coulter,
at JSM 2018 (Vancouver, BC, Canada)