Organized by ASA BIOP 2024 Distance Learning Committee
Chair: Herb Pang, Genentech
Associate Chair: So Young Park, Eli Lilly
Open-Source Software for Regulatory Submissions/Environments & Introducing openstatsware BIOP working group
Paul Scheutte, Ya Wang
February 23
Below are the titles and abstracts of the two talks by Paul and Ya, respectively.
Title: Open-Source Software for Regulatory Submissions and Regulatory Environments
Abstracts: Regulatory submissions and regulatory computing environments have traditionally been associated with the use of proprietary software packages. While academic institutions have embraced open-source software, both industry and government have been slower to adopt open-source alternatives. I will discuss some of the challenges and issues with using open-source software in a regulatory environment, followed by some of the lessons learned in the ongoing R Consortium R Submission Pilot, as well as emerging issues.
Title: Introducing openstatsware: Who we are and what we build together
Abstracts: In this talk, we would like to introduce openstatsware, an official working group of the American Statistical Association (ASA) Biopharmaceutical Section. The working group has a primary objective to engineer R packages that implement important statistical methods, and a secondary objective to develop and disseminate best practices for engineering high-quality open-source statistical software. We will talk about what R packages we have been developing and what we have done to disseminate the best practices, as well as our long-term perspective and next steps.
We would also like to give an overview of our three active workstreams. The MMRM R package development workstream aims to develop a comprehensive R package for mixed models for repeated measures (MMRM) that is robust, well documented, and thoroughly tested. The Bayesian MMRM R package development workstream aims to develop an R package for Bayesian MMRM to support robust analysis of longitudinal clinical data. The HTA-R workstream aims to develop open-source R tools of good quality to support crucial analytic topics in Health Technology Assessment (HTA) dossier submission across various countries.
Real-World Evidence NIHU01 FDA grantees series Part 1
Marie Bradley, Tianxi Cai, Ashita Batavia
March 8
Below are the titles and abstracts of the talks by Marie, Tianxi, and Ashita, respectively.
Title: Overview of CDER’s Real-World Evidence Demonstration Projects
Abstract: Aligned with the U.S. 21st Century Cures Act, FDA established a program to evaluate the potential use of real-world evidence (RWE) in regulatory decision-making. The program is multifaceted and supports activities such as demonstration (research) projects, guidance development, internal Agency processes, external stakeholder engagement, and the Advancing Real-World Evidence initiative. This talk will present an overview of several RWE demonstration projects and will describe, for select projects, how learnings directly or indirectly serve to support FDA regulatory decision-making in evaluating the effectiveness and safety of medical products.
Title: Deriving reliable Real world evidence with electronic health records data
Abstract: Real-world clinical data hold tremendous potential to advance our understanding on the efficacy and safety of therapeutic interventions in broader populations, including disease modifying therapies for chronic diseases. However, these data remain underutilized due to methodological constraints and the inability to efficiently link and integrate data sources across study types and healthcare settings. This talk will discuss opportunities and challenges in leveraging electronic health records data to derive reliable real world evidence.
Title: Novel methods for aligning real-world progression-free survival (rwPFS) and clinical trial PFS endpoints in Multiple Myeloma
Abstract: Randomized clinical trials remain the gold standard for evaluating treatment efficacy because of their rigorous design and data collection parameters, which reduce bias and allow for valid inference of causal relationships. In Multiple Myeloma (MM), the brisk pace of drug development has seen twelve new therapies approved in the past decade - many of these were accelerated approvals based on single arm trials. Robust Real World Evidence (RWE) can enhance the interpretation of single arm studies, however comparisons between real world and clinical trial endpoints are limited by measurement bias.
J&J Innovative Medicine has established a consortium that includes leading academic RWE methodology experts, MM clinician scientists and Flatiron Health to develop novel treatment-agnostic methods for aligning rwPFS and clinical trial PFS. We will discuss our research approach, inclusion of under-represented minorities, and potential future applications of this work in this ASA Biopharma webinar.
40 years of Contributions to Statistical and Regulatory Sciences - Honoring Professor Chow
Thomas Gwise, Mark Chang, Chinfu Hsiao
Apr 5
Abstract: Come learn about and celebrate a statistician's 40 years' journey and contributions to statistical and regulatory sciences. Part of a session at the Duke-Industry Statistics Symposium 2024 will be streamed online. This includes speakers, Thomas Gwise (formerly FDA), Mark Chang (Boston University), and Chinfu Hsiao (NHRI), who will share their stories with you.
Backfilling Patients in Phase I Dose Escalation Trials
Ying Yuan
May 31
Abstract: In recent years there has been increased interest in incorporation of backfilling into dose escalation clinical trials, which involves concurrently assigning patients to doses that has been previously cleared for safety by the dose escalation design. Backfilling generates additional information on safety, tolerability, and preliminary activity on a range of doses below the maximum tolerated dose, which is relevant for selection of the recommended phase 2 dose and dose optimization. However, in practice, backfilling may not be rigorously defined in trial protocols and implemented consistently. Furthermore, backfilling designs require careful planning to minimize the probability of treating additional patients with potentially inactive agents (and/or subtherapeutic doses).
In this talk, I will propose a simple and principled approach to incorporate backfilling into the Bayesian optimal interval design (BOIN). The design integrates data from the dose escalation and backfilling components of the design and ensures that the additional patients are treated at doses where some activity has been seen. Simulation studies demonstrated that the proposed backfilling BOIN design (BF-BOIN) generates additional data for future dose optimization, maintains the accuracy of the maximum tolerated dose identification, and improves patient safety without prolonging the trial duration. The application of the design will be illustrated using an FDA-accepted trial.
Overview of HTA framework and commonly used statistical methods
Min-Hua Jen, Weili He
Jun 21
Abstract: Health Technology Assessment (HTA) is a systematic evaluation process that examines health technologies, including medications, medical devices, and prevention methods. It considers various factors such as medical, economic, social, and ethical aspects. The primary goal of HTA is to provide evidence-based information to national health authorities for decision-making on reimbursement and pricing in comparison to other available therapies. Notable advancements in HTA include mandatory joint clinical assessments (JCA) of new oncology and advanced therapies by the European Network for Health Technology Assessment (EUnetHTA) in 2025. In the United States, the US Inflation Reduction Act (IRA) will allow Medicare to negotiate drug prices directly with manufacturers starting in 2026. While HTA requirements may differ by region, the fundamental principles remain consistent. However, in the US, where there is no single payer for HTA, the evaluation process and its components are not broadly understood by drug developers, including statisticians. To address this knowledge gap, the American Statistical Association (ASA) Biopharmaceutical Section (BIOP) Health Technology Assessment (HTA) Scientific Working Group (SWG) has conducted an assessment of the HTA landscape in major markets worldwide. This webinar, hosted by the ASA BIOP HTA SWG and facilitated by the BIOP Section Distance Learning Committee, introduces the conceptual framework of HTA evaluations and commonly used statistical methodologies. Statisticians play a critical role in the reimbursement strategy for patient access and HTA submissions, making this webinar essential for statisticians in the US and worldwide.
Real-World Evidence HHSU01 FDA grantees series Part 2
Xiaofei Wang, Shu Yang, Matthew Secrest
Aug 30
Below are the titles and abstracts of the two talks by Wang & Yang, and Secrest.
Title: Methods to Improve Efficiency and Robustness of Clinical Trials Using Information from Real-World Data with Hidden Bias (Wang & Yang)
Abstract: The use of external controls (ECs) from real-world data to supplement clinical trials has the potential to expedite the development of therapies for patients. The majority of the existing methods are unable to address unmeasured differences between the external control subjects and the trial population which can lead to biased treatment effect estimates. Our U01 project aims to develop innovative statistical methods to address hidden biases when integrating real-world data (RWD) to improve the efficiency and robustness of clinical trials.
The first part of the presentation will review the research aims of the project: (a) develop a novel sensitivity analysis framework for the use of ECs from RWD sources to assess the robustness of results regarding hidden biases, b) develop efficient analytic methods that selectively borrow and adjust for data discrepancies to mitigate the impact of hidden biases, and (c) disseminate these methods through representative applications, software, website, and workshops and tutorial sessions. We will also briefly introduce the project team and summarize progress to date.
The second part of the presentation will be more technically focused. We will present a data-driven integrative framework capable of addressing unknown biases associated with the use of ECs. The adaptive nature is achieved by dynamically sorting out a set of comparable ECs via bias penalization. The proposed method can simultaneously achieve (a) the semiparametric efficiency bound when the ECs are comparable and (b) selective borrowing that mitigates the impact of the existence of noncomparable ECs. Furthermore, we establish statistical guarantees, including consistency, asymptotic distribution, and inference, providing type-I error control and good power. Extensive simulations and two real-data applications show that the proposed method leads to improved performance over the trial-only estimator across various bias-generating scenarios.
Title: Bayesian dynamic borrowing and an R package for the design and analysis of hybrid control studies (Secrest)
Abstract: While randomized controlled trials (RCTs) remain the gold standard for evaluating novel therapies, integrating external control data can enhance study power, reduce trial duration, and allow more subjects to receive the experimental therapy. However, external data can introduce bias if the RCT control and external control arms are not comparable. Bayesian dynamic borrowing (BDB) offers a solution by incorporating external data while mitigating bias.
In this webinar, we will provide a high-level overview of BDB, focusing on priors such as the power prior, commensurate prior, and robust Meta-Analytic-Predictive (rMAP) prior. These methods aim to reduce type I error and increase the power of hybrid control studies.
We will also introduce the R package {psborrow2}, designed to simplify the simulation and analysis of hybrid control data. This open-source tool eliminates the need for users to create their own MCMC samplers, lowering the technical barriers to adopting BDB methods. The package, available on GitHub (github.com/Genentech/psborrow2), provides a user-friendly interface for conducting BDB analyses and facilitates simulation studies to evaluate various trial parameters' impact on study power, type I error, and other operating characteristics.
This effort is also one of the U01 project aims and is relevant to biostatisticians in the biopharmaceutical industry and academia. By leveraging dynamic borrowing techniques and tools like {psborrow2}, stakeholders can expedite the delivery of efficacious medicines to patients while maintaining the integrity of the evidence generation process.
It Is Never Too Early to Think About Statistical Leadership
Richard C. Zink
Sep 20
Abstract: (Bio)statisticians and data scientists are technically proficient. This should come as no surprise, since their education focuses on developing quantitative scientists of the highest order. By the time one completes all of the required courses in methodology, research, programming, and technology, there is often little room for the development of soft skills. Notably, soft skills such as presenting, public speaking, scientific writing, negotiating, influencing, and relating to others are as important to one’s success as technical prowess, and these skills are of particular importance in a multidisciplinary environment. (Bio)statisticians and data scientists must communicate technical concepts clearly and convincingly to an audience that possesses little to no statistical background, often containing individuals with some underlying fear or distaste of math and statistics. Ultimately, success with non-statisticians determines how readily the quantitative scientist can contribute to important decisions in their daily work. Career growth and a healthy and thriving statistics discipline depend on the continued development and broadening of statistical influence and leadership. It is important to start early! Even as a student, it is never too early to think about statistical leadership.
This is a presentation of the Leadership in Practice Committee (LiPCom) of the Biopharmaceutical Section of the American Statistical Association. Students and young professionals are especially encouraged to attend.
FDA's Patient-Reported Outcome (PRO) Guidances & Responder Analysis of PROs
Laura Lee Johnson, Joseph C Cappelleri
Oct 25
Below are the titles and abstracts of the two talks by Laura Lee and Joe, respectively.
Title: Statistical Highlights of Recent FDA Guidances: Piecing Together Clinical Outcome Assessments, Patient Reported Outcome Measures, Endpoints, and Analysis Decisions
Abstract: The US Food and Drug Administration’s Patient-Focused Drug Development (PFDD) guidance series is an important part of biopharmaceutical statistical knowledge for anyone working with patient experience data, and in particular clinical outcome assessments (COAs). When final, the four-part PFDD guidance series will replace the guidance for industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims (published December 2009). We will briefly cover current FDA thinking on topics including considerations for constructing and analyzing an endpoint, missing data plans, nonrandomized trial design with a COA-based endpoint, and if time allows a brief introduction on evaluating the meaningfulness of clinical benefit.
Title: Adjusting for Bias in Responder Analysis of Patient-Reported Outcomes
Abstract:
Introduction: Quantitative patient-reported outcome (PRO) measures ideally are analyzed on their original scales and responder analyses are used to aid the interpretation of those primary analyses. As stated in the FDA PRO Guidance for Medical Product Development (2009), one way to lend meaning and interpretation to such a PRO measure is to dichotomize between values where within-patient changes are considered clinically important and those that are not. But even a PRO scale with a cutoff score that discriminates well between responder and non-responders is fraught with some misclassification.
Methods: Using estimates of sensitivity and specificity on classification of responder status from a PRO instrument, formulas are provided to correct for such responder misclassification under the assumption of no treatment misclassification. Two case studies from sexual medicine illustrate the methodology.
Results: Adjustment formulas on cell counts for responder misclassification are a direct extension of correction formulas for misclassification on disease from a two-way cross-classification table of disease (yes, no) and exposure (yes, no). Unadjusted and adjusted estimates of treatment effect are compared in terms of odds ratio, response ratio, and response difference. In the two case studies there was considerable underestimation of treatment effect.
Discussion and Conclusions: The methodology can be applied to different therapeutic areas. Limitations of the methodology, such as when adjusted cell estimates become negative, are highlighted. The role of anchor-based methodology is discussed for obtaining estimates of sensitivity and specificity on responder classification. Correction for treatment effect bias from misclassification of responder status on PRO measures can lead to more trustworthy interpretation and effective decision-making.
AI/ML in regulatory setting & Brief intro to LLMs
Elena Rantou, Emily Getzen
Nov 22
Below are the titles and abstracts of the two talks by Elena and Emily, respectively.
Title: Using AI/ML in a Regulatory Environment - Elena Rantou and Paul Schuette
Abstract: Artificial Intelligence and Machine Learning (AI/ML) methods assist in analyzing a huge volume of patient data and can potentially transform biopharmaceutical development. But how and to which extent are these methods used in a regulatory environment? This talk presents the available regulatory publications and highlights cases where AI/ML has been used from reviewers and scientists at FDA.
Reference will be made to FDA-Health Canada and EMA ‘Good Machine Learning Practice for Medical Device Development: Guiding Principles’ and how these principles can be suitably adaptable to drugs and biologics, as well as a recently published discussion paper from FDA centers CDER, CBER, CDRH and DHCoE: “Using Artificial Intelligence & Machine Learning in the Development of Drugs and Biological Products”.
Additionally, the talk will cover cases where AI/ML methods have been successfully employed to address topics such as data integrity and suspicious clinical sites identification, post-marketing drug evaluation, pharmacokinetic/pharmacodynamic (PK/PD) studies, precision medicine, patient enrichment and Real World Evidence (RWE), among others. The review of such cases of application of AI/ML in clinical development is summarized in Köchert et al. (2024).
Title: A brief introduction to the fundamentals of LLMs - Emily Getzen
Abstract: In this talk, we will explore large language models (LLMs) and their impact on the field of medicine. We will begin with an overview of early language models, delve into some of the core principles of the transformer architecture, and differentiate between encoding and decoding models. We will also focus heavily on biomedical applications of LLMs– medical question-answering systems, diagnostic accuracy, and identification of social determinants of health as well as exploring the pivotal role of LLMs in accelerating drug discovery and their multimodal capabilities which combine text, imaging, and other data types to create comprehensive tools for medical practice.