We are excited to announce the launch of the SPES Webinar Series! Our first speaker will be Dr. Ying Jin from the Harvard Data Science Initiative. We also have an excellent lineup of upcoming speakers, including Drs. Kamran Paynabar, Peter Goos, Abby Nachtsheim, William Brenneman, and Peter Parker. Stay tuned!
Webinar Details
Date & Time: March 13, 2025, at 12:00 PM Eastern Time (US & Canada)
Registration: Register in advance here
https://queensu.zoom.us/meeting/register/0trqReAUQySAlOy0_UaLjA
Upon registering, you will receive a confirmation email with details on how to join the webinar.
Talk Information
Title: Conformal Alignment: Knowing when to trust AI models with guarantees
Speaker: Dr. Ying Jin (Harvard Data Science Initiative)
Time: 12:00 PM – 1:00 PM Eastern US & Canada Time, Thursday, March 13
Abstract: Before deploying outputs from foundation models in high-stakes tasks, it is imperative to ensure that they align with human values. For instance, in radiology report generation, reports generated by a vision-language model must align with human evaluations before their use in medical decision-making. In this talk, I introduce Conformal Alignment, a general framework for identifying units whose outputs meet a user-specified alignment criterion. Applying the conformal selection framework, we quantify the uncertainty in model prediction with conformal prediction, and select units with strong confidence with multiple testing techniques. It is guaranteed that on average, a prescribed fraction of selected units indeed meet the alignment criterion, regardless of the foundation model or the data distribution. In applications to question answering and radiology report generation, our method accurately identifies units with trustworthy outputs via lightweight training over a moderate amount of golden-label reference data. En route, we investigate the informativeness of various features in alignment prediction and combine them with standard models to probe the confidence of model predictions.
We look forward to your participation!
SPES Executive Committee
-
Chair: Devon Lin (Queen's University)
-
Past Chair: Jonathan Stallrich (North Carolina State University)
-
Chair-Elect: Arman Sabbaghi (Purdue University)
-
Treasurer: Youngdeok Hwang (Baruch College, City University of New York)
-
JSM Program Chair-Elect: Simon Mak (Duke University)
-
JSM Program Chair: Oksana Chkrebtii (Ohio State University)
-
Webmaster: Alan Gao (Ohio State University)
-
Council Representative: Annie Booth (Virginia Tech)
-
Council of Sections Governing Board Vice Chair: Andrea Cook (University of Washington)
-
Industrial Speakers Marquardt Program Chair: HaiYing Wang (University of Connecticut)
-
Chair of SPES Award Committee: E. Christi Thompson (Los Alamos National Lab)
-
SRC Management Committee Chair: Xinwei Deng (Virginia Tech)
-
Fall Technical Conference Steering Committee Representative: Anne Driscoll (Virginia Tech)
------------------------------
C. Devon Lin
Professor
Queen's University
------------------------------