Dear Colleagues,
This is a reminder that May webinar from ASA Statistical Learning and Data Science Section is on this Thursday. There is still time to register.
Title: Benign Overfitting in Two-layer Convolutional Neural Networks
Speakers: Quanquan Gu, Ph.D., Assistant Professor of Computer Science, UCLA
Date and Time: May 26, 2022, 1:00 to 2:30 pm Eastern Time
Registration Link: ASA SLDS Webinar Registration Link [eventbrite.com]
Abstract: Modern neural networks often have great expressive power and can be trained to overfit the training data, while still achieving a good test performance. This phenomenon is referred to as "benign overfitting". Recently, there emerged a line of works studying "benign overfitting" from the theoretical perspective for linear models and kernel/random feature models, while there is still a lack of theoretical understanding about when and how benign overfitting occurs in neural network models.
This talk will present a result about benign overfitting for two-layer convolutional neural networks (CNNs). We precisely characterize the conditions under which benign overfitting can occur in training two-layer CNNs. In detail, we show that when the signal-to-noise ratio satisfies a certain condition, a two-layer CNN trained by gradient descent can achieve arbitrarily small training and test losses. On the other hand, when this condition does not hold, overfitting becomes harmful and the obtained CNN can only achieve a constant level test loss. These together demonstrate a sharp phase transition between benign overfitting and harmful overfitting, driven by the signal-to-noise ratio of the data. This talk is based on joint work with Yuan Cao, Zixiang Chen and Mikhail Belkin.
Presenter: Quanquan Gu is an Assistant Professor of Computer Science at UCLA. His research is in the area of artificial intelligence and machine learning, with a focus on developing and analyzing nonconvex optimization algorithms for machine learning to understand large-scale, dynamic, complex, and heterogeneous data and building the theoretical foundations of deep learning and reinforcement learning. He received his Ph.D. degree in Computer Science from the University of Illinois at Urbana-Champaign in 2014. He is a recipient of the Sloan Research Fellowship, NSF CAREER Award, Simons Berkeley Research Fellowship among other industrial research awards.
------------------------------
Zhihua Su, PhD
Associate Professor
Department of Statistics
University of Florida
zhihuasu@stat.ufl.edu------------------------------