Dear Colleagues,
You are cordially invited to join 2023 Dartmouth Summer Data Science and AI Webinar Series - Week 3. Please take a look at the details of the event below.
Title: Predicting and improving generalization by measuring loss landscapes and weight matrices
Date/Time: July 20, Thursday, 11:00 AM – 12:00 PM EST.
Abstract:
This talk will present several useful metrics obtained from loss landscapes and weight matrices of neural networks. I will show how one can use these metrics to do model selection without access to any training or testing data and improve neural network test-time performance. The talk will start with a brief recap of the "phase plots" often studied in statistical physics, which can be used to taxonomize and diagnose machine learning problems based on the structure of optimization loss landscapes. Then, I will use these phase plots to motivate new ideas of network pruning and ensemble learning. Finally, I will show how to use generalization metrics from the heavy-tail analysis of neural network weight matrices to predict the quality of large language models, e.g., to do model selection on Huggingface Transformers. I will also connect these generalization metrics to prior work in statistical physics and information theory.
Bio of the speaker:
Yaoqing Yang is an assistant professor at Dartmouth College. Before that, he was a postdoctoral researcher at the RISE Lab at UC Berkeley. He received his PhD from Carnegie Mellon University and B.S. from Tsinghua University, China. He studies the fundamental aspects of machine learning, and his main contributions are towards improving reliability and generalization in the face of uncertainty, both in the data and the compute platform. His recent works focus on generalization and robustness of deep neural networks, and he also applies these studies to practical data analytics, such as 3D point clouds and graph neural networks. His works have won the best paper finalist at ICDCS and have been published multiple times in NeurIPS, ICML, CVPR and IEEE Transactions on Information Theory. He has worked as a research intern at Microsoft, MERL and Bell Labs, and two of his joint CVPR papers with MERL have both received more than 500 citations. He is also the recipient of the 2015 John and Claire Bertucci Fellowship.
This webinar will be offered online via Zoom. Please register to receive the Zoom link (one day before the webinar).
Registration link: libcal.dartmouth.edu/calendar/itc/2023DSAIW3
Look forward to seeing you at the webinar!
------------------------------
Jianjun Hua
Dartmouth College
------------------------------