Dear Colleagues,
The ASA Statistical Learning and Data Science Section is pleased to announce the August webinar, presented by Dr. Linglong on August 30, 2022.
Title: Exploration and Optimization in Deep Reinforcement Learning
Speakers: Dr. Linglong Kong, Department of Mathematical and Statistical Sciences, University of Alberta, Canada
Date and Time: August 30, 2022, 2:00 to 3:30 pm Eastern Time
Registration Link: ASA SLDS Webinar Registration Link [eventbrite.com]
Abstract: Reinforcement Learning (RL) is a mathematical framework to develop intelligent agents that can learn the optimal behaviour that maximizes the cumulative reward by interacting with the environment. There are numerous successful applications in many fields. Statistics and optimization are becoming important tools for RL. In this talk, we will look at two of our recent developments. In the first example, we employ distributional RL for efficient exploration. In distributional RL, the estimated distribution of value function models both the parametric and intrinsic uncertainties. We propose a novel and efficient exploration method for deep RL that has two components: a decaying schedule to suppress the intrinsic uncertainty and an exploration bonus calculated from the upper quantiles of the learned distribution. In the second example, we study damped Anderson mixing for deep RL. Anderson mixing has been heuristically applied to RL algorithms for accelerating convergence and improving the sampling efficiency of deep RL. Motivated by that, we provide a rigorous mathematical justification for the benefits of Anderson mixing in RL. Our main results establish a connection between Anderson mixing and quasi-Newton methods, prove that Anderson mixing increases the convergence radius of policy iteration schemes by an extra contraction factor, and propose a stabilization strategy. Besides the two examples, we will discuss some current progress and future directions on statistics and optimization in RL.
Presenter: Dr. Linglong Kong is a professor at the department of Mathematical and Statistical Sciences of the University of Alberta. He is a Canada Research Chair in Statistical Learning, Fellow of Alberta Machine Intelligence Institute (Amii). He has published more than 70 peer-reviewed manuscripts including top journals AOS, JASA and JRSSB, and top conferences NeurIPS, ICML, ICDM, AAAI and IJCAI. Currently, Linglong is serving as associate editors of the Journal of the American Statistical Association, Canadian Journal of Statistics, and Statistics and its Interface, and guest editor of Statistics and its Interface, the Western North American Region of the International Biometric Society, and the ASA Statistical Computing Session program chair and webinar committee chair. He served as a guest editor of Canadian Journal of Statistics, associate editor of International Journal of Imaging Systems and Technology, guest associate editor of the Frontiers of Neurosciences, the ASA Statistical Imaging Session program chair, and member of the Board of Directors of the Statistics Society of Canada. His research interests include high-dimensional and neuroimaging data analysis, statistical machine learning, robust statistics and quantile regression, AI for smart health.
------------------------------
Zhihua Su, PhD
Associate Professor
Department of Statistics
University of Florida
zhihuasu@stat.ufl.edu------------------------------