Learning-rate-free Momentum SGD with Reshuffling Converges in Nonsmooth Nonconvex Optimization 发布时间：2025-10-28
- 活动时间：
- 活动地址：
- 主讲人：
题 目：Learning-rate-free Momentum SGD with Reshuffling Converges in Nonsmooth Nonconvex Optimization
嘉 宾：肖纳川 助理教授 香港中文大学（深圳）
主持人：邓琪 副教授 上海交通大学安泰经济与管理学院
时 间：2025年10月31日（周五）14:30-16:00
地 点：徐汇校区安泰浩然308室
内容简介:
In this talk, we propose a generalized framework for developing learning-rate-free momentum stochastic gradient descent (SGD) methods in the minimization of nonsmooth nonconvex functions, especially in training nonsmooth neural networks. Our framework adaptively generates learning rates based on the historical data of stochastic subgradients and iterates. Under mild conditions, we prove that our proposed framework enjoys global convergence to the stationary points of the objective function in the sense of the conservative field, hence providing convergence guarantees for training nonsmooth neural networks. Based on our proposed framework, we propose a novel learning-rate-free momentum SGD method (LFM). Preliminary numerical experiments reveal that LFM performs comparably to the state-of-the-art learning-rate-free methods (which have not been shown theoretically to be convergent) across well-known neural network training benchmarks.
演讲人简介：
Dr. Nachuan Xiao is an Assistant Professor at the School of Data Science, Chinese University of Hong Kong (Shenzhen). He received his Ph.D. from the Academy of Mathematics and Systems Science, Chinese Academy of Sciences in 2021 under the supervision of Prof. Ya-xiang Yuan. After that, he joined the Institute of Operations Research and Analytics at the National University of Singapore as Research Fellow, working with Prof. Kim-Chuan Toh. His research focuses computational optimization, especially on developing efficient numerical methods for nonconvex constrained optimization problems. His work has been published in top Journals in the fields of optimization and machine learning, including SIAM Journal on Optimization, Mathematics of Operations Research, Journal of Machine Learning Research, Mathematical Programming Computation, and IMA Journal of Numerical Analysis.
欢迎广大师生参加!
