Optimization Methods for Large-Scale Machine Learning

Date and Time: October 6, 2017 at 3:00 PM

Place: 235 Weir Hall

Speaker: Dao Nguyen, Ph.D. Assistant Professor. Department of Mathematics. University of Mississippi.

 

Abstract: This presentation reviews the past, present, and future of numerical optimization algorithms in the context of machine learning through a case study of the training of deep neural networks. It discusses how optimization problems arise in machine learning and what makes them challenging. Large-scale machine learning represents a distinctive setting in which the stochastic gradient (SG) method has traditionally played a central role while conventional gradient-based nonlinear optimization techniques typically falter. This leads to a discussion about the next generation of optimization methods for large-scale machine learning, including an investigation of two main streams of research on techniques that diminish noise in the stochastic directions and methods that make use of second-order derivative approximations.

 

Bio: Dao Nguyen is Assistant Professor of Statistics in the Department of Mathematics at The University of Mississippi. Dr. Nguyen received his Ph.D. at the University of Michigan. Before coming to The University of Mississippi, he worked as postdoc at University of California, Berkeley. He specializes in computational statistics, simulation-based inference, stochastic optimization and machine learning.