CISC 371/3.0 Nonlinear Optimization
Original Author: Randy Ellis
Last Revised: 2019-03-20
Methods for computational optimization, particularly examining nonlinear functions of vectors. Topics may include: unconstrained optimization; first-order methods; second-order methods; convex problems; equality constraints; inequality constraints; applications in machine learning.
Learning hours: 120 (36L; 84P)
This course is required for the
focus of the COMP degree plan.
This course is a direct prerequisite to:
- CISC 372/3.0 (Advanced Data Analytics)
- CISC 473/3.0 (Deep Learning)
Introduction to Nonlinear Optimization:
Theory, Algorithms, and Applications with MATLAB Algebra (2014).
SIAM Press. ISBN: 978-1-61197-364-8
S. Boyd and L. Vandenberghe. Convex Optimization (2009).
Cambridge University Press.
E.K.P. Chong and S.H. Zak. An Introduction to Optimization (2011). Wiley.
- introduction to optimization, optimality conditions (1 week)
- first order methods, gradient descent, backtracking (2 weeks)
- second-order methods, Newton's method (1 week)
- convex sets, level sets (2 weeks)
- convex optimization, gradient projection (1 week)
- linear constraints and inequality constraints (1 week)
- support vector machines, dual formulations (2 weeks)
- neural networks as problems in optimization (2 weeks)