CISC 371/3.0 Nonlinear Optimization

Original Author: Randy Ellis
Last Revised: 2019-03-20

Calendar Description

Methods for computational optimization, particularly examining nonlinear functions of vectors. Topics may include: unconstrained optimization; first-order methods; second-order methods; convex problems; equality constraints; inequality constraints; applications in machine learning.

Prerequisites: CISC 271/3.0

Exclusions: CISC 351/3.0

Learning hours: 120 (36L; 84P)

Degree Planning

  • This course is required for the Data Analytics focus of the COMP degree plan.
  • This course is a direct prerequisite to:
    • CISC 372/3.0 (Advanced Data Analytics)
    • CISC 473/3.0 (Deep Learning)

Possible Texts

  • Amir Beck. Introduction to Nonlinear Optimization: Theory, Algorithms, and Applications with MATLAB Algebra (2014). SIAM Press. ISBN: 978-1-61197-364-8
  • S. Boyd and L. Vandenberghe. Convex Optimization (2009). Cambridge University Press. ISBN: 978-0-521-83378-3
  • E.K.P. Chong and S.H. Zak. An Introduction to Optimization (2011). Wiley. ISBN: 978-0-47175-800-6


  • introduction to optimization, optimality conditions (1 week)
  • first order methods, gradient descent, backtracking (2 weeks)
  • second-order methods, Newton's method (1 week)
  • convex sets, level sets (2 weeks)
  • convex optimization, gradient projection (1 week)
  • linear constraints and inequality constraints (1 week)
  • support vector machines, dual formulations (2 weeks)
  • neural networks as problems in optimization (2 weeks)