24-786   Advanced Optimization for Engineering

Optimization is a fundamental workhorse of today’s technological progress, be it in robotics, design, advanced manufacturing, or deep learning. This class will introduce students to concepts and tools relevant to modern-day optimization. In the first part, we will review the fundamental theory of unconstrained optimization, drawing connections with differentiable programming. We will then move on to state-of- the-art gradient-based methods, including concepts such as momentum and the Augmented Lagrangian, enabling students to understand popular solvers like Adam and ADMM. In the second part, we will cover constrained optimization. We will study in particular (sequential) quadratic programming, interior-point solvers and primal-dual formulations. In the last part of the class, we take a deep dive into global optimization. We cover, in particular, semidefinite programming for global optimality certificates, branch-and- bound methods, blackbox methods, as well as methods for sequential decision making. Throughout the class, students will program their own solvers and apply them to real- world problems, from trajectory optimization for robotics to training neural networks, using Python. There will be regular homework assignments and in-person quizzes. Knowledge of basic concepts such as convexity, gradient descent, and  multivariate calculus is required. Recommended prerequisite: 24-685 / 24-785 or similar.