Description: D:\Users\sturler\Documents\My Lectures\math5486\InstabMaxLeft_sm.jpg

MATH 5486 / CS 5486

Numerical Analysis and Software II


Numerical Analysis and Software II

Large-Scale Optimization and Nonlinear Systems of Equations

Many of the methods discussed in part II are closely related to the Krylov methods discussed in part I of this course. However, part II can be taken independently of part I.

Large nonlinear systems of equations and optimization problems arise in the solution of nonlinear differential and integral equations, the computation of steady states of dynamical systems, parameter estimation, the optimal design of structures, inverse problems, and a range of other problems from science and engineering. Nowadays, such problems are almost a standard ingredient of any large-scale simulation or design problem. We look at several variants of Newton’s method suitable for very large problems, and we consider extensions that lead to robust methods. Robust methods are guaranteed to converge to a solution or local minimum from any starting guess (under weak and reasonable assumptions on the problem). In many cases, we will use iterative methods to solve approximately the linear systems arising in Newton-type methods, and we analyze the properties of these methods, conditions for convergence, and rates of convergence. We will also consider several methods that avoid the (in general) expensive computation of Jacobians or Hessians, such as secant methods (like Broyden’s method) and Newton-Krylov methods. For robustness, we consider line search and trust region methods. Although theoretical considerations are an inherent part of the course, we focus on practical methods for real-world applications.

This course goes substantially beyond what is taught in our 4000/5000 numerical analysis courses (which do provide a good background).



1.      Required textbook: Numerical Optimization, Nocedal and Wright, Springer

2.      Optional/Recommended textbook: Solving Nonlinear Equations with Newton’s Method, C.T. Kelley, Society for Industrial and Applied Mathematics (SIAM), 2003


The lecture notes will cover significant additional material:


Lecture Notes for Iterative Methos (part I)

Lecture Notes for Nonlinear Systems (part IIa)

Lecture Notes on Optimization (part III) – to be developed


Other useful books on nonlinear systems of equations and/or optimization are:

Quizzes will only be given on request J

Course Policies

Class Projects (link will be added shortly)