
MATH 5486 / CS 5486
Numerical Analysis and
Software II

Numerical
Analysis and Software II
LargeScale
Optimization and Nonlinear Systems of Equations
Many of the methods discussed in part II are
closely related to the Krylov methods discussed in part I of this course.
However, part II can be taken independently of part I.
Large nonlinear systems of equations and optimization problems arise in
the solution of nonlinear differential and integral equations, the computation
of steady states of dynamical systems, parameter estimation, the optimal design
of structures, inverse problems, and a range of other problems from science and
engineering. Nowadays, such problems are almost a standard ingredient of any
largescale simulation or design problem. We look at several variants of
Newton’s method suitable for very large problems, and we consider extensions
that lead to robust methods. Robust methods are guaranteed to converge to a
solution or local minimum from any starting guess (under weak and reasonable
assumptions on the problem). In many cases, we will use iterative methods to
solve approximately the linear systems arising in Newtontype methods, and we
analyze the properties of these methods, conditions for convergence, and rates
of convergence. We will also consider several methods that avoid the (in
general) expensive computation of Jacobians or Hessians, such as secant methods
(like Broyden’s method) and NewtonKrylov methods.
For robustness, we consider line search and trust region methods. Although
theoretical considerations are an inherent part of the course, we focus on
practical methods for realworld applications.
This course goes substantially beyond what is taught in our 4000/5000
numerical analysis courses (which do provide a good background).
Instructor:
Textbooks:
1. Required textbook: Numerical Optimization, Nocedal and Wright, Springer
2. Optional/Recommended textbook: Solving Nonlinear Equations with Newton’s Method, C.T. Kelley, Society for Industrial and Applied Mathematics (SIAM), 2003
The lecture notes will cover significant additional material:
Lecture Notes for Iterative Methos (part I)
Lecture Notes for Nonlinear Systems (part IIa)
Lecture Notes on Optimization (part III) – to be developed
Other useful books on nonlinear systems of equations and/or optimization are:
Quizzes will only be given on request J
Class Projects (link will be added shortly)